Automatic Tuning of Machine Learning Systems for Novelty Detection

Abstract

Novelty detection is the ability to identify test data that are different in some aspects to the usual normal data. This ability is particularly useful for applications such as fraud detection, failure detection, medical diagnosis, video surveillance, selective learning, and obstacle detection in robotics, among others. Novelty detection often uses machine learning algorithms, where their hyperparameter values define their performance. Therefore, it emerges the need for automatic configuration techniques. This thesis shows the application of automatic design configuration tools to novelty detection problems and related areas. We showed the advantage of these tools in two study cases. In the first case, we adopted the artificial bee colony algorithm for tuning novelty detectors in robotics, specifically the grow-when-required neural networks and the simple evolving connectionist systems. We trained the novelty detectors in an outdoor environment with images captured by an unmanned aerial vehicle. Then, we added some objects to the environment that should be detected as novel objects. Under this setup, we explored the performance of traditional visual features such as color histograms, GIST descriptors, and color angular indexing. We also proposed using the pre-trained MobileNetV2 as a feature extractor. Our results showed the benefits of using tuned novelty detectors with the features extracted by the MobileNetV2. Our second study proposed an automatic design methodology based on genetic programming to select and combine saliency detection algorithms using fuzzy logic combination rules, morphological operations, and image processing filters. Saliency detection is strongly related to novelty localization in images. The improvements offered by the combination models are demonstrated by comparing their performance against several state-of-the-art saliency detection methods on four benchmark datasets. The results were analyzed using two statistical tests. Both tests confirmed that the proposed method outperforms all of the other algorithms and that its performance advantage is statistically significant.