Novelty Detection for Multivariate Data Streams with Probabilistic Models

Mehr zum Buch

Novelty detection refers to the autonomous identification of unexpected changes in data, often utilizing multivariate data streams from multiple sensors. Such changes can indicate events like cardiac arrhythmias, power failures, storms, or network attacks, impacting both systems and their environments. This doctoral thesis explores online novelty detection methods in multivariate data streams, introducing the CANDIES methodology. A key aspect of CANDIES is the separation of the input space of a probabilistic model into High-Density Regions (HDR) and Low-Density Regions (LDR), with tailored detection techniques for each. Unlike traditional detectors that primarily identify novelties in LDR, CANDIES can also detect novelties in HDR, while effectively managing concept drift and noise in data streams. Additionally, CANDIES conceptualizes novelties as clusters of anomalies with spatial or temporal relationships. The thesis emphasizes the experimental evaluation of novelty detection algorithms, presenting a data generator for synthesizing data streams and novelties, alongside a new evaluation measure, the FDS, specifically for assessing novelty detection methods. All developed methods, algorithms, and tools are publicly and freely accessible online.