Real-time Anomaly Detection from Environmental Data Streams

Modern sensor networks monitor a wide range of phenomena. They are applied in environmental monitoring, health care, optimization of industrial processes, social media, smart city solutions, and many other domains. All in all, they provide a continuously pulse of the almost infinite activities that are happening in the physical space—and in cyber space. The handling of the massive amounts of generated measurements poses a series of (Big Data) challenges. Our work addresses one of these challenges: the detection of anomalies in real-time. In this paper, we propose a generic solution to this problem, and introduce a system that is capable of detecting anomalies, generating notifications, and displaying the recent situation to the user. We apply CUSUM a statistical control algorithm and adopt it so that it can be used inside the Storm framework—a robust and scalable real-time processing framework. We present a proof of concept implementation from the area of environmental monitoring.

SCHADE Sven;  BELMONTE Oscar;  HUERTA Joaquin; 

TRILLES Sergi; 

2015-06-26

Springer International Publishing

JRC94919

978-3-319-16786-2,   

1863-2246,   

http://link.springer.com/chapter/10.1007/978-3-319-16787-9_8,    https://publications.jrc.ec.europa.eu/repository/handle/JRC94919,   

10.1007/978-3-319-16787-9,