Robust Anomaly Detection on Unreliable Data

Abstract : Abstract—Classification algorithms have been widely adopted to detect anomalies for various systems, e.g., IoT and cloud, under the common assumption that the data source is clean, i.e., features and labels are correctly set. However, data collected from the field can be unreliable due to careless annotations or malicious data transformation for incorrect anomaly detection. In this paper, we present a two-layer learning framework for robust anomaly detection (RAD) in the presence of unreliable anomaly labels. The first layer of quality model filters the suspicious data, where the second layer of classification model detects the anomaly types. We specifically focus on two use cases, (i) detecting 10classes of IoT attacks and (ii) predicting 4 classes of task failures of big data jobs. Our evaluation results show that RAD can robustly improve the accuracy of anomaly detection, to reach up to 98% for IoT device attacks (i.e., +11%) and up to 83% for cloud task failures (i.e., +20%), under a significant percentage of altered anomaly labels.
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-02056558
Contributor : Zilong Zhao <>
Submitted on : Monday, March 4, 2019 - 4:35:14 PM
Last modification on : Tuesday, September 17, 2019 - 10:39:27 AM

Identifiers

  • HAL Id : hal-02056558, version 1

Citation

Zilong Zhao, Sophie Cerf, Robert Birke, Bogdan Robu, Sara Bouchenak, et al.. Robust Anomaly Detection on Unreliable Data. 49th IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2019), Jun 2019, Portland, Oregon, United States. ⟨hal-02056558⟩

Share

Metrics

Record views

351