Abstract
Large network logs, recording multivariate time series generated from heterogeneous devices and sensors in a network, can reveal important information about abnormal activities, such as network intrusions and packet losses. Existing machine learning methods for anomaly detection on multiple multivariate time series typically assume that 1) infrequent behaviors beyond some inference threshold are anomalous for unsupervised models or 2) require a large set of labeled normal and abnormal sequences for supervised models. However, in practice, the reported abnormal events might be available but incomplete and sparse (i.e., much fewer than normal cases). This paper presents a novel semi-supervised approach, SNetAD, that takes advantage of the incomplete and imbalanced labels to effectively learn separable feature embeddings of network activities representing normal and abnormal events. Specifically, SNetAD first generates network representations by capturing relationships across time points and between network devices. Then SNetAD encourages the embeddings to form two clusters using contrastive center loss and improves the separability of the learned clusters using labeled and unlabeled samples in a semi-supervised manner. The experiments demonstrate that SNetAD significantly outperforms state-of-the-art approaches for abnormal event prediction on a large real-world network log.
Original language | English (US) |
---|---|
Title of host publication | Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022 |
Editors | Shusaku Tsumoto, Yukio Ohsawa, Lei Chen, Dirk Van den Poel, Xiaohua Hu, Yoichi Motomura, Takuya Takagi, Lingfei Wu, Ying Xie, Akihiro Abe, Vijay Raghavan |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 1024-1033 |
Number of pages | 10 |
ISBN (Electronic) | 9781665480451 |
DOIs | |
State | Published - 2022 |
Event | 2022 IEEE International Conference on Big Data, Big Data 2022 - Osaka, Japan Duration: Dec 17 2022 → Dec 20 2022 |
Publication series
Name | Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022 |
---|
Conference
Conference | 2022 IEEE International Conference on Big Data, Big Data 2022 |
---|---|
Country/Territory | Japan |
City | Osaka |
Period | 12/17/22 → 12/20/22 |
Bibliographical note
Funding Information:This material is based upon work supported in part by the NTT Global Networks and NVIDIA Corporation.
Publisher Copyright:
© 2022 IEEE.
Keywords
- multivariate time series
- network event prediction
- semi-supervised