A Semi-Supervised Learning Approach for Abnormal Event Prediction on Large Network Operation Time-Series Data

Yijun Lin, Yao Yi Chiang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

Large network logs, recording multivariate time series generated from heterogeneous devices and sensors in a network, can reveal important information about abnormal activities, such as network intrusions and packet losses. Existing machine learning methods for anomaly detection on multiple multivariate time series typically assume that 1) infrequent behaviors beyond some inference threshold are anomalous for unsupervised models or 2) require a large set of labeled normal and abnormal sequences for supervised models. However, in practice, the reported abnormal events might be available but incomplete and sparse (i.e., much fewer than normal cases). This paper presents a novel semi-supervised approach, SNetAD, that takes advantage of the incomplete and imbalanced labels to effectively learn separable feature embeddings of network activities representing normal and abnormal events. Specifically, SNetAD first generates network representations by capturing relationships across time points and between network devices. Then SNetAD encourages the embeddings to form two clusters using contrastive center loss and improves the separability of the learned clusters using labeled and unlabeled samples in a semi-supervised manner. The experiments demonstrate that SNetAD significantly outperforms state-of-the-art approaches for abnormal event prediction on a large real-world network log.

Original languageEnglish (US)
Title of host publicationProceedings - 2022 IEEE International Conference on Big Data, Big Data 2022
EditorsShusaku Tsumoto, Yukio Ohsawa, Lei Chen, Dirk Van den Poel, Xiaohua Hu, Yoichi Motomura, Takuya Takagi, Lingfei Wu, Ying Xie, Akihiro Abe, Vijay Raghavan
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1024-1033
Number of pages10
ISBN (Electronic)9781665480451
DOIs
StatePublished - 2022
Event2022 IEEE International Conference on Big Data, Big Data 2022 - Osaka, Japan
Duration: Dec 17 2022Dec 20 2022

Publication series

NameProceedings - 2022 IEEE International Conference on Big Data, Big Data 2022

Conference

Conference2022 IEEE International Conference on Big Data, Big Data 2022
Country/TerritoryJapan
CityOsaka
Period12/17/2212/20/22

Bibliographical note

Funding Information:
This material is based upon work supported in part by the NTT Global Networks and NVIDIA Corporation.

Publisher Copyright:
© 2022 IEEE.

Keywords

  • multivariate time series
  • network event prediction
  • semi-supervised

Fingerprint

Dive into the research topics of 'A Semi-Supervised Learning Approach for Abnormal Event Prediction on Large Network Operation Time-Series Data'. Together they form a unique fingerprint.

Cite this