A Semi-Supervised Learning Approach for Abnormal Event Prediction on Large Network Operation Time-Series Data

Large network logs, recording multivariate time series generated from heterogeneous devices and sensors in a network, can reveal important information about abnormal activities, such as network intrusions and packet losses. Existing machine learning methods for anomaly detection on multiple multivariate time series typically assume that 1) infrequent behaviors beyond some inference threshold are anomalous for unsupervised models or 2) require a large set of labeled normal and abnormal sequences for supervised models. However, in practice, the reported abnormal events might be available but incomplete and sparse (i.e., much fewer than normal cases). This paper presents a novel semi-supervised approach, SNetAD, that takes advantage of the incomplete and imbalanced labels to effectively learn separable feature embeddings of network activities representing normal and abnormal events. Specifically, SNetAD first generates network representations by capturing relationships across time points and between network devices. Then SNetAD encourages the embeddings to form two clusters using contrastive center loss and improves the separability of the learned clusters using labeled and unlabeled samples in a semi-supervised manner. The experiments demonstrate that SNetAD significantly outperforms state-of-the-art approaches for abnormal event prediction on a large real-world network log.