Anomaly detection in network traffic using dynamic graph mining with a sparse autoencoder.

Jia, Guanbo; Miller, Paul; Hong, Xin; Kalutarage, Harsha; Ban, Tao

Abstract

Network based attacks on ecommerce websites can have serious economic consequences. Hence, anomaly detection in dynamic network traffic has become an increasingly important research topic in recent years. This paper proposes a novel dynamic Graph and sparse Autoencoder based Anomaly Detection algorithm named GAAD. In GAAD, the network traffic over contiguous time intervals is first modelled as a series of dynamic bipartite graph increments. One mode projection is performed on each bipartite graph increment and the adjacency matrix derived. Columns of the resultant adjacency matrix are then used to train a sparse autoencoder to reconstruct it. The sum of squared errors between the reconstructed approximation and original adjacency matrix is then calculated. An online learning algorithm is then used to estimate a Gaussian distribution that models the error distribution. Outlier error values are deemed to represent anomalous traffic flows corresponding to possible attacks. In the experiment, a network emulator was used to generate representative ecommerce traffic flows over a time period of 225 minutes with five attacks injected, including SYN scans, host emulation and DDoS attacks. ROC curves were generated to investigate the influence of the autoencoder hyper-parameters. It was found that increasing the number of hidden nodes and their activation level, and increasing sparseness resulted in improved performance. Analysis showed that the sparse autoencoder was unable to encode the highly structured adjacency matrix structures associated with attacks, hence they were detected as anomalies. In contrast, SVD and variants, such as the compact matrix decomposition, were found to accurately encode the attack matrices, hence they went undetected.

Citation

JIA, G., MILLER, P., HONG, X., KALUTARAGE, H. and BAN, T. 2019. Anomaly detection in network traffic using dynamic graph mining with a sparse autoencoder. In Proceedings of 18th Institution of Electrical and Electronics Engineers (IEEE) international Trust, security and privacy in computing and communications conference, co-located with 13th IEEE international Big data science and engineering conference (TrustCom/BigDataSE), 5-8 August 2019, Rotorua, New Zealand. Piscataway: IEEE [online], pages 458-465. Available from: https://doi.org/10.1109/TrustCom/BigDataSE.2019.00068

• Conference Name: 18th Institution of Electrical and Electronics Engineers (IEEE) international Trust, security and privacy in computing and communications conference, co-located with 13th IEEE international Big data science and engineering conference (TrustCom/BigDataSE)
• Conference Location: Rotorua, New Zealand
• Start Date: Aug 5, 2019
• End Date: Aug 8, 2019
• Acceptance Date: Apr 27, 2019
• Online Publication Date: Aug 8, 2019
• Publication Date: Oct 31, 2019
• Deposit Date: Nov 21, 2019
• Publicly Available Date: Nov 21, 2019
• Publisher: Institute of Electrical and Electronics Engineers (IEEE)
• Pages: 458-465
• Series ISSN: 2324-9013
• ISBN: 9781728127767
• DOI: https://doi.org/10.1109/TrustCom/BigDataSE.2019.00068
• Keywords: Anomaly detection; Network traffic; Network security; Bipartite graph; Sparse autoencoder; Dynamic graph
• Public URL: https://rgu-repository.worktribe.com/output/742731