Skip to main content

Research Repository

Advanced Search

Anomaly detection in network traffic using dynamic graph mining with a sparse autoencoder.

Jia, Guanbo; Miller, Paul; Hong, Xin; Kalutarage, Harsha; Ban, Tao


Guanbo Jia

Paul Miller

Xin Hong

Harsha Kalutarage

Tao Ban


Network based attacks on ecommerce websites can have serious economic consequences. Hence, anomaly detection in dynamic network traffic has become an increasingly important research topic in recent years. This paper proposes a novel dynamic Graph and sparse Autoencoder based Anomaly Detection algorithm named GAAD. In GAAD, the network traffic over contiguous time intervals is first modelled as a series of dynamic bipartite graph increments. One mode projection is performed on each bipartite graph increment and the adjacency matrix derived. Columns of the resultant adjacency matrix are then used to train a sparse autoencoder to reconstruct it. The sum of squared errors between the reconstructed approximation and original adjacency matrix is then calculated. An online learning algorithm is then used to estimate a Gaussian distribution that models the error distribution. Outlier error values are deemed to represent anomalous traffic flows corresponding to possible attacks. In the experiment, a network emulator was used to generate representative ecommerce traffic flows over a time period of 225 minutes with five attacks injected, including SYN scans, host emulation and DDoS attacks. ROC curves were generated to investigate the influence of the autoencoder hyper-parameters. It was found that increasing the number of hidden nodes and their activation level, and increasing sparseness resulted in improved performance. Analysis showed that the sparse autoencoder was unable to encode the highly structured adjacency matrix structures associated with attacks, hence they were detected as anomalies. In contrast, SVD and variants, such as the compact matrix decomposition, were found to accurately encode the attack matrices, hence they went undetected.

Start Date Aug 5, 2019
Publication Date Oct 31, 2019
Publisher Institute of Electrical and Electronics Engineers
Pages 458-465
Series ISSN 2324-9013
ISBN 9781728127767
Institution Citation JIA, G., MILLER, P., HONG, X., KALUTARAGE, H. and BAN, T. 2019. Anomaly detection in network traffic using dynamic graph mining with a sparse autoencoder. In Proceedings of 18th Institution of Electrical and Electronics Engineers (IEEE) International trust, security and privacy in computing and communications conference, co-located with 13th Institution of Electrical and Electronics Engineers (IEEE) International big data science and engineering conference (TrustCom/BigDataSE), 5-8 August 2019, Rotorua, New Zealand. Piscataway: IEEE [online], pages 458-465. Available from:
Keywords Anomaly detection; Network traffic; Network security; Bipartite graph; Sparse autoencoder; Dynamic graph


You might also like

Downloadable Citations