A comparative study of anomaly detection methods for gross error detection problems.

Dobos, Daniel; Nguyen, Tien Thanh; Dang, Truong; Wilson, Allan; Corbett, Helen; McCall, John; Stockton, Phil

Abstract

The chemical industry requires highly accurate and reliable measurements to ensure smooth operation and effective monitoring of processing facilities. However, measured data inevitably contains errors from various sources. Traditionally in flow systems, data reconciliation through mass balancing is applied to reduce error by estimating balanced flows. However, this approach can only handle random errors. For non-random errors (called gross errors, GEs) which are caused by measurement bias, instrument failures, or process leaks, among others, this approach would return incorrect results. In recent years, many gross error detection (GED) methods have been proposed by the research community. It is recognised that the basic principle of GED is a special case of the detection of outliers (or anomalies) in data analytics. With the developments of Machine Learning (ML) research, patterns in the data can be discovered to provide effective detection of anomalous instances. In this paper, we present a comprehensive study of the application of ML-based Anomaly Detection methods (ADMs) in the GED context on a number of synthetic datasets and compare the results with several established GED approaches. We also perform data transformation on the measurement data and compare its associated results to the original results, as well as investigate the effects of training size on the detection performance. One class Support Vector Machine outperformed other ADMs and five selected statistical tests for GED on Accuracy, F1 Score, and Overall Power while Interquartile Range (IQR) method obtained the best selectivity outcome among the top 6 AMDs and the five statistical tests. The results indicate that ADMs can potentially be applied to GED problems.

Citation

DOBOS, D., NGUYEN, T.T., DANG, T., WILSON, A., CORBETT, H., MCCALL, J. and STOCKTON, P. 2023. A comparative study of anomaly detection methods for gross error detection problems. Computers and chemical engineering [online], 175, article 108263. Available from: https://doi.org/10.1016/j.compchemeng.2023.108263

• Journal Article Type: Article
• Acceptance Date: Apr 17, 2023
• Online Publication Date: Apr 19, 2023
• Publication Date: Jul 31, 2023
• Deposit Date: Apr 19, 2023
• Publicly Available Date: Apr 19, 2023
• Journal: Computers and chemical engineering
• Print ISSN: 0098-1354
• Electronic ISSN: 1873-4375
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 175
• Article Number: 108263
• DOI: https://doi.org/10.1016/j.compchemeng.2023.108263
• Keywords: Gross error detection; Anomaly detection; Machine learning; Deep learning
• Public URL: https://rgu-repository.worktribe.com/output/1942517
• Additional Information: This article has been published with separate supporting information. This supporting information has been incorporated into a single file on this repository and can be found at the end of the file associated with this output.