Multi-layer heterogeneous ensemble with classifier and feature selection.

Nguyen, Tien Thanh; Van Pham, Nang; Dang, Manh Truong; Luong, Anh Vu; McCall, John; Liew, Alan Wee Chung

Abstract

Deep Neural Networks have achieved many successes when applying to visual, text, and speech information in various domains. The crucial reasons behind these successes are the multi-layer architecture and the in-model feature transformation of deep learning models. These design principles have inspired other sub-fields of machine learning including ensemble learning. In recent years, there are some deep homogenous ensemble models introduced with a large number of classifiers in each layer. These models, thus, require a costly computational classification. Moreover, the existing deep ensemble models use all classifiers including unnecessary ones which can reduce the predictive accuracy of the ensemble. In this study, we propose a multi-layer ensemble learning framework called MUlti-Layer heterogeneous Ensemble System (MULES) to solve the classification problem. The proposed system works with a small number of heterogeneous classifiers to obtain ensemble diversity, therefore being efficiency in resource usage. We also propose an Evolutionary Algorithm-based selection method to select the subset of suitable classifiers and features at each layer to enhance the predictive performance of MULES. The selection method uses NSGA-II algorithm to optimize two objectives concerning classification accuracy and ensemble diversity. Experiments on 33 datasets confirm that MULES is better than a number of well-known benchmark algorithms.

Citation

NGUYEN, T.T., VAN PHAM, N., DANG, M.T., LUONG, A.V., MCCALL, J. and LIEW, A.W.C. 2020. Multi-layer heterogeneous ensemble with classifier and feature selection. In GECCO '20: proceedings of the Genetic and evolutionary computation conference (GECCO 2020), 8-12 July 2020, Cancun, Mexico. New York: ACM [online], pages 725-733. Available from: https://doi.org/10.1145/3377930.3389832

• Conference Name: 2020 Genetic and evolutionary computation conference (GECCO 2020)
• Conference Location: Cancún Mexico
• Start Date: Jul 8, 2020
• End Date: Jul 12, 2020
• Acceptance Date: Mar 20, 2020
• Online Publication Date: Jun 25, 2020
• Publication Date: Jun 30, 2020
• Deposit Date: May 14, 2020
• Publicly Available Date: May 14, 2020
• Publisher: Association for Computing Machinery (ACM)
• Pages: 725-733
• Book Title: GECCO '20: proceedings of the 2020 Genetic and evolutionary computation conference, 8-12 July 2020, Cancún, Mexico
• ISBN: 9781450371285
• DOI: https://doi.org/10.1145/3377930.3389832
• Keywords: Ensemble method; Deep learning; Multiple classifiers; Ensemble of classifiers; Feature selection; Classifier selection
• Public URL: https://rgu-repository.worktribe.com/output/905846