Skip to main content

Research Repository

Advanced Search

Multi-layer heterogeneous ensemble with classifier and feature selection.

Nguyen, Tien Thanh; Pham, Nang Van ; Dang, Manh Truong; Luong, Anh Vu; McCall, John; Liew, Alan Wee-Chung


Nang Van Pham

Manh Truong Dang

Anh Vu Luong

John McCall

Alan Wee-Chung Liew


Deep Neural Networks have achieved many successes when applying to visual, text, and speech information in various domains. The crucial reasons behind these successes are the multi-layer architecture and the in-model feature transformation of deep learning models. These design principles have inspired other sub-fields of machine learning including ensemble learning. In recent years, there are some deep homogenous ensemble models introduced with a large number of classifiers in each layer. These models, thus, require a costly computational classification. Moreover, the existing deep ensemble models use all classifiers including unnecessary ones which can reduce the predictive accuracy of the ensemble. In this study, we propose a multi-layer ensemble learning framework called MUlti-Layer heterogeneous Ensemble System (MULES) to solve the classification problem. The proposed system works with a small number of heterogeneous classifiers to obtain ensemble diversity, therefore being efficiency in resource usage. We also propose an Evolutionary Algorithm-based selection method to select the subset of suitable classifiers and features at each layer to enhance the predictive performance of MULES. The selection method uses NSGA-II algorithm to optimize two objectives concerning classification accuracy and ensemble diversity. Experiments on 33 datasets confirm that MULES is better than a number of well-known benchmark algorithms.

Start Date Jul 8, 2020
Publication Date Jun 30, 2020
Publisher Association for Computing Machinery
Pages 725-733
Book Title GECCO '20: proceedings of the 2020 Genetic and evolutionary computation conference, 8-12 July 2020, CancĂșn, Mexico
ISBN 9781450371285
Institution Citation NGUYEN, T.T., PHAM, N.V., DANG, M.T., LUONG, A.V., MCCALL, J. and LIEW, A. W.-C. 2020. Multi-layer heterogeneous ensemble with classifier and feature selection. In GECCO '20: proceedings of the Genetic and evolutionary computation conference (GECCO 2020), 8-12 July 2020, Cancun, Mexico. New York: ACM [online], pages 725-733. Available from:
Keywords Ensemble method; Deep learning; Multiple classifiers; Ensemble of classifiers; Feature selection; Classifier selection


You might also like

Downloadable Citations