VEGAS: a variable length-based genetic algorithm for ensemble selection in deep ensemble learning.

Han, Kate; Pham, Tien; Vu, Trung Hieu; Dang, Truong; McCall, John; Nguyen, Tien Thanh

Abstract

In this study, we introduce an ensemble selection method for deep ensemble systems called VEGAS. The deep ensemble models include multiple layers of the ensemble of classifiers (EoC). At each layer, we train the EoC and generates training data for the next layer by concatenating the predictions for training observations and the original training data. The predictions of the classifiers in the last layer are combined by a combining method to obtain the final collaborated prediction. We further improve the prediction accuracy of a deep ensemble model by searching for its optimal configuration, i.e., the optimal set of classifiers in each layer. The optimal configuration is obtained using the Variable-Length Genetic Algorithm (VLGA) to maximize the prediction accuracy of the deep ensemble model on the validation set. We developed three operators of VLGA: roulette wheel selection for breeding, a chunk-based crossover based on the number of classifiers to generate new offsprings, and multiple random points-based mutation on each offspring. The experiments on 20 datasets show that VEGAS outperforms selected benchmark algorithms, including two well-known ensemble methods (Random Forest and XgBoost) and three deep learning methods (Multiple Layer Perceptron, gcForest, and MULES).

Citation

HAN, K., PHAM, T., VU, T.H., DANG, T., MCCALL, J. and NGUYEN, T.T. 2021. VEGAS: a variable length-based genetic algorithm for ensemble selection in deep ensemble learning. In Nguyen, N.T., Chittayasothorn, S., Niyato, D. and Trawiński, B. (eds.) Intelligent information and database systems: proceedings of the 13th Asian conference on intelligent information and database systems 2021 (ACCIIDS 2021), 7-10 April 2021, [virtual conference]. Lecture Notes in Computer Science, 12672. Cham: Springer [online], pages 168–180. Available from: https://doi.org/10.1007/978-3-030-73280-6_14

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 13th Asian conference on intelligent information and database systems 2021 (ACIIDS 2021)
• Start Date: Apr 7, 2021
• End Date: Apr 10, 2021
• Acceptance Date: Dec 30, 2020
• Online Publication Date: Apr 5, 2021
• Publication Date: Dec 31, 2021
• Deposit Date: May 31, 2021
• Publicly Available Date: May 31, 2021
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Pages: 168-180
• Series Title: Lecture notes in computer science
• Series Number: 12672
• Series ISSN: 0302-9743
• Book Title: Intelligent information and database systems
• ISBN: 9783030732790
• DOI: https://doi.org/10.1007/978-3-030-73280-6_14
• Keywords: Deep learning; Ensemble learning; Ensemble selection; Classifier selection; Ensemble of classifiers; Genetic algorithum
• Public URL: https://rgu-repository.worktribe.com/output/1335449