VISTA: a variable length genetic algorithm and LSTM-based surrogate assisted ensemble selection algorithm in multiple layers ensemble system.

Han, Kate; Nguyen, Truong Thanh; Vu, Viet Anh; Liew, Alan Wee-Chung; Dang, Truong; Nguyen, Tien Thanh

Abstract

We proposed a novel ensemble selection method called VISTA for multiple layers ensemble systems (MLES). Our ensemble model consists of multiple layers of ensemble of classifiers (EoC) in which the EoC in each layer is trained on the data generated by a concatenation of the original training data and the predictions by classifiers of the previous layer. The predictions of the EoC in the final layer are aggregated to obtain the final prediction. To enhance the accuracy of the MLES, we used the Variable-Length Genetic Algorithm (VLGA) to search for the optimal configuration of EoC in each layer. Since the optimisation process is computationally intensive, we use Surrogate-Assisted Evolutionary Algorithms (SAEA) to reduce the training time. Most surrogate models developed in the literature require a fixed-length input, which limits their applications when the encoding is of variable length. In this paper, we proposed to use a Long Short-Term Memory (LSTM)-based surrogate model, in which the LSTM transforms the variable-length encoding to a fixed-size representation which will then be used by the surrogate model to predict the fitness values in VLGA. For the surrogate model, we adopted Radial Basis Function (RBF) for surrogation. We first conducted experiments in comparing two types of LSTM converters, and the results suggest that the proposed chunk-based LSTM converter provides better results compared to the normal LSTM converter. Our experiments on 15 datasets show that VISTA outperforms several benchmark algorithms.

Citation

HAN, K., NGUYEN, T.T., VU, V.A., LIEW, A.W.-C., DANG, T. and NGUYEN, T.T. 2024. VISTA: a variable length genetic algorithm and LSTM-based surrogate assisted ensemble selection algorithm in multiple layers ensemble system. In Proceedings of the 2024 IEEE (Institute of Electrical and Electronics Engineers) Congress on evolutionary computation (CEC 2024), 30 June - 5 July 2024, Yokohama, Japan. Piscataway: IEEE [online], article 10612029. Available from: https://doi.org/10.1109/CEC60901.2024.10612029

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 2024 IEEE (Institute of Electrical and Electronics Engineers) Congress on evolutionary computation (CEC 2024)
• Start Date: Jun 30, 2024
• End Date: Jul 5, 2024
• Acceptance Date: Mar 15, 2024
• Online Publication Date: Jul 5, 2024
• Publication Date: Dec 31, 2024
• Deposit Date: Aug 16, 2024
• Publicly Available Date: Aug 16, 2024
• Publisher: Institute of Electrical and Electronics Engineers (IEEE)
• Peer Reviewed: Peer Reviewed
• DOI: https://doi.org/10.1109/CEC60901.2024.10612029
• Keywords: Ensemble learning; Ensemble selection; Classifier selection; Ensemble of classifiers; Surrogate model
• Public URL: https://rgu-repository.worktribe.com/output/2434463