A feature transformation technique for improving ensemble learning systems.

Nguyen, Truong Thanh; Vu, Hieu; Dang, Truong; Elyan, Eyad; Vu, Thanh Son; Nguyen, Tien Thanh

Abstract

In this study, we propose a feature transformation approach to improve the performance of Ensemble Learning Systems. Our method operates on the predictions of base classifiers within an ensemble system, known as meta-data, which are produced by applying a Cross-Validation procedure on the training data using various learning algorithms. The goal is to transform the meta-data to obtain a better combined output. Specifically, we compute the center vectors associated with each class label on the meta-data. We then increase the dimensionality of the meta-data by concatenating it with residuals, which are obtained by subtracting each vector in the meta-data from the associated center vector. A classifier is then trained on this transformed meta-data to create the combiner which will be used to combine outputs of base classifiers for ensemble prediction. Our method is compared with three well-known ensemble methods: deep forest (gcForest), Decision Template (DT), Sum rule, and the same ensemble without transformation. We evaluated the experimental methods on 22 UCI datasets and used the Friedman and Nemenyi tests to statistically compare their performances. The results show that our method outperforms all base classifiers and the benchmark algorithms on experimental datasets.

Citation

NGUYEN, T.T., VU, H., DANG, T., ELYAN, E., VU, T.S. and NGUYEN, T.T. 2025. A feature transformation technique for improving ensemble learning systems. In Nguyen, N.T., Matsuo, T., Gaol, F.L. et al (eds.) Intelligent information and database systems: proceedings of the 17th Asian conference on intelligent information database systems 2025 (ACIIDS 2025), 23-25 April 2025, Kitakyushu, Japan. Lecture notes in computer science, 15684. Singapore: Springer [online], part II, pages 292-306. Available from: https://doi.org/10.1007/978-981-96-6005-6_21

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 17th Asian conference on intelligent information and database systems 2025 (ACIIDS 2025)
• Start Date: Apr 23, 2025
• End Date: Apr 25, 2025
• Acceptance Date: Feb 10, 2025
• Online Publication Date: Apr 21, 2025
• Publication Date: Apr 21, 2025
• Deposit Date: Aug 7, 2025
• Publicly Available Date: Apr 22, 2026
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Volume: part II
• Pages: 292-306
• Series Title: Lecture notes in computer science
• Series Number: 15684
• Book Title: Intelligent information and database systems
• ISBN: 9789819660049
• DOI: https://doi.org/10.1007/978-981-96-6005-6_21
• Keywords: Ensemble learning; Ensemble combining; Ensemble aggregation; Feature transformation; Ensemble of classifiers
• Public URL: https://rgu-repository.worktribe.com/output/2836763