Skip to main content

Research Repository

Advanced Search

DEFEG: deep ensemble with weighted feature generation.

Luong, Anh Vu; Nguyen, Tien Thanh; Han, Kate; Vu, Trung Hieu; McCall, John; Liew, Alan Wee-Chung

Authors

Anh Vu Luong

Kate Han

Trung Hieu Vu

Alan Wee-Chung Liew



Abstract

With the significant breakthrough of Deep Neural Networks in recent years, multi-layer architecture has influenced other sub-fields of machine learning including ensemble learning. In 2017, Zhou and Feng introduced a deep random forest called gcForest that involves several layers of Random Forest-based classifiers. Although gcForest has outperformed several benchmark algorithms on specific datasets in terms of classification accuracy and model complexity, its input features do not ensure better performance when going deeply through layer-by-layer architecture. We address this limitation by introducing a deep ensemble model with a novel feature generation module. Unlike gcForest where the original features are concatenated to the outputs of classifiers to generate the input features for the subsequent layer, we integrate weights on the classifiers’ outputs as augmented features to grow the deep model. The usage of weights in the feature generation process can adjust the input data of each layer, leading the better results for the deep model. We encode the weights using variable-length encoding and develop a variable-length Particle Swarm Optimisation method to search for the optimal values of the weights by maximizing the classification accuracy on the validation data. Experiments on a number of UCI datasets confirm the benefit of the proposed method compared to some well-known benchmark algorithms.

Citation

LUONG, A.V., NGUYEN, T.T., HAN, K., VU, T.H., MCCALL, J. and LIEW, A.W.-C. 2023. DEFEG: deep ensemble with weighted feature generation. Knowledge-based systems [online], 275, article 110691. Available from: https://doi.org/10.1016/j.knosys.2023.110691

Journal Article Type Article
Acceptance Date May 28, 2023
Online Publication Date Jun 2, 2023
Publication Date Sep 5, 2023
Deposit Date Jun 5, 2023
Publicly Available Date Mar 29, 2024
Journal Knowledge-based systems
Print ISSN 0950-7051
Publisher Elsevier
Peer Reviewed Peer Reviewed
Volume 275
Article Number 110691
DOI https://doi.org/10.1016/j.knosys.2023.110691
Keywords Ensemble method; Deep learning; Multiple classifiers; Ensemble of classifiers; Random forest; Feature generation
Public URL https://rgu-repository.worktribe.com/output/1977787

Files





You might also like



Downloadable Citations