Deep heterogeneous ensemble.

Nguyen, Tien Thanh; Dang, Manh Truong; Pham, Tien Dung; Dao, Lan Phuong; Luong, Anh Vu; McCall, John; Liew, Alan Wee Chung

Abstract

In recent years, deep neural networks (DNNs) have emerged as a powerful technique in many areas of machine learning. Although DNNs have achieved great breakthrough in processing images, video, audio and text, it also has some limitations such as needing a large number of labeled data for training and having a large number of parameters. Ensemble learning, meanwhile, provides a learning model by combining many different classifiers such that an ensemble of classifiers is better than using single classifier. In this study, we propose a deep ensemble framework called Deep Heterogeneous Ensemble (DHE) for supervised learning tasks. In each layer of our algorithm, the input data is passed through a feature selection method to remove irrelevant features and prevent overfitting. The cross-validation with K learning algorithms is applied to the selected data, in order to obtain the meta-data and the K base classifiers for the next layer. In this way, one layer will output the meta-data as the input data for the next layer, the base classifiers, and the indices of the selected meta-data. A combining algorithm is then applied on the meta-data of the last layer to obtain the final class prediction. Experiments on 30 datasets confirm that the proposed DHE is better than a number of well-known benchmark algorithms.

Citation

NGUYEN, T.T., DANG, M.T., PHAM, T.D., DAO, L.P., LUONG, A.V., MCCALL, J. and LIEW, A.W.C. 2019. Deep heterogeneous ensemble. Australian journal of intelligent information processing systems [online], 16(1): special issue on neural information processing: proceedings of the 26th International conference on neural information processing (ICONIP 2019), 12-15 December 2019, Sydney, Australia, pages 1-9. Available from: http://ajiips.com.au/papers/V16.1/v16n1_5-13.pdf

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 26th International conference on neural information processing (ICONIP 2019)
• Acceptance Date: Sep 14, 2019
• Online Publication Date: Dec 31, 2019
• Publication Date: Dec 31, 2019
• Deposit Date: May 5, 2020
• Publicly Available Date: May 6, 2020
• Journal: Australian journal of intelligent information processing systems
• Print ISSN: 1321-2133
• Electronic ISSN: 1321-2133
• Publisher: Australian Journal of Intelligent Processing Systems
• Peer Reviewed: Peer Reviewed
• Volume: 16
• Issue: 1
• Pages: 1-9
• Keywords: Deep neural networks; Ensemble learning systems; Ensemble systems; Machine learning
• Public URL: https://rgu-repository.worktribe.com/output/905693
• Publisher URL: http://ajiips.com.au/papers/V16.1/v16n1_5-13.pdf