In recent years, deep neural networks (DNNs) have emerged as a powerful technique in many areas of machine learning. Although DNNs have achieved great breakthrough in processing images, video, audio and text, it also has some limitations such as needing a large number of labeled data for training and having a large number of parameters. Ensemble learning, meanwhile, provides a learning model by combining many different classifiers such that an ensemble of classifiers is better than using single classifier. In this study, we propose a deep ensemble framework called Deep Heterogeneous Ensemble (DHE) for supervised learning tasks. In each layer of our algorithm, the input data is passed through a feature selection method to remove irrelevant features and prevent overfitting. The cross-validation with K learning algorithms is applied to the selected data, in order to obtain the meta-data and the K base classifiers for the next layer. In this way, one layer will output the meta-data as the input data for the next layer, the base classifiers, and the indices of the selected meta-data. A combining algorithm is then applied on the meta-data of the last layer to obtain the final class prediction. Experiments on 30 datasets confirm that the proposed DHE is better than a number of well-known benchmark algorithms.
NGUYEN, T.T., DANG, M.T., PHAM, T.D., DAO, L.P., LUONG, A.V., MCCALL, J. and LIEW, A.W.C. 2019. Deep heterogeneous ensemble. Australian journal of intelligent information processing systems [online], 16(1): special issue on neural information processing: proceedings of the 26th International conference on neural information processing (ICONIP 2019), 12-15 December 2019, Sydney, Australia, pages 1-9. Available from: http://ajiips.com.au/papers/V16.1/v16n1_5-13.pdf