Benjamin Lacroix
Limitations of benchmark sets and landscape features for algorithm selection and performance prediction.
Lacroix, Benjamin; McCall, John
Abstract
Benchmark sets and landscape features are used to test algorithms and to train models to perform algorithm selection or configuration. These approaches are based on the assumption that algorithms have similar performances on problems with similar feature sets. In this paper, we test different configurations of differential evolution (DE) against the BBOB set. We then use the landscape features of those problems and a case base reasoning approach for DE configuration selection. We show that, although this method obtains good results for BBOB problems, it fails to select the best configurations when facing a new set of optimisation problems with a distinct array of landscape features. This demonstrates the limitations of the BBOB set for algorithm selection. Moreover, by examination of the relationship between features and algorithm performance, we show that there is no correlation between the feature space and the performance space. We conclude by identifying some important open questions raised by this work.
Citation
LACROIX, B. and MCCALL, J. 2019. Limitations of benchmark sets and landscape features for algorithm selection and performance prediction. In López-Ibáñe, M. (ed.) Proceedings of the 2019 Genetic and evolutionary computation conference (GECCO 2019) companion, 13-17 July 2019, Prague, Czech Republic. New York: Association for Computing Machinery [online], pages 261-262. Available from: https://doi.org/10.1145/3319619.3322051
Conference Name | 2019 Genetic and evolutionary computation conference (GECCO 2019) |
---|---|
Conference Location | Prague, Czech Republic |
Start Date | Jul 13, 2019 |
End Date | Jul 17, 2019 |
Acceptance Date | Mar 20, 2019 |
Online Publication Date | Jul 13, 2019 |
Publication Date | Jul 13, 2019 |
Deposit Date | Sep 24, 2019 |
Publicly Available Date | Sep 24, 2019 |
Publisher | ACM Association for Computing Machinery |
Pages | 261-262 |
Book Title | Proceedings of the 2019 Genetic and evolutionary computation conference (GECCO 2019) companion |
ISBN | 9781450367486 |
DOI | https://doi.org/10.1145/3319619.3322051 |
Keywords | Continuous optimisation; Numerical optimisation; Benchmark functions; Algorithm selection; Exploratory landscape analysis |
Public URL | https://rgu-repository.worktribe.com/output/325306 |
Files
LACROIX 2019 Limitations of benchmark
(1.5 Mb)
PDF
Publisher Licence URL
https://creativecommons.org/licenses/by-nc/4.0/
You might also like
Ensemble of deep learning models with surrogate-based optimization for medical image segmentation.
(2022)
Conference Proceeding
Facility location problem and permutation flow shop scheduling problem: a linked optimisation problem.
(2022)
Conference Proceeding
Towards explainable metaheuristics: PCA for trajectory mining in evolutionary algorithms.
(2021)
Conference Proceeding
Towards the landscape rotation as a perturbation strategy on the quadratic assignment problem.
(2021)
Conference Proceeding