Dr Lee Christie l.a.christie@rgu.ac.uk
Research Fellow
Investigating benchmark correlations when comparing algorithms with parameter tuning.
Christie, Lee A.; Brownlee, Alexander E.I.; Woodward, John R.
Authors
Alexander E.I. Brownlee
John R. Woodward
Contributors
Hern�n E. Aguirre
Editor
Abstract
Benchmarks are important for comparing performance of optimisation algorithms, but we can select instances that present our algorithm favourably, and dismiss those on which our algorithm under-performs. Also related are automated design of algorithms, which use problem instances (benchmarks) to train an algorithm: careful choice of instances is needed for the algorithm to generalise. We sweep parameter settings of differential evolution to applied to the BBOB benchmarks. Several benchmark functions are highly correlated. This may lead to the false conclusion that an algorithm performs well in general, when it performs poorly on a few key instances. These correlations vary with the number of evaluations.
Citation
CHRISTIE, L.A., BROWNLEE, A.E.I. and WOODWARD, J.R. 2018. Investigating benchmark correlations when comparing algorithms with parameter tuning. In Aguirre, H.E. (ed.) Proceedings of the 2018 Genetic and evolutionary computation conference companion (GECCO'18 companion), 15-19 July 2018, Kyoto, Japan. New York: Association for Computing Machinery [online], pages 209-210. Available from: https://doi.org/10.1145/3205651.3205747
Conference Name | 2018 Genetic and evolutionary computation conference (GECCO'18) |
---|---|
Conference Location | Kyoto, Japan |
Start Date | Jul 15, 2018 |
End Date | Jul 19, 2018 |
Acceptance Date | Mar 24, 2018 |
Online Publication Date | Jul 6, 2018 |
Publication Date | Jul 31, 2018 |
Deposit Date | Mar 9, 2020 |
Publicly Available Date | Mar 28, 2024 |
Publisher | Association for Computing Machinery (ACM) |
Pages | 209-210 |
Book Title | Proceedings of the 2018 Genetic and evolutionary computation conference companion (GECCO'18 companion) |
ISBN | 9781450357647 |
DOI | https://doi.org/10.1145/3205651.3205747 |
Keywords | Benchmarking; Benchmarks; Black-Box Optimization Benchmarking (BBOB); Differential evolution; Continuous optimisation; Algorithm automation |
Public URL | https://rgu-repository.worktribe.com/output/876279 |
Related Public URLs | https://rgu-repository.worktribe.com/output/876294 |
Files
CHRISTIE 2018 Investigating benchmark correlations (PAPER)
(139 Kb)
PDF
Copyright Statement
© Authors 2018. Personal use only. Not for redistribution.
Related Outputs
You might also like
Towards explainable metaheuristics: feature extraction from trajectory mining.
(2023)
Journal Article
Explaining a staff rostering genetic algorithm using sensitivity analysis and trajectory analysis.
(2023)
Conference Proceeding
On discovering optimal trade-offs when introducing new routes in existing multi-modal public transport systems.
(2022)
Conference Proceeding
Towards explainable metaheuristics: PCA for trajectory mining in evolutionary algorithms.
(2021)
Conference Proceeding
Downloadable Citations
About OpenAIR@RGU
Administrator e-mail: publications@rgu.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search