Skip to main content

Research Repository

Advanced Search

Investigating benchmark correlations when comparing algorithms with parameter tuning.

Christie, Lee A.; Brownlee, Alexander E.I.; Woodward, John R.

Authors

Alexander E.I. Brownlee

John R. Woodward



Contributors

Hernán E. Aguirre
Editor

Abstract

Benchmarks are important for comparing performance of optimisation algorithms, but we can select instances that present our algorithm favourably, and dismiss those on which our algorithm under-performs. Also related are automated design of algorithms, which use problem instances (benchmarks) to train an algorithm: careful choice of instances is needed for the algorithm to generalise. We sweep parameter settings of differential evolution to applied to the BBOB benchmarks. Several benchmark functions are highly correlated. This may lead to the false conclusion that an algorithm performs well in general, when it performs poorly on a few key instances. These correlations vary with the number of evaluations.

Citation

CHRISTIE, L.A., BROWNLEE, A.E.I. and WOODWARD, J.R. 2018. Investigating benchmark correlations when comparing algorithms with parameter tuning. In Aguirre, H.E. (ed.) Proceedings of the 2018 Genetic and evolutionary computation conference companion (GECCO'18 companion), 15-19 July 2018, Kyoto, Japan. New York: Association for Computing Machinery [online], pages 209-210. Available from: https://doi.org/10.1145/3205651.3205747

Conference Name 2018 Genetic and evolutionary computation conference (GECCO'18)
Conference Location Kyoto, Japan
Start Date Jul 15, 2018
End Date Jul 19, 2018
Acceptance Date Mar 24, 2018
Online Publication Date Jul 6, 2018
Publication Date Jul 31, 2018
Deposit Date Mar 9, 2020
Publicly Available Date Mar 17, 2020
Publisher Association for Computing Machinery
Pages 209-210
Book Title Proceedings of the 2018 Genetic and evolutionary computation conference companion (GECCO'18 companion)
ISBN 9781450357647
DOI https://doi.org/10.1145/3205651.3205747
Keywords Benchmarking; Benchmarks; Black-Box Optimization Benchmarking (BBOB); Differential evolution; Continuous optimisation; Algorithm automation
Public URL https://rgu-repository.worktribe.com/output/876279
Related Public URLs https://rgu-repository.worktribe.com/output/876294

Files

CHRISTIE 2018 Investigating benchmark correlations (PAPER) (139 Kb)
PDF

Copyright Statement
© Authors 2018. Personal use only. Not for redistribution.





You might also like



Downloadable Citations