The suitability of a quasi-Newton algorithm for estimating fitness-fatigue models: sensitivity, troublesome local optima and implications for future research (an in silico experimental design).

Stephens Hemingway, Ben; Swinton, Paul; Ogorek, Ben

Abstract

Fitting an FFM via NLS in practice assumes that a unique optimal solution exists and can be found by the algorithm applied. However, this idealistic scenario may not hold for two reasons: 1) the absolute minimum may not be unique; and 2) local minima, saddle points, and/or plateau features may exist that cause problems for certain algorithms. If there exist different parameter sets in the domain that share the same global minimum under standard NLS, then there is a situation where parameters aren't uniquely identified without additional constraints or regularisation terms. However, more likely is that problems with the typical FFM fitting process will stem from the existence of local minima, saddles, or plateau features that cause the algorithm to converge to a solution not equal to the global minimum. Local optima can provoke sensitivities in the fitting process for first and second-order algorithms that are by definition local optimisers. This manifests as sensitivity to initial parameter estimates (i.e. the starting point the algorithm initialises the search from). The extent of starting point sensitivity is largely unknown in the context of FFMs for common algorithms adopted and has not been studied directly. Given this concern, research reporting a single model solution derived from 'one shot' minimisation of NLS via typical first and second-order algorithms is fundamentally limited by possible uncertainty as to the suitability of fitted estimates as global minimisers. Therefore, the primary aim of this study was to investigate the sensitivity of a classical first-order search algorithm to selection of initial estimates when fitting a fitness-fatigue model (FFM) via nonlinear least-squares (NLS), and to subsequently assess the existence of local optima. A secondary aim of this study was to examine the implications of any findings in relation to previous research and provide considerations for future experimentation. The aims of the study were addressed through a computer experiment (in silico) approach that adopted a deterministic assumption the FFM completely specified athlete response. Under this assumption, two FFMs (standard, and fitness-delay) were simulated under a set of hypothetical model inputs and manually selected 'true' parameter values (for each FFM), generating a set of synthetic performance data. The two FFMs were refitted to the synthetic performance data without noise (and under the same model inputs) by the quasi-Newton L-BFGS-B algorithm in a repetitive fashion initiated from multiple starting points in the parameter space, attempting to at each search recover the true parameter values. Estimates obtained from this process were then further transformed into prediction errors quantifying in-sample model fit across the iterations and non-true solutions. Within the standard model scenarios, 69.1-70.3% of solutions found were the true parameters. In contrast, within the fitness-delay model scenarios, 17.6-17.9% of solutions found were the true parameters. A large number of unique non-true solutions were found for both the standard model (N=275-353) and the fitness-delay model (N=383-550) in this idealistic environment. Many of the non-true extrema found by the algorithm were local minima or saddles. Strong in-sample model fit was also observed across non-true solutions for both models. Collectively, these results indicate the typical NLS approach to fitting FFMs is harder for a hill-climbing algorithm to solve than previously recognised in the literature, particularly for models of higher complexity. The findings of this study add weight to the hypothesis that there exists substantial doubt in reported estimates across prior literature where local optimisers have been used or models more complex than the standard FFM applied, particularly when optimisation procedures reported have lacked the relevant detail to indicate that these issues have been considered. Future research should consider the use of global optimisation algorithms, hybrid approaches, or different perspectives (e.g. Bayesian optimisation).

Citation

STEPHENS HEMINGWAY, B., SWINTON, P. and OGOREK, B. 2021. The suitability of a quasi-Newton algorithm for estimating fitness-fatigue models: sensitivity, troublesome local optima and implications for future research (an in silico experimental design). SportRxiv [online]. Available from: https://doi.org/10.31236/osf.io/dx7gm

• Working Paper Type: Preprint
• Deposit Date: Jan 30, 2023
• Publicly Available Date: Jan 30, 2023
• DOI: https://doi.org/10.31236/osf.io/dx7gm
• Keywords: Fitness-fatigue; Athletic performance modelling; Sport performance modelling
• Public URL: https://rgu-repository.worktribe.com/output/1341424