Locality sensitive batch selection for triplet networks.
Martin, Kyle; Wiratunga, Nirmalie; Sani, Sadiq
Professor Nirmalie Wiratunga email@example.com
Triplet networks are deep metric learners which learn to optimise a feature space using similarity knowledge gained from training on triplets of data simultaneously. The architecture relies on the triplet loss function to optimise its weights based upon the distance between triplet members. Composition of input triplets therefore directly impacts the quality of the learned representations, meaning that a training scheme which optimises their formation is crucial. However, an exhaustive search for the best triplets is prohibitive unless the search for triplets is confined to smaller training regions or batches. Accordingly, current triplet mining approaches use informed selection applied only to a random minibatch, but the resulting view fails to exploit areas of complexity in the feature space. In this work, we introduce a locality-sensitive batching strategy, which uses the locality of examples to create batches as an alternative to the commonly adopted randomly minibatching. Our results demonstrate this method to offer better performance on three image and two text classification tasks with statistical significance. Importantly most of these gains are incrementally realised with as little as 25% of the training iterations.
|Start Date||Jul 19, 2020|
|Publication Date||Sep 28, 2020|
|Publisher||Institute of Electrical and Electronics Engineers|
|Book Title||Proceedings of the 2020 Institute of Electrical and Electronics Engineers (IEEE) International joint conference on neural networks (IEEE IJCNN 2020), part of the 2020 IEEE World congress on computational intelligence (IEEE WCCI 2020) and co-located with t|
|Institution Citation||MARTIN, K., WIRATUNGA, N. and SANI, S. 2020. Locality sensitive batch selection for triplet networks. In Proceedings of the 2020 Institute of Electrical and Electronics Engineers (IEEE) International joint conference on neural networks (IEEE IJCNN 2020), part of the 2020 IEEE World congress on computational intelligence (IEEE WCCI 2020) and co-located with the 2020 IEEE congress on evolutionary computation (IEEE CEC 2020) and the 2020 IEEE International fuzzy systems conference (FUZZ-IEEE 2020), 19-24 July 2020, [virtual conference]. Piscataway: IEEE [online], article ID 9207538. Available from: https://doi.org/10.1109/IJCNN48605.2020.9207538|
|Keywords||Deep metric learning; Triplet network; Approximate nearest neighbour; Locality sensitive hashing; Batch selection; Self-paced learning|
MARTIN 2020 Locality sensitive
You might also like
Evaluating the transferability of personalised exercise recognition models.
Learning to recognise exercises for the self-management of low back pain.
A knowledge-light approach to personalised and open-ended human activity recognition.
Human activity recognition with deep metric learners.