Informed pair selection for self-paced metric learning in Siamese neural networks.

Martin, Kyle; Wiratunga, Nirmalie; Massie, Stewart; Clos, Jérémie

Abstract

Siamese Neural Networks (SNNs) are deep metric learners that use paired instance comparisons to learn similarity. The neural feature maps learnt in this way provide useful representations for classification tasks. Learning in SNNs is not reliant on explicit class knowledge; instead they require knowledge about the relationship between pairs. Though often ignored, we have found that appropriate pair selection is crucial to maximising training efficiency, particularly in scenarios where examples are limited. In this paper, we study the role of informed pair selection and propose a 2-phased strategy of exploration and exploitation. Random sampling provides the needed coverage for exploration, while areas of uncertainty modeled by neighbourhood properties of the pairs drive exploitation. We adopt curriculum learning to organise the ordering of pairs at training time using similarity knowledge as a heuristic for pair sorting. The results of our experimental evaluation show that these strategies are key to optimising training.

Citation

MARTIN, K., WIRATUNGA, N., MASSIE, S. and CLOS, J. 2018. Informed pair selection for self-paced metric learning in Siamese neural networks. In Bramer, M. and Petridis, M. (eds.) Artificial intelligence XXXV: proceedings of the 38th British Computer Society's Specialist Group on Artificial Intelligence (SGAI) International conference on innovative techniques and applications of artificial intelligence (AI-2018), 11-13 December 2018, Cambridge, UK. Lecture notes in computer science, 11311. Cham: Springer [online], pages 34-49. Available from: https://doi.org/10.1007/978-3-030-04191-5_3

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 38th British Computer Society's Specialist Group on Artificial Intelligence (SGAI) International conference on innovative techniques and applications of artificial intelligence (AI-2018)
• Start Date: Dec 11, 2018
• End Date: Dec 13, 2018
• Acceptance Date: Sep 3, 2018
• Online Publication Date: Nov 16, 2018
• Publication Date: Dec 31, 2018
• Deposit Date: Jan 21, 2019
• Publicly Available Date: Nov 17, 2019
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Pages: 34-49
• Series Title: Lecture notes in computer science
• Series Number: 11311
• Series ISSN: 0302-9743
• Book Title: Artificial intelligence XXXV
• ISBN: 9783030041908
• DOI: https://doi.org/10.1007/978-3-030-04191-5_3
• Keywords: Deep learning; Siamese neural networks; Active learning; Case based reasoning; Machine learning; Metric learning
• Public URL: http://hdl.handle.net/10059/3269
• Contract Date: Jan 21, 2019