Food places classification in egocentric images using Siamese neural networks.

Sarker, Md Mostafa Kamal; Banu, Syeda Furruka; Rashwan, Hatem A.; Abdel-Nasser, Mohamed; Singh, Vivek Kumar; Chambon, Sylvie; Radeva, Petia; Puig, Domenec

Abstract

Wearable cameras have become more popular in recent years for capturing unscripted moments in the first-person, which help in analysis of the user's lifestyle. In this work, we aim to identify the daily food patterns of a person through recognition of places relating to food in person-focused images ("selfies"). This has the potential for a system that can assist with improvements to eating habits and prevention of diet-related conditions. In this paper, we use Siamese Neural Networks (SNN) to learn similarities between images with one-shot "food places" classification. We tested our proposed method with "MiniEgoFoodPlaces", using 15 food-related locations. The proposed SNN model with MobileNet achieved an overall classification accuracy of 76.74% and 77.53% on the validation and test sets of the "MiniEgoFoodPlaces" dataset, outperforming the base models such as ResNet50, InceptionV3 and InceptionResNetV2.

Citation

SARKER, M.M.K., BANU, S.F., RASHWAN, H.A., ABDEL-NASSER, M., SINGH, V.K., CHAMBON, S., RADEVA, P. and PUIG, D. 2019. Food places classification in egocentric images using Siamese neural networks. In Sabater-Mir, J., Torra, V., Aguiló, I. and González-Hidalgo, M. (eds.) Artificial intelligence research and development: proceedings of the 22nd International conference of the Catalan Association for Artificial Intelligence (CCIA 2019), 23-25 October 2019, Colònia de Sant Jordi, Spain. Frontiers in artificial intelligence and applications, 319. Amsterdam: IOS Press [online], pages 145-151. Available from: https://doi.org/10.3233/FAIA190117

• Conference Name: 22nd International conference of the Catalan Association for Artificial Intelligence (CCIA 2019)
• Conference Location: Colònia de Sant Jordi, Spain
• Start Date: Oct 23, 2019
• End Date: Oct 25, 2019
• Acceptance Date: Oct 2, 2019
• Online Publication Date: Dec 31, 2019
• Publication Date: Dec 31, 2019
• Deposit Date: Dec 3, 2021
• Publicly Available Date: Mar 2, 2022
• Publisher: IOS Press
• Pages: 145-151
• Series Title: Frontiers in artificial intelligence and applications
• Series Number: 319
• Series ISSN: 0922-6389 ; 1879-8314
• Book Title: Artificial intelligence research and development
• ISBN: 9781643680149
• DOI: https://doi.org/10.3233/FAIA190117
• Keywords: Image recognition; Image classification; Artificial intelligence; Computer vision; Artificial neural networks
• Public URL: https://rgu-repository.worktribe.com/output/1542012