Hierarchical approach to classify food scenes in egocentric photo-streams.

Martinez, Estefan�a Talavera; Leyva-Vallina, Mar�a; Sarker, Md. Mostafa Kamal; Puig, Domenec; Petkov, Nicolai; Radeva, Petia

Abstract

Recent studies have shown that the environment where people eat can affect their nutritional behavior. In this paper, we provide automatic tools for personalized analysis of a person's health habits by the examination of daily recorded egocentric photo-streams. Specifically, we propose a new automatic approach for the classification of food-related environments, which is able to classify up to 15 such scenes. In this way, people can monitor the context around their food intake in order to get an objective insight into their daily eating routine. We propose a model that classifies food-related scenes organized in a semantic hierarchy. Additionally, we present and make available a new egocentric dataset composed of more than 33,000 images recorded by a wearable camera, over which our proposed model has been tested. Our approach obtains an accuracy and F-score of 56% and 65%, respectively, clearly outperforming the baseline methods.

Citation

MARTINEZ, E.T., LEYVA-VALLINA, M., SARKER, M.M.K., PUIG, D., PETKOV, N. and RADEVA, P. 2020. Hierarchical approach to classify food scenes in egocentric photo-streams. IEEE journal of biomedical and health informatics [online], 24(3), pages 866-877. Available from: https://doi.org/10.1109/JBHI.2019.2922390

• Journal Article Type: Article
• Acceptance Date: Jun 5, 2019
• Online Publication Date: Jun 12, 2019
• Publication Date: Mar 31, 2020
• Deposit Date: Dec 4, 2021
• Publicly Available Date: Mar 2, 2022
• Journal: IEEE journal of biomedical and health informatics
• Print ISSN: 2168-2194
• Electronic ISSN: 2168-2208
• Publisher: Institute of Electrical and Electronics Engineers (IEEE)
• Peer Reviewed: Peer Reviewed
• Volume: 24
• Issue: 3
• Pages: 866-877
• DOI: https://doi.org/10.1109/JBHI.2019.2922390
• Keywords: Image recognition; Image classification; Artificial intelligence; Machine learning
• Public URL: https://rgu-repository.worktribe.com/output/1542047