Multimodal image and spectral feature learning for efficient analysis of water-suspended particles.

Takahashi, Tomoko; Liu, Zonghua; Thevar, Thangavel; Burns, Nicholas; Lindsay, Dhugal; Watson, John; Mahajan, Sumeet; Yukioka, Satoru; Tanaka, Shuhei; Nagai, Yukiko; Thornton, Blair

Abstract

We have developed a method to combine morphological and chemical information for the accurate identification of different particle types using optical measurement techniques that require no sample preparation. A combined holographic imaging and Raman spectroscopy setup is used to gather data from six different types of marine particles suspended in a large volume of seawater. Unsupervised feature learning is performed on the images and the spectral data using convolutional and single-layer autoencoders. The learned features are combined, where we demonstrate that non-linear dimensional reduction of the combined multimodal features can achieve a high clustering macro F1 score of 0.88, compared to a maximum of 0.61 when only image or spectral features are used. The method can be applied to long-term monitoring of particles in the ocean without the need for sample collection. In addition, it can be applied to data from different types of sensor measurements without significant modifications.

Citation

TAKAHASHI, T., LIU, Z., THEVAR, T., BURNS, N., LINDSAY, D., WATSON, J., MAHAJAN, S., YUKIOKA, S., TANAKA, S., NAGAI, Y. and THORNSTON, B. 2023. Multimodal image and spectral feature learning for efficient analysis of water-suspended particles. Optics express [online], 31(5), pages 7492-7504. Available from: https://doi.org/10.1364/OE.470878

• Journal Article Type: Article
• Acceptance Date: Jan 11, 2023
• Online Publication Date: Feb 15, 2023
• Publication Date: Feb 27, 2023
• Deposit Date: Feb 1, 2024
• Publicly Available Date: Feb 1, 2024
• Journal: Optics express
• Electronic ISSN: 1094-4087
• Publisher: Optical Society of America
• Peer Reviewed: Peer Reviewed
• Volume: 31
• Issue: 5
• Pages: 7492-7504
• DOI: https://doi.org/10.1364/OE.470878
• Keywords: Analytical techniques; Digital holographic imaging; Fourier transform infrared spectroscopy; Optical measurement; Raman spectroscopy; Spectral imaging
• Public URL: https://rgu-repository.worktribe.com/output/2114591