Tracking and estimation of surgical instrument position and angle in surgical robot using vision system.

Muthukrishnan, Ramkumar; Kannan, Somasundar; Prabhu, Radhakrishna; Zhao, Yafan; Bhowmick, Parijat; Hasan, Md Junayed

Abstract

A da Vinci robot endoscopic-camera gives surgeons a magnified 2D view of the operating area, but additional time is required to detect and estimate the location of the surgical-instrument during an operation. The main focus and novelty of this research is to develop a new virtual coloured marker-based tracking algorithm for estimating the posture and orientation of the instrument. Initially, the developed algorithm begins by determining the coloured area of the instrument as reference-contour. Followed by a new Virtual-Rotating Bounding Rectangle (V-RBR) created over the reference-contour by meeting the minimum area of contour criteria. Additionally, a new Virtual Dynamic Multi-line Crossbar and a Virtual Static Graph (VDMC-VSG) was constructed to trace the movement of V-RBR, which helps to estimate the pose and angle of the targeted instrument in 2D during observations. V-RBR is considered as virtual coloured marker, it avoids ambient illumination-related difficulties. The proposed approach performed excellently in Gazebo-simulation and the overall accuracy is 91.3 % obtained by comparing with Robot Operating System (ROS)-based Transform measuring system, which uses robot kinematics.

Citation

MUTHUKRISHNAN, R., KANNAN, S., PRABHU, R., ZHAO, Y., BHOWMICK, P. and HASAN, M.J. 2023. Tracking and estimation of surgical instrument position and angle in surgical robot using vision system. In Proceedings of the 2023 Network, multimedia and information technology international conference (NMITCON 2023), 1-2 September 2023, Bengaluru, India. Piscataway: IEEE [online], article number 10275983. Available from: https://doi.org/10.1109/NMITCON58196.2023.10275983

• Conference Name: 2023 Network, multimedia and information technology international conference (NMITCON 2023)
• Conference Location: Bengaluru, India
• Start Date: Sep 1, 2023
• End Date: Sep 2, 2023
• Acceptance Date: Jul 30, 2023
• Online Publication Date: Sep 2, 2023
• Publication Date: Oct 17, 2023
• Deposit Date: Nov 13, 2023
• Publicly Available Date: Nov 13, 2023
• Publisher: Institute of Electrical and Electronics Engineers (IEEE)
• ISBN: 9798350300826
• DOI: https://doi.org/10.1109/NMITCON58196.2023.10275983
• Keywords: Image processing; Robot operating system; Gazebo simulation; Surgical robotics
• Public URL: https://rgu-repository.worktribe.com/output/2139465