Research advance of surgical robots based on monocular vision tracking and positioning system

doi:10.19438/j.cjoms.2024.05.013

Abstract

Abstract: With the rapid development of digital surgical technology, the application of surgical robotic systems in oral and maxillofacial surgery has become an inevitable trend due to the demand for minimally invasive and precise treatment. Among the filed of computer vision-based target tracking and positioning, monocular vision system has the advantages of simple structure, fast speed, high accuracy and few calibration steps, and has been attempted in surgical robots. In this paper, the development history and application status of surgical robots was reviewed, the advantages, limitations and research progress of monocular vision localization technology were summarized, and then the progress and prospects of surgical robots with monocular vision system was discussed.

Key words: Surgical robot, Target tracking and positioning, Monocular vision, Craniomaxillofacial surgery

CLC Number:

R782.05

HUANG Xin-yi, LI Wen-hao, LEI Yun-dan, HUANG Jing-yang, YU Hong-bo. Research advance of surgical robots based on monocular vision tracking and positioning system[J]. China Journal of Oral and Maxillofacial Surgery, 2024, 22(5): 487-492.

References

[1] Wu J, Hui W, Niu J, et al.Collaborative control method and experimental research on robot-assisted craniomaxillofacial osteotomy based on the force feedback and optical navigation[J]. J Craniofac Surg, 2022, 33(7): 2011-2018.
[2] Lin HH, Lonic D, Lo LJ.3D printing in orthognathic surgery - a literature review[J]. J Formos Med Assoc, 2018, 117(7): 547-558.
[3] Ahn J, Choi H, Hong J, et al. Tracking accuracy of a stereo camera-based augmented reality navigation system for orthognathic surgery[J]. J Oral Maxillofac Surg, 2019, 77(5): 1070.e1-1070.e11.
[4] Sun T, Lei B, Zhong Y, et al.Accuracy and security analysis of a cranio-maxillofacial plastic surgery robot equipped with piezosurgery in genioplasty[J]. J Craniofac Surg, 2022, 33(5):1533-1536.
[5] 赵琛, 曹煜桢, 徐凯. 手术机器人的前世今生[J]. 世界科学, 2023(2): 35-38.
Zhao C, Cao YZ, Xu K.The past and present lives of surgical robots[J]. World Science, 2023(2): 35-38.
[6] Hemli JM, Patel NC.Robotic cardiac surgery[J]. Surg Clin North Am, 2020, 100(2): 219-236.
[7] Lin J, Yan S, Ye Z, et al.A systematic review of MAKO-assisted unicompartmental knee arthroplasty[J]. Int J Med Robot, 2020, 16(5): 1-7.
[8] Fatima N, Massaad E, Hadzipasic M, et al.Safety and accuracy of robot-assisted placement of pedicle screws compared to conventional free-hand technique: a systematic review and meta analysis[J]. Spine J , 2021, 21(2): 181-192.
[9] Kanzaki M.Current status of robot-assisted thoracoscopic surgery for lung cancer[J]. Surg Today, 2019, 49(10): 795-802.
[10] Han JJ, Woo SY, Yi WJ, et al.A robot arm and image-guided navigation assisted surgical system for maxillary repositioning in orthognathic surgery: a phantom skull-based trial[J]. Appl Sci, 2020, 10(4): 1549.
[11] 段星光, 高亮, 李建玺, 等. 颅颌面外科手术机器人空间配准及实验[J]. 机器人, 2018, 40(1): 64-71, 80.
Duan XG, Gao L, Li JX, et al.Spatial registration of a cranio-maxillofacial surgery robot and its experiments[J]. Robot, 2018, 40(1): 64-71, 80.
[12] Li H, Zhang Y, Ye B, et al.A Monocular vision positioning and tracking system based on deep neural network[J]. J Eng, 2023(3): e12246.
[13] Truong MTN, Pak M, Kim S.Single object tracking using particle filter framework and saliency-based weighted color histogram[J]. Multimed Tools Appl, 2018, 77(22) : 30067-30088.
[14] Wang H, Li R, Gao Y, et al.Target recognition and localization of mobile robot with monocular PTZ camera[J]. J Robot, 2019: 8789725.
[15] 张晓丽, 张龙信, 肖满生, 等. 快速多域卷积神经网络和光流法融合的目标跟踪[J]. 计算机工程与科学, 2020, 42(12): 2217-2222.
Zhang XL, Zhang LX, Xiao MS, et al.Target tracking by deep fusion of fast multi-domain convolutional neural network and optical flow method[J]. Computer Engineering and Science, 2020, 42(12): 2217-2222.
[16] Shi L,Deng X,Wang J,et al.Multiple object tracking based on optical flow and Kalman filtering[J]. J Comput Appl, 2017, 37(S1): 131-136.
[17] Liu D, Liu WJ, Fei BW, et al.A new method of anti-interference matching under foreground constraint for target tracking[J]. Acta Automatica Sinica, 2018, 44(6): 1138-1152.
[18] Topkaya IS, Erdogan H.Using spatial overlap ratio of independent classifiers for likelihood map fusion in mean-shift tracking[J]. SIViP, 2019, 13(1): 61-67.
[19] Wang H, Wang XY, Yu LJ, et al.Design of mean shift tracking algorithm based on target position prediction[C]. Tianjin: IEEE International Conference on Mechatronics and Automation (ICMA), 2019: 1114-1119.
[20] 付灿, 袁亮, 张建杰. 基于Camshift算法的神经丝蛋白自动跟踪[J]. 机械设计与制造, 2021, 360(2): 174-177.
Fu C,Yuan L, Zhang JJ.Neurofilament protein automaticaly tracking using camshift algorithm[J]. Machinery Design & Manufacture, 2021, 360(2): 174-177.
[21] Gong H, Ren H, Shi T, et al.Sparse subspace single target tracking algorithm based on improved particle filtering[J]. Mod Electron Tech, 2018, 41(13): 10-13.
[22] Liu TJ.Adaptive hierarchical particle filter in dynamic tracking scenarios[J]. J Electron Compon Inf Technol, 2018, 1(1) :17-21.
[23] Chen Z, Bo Y, Tian M, et al.Dynamic perceptive bat algorithm used to optimize particle filter for tracking multiple targets[J]. J Aerosp Eng, 2018, 31(3) : 04018015.
[24] Bolme DS, Beveridge JR, Draper BA, et al.Visual object tracking using adaptive correlation filters[C]. San Francisco: CVRP 2010—IEEE Computer Society Conference On Computer Vision And Pattern Recognition , 2010: 2544-2550.
[25] Henriques JF, Caseiro R, Martins P, et al.Exploiting the circulant structure of tracking-by-detection with kernels[C]. Florence: 12th European Conference on Computer Vision, 2012: 702-715.
[26] Sun Y, Pan B, Fu Y.Correlation filters tissue tracking with application to robotic minimally invasive surgery[J]. Int J Med Robot, 2022, 18(6): e2440.
[27] Li B, Yan J, Wu W, et al.High performance visual tracking with siamese region proposal network[C]. Salt Lake City: IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
[28] Guo D, Wang J, Cui Y, et al.SiamCAR: Siamese fully convolutional classification and regression for visual tracking[C]. Seattle: IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[29] Li B, Wu W, Wang Q, et al.SiamRPN++: evolution of siamese visual tracking with very deep networks[C]. Los Angeles: IEEE Conference on Computer Vision and Pattern Recognition, 2019:4282-4291.
[30] Kubo A, Meshgi K, lshi S. A meta-Q-learning approach to discriminative correlation filter based visual tracking[J]. J Intell Robot Syst, 2021,101(1):article No.11.
[31] Garcia Nespolo R, Yi D, Cole E, et al.Evaluation of artificial intelligence-based intraoperative guidance tools for phacoemulsification cataract surgery[J]. JAMA Ophthalmol, 2022, 140(2): 170-177.
[32] Zhao Z, Chen Z, Voros S, et al.Real-time tracking of surgical instruments based on spatio-temporal context and deep learning[J]. Comput Assist Surg (Abingdon), 2019, 24(Sup1): 20-29.
[33] Nwoye CI, Mutter D, Marescaux J, et al.Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos[J]. Int J Comput Assist Radiol Surg, 2019, 14(6): 1059-1067.
[34] 李松涛, 宋光明, 李雅雯, 等. 基于导线纹理的架空导线巡检机器人视觉定位[J]. 仪表技术与传感器, 2022, 468(1): 77-81.
Li ST, Song GM, Li YW, et al.Visual positioning for overhead transmission line inspection robot based on transmission line texture[J]. Instrument Technique and Sensor, 2022, 468(1): 77-81.
[35] Wu X, Guan F, Xu A.Passive ranging based on planar homography in a monocular vision system[J]. J Inf Process Syst, 2020, 16(1): 155-170 .
[36] 王巍, 白天宇. 基于单目视觉的手术器械位姿估计模型研究[J]. 激光杂志, 2023, 44(1): 32-41.
Wang W, Bai TY.Research on the pose estimation model of surgical instruments based on monocular vision[J]. Laser Journal, 2023, 44(1): 32-41.
[37] Zhao YZ, Geng SL.Face occlusion detection algorithm based on yolov5[J]. J Phys Conf Ser, 2021, 2031: 012053.
[38] Zhao Y, Zhang XH, Yan JW, et al.A wheat spike detection method in UAV images based on improved YOLOv5[J]. Remote Sens, 2021, 13(16): 3095.