基于单目视觉跟踪定位系统的手术机器人研究进展

doi:10.19438/j.cjoms.2024.05.013

中国口腔颌面外科杂志 ›› 2024, Vol. 22 ›› Issue (5): 487-492.doi: 10.19438/j.cjoms.2024.05.013

基于单目视觉跟踪定位系统的手术机器人研究进展

黄心怡¹, 李文皓¹, 雷云丹², 黄景揚^3,*, 于洪波^3,*

1.上海交通大学医学院,上海 200025;
2.上海交通大学生物医学工程学院,上海 200241;
3.上海交通大学医学院附属第九人民医院口腔颅颌面科,上海交通大学口腔医学院,国家口腔医学中心, 国家口腔疾病临床医学研究中心,上海市口腔医学重点实验室,上海市口腔医学研究所,上海 200011

收稿日期:2023-07-03 修回日期:2024-03-09 出版日期:2024-09-20 发布日期:2024-09-29
通讯作者: 于洪波,E-mail: yhb3508@163.com;黄景揚,E-mail: jeffhuang0816@163.com。^*共同通信作者
作者简介:黄心怡（2001-）,女,本科,E-mail: hxy79102023@163.com
基金资助:
国家自然科学基金（81571022）; 上海市自然科学基金（23ZR1438100）; 上海交通大学医学院科研计划项目（DLY201808）

Research advance of surgical robots based on monocular vision tracking and positioning system

HUANG Xin-yi¹, LI Wen-hao¹, LEI Yun-dan², HUANG Jing-yang³, YU Hong-bo³

1. Shanghai Jiao Tong University School of Medicine. Shanghai 200025;
2. Shanghai Jiao Tong University School of Biomedical Engineering. Shanghai 200241;
3. Department of Craniomaxillofacial Surgery, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine; College of Stomatology, Shanghai Jiao Tong University; National Center for Stomatology; National Clinical Research Center for Oral Diseases; Shanghai Key Laboratory of Stomatology; Shanghai Research Institute of Stomatology. Shanghai 200011, China

Received:2023-07-03 Revised:2024-03-09 Online:2024-09-20 Published:2024-09-29

摘要/Abstract

摘要： 随着数字化外科技术的迅速发展,因诊疗微创、精准化的需求,手术机器人系统应用于口腔颌面部手术成为必然趋势。基于计算机视觉的目标跟踪定位中,单目视觉系统具有结构简单、速度快、误差小和标定步骤少等优点,逐渐用于手术机器人。本文回顾手术机器人发展历史及应用现状,讨论单目视觉定位技术的优势、局限性以及研究进展,并就单目视觉系统手术机器人的进展及展望进行综述。

关键词: 手术机器人, 目标跟踪定位, 单目视觉, 颅颌面手术

Abstract: With the rapid development of digital surgical technology, the application of surgical robotic systems in oral and maxillofacial surgery has become an inevitable trend due to the demand for minimally invasive and precise treatment. Among the filed of computer vision-based target tracking and positioning, monocular vision system has the advantages of simple structure, fast speed, high accuracy and few calibration steps, and has been attempted in surgical robots. In this paper, the development history and application status of surgical robots was reviewed, the advantages, limitations and research progress of monocular vision localization technology were summarized, and then the progress and prospects of surgical robots with monocular vision system was discussed.

Key words: Surgical robot, Target tracking and positioning, Monocular vision, Craniomaxillofacial surgery

中图分类号:

R782.05

黄心怡, 李文皓, 雷云丹, 黄景揚, 于洪波. 基于单目视觉跟踪定位系统的手术机器人研究进展[J]. 中国口腔颌面外科杂志, 2024, 22(5): 487-492.

HUANG Xin-yi, LI Wen-hao, LEI Yun-dan, HUANG Jing-yang, YU Hong-bo. Research advance of surgical robots based on monocular vision tracking and positioning system[J]. China J Oral Maxillofac Surg, 2024, 22(5): 487-492.

参考文献

[1] Wu J, Hui W, Niu J, et al.Collaborative control method and experimental research on robot-assisted craniomaxillofacial osteotomy based on the force feedback and optical navigation[J]. J Craniofac Surg, 2022, 33(7): 2011-2018.
[2] Lin HH, Lonic D, Lo LJ.3D printing in orthognathic surgery - a literature review[J]. J Formos Med Assoc, 2018, 117(7): 547-558.
[3] Ahn J, Choi H, Hong J, et al. Tracking accuracy of a stereo camera-based augmented reality navigation system for orthognathic surgery[J]. J Oral Maxillofac Surg, 2019, 77(5): 1070.e1-1070.e11.
[4] Sun T, Lei B, Zhong Y, et al.Accuracy and security analysis of a cranio-maxillofacial plastic surgery robot equipped with piezosurgery in genioplasty[J]. J Craniofac Surg, 2022, 33(5):1533-1536.
[5] 赵琛, 曹煜桢, 徐凯. 手术机器人的前世今生[J]. 世界科学, 2023(2): 35-38.
Zhao C, Cao YZ, Xu K.The past and present lives of surgical robots[J]. World Science, 2023(2): 35-38.
[6] Hemli JM, Patel NC.Robotic cardiac surgery[J]. Surg Clin North Am, 2020, 100(2): 219-236.
[7] Lin J, Yan S, Ye Z, et al.A systematic review of MAKO-assisted unicompartmental knee arthroplasty[J]. Int J Med Robot, 2020, 16(5): 1-7.
[8] Fatima N, Massaad E, Hadzipasic M, et al.Safety and accuracy of robot-assisted placement of pedicle screws compared to conventional free-hand technique: a systematic review and meta analysis[J]. Spine J , 2021, 21(2): 181-192.
[9] Kanzaki M.Current status of robot-assisted thoracoscopic surgery for lung cancer[J]. Surg Today, 2019, 49(10): 795-802.
[10] Han JJ, Woo SY, Yi WJ, et al.A robot arm and image-guided navigation assisted surgical system for maxillary repositioning in orthognathic surgery: a phantom skull-based trial[J]. Appl Sci, 2020, 10(4): 1549.
[11] 段星光, 高亮, 李建玺, 等. 颅颌面外科手术机器人空间配准及实验[J]. 机器人, 2018, 40(1): 64-71, 80.
Duan XG, Gao L, Li JX, et al.Spatial registration of a cranio-maxillofacial surgery robot and its experiments[J]. Robot, 2018, 40(1): 64-71, 80.
[12] Li H, Zhang Y, Ye B, et al.A Monocular vision positioning and tracking system based on deep neural network[J]. J Eng, 2023(3): e12246.
[13] Truong MTN, Pak M, Kim S.Single object tracking using particle filter framework and saliency-based weighted color histogram[J]. Multimed Tools Appl, 2018, 77(22) : 30067-30088.
[14] Wang H, Li R, Gao Y, et al.Target recognition and localization of mobile robot with monocular PTZ camera[J]. J Robot, 2019: 8789725.
[15] 张晓丽, 张龙信, 肖满生, 等. 快速多域卷积神经网络和光流法融合的目标跟踪[J]. 计算机工程与科学, 2020, 42(12): 2217-2222.
Zhang XL, Zhang LX, Xiao MS, et al.Target tracking by deep fusion of fast multi-domain convolutional neural network and optical flow method[J]. Computer Engineering and Science, 2020, 42(12): 2217-2222.
[16] Shi L,Deng X,Wang J,et al.Multiple object tracking based on optical flow and Kalman filtering[J]. J Comput Appl, 2017, 37(S1): 131-136.
[17] Liu D, Liu WJ, Fei BW, et al.A new method of anti-interference matching under foreground constraint for target tracking[J]. Acta Automatica Sinica, 2018, 44(6): 1138-1152.
[18] Topkaya IS, Erdogan H.Using spatial overlap ratio of independent classifiers for likelihood map fusion in mean-shift tracking[J]. SIViP, 2019, 13(1): 61-67.
[19] Wang H, Wang XY, Yu LJ, et al.Design of mean shift tracking algorithm based on target position prediction[C]. Tianjin: IEEE International Conference on Mechatronics and Automation (ICMA), 2019: 1114-1119.
[20] 付灿, 袁亮, 张建杰. 基于Camshift算法的神经丝蛋白自动跟踪[J]. 机械设计与制造, 2021, 360(2): 174-177.
Fu C,Yuan L, Zhang JJ.Neurofilament protein automaticaly tracking using camshift algorithm[J]. Machinery Design & Manufacture, 2021, 360(2): 174-177.
[21] Gong H, Ren H, Shi T, et al.Sparse subspace single target tracking algorithm based on improved particle filtering[J]. Mod Electron Tech, 2018, 41(13): 10-13.
[22] Liu TJ.Adaptive hierarchical particle filter in dynamic tracking scenarios[J]. J Electron Compon Inf Technol, 2018, 1(1) :17-21.
[23] Chen Z, Bo Y, Tian M, et al.Dynamic perceptive bat algorithm used to optimize particle filter for tracking multiple targets[J]. J Aerosp Eng, 2018, 31(3) : 04018015.
[24] Bolme DS, Beveridge JR, Draper BA, et al.Visual object tracking using adaptive correlation filters[C]. San Francisco: CVRP 2010—IEEE Computer Society Conference On Computer Vision And Pattern Recognition , 2010: 2544-2550.
[25] Henriques JF, Caseiro R, Martins P, et al.Exploiting the circulant structure of tracking-by-detection with kernels[C]. Florence: 12th European Conference on Computer Vision, 2012: 702-715.
[26] Sun Y, Pan B, Fu Y.Correlation filters tissue tracking with application to robotic minimally invasive surgery[J]. Int J Med Robot, 2022, 18(6): e2440.
[27] Li B, Yan J, Wu W, et al.High performance visual tracking with siamese region proposal network[C]. Salt Lake City: IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
[28] Guo D, Wang J, Cui Y, et al.SiamCAR: Siamese fully convolutional classification and regression for visual tracking[C]. Seattle: IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[29] Li B, Wu W, Wang Q, et al.SiamRPN++: evolution of siamese visual tracking with very deep networks[C]. Los Angeles: IEEE Conference on Computer Vision and Pattern Recognition, 2019:4282-4291.
[30] Kubo A, Meshgi K, lshi S. A meta-Q-learning approach to discriminative correlation filter based visual tracking[J]. J Intell Robot Syst, 2021,101(1):article No.11.
[31] Garcia Nespolo R, Yi D, Cole E, et al.Evaluation of artificial intelligence-based intraoperative guidance tools for phacoemulsification cataract surgery[J]. JAMA Ophthalmol, 2022, 140(2): 170-177.
[32] Zhao Z, Chen Z, Voros S, et al.Real-time tracking of surgical instruments based on spatio-temporal context and deep learning[J]. Comput Assist Surg (Abingdon), 2019, 24(Sup1): 20-29.
[33] Nwoye CI, Mutter D, Marescaux J, et al.Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos[J]. Int J Comput Assist Radiol Surg, 2019, 14(6): 1059-1067.
[34] 李松涛, 宋光明, 李雅雯, 等. 基于导线纹理的架空导线巡检机器人视觉定位[J]. 仪表技术与传感器, 2022, 468(1): 77-81.
Li ST, Song GM, Li YW, et al.Visual positioning for overhead transmission line inspection robot based on transmission line texture[J]. Instrument Technique and Sensor, 2022, 468(1): 77-81.
[35] Wu X, Guan F, Xu A.Passive ranging based on planar homography in a monocular vision system[J]. J Inf Process Syst, 2020, 16(1): 155-170 .
[36] 王巍, 白天宇. 基于单目视觉的手术器械位姿估计模型研究[J]. 激光杂志, 2023, 44(1): 32-41.
Wang W, Bai TY.Research on the pose estimation model of surgical instruments based on monocular vision[J]. Laser Journal, 2023, 44(1): 32-41.
[37] Zhao YZ, Geng SL.Face occlusion detection algorithm based on yolov5[J]. J Phys Conf Ser, 2021, 2031: 012053.
[38] Zhao Y, Zhang XH, Yan JW, et al.A wheat spike detection method in UAV images based on improved YOLOv5[J]. Remote Sens, 2021, 13(16): 3095.

基于单目视觉跟踪定位系统的手术机器人研究进展

Research advance of surgical robots based on monocular vision tracking and positioning system

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	高鹏飞, 林洁. 区域神经阻滞技术用于口腔癌缺损皮瓣修复手术的研究进展[J]. 中国口腔颌面外科杂志, 2024, 22(4): 400-403.
[2]	周循, 索璐璐, 刘锦星, 周徐慧, 仇琳, 李静洁, 郑永超, 劳蔚. 颌面部鲜红斑痣患儿光动力治疗苏醒期躁动的影响因素及药物干预措施的观察性研究[J]. 中国口腔颌面外科杂志, 2024, 22(3): 261-267.
[3]	孟宇翔, 宋凯, 何宗轩, 王琳, 葛胜优, 魏晨珂, 尚伟. 微血管吻合器在股前外侧皮瓣修复口腔恶性肿瘤术后缺损中的应用效果[J]. 中国口腔颌面外科杂志, 2024, 22(3): 268-273.
[4]	范晶娴, 吴嘉骏, 葛奎, 徐兵, 王齐, 杨天页, 王繁麟. 口腔颌面间隙感染患者全麻术后机械通气时间延长的危险因素分析[J]. 中国口腔颌面外科杂志, 2024, 22(2): 158-164.
[5]	康铖, 朱挺, 韩爽, 娄伊伊. 笑气联合局麻在阻生牙拔除中的应用效果评价[J]. 中国口腔颌面外科杂志, 2024, 22(2): 165-169.
[6]	王家慧, 段晓雯, 周徐慧, 仇琳, 李静洁, 蔡美华, 郑永超. 儿童阻塞性睡眠呼吸暂停综合征全麻苏醒期躁动风险预测模型的构建与验证[J]. 中国口腔颌面外科杂志, 2024, 22(2): 175-180.
[7]	许鑫, 夏明, 金晨昱, 周韧, 徐辉. 经鼻腔气管插管固定器用于口腔颌面外科全麻患者鼻保护的效果评价[J]. 中国口腔颌面外科杂志, 2023, 21(5): 486-490.
[8]	曹爽, 康华, 陈洁. 纤维支气管镜引导对经鼻腔气管插管患者术后拔管时鼻出血的影响[J]. 中国口腔颌面外科杂志, 2023, 21(3): 253-257.
[9]	王俊安, 顾海林, 史文涛, 孙宇. 超声测量舌根至甲状软骨夹角预测喉镜暴露困难的临床研究[J]. 中国口腔颌面外科杂志, 2023, 21(2): 142-145.
[10]	冯婉晴, 李静洁, 刘锦星, 劳蔚. 儿童阻塞性睡眠呼吸暂停综合征手术苏醒期疼痛危险因素分析[J]. 中国口腔颌面外科杂志, 2023, 21(2): 175-179.
[11]	施金雅, 李静洁, 刘锦星, 仇琳. 阻塞性睡眠呼吸暂停综合征患儿全麻术后拔管时间的影响因素分析[J]. 中国口腔颌面外科杂志, 2023, 21(2): 180-185.
[12]	马东洋, 庞超远, 张文凯, 王兵武, 刘东, 张琪, 张舒萌. 基于术中CT影像的计算机导航技术在颌面深部小异物取出中的应用效果评价[J]. 中国口腔颌面外科杂志, 2022, 20(6): 581-585.
[13]	闫星泉, 南欣荣, 任晓彦, 赵佳雄, 孟兵. 3D显微镜在35例口腔颌面外科手术中的应用效果评价[J]. 中国口腔颌面外科杂志, 2022, 20(6): 590-593.
[14]	姚侃, 谢轶伦, 于雯雯, 魏思龙, 夏亮, 郑彤, 卢晓峰. 清醒及麻醉状态下5种口内扁桃体肥大评估方法可靠性比较[J]. 中国口腔颌面外科杂志, 2022, 20(5): 461-465.
[15]	刘迪, 张思敏, 阿迪力江·赛买提, 祖丽胡玛·卡迪, 木合森·牙生江, 尼加提·努尔穆罕默德, 买买提吐逊·吐尔地. 血栓弹力图检测在70例口腔颌面部缺损游离皮瓣修复围术期中的效果评价[J]. 中国口腔颌面外科杂志, 2022, 20(3): 244-250.