Putting the reality in virtual reality with robot-assisted surgery

Cardiac surgeons may be able to better plan operations and improve their surgical field view with the help of a robot.

Controlled through a virtual reality parallel system as a digital twin, the robot can accurately image a patient through ultrasound without the hand cramping or radiation exposure that hinder human operators. The international research team published their method in IEEE/CAA Journal of Automatica Sinica.

Intra-operative ultrasound is especially useful, as it can guide the surgery by providing real-time images of otherwise hidden devices and anatomy. However, the need for highly specialized skills is always a barrier for reliable and repeatable acquisition."

Fei-Yue Wang, Study Author and Director, State Key Laboratory of Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences

Wang noted that the availability of onsite sonographers can be limited, and that many procedures requiring intra-operative ultrasound also often require X-ray imaging, which could expose the operator to harmful radiation. To mitigate these challenges, Wang and his team developed a platform for robotic intra-operative trans-esophageal echocardiography (TEE), an imaging technique widely used to diagnose heart disease and guide cardiac surgical procedures.

"Our result has indicated the use of robot with a simulation platform could potentially improve the general usability of intra-operative ultrasound and assist operators with less experience," Wang said.

The researchers employed parallel control and intelligence to pair an operator with the robot in a virtual environment that accurately represents the real environment. Equipped with a database of ultrasound images and a digital platform capable of reconstructing anatomy, the robot could navigate the target areas for the operator to better visualize and plan potential surgical corrections in computational experiments.

"Such a system can be used for view definition and optimization to assist pre-planning, as well as algorithm evaluations to facilitate control and navigation in real-time," Wang said.

Next, the researchers plan to further integrate the currently proposed parallel real/virtual system with specific clinical needs to assist the translational research of such imaging robots.

"The ultimate goal is to integrate the virtual system and the physical robot for in-vivo clinical tests, so as to propose a new diagnosis and treatment protocol using parallel intelligence in medical operations," Wang said.