For decades, surgeons have dedicated extensive time to study and practice to acquire sophisticated surgical skills, meticulously perfecting each movement to ensure patient safety. However, advanced robots with artificial intelligence (AI) technology are now learning to perform surgical tasks simply by observing videos, heralding a promising era of automated surgery.
Researchers at Johns Hopkins University (JHU) and Stanford University have made a breakthrough in the field of robotic surgery. They have taught a robotic system called the “da Vinci Surgical System” to perform a variety of surgical tasks by training it using video footage. The da Vinci system is an advanced robotic platform often operated remotely by surgeons to execute precise maneuvers such as dissections, suctioning, cutting, and suturing. Such systems typically come at a high cost, exceeding $2 million, not including accessory costs, sterilization equipment, or training expenses.
However, JHU’s research has demonstrated the potential of imitation learning, a new machine learning method. Through this approach, the robotic system can learn to perform surgical tasks by watching videos of sample surgeries.
With the help of an AI model trained on surgical videos, a robotic system has successfully performed complex surgical tasks skillfully, akin to human capabilities.
Through imitation learning, the research team has enabled the da Vinci Surgical System to learn how to execute three typical surgical tasks: needle manipulation, tissue lifting, and wound suturing. Notably, the robot not only achieved these tasks with high precision but also demonstrated the ability to self-correct errors that arose during the process.
Axel Krieger, an assistant professor at JHU and co-author of the study, shared that the system was able to automatically pick up a needle when it was accidentally dropped, a maneuver that the research team had not programmed. This capability marks a significant advancement, as previous robotic systems often required human intervention when errors occurred.
da Vinci Surgical System at work.
To control the da Vinci Surgical System, the research team applied an advanced AI model similar to what is used in chatbots like ChatGPT. However, instead of processing text, the robotic system processes kinematic information – a specialized language used to describe motion. With complex mathematical elements such as data and equations, this language facilitates the simulation and control of the robotic arm’s precise movements during surgical procedures.
The system was trained using hundreds of videos documenting surgeries from the perspective of a camera mounted on the robotic arm. Through imitation learning, the robot can learn to replicate the surgeon’s maneuvers and even develop self-correction capabilities when encountering errors. This ability helps minimize mistakes and enhance accuracy during surgery.
Krieger and his team hope that this method can open up possibilities for training robots to perform any type of surgery much faster and more efficiently compared to traditional step-by-step coding methods. Instead of spending years programming each action, the AI system can now quickly learn from video data. This could help realize the goal of automated surgery faster than anticipated.
Krieger stated: “What’s new here is that we only need to collect data from various procedures, and we can train the robot to learn within a few days. This allows us to move towards the goal of automating surgery, reducing medical errors and improving accuracy.”
The robotic surgical system will work alongside a human surgeon to guide each step, allowing it to operate with higher precision.
Despite achieving remarkable milestones, automated surgery still faces numerous challenges. Currently, some advanced robotic surgical systems, such as the CorPath system by Corindus, have been utilized in complex cardiovascular procedures. However, the capabilities of these systems remain limited to certain stages of the surgical process.
According to Krieger, coding each step in surgery for a robotic system is a complex and time-consuming process. “An average person could take up to a decade just to model the suturing process for a single type of surgery,” he noted. This is why developing a system that can learn through observation is paving the way for entirely new approaches.
In 2022, Krieger’s team developed an autonomous robotic system named Smart Tissue Autonomous Robot (STAR), capable of suturing complex intestinal tissues in pigs without human intervention. STAR employs a 3D camera and machine learning tracking algorithms to complete intricate surgical tasks.
Researchers at JHU continue to develop robots to perform comprehensive surgical procedures. While fully automated surgery is still a distant goal, these advancements will contribute to enhancing surgical safety and accuracy. With progress in AI and machine learning, complex surgical procedures may become more accessible to many patients around the world.
However, this also raises questions about the role of surgeons in the future. Will robots completely replace humans on the operating table? Krieger believes that humans and robots will coexist and support each other in medicine. Robotic surgery is not about replacing humans but assisting and enabling surgeons to perform difficult, precise tasks more quickly and effectively.
