The robot watches instructional videos and becomes an experienced surgeon

BALTIMORE – How-to videos are great for people who want to pick up a few skills quickly. Now it turns out they're also great for robots who want to become great surgeons. In a groundbreaking step toward robotic autonomy, researchers have developed an AI system that can perform complex surgeries with the same precision as experienced human doctors. The secret? Let the robots learn by watching the professionals do it first.

Traditionally, programming robots to perform even the simplest surgical maneuvers required careful programming of each movement by hand. Now the Johns Hopkins University team led by Axel Krieger has cracked the code using a revolutionary technique called imitation learning.

“It's really magical to have this model and all we do is give it camera input and it can predict the robotic movements required for the operation,” Krieger said in a media release. “We believe this represents a significant step forward toward a new frontier in medical robotics.”

The researchers trained their model using a wealth of footage captured by wrist-mounted cameras on the popular da Vinci Surgical System robots. With nearly 7,000 of these robots in use worldwide and over 50,000 surgeons trained on the platform, the team was able to draw on an extensive archive of surgical procedures.

“All we need is an image input and then this AI system finds the right action,” explains lead author Ji Woong “Brian” Kim. “We find that even with a few hundred demos, the model is able to learn the procedure and generalize to new environments it has not yet encountered.”

The model was capable of completing three basic surgical tasks: needle manipulation, tissue tightening and suturing. In each case, the robot performance was comparable to that of human doctors.

“The model here is so good at learning things we didn’t teach it,” says Krieger. “For example, if it drops the needle, it will automatically pick it up and continue. I didn’t teach him that.”