Robotic arms with a brain-machine interface allows partially paralyzed man to feed himself

Recent advances in neural science, robotics, and software have enabled scientists to develop a robotic system that responds to muscle movement signals from a partially paralyzed person relayed through a brain-machine interface. Human and robot act as a team to make performing some tasks a piece of cake.

Two robotic arms – a fork in one hand, a knife in the other – flank a seated man, who sits in front of a table, with a piece of cake on a plate. A computerized voice announces each action: “moving fork to food” and “retracting knife.” Partially paralyzed, the man makes subtle motions with his right and left fists at certain prompts, such as “select cut location”, so that the machine slices off a bite-sized piece. Now: “moving food to mouth” and another subtle gesture to align the fork with his mouth.

In less than 90 seconds, a person with very limited upper body mobility who hasn’t been able to use his fingers in about 30 years, just fed himself dessert using his mind and some smart robotic hands.

A team led by researchers at the Johns Hopkins Applied Physics Laboratory (APL), in Laurel, Maryland, and the Department of Physical Medicine and Rehabilitation (PMR) in the Johns Hopkins School of Medicine, published a paper in the journal Frontiers in Neurorobotics that described this latest feat using a brain-machine interface (BMI) and a pair of modular prosthetic limbs.

Free daily newsletter

Also sometimes referred to as a brain-computer interface, BMI systems provide a direct communication link between the brain and a computer, which decodes neural signals and ‘translates’ them to perform various external functions, from moving a cursor on a screen to now enjoying a bite of cake. In this particular experiment, muscle movement signals from the brain helped control the robotic prosthetics.

A new approach

The study built on more than 15 years of research in neural science, robotics, and software, led by APL in collaboration with the Department of PMR, as part of the Revolutionizing Prosthetics program, which was originally sponsored by the US Defense Advanced Research Project Agency (DARPA). The new paper outlines an innovative model for shared control that enables a human to maneuver a pair of robotic prostheses with minimal mental input.

“This shared control approach is intended to leverage the intrinsic capabilities of the brain machine interface and the robotic system, creating a ‘best of both worlds’ environment where the user can personalize the behavior of a smart prosthesis,” said Dr Francesco Tenore, a senior project manager in APL’s Research and Exploratory Development Department. The paper’s senior author, Tenore focuses on neural interface and applied neuroscience research.

“Although our results are preliminary, we are excited about giving users with limited capability a true sense of control over increasingly intelligent assistive machines,” he added.

► Read original article
► Download original article (pdf)

Helping people with disabilities

One of the most important advances in robotics demonstrated in the paper is combining robot autonomy with limited human input, with the machine doing most of the work while enabling the user to customize robot behavior to their liking, according to Dr David Handelman, the paper’s first author and a senior roboticist in the Intelligent Systems Branch of the Research and Exploratory Development Department at APL.

“In order for robots to perform human-like tasks for people with reduced functionality, they will require human-like dexterity. Human-like dexterity requires complex control of a complex robot skeleton,” he explained. “Our goal is to make it easy for the user to control the few things that matter most for specific tasks.”

Dr Pablo Celnik, project principal investigator in the department of PMR said: “The human-machine interaction demonstrated in this project denotes the potential capabilities that can be developed to help people with disabilities.”

Closing the loop

While the DARPA program officially ended in August 2020, the team at APL and at the Johns Hopkins School of Medicine continues to collaborate with colleagues at other institutions to demonstrate and explore the potential of the technology.

The next iteration of the system may integrate previous research that found providing sensory stimulation to amputees enabled them to not only perceive their phantom limb, but use muscle movement signals from the brain to control a prosthetic. The theory is that the addition of sensory feedback, delivered straight to a person’s brain, may help him or her perform some tasks without requiring the constant visual feedback in the current experiment.

“This research is a great example of this philosophy where we knew we had all the tools to demonstrate this complex bimanual activity of daily living that non-disabled people take for granted,” Tenore said. “Many challenges still lie ahead, including improved task execution, in terms of both accuracy and timing, and closed-loop control without the constant need for visual feedback.”

Celnik added: “Future research will explore the boundaries of these interactions, even beyond basic activities of daily living.”