Use virtual reality
“Handling the VR arm is very exciting because when I first started doing it I tended to try to move my own arm,” Imbria said. “Now, I’ve learned that just thinking about moving it makes it work a lot better. Sometimes when I learn to do a new task with the virtual arm and hand, I have to think about how I would move my real hand and arm just to put it in my mind, so I can imagine doing it with the virtual arm. Sometimes I lose control of the virtual arm, so I hum or close my eyes and think about how to do it. It’s exciting. There are a lot of different methods that I try to make things work when they don’t work very well.
During her four-hour testing sessions, Imbria performs activities that seem repetitive. With a VR headset slipped over his eyes, he uses his mind to move a floating hand in a virtual reality environment. For this activity, he reaches out to pick up a cup from a table, moves it to line it up with a target presented by the system, and puts it down. Later, Imbria will focus as researchers feed his brain with sensory information as he reports where he feels the sensation and what it looks like.
“There are 62 different channels that all affect my sense of touch in a different place,” Imbria explained. “I can feel some of it on the surface in some places, or maybe deep down, like under the nail. And there are distinct differences in the feelings as well, like with the ones right on my fingertip, on the surface it looks like a paperclip or a needle pushing on it. But that other on the edge of my thumb, I feel like I’m touching the evergreens to clean the needles, and they sting me.
Even just a few months after her operation, Imbria progressed rapidly in the study.
“Scott can do a variety of different jobs right now,” Downey said. “He can reach and position the hand in space, grab and move objects, and even grab objects with different forces. He also performs sensory tasks, which involve us stimulating his brain, and he reports what the sensations look like, how strong they are, whether it’s tingling or pressure. We’re using it to figure out exactly how we can change the way we stimulate the brain to change how Scott feels and try to make his hand feel like it’s touching a natural object.
All of the data from these repeated trials is then fed into modeling systems that the research team uses to help refine the computer programs that translate Imbrian thoughts into motion and provide him with a sense of touch. The hope is to use this information to inform not only future work with Imbria, but also future studies and possibly to develop technologies that can be widely and regularly used by those paralyzed or missing.
“All of the sensory signals that come from the hand give you information about the objects you interact with and your interactions with them,” said Bensmaia, professor of biology and anatomy of the James and Karen Frank family organisms. “Our goal is to give the users of these robots the dexterity that we are endowed with with our own native hands. On the motor side, we want to understand how the brain naturally controls the hand. How can we speak the language of the brain and be able to recognize this language by moving this robotic hand? On the sensory side, when you touch something, how is it that the nervous system reacts to it? And how does this neural response give rise to a perceptual experience? “
While the researchers hope to find other quadriplegic subjects who may be interested in participating in this study, they recognize that not all applicants will have the same level of functionality as Umbria. Imbria was an ideal candidate for this trial due to his pre-existing spinal cord injury, limited use and feel of his hands, and his willingness to devote time to the study – he’s in the lab 15 hours a day. week to do experiments.
In addition, he enjoys the work.
“Scott is a gift,” Bensmaia said. “He’s someone with unbridled enthusiasm, and he’s so generous and so positive and so motivated. He came over, got these implants in his brain, and the next day he was like, “When can we start? He just hit the ground running and is already excellent at everything. “