July 2020 Massive Science
Surrounded by the buzz of medical equipment, a blind man raises his hand to a touch screen. Pop! A vision of the letter “N,” placed in his brain, flashes through his mind. He traces his finger across the screen, replicating the vision with perfect form. It sounds like science fiction. But in a recent study at Baylor College of Medicine, researchers made the blind see. A team led by neurosurgeon Daniel Yoshor “drew” letters of the alphabet on blind people’s brains by giving them specific patterns of electrical zaps. These patterns caused the participants to “see” the letters in their mind’s eye. The results could improve medical devices for people who have experienced other types of sensory or motor loss.
A participant in the study is shown drawing a letter that was stimulated in his brain
The researchers accomplished this by giving patterns of small electrical stimulations to the visual cortex. The visual cortex is one of the hubs in the brain that responds to what we see. This region contains a spatial map of our field of view, meaning particular sets of cells respond to visual information coming from particular locations in our line of sight. Turn on a light on the left side of your field of view, and one set of cells will respond by shooting off an electrical signal. Turn on a light on the right side, and a different set of cells will respond.
Yoshor’s team took advantage of this map in a clever way. Because the cells in the visual cortex respond to patterns of light in space, the scientists could reverse the process — give a tiny electrical zap to a particular group of cells and cause someone to perceive a spot of light at a specific location. They performed brain surgery on blind adults to implant a small electrical device with several points of contact to the visual cortex. Each point could be activated individually or in combination with others to stimulate the brain in precise patterns. By carefully controlling the combinations of activated areas, the researchers could cause someone to “see” a specific shape, such as a letter of the alphabet.
But there was one problem. Other researchers had tried similar experiments, but they hadn’t been successful in creating a detailed image in someone’s mind. These previous experiments stimulated discrete patches of cells, like a shape drawn with dotted lines. But instead of seeing a continuous shape, people just saw several blobs of light. In order to create a detailed mental image, Yoshor’s team had to improve on existing techniques for stimulating the brain.
They realised that they could borrow a technique that’s common in cochlear implants and auditory therapy to improve the way they stimulated the brain’s visual centre. This new technique allowed them to replace their dotted sketch with a single continuous line to “draw” shapes such as letters on the brain. They reasoned that the brain might be able to make more sense of a stimulation pattern that is drawn continuously, since the pattern would more closely match the appearance of most letters in daily life.
To achieve this continuous drawing in a person’s brain, the researchers gradually changed the level of activation of each point of contact. Instead of turning each point just on or off, they raised or lowered the dial on each one to create a wide range of activity in the corresponding brain cells. When they gradually increased the activation of one point of contact while gradually decreasing the activation of another, they caused people to “see” a continuous line of light corresponding to the space between the two active points of contact in the brain. Participants described the image “like a line being drawn” in their mind’s eye.
Now that the researchers had improved their technique for stimulating the brain, they were able to test whether complex activation patterns could cause people to perceive shapes like letters. They stimulated groups of several points in the brain, adjusting the dial at each point to draw lines in spatial arrangements that matched the form of real letters. They asked people to replicate the shapes they saw on a touch screen, and also to respond aloud to indicate which letters they perceived.
The participants were consistently able to reproduce and name letters that matched the stimulation pattern drawn on their brains. When asked to name letters in rapid succession, they accurately named nearly 90 percent at a rate of 85 shapes per minute. In other words, the new stimulation technique successfully generated clearly shaped letters in people’s minds.
But the technique isn’t perfect. The activation patterns aren’t precise enough to allow people to tell the difference between similar shapes, like the letters “C” and “O”. And the participants in this experiment became blind as adults, meaning they already knew what the letters looked like. It’s likely that it would have been more difficult to replicate the experiment’s results in someone who had been born blind and had never seen letters or other shapes.
Nevertheless, this experiment is a promising step forward in scientists’ ongoing effort to create a device that could restore vision to those who have lost it. With further improvements in the technology, scientists might be able to stimulate the brain with a finer resolution and generate more detailed mental images. The results could have implications for other types of sensory function, too. Finer stimulation methods could improve pitch perception in users of cochlear implants, where a version of this technique has already been tried, and could help restore a detailed sense of touch in cases of nerve damage. Besides sensory systems, the results might also improve medical devices that help with motor control. For example, prosthetic limbs rely on technology that reads activity in the motor regions of the brain. Scientists could use their new knowledge about stimulating the brain to read and interpret the brain activity that controls a prosthetic, allowing people to control their new limbs in more precise ways.
It’s clear that the results uncovered by Yoshor’s team are a crucial development in the ongoing quest to create a visual aid for the blind. In the future, the technology might allow blind people to see the complex objects that surround them in real time.