New Wearable Device Helps Blind Patients See Shapes and Colors
Dean Lloyd started losing his sight in his mid-twenties, while he was studying medicine in the early 1960s. His doctors diagnosed him with a debilitating condition known as Usher Syndrome. They told him he’d be blind and deaf in months, and that dementia would follow.
“My prognosis didn’t look good,” Lloyd recalls with a chuckle.
As it turns out, Lloyd had something else altogether. He’d been born with a genetic condition known as retinitis pigmentosa, a disease that eats away at the retina, specifically targeting those cells that detect light. Once these cells die, the eyes can’t transform and decode the incoming light in order to produce sight.
Lloyd’s condition worsened very slowly. Though his night vision wasn’t great, he doesn’t recall having other difficulties at first. He transitioned into biochemistry research at Stanford, then moved on to be a software engineer. When his loss of vision began to make coding on a computer screen too difficult — there were no aids for visually impaired programmers back then, he says — he decided to try law school. With the help of two readers, Lloyd passed the bar and started practicing law in 1982. At the time, he had very limited vision.
Then in 1989, he finally lost the ability to see.
“There was nothing for 17 years,” he says. “Until I got the Argus in 2007.”
The Argus, named after the all-seeing Greek god with 100 eyes, is a wearable computer that helps blind people see borders and boundaries at very low resolution. We first reported on the device some eight years ago, when it was still in testing. Now, at long last, the Food and Drug Administration has approved its use in the U.S. for people with Lloyd’s condition, and the device will begin selling shortly at select medical centers, including the University of California, San Francisco.
The Argus is not a true “eyes for the blind” device — patients can’t see objects in the same way people with normal eyesight can. Instead, they see black-and-white edges and contrast points, and the brain can be trained to use this artificial data as a visual guide. It provides enough visual information for the patient to gain some independence, allowing them to cross a street safely, or navigate an unfamiliar room.
“You have to learn to see again, but people who have this implant were people that used to see,” said Lloyd, one of the first patients to get the Argus. “As you go through life, you still have pictures in your brain of everything you’ve seen before. So, you’re creating yourself an image that matches what’s in your memory. It’s a concept that a lot of people don’t get when they think about this device.”
An array of hardware makes this possible. Central to the Argus system is a grid of 60 tiny electrodes, which are implanted in a patient’s retina. The patient also wears a pair of sunglasses outfitted with a video camera and a Texas Instruments digital signal processor. The chip translates the light picked up by the video camera into patterns of electrical activity, and those patterns get beamed wirelessly to a receiver connected to the grid of electrodes inside the eye. These electrodes — basically miniature glass needles — stimulate nerve cells in the retina, which then send signals to the part of the brain that processes visual information. Through this stimulation, the perception of sight is produced. Since it requires the optic nerve to be healthy, it won’t work on patients who have nerve damage or suffer from other kinds of blindness.
The device is the result of about 20 years of research and clinical trials, and its development has benefited tremendously from innovations in consumer electronics. The tiny video camera, for example, is very close to what you’d find on a cellphone, says Dr. Robert Greenberg of Second Sight, the Sylmar, California-based company that developed Argus. The images the camera captures are then processed by a DSP chip, a component common in consumer-grade electronics.
Even though the Argus is the first of its kind in the medical world, the hardware components aren’t exactly state-of-the-art. The DSP chip it’s currently using is 10 years old. Getting a medical device cleared by the FDA is a long and tedious process, and approval is only granted for a specific device. So if it’s tweaked at all, the FDA needs to take a second look. The medical space isn’t like the consumer world, where new and shiny devices are prototyped and brought to market within months. Right now, Second Sight is considering upgrading the Argus to an ARM processor or a new DSP chip for image processing, but such changes would likely result in a couple of years of testing and regulatory approvals.
“The computer technology [we use] tends to not be the latest cutting edge because of the amount of testing required,” said Greenberg.
To speed up the development process, Second Sight is turning to software instead. The company is working on a software platform called Acuboost that would make updating previously manufactured Argus models as easy as updating your computer’s operating system, says Greenberg. This is especially important because the Argus is an implanted device, and installing it inside a patient’s eye requires pretty invasive surgery. Swapping out the electrode array like you would a cellphone, a tablet, or a pair of augmented-reality goggles isn’t really an option. So software upgrades would benefit both new patients and patients who already have the device implanted.
The company is currently developing algorithms to improve resolution, image focus and zooming. Their latest software can also automate brightness adjustments and enable color recognition.
Thus far, scientists at Second Sight have been able to produce the perception of multiple colors in the lab by sending different patterns of stimulation to each electrode in the retinal implant. When the Argus camera picks up red or green, that information would be encoded through different patterns of electrical activity, which would be sent to the electrodes in the patient’s eye, creating the perception of color. The team is testing whether varying the frequency or the delay between each stimulus plays a roll in producing color “vision.” Because each patient is different, they also have to optimize the algorithm to determine each person’s color map — that is, which stimuli can reliably result in the particular patient seeing which colors.
This new direction of research grew out of feedback from current Argus patients who reported seeing colors. Because the cells that are tuned to detect specific colors are dead in these patients, the scientists at Second Sight didn’t expect any color perception to be reported.
“That’s what led to the investigation,” said USC’s Dr. Mark Humayun, the ophthalmologist and biomedical engineer who developed Argus. These new color-detecting algorithms aren’t quite ready yet, but the Argus team hopes to roll them out shortly.
Meanwhile, researchers continue to learn a lot about how an artificial sight mechanism relays the information it gathers to the brain.
“It’s different,” says Humayun. “It’s not the way we normally see, but it activates the same function.”