Nov. ‘08 edition of Scientific American – promotion of BMIs and brain implants

From the November 2008 edition of Scientific American
PDF File (25.28MB) – alternative download.

Front cover – http://i36.tinypic.com/4sbdky.jpg

Article on brain chip – http://i38.tinypic.com/2ecjbco.jpg

————————-

November, 2008
Jacking into the Brain–Is the Brain the Ultimate Computer Interface?

How far can science advance brain-machine interface technology? Will we one day pipe the latest blog entry or NASCAR highlights directly into the human brain as if the organ were an outsize flash drive?
By Gary Stix

Key Concepts

- Futurists and science-fiction writers speculate about a time when brain activity will merge with computers.

- Technology now exists that uses brain signals to control a cursor or prosthetic arm. How much further development of brain-machine interfaces might progress is still an imponderable.

- It is at least possible to conceive of inputting text and other high-level information into an area of the brain that helps to form new memories. But the technical hurdles to achieving this task probably require fundamental advances in understanding the way the brain functions.

The cyberpunk science fiction that emerged in the 1980s routinely paraded “neural implants” for hooking a computing device directly to the brain: “I had hundreds of megabytes stashed in my head,” proclaimed the protagonist of “Johnny Mnemonic,” a William Gibson story that later became a wholly forgettable movie starring Keanu Reeves.

The genius of the then emergent genre (back in the days when a megabyte could still wow) was its juxtaposition of low-life retro culture with technology that seemed only barely beyond the capabilities of the deftest biomedical engineer. Although the implants could not have been replicated at the Massachusetts Institute of Technology or the California Institute of Technology, the best cyberpunk authors gave the impression that these inventions might yet materialize one day, perhaps even in the reader’s own lifetime.

In the past 10 years, however, more realistic approximations of technologies originally evoked in the cyberpunk literature have made their appearance. A person with electrodes implanted inside his brain has used neural signals alone to control a prosthetic arm, a prelude to allowing a human to bypass limbs immobilized by amyotrophic lateral sclerosis or stroke. Researchers are also investigating how to send electrical messages in the other direction as well, providing feedback that enables a primate to actually sense what a robotic arm is touching.

But how far can we go in fashioning replacement parts for the brain and the rest of the nervous system? Besides controlling a computer cursor or robot arm, will the technology somehow actually enable the brain’s roughly 100 billion neurons to function as a clandestine repository for pilfered industrial espionage data or another plot element borrowed from Gibson?

Will Human Become Machine?

Today’s Hollywood scriptwriters and futurists, less skilled heirs of the original cyberpunk tradition, have embraced these neurotechnologies. The Singularity Is Near, scheduled for release next year, is a film based on the ideas of computer scientist Ray Kurzweil, who has posited that humans will eventually achieve a form of immortality by transferring a digital blueprint of their brain into a computer or robot.

Yet the dream of eternity as a Max Headroom–like avatar trapped inside a television set (or as a copy-and-paste job into the latest humanoid bot) remains only slightly less distant than when René Descartes ruminated on mind-body dualism in the 17th century. The wholesale transfer of self—a machine-based facsimile of the perception of the ruddy hues of a sunrise, the constantly shifting internal emotional palette and the rest of the mix that combines to evoke the uniquely subjective sense of the world that constitutes the essence of conscious life—is still nothing more than a prop for fiction writers.

Hoopla over thought-controlled prostheses, moreover, obscures the lack of knowledge of the underlying mechanisms of neural functioning needed to feed information into the brain to re-create a real-life cyberpunk experience. “We know very little about brain circuits for higher cognition,” says Richard A. Andersen, a neuroscientist at Caltech.

What, then, might realistically be achieved by interactions between brains and machines? Do the advances from the first EEG experiment to brain-controlled arms and cursors suggest an inevitable, deterministic progression, if not toward a Kurzweilian singularity, then perhaps toward the possibility of inputting at least some high-level cognitive information into the brain? Could we perhaps download War and Peace or, with a nod to The Matrix, a manual of how to fly a helicopter? How about inscribing the sentence “See Spot run” into the memory of someone who is unconscious of the transfer? How about just the word “see”?

These questions are not entirely academic, although some wags might muse that it would be easier just to buy a pair of reading glasses and do things the old-fashioned way. Even if a pipeline to the cortex remains forever a figment of science fiction, an understanding of how photons, sound waves, scent molecules and pressure on the skin get translated into lasting memories will be more than mere cyberpunk entertainment. A neural prosthesis built from knowledge of these underlying processes could help stroke victims or Alz­heimer’s patients form new memories.

Primitive means of jacking in already reside inside the skulls of thousands of people. Deaf or profoundly hearing-impaired individuals carry cochlear implants that stimulate the auditory nerve with sounds picked up by a microphone—a device that neuroscientist Michael S. Gaz­zaniga of the University of California, Santa Barbara, has characterized as the first successful neuroprosthesis in humans. Arrays of electrodes that serve as artificial retinas are in the laboratory. If they work, they might be tweaked to give humans night vision.

The more ambitious goal of linking Amazon.com directly to the hippocampus, a neural structure involved with forming memories, requires technology that has yet to be invented. The bill of particulars would include ways of establishing reliable connections between neurons and the extracranial world—and a means to translate a digital version of War and Peace into the language that neurons use to communicate with one another. An inkling of how this might be done can be sought by examining leading work on brain-machine interfaces.

Your Brain on Text

Jacking text into the brain requires consideration of whether to insert electrodes directly into tissue, an impediment that might make neural implants impractical for anyone but the disabled. As has been known for nearly a century, the brain’s electrical activity can be detected without cracking bone. What looks like a swimming cap studded with electrodes can transmit signals from a paralyzed patient, thereby enabling typing of letters on a screen or actual surfing of the Web. Niels Birbaumer of the University of Tübingen in Germany, a leading developer of the technology, asserts that trial-and-error stimulation of the cortex using a magnetic signal from outside the skull, along with the electrode cap to record which neurons are activated, might be able to locate the words “see” or “run.” Once mapped, these areas could be fired up again to evoke those memories—at least in theory.

Some neurotechnologists think that if particular words reside in specific spots in the brain (which is debatable), finding those spots would probably require greater precision than is afforded by a wired swim cap. One of the ongoing experiments with invasive implants could possibly lead to the needed fine-level targeting. Philip R. Kennedy of Neural Signals and his colleagues designed a device that records the output of neurons. The hookup lets a stroke victim send a signal, through thought alone, to a computer that interprets it as, say, a vowel, which can then be vocalized by a speech synthesizer, a step toward forming whole words. This type of brain-machine interface might also eventually be used for activating individual neurons.

Still more precise hookups might be furnished by nanoscale fibers, measuring 100 nanometers or less in diameter, which could easily tap into single neurons because of their dimensions and their electrical and mechanical properties. Jun Li of Kansas State University and his colleagues have crafted a brushlike structure in which nano­fiber bristles serve as electrodes for stimulating or receiving neural signals. Li foresees it as a way to stimulate neurons to allay Parkinson’s disease or depression, to control a prosthetic arm or even to flex astronauts’ muscles during long spaceflights to prevent the inevitable muscle wasting that occurs in zero gravity.

Learning the Language

Fulfilling the fantasy of inputting a calculus text—or even plugging in Traveler’s French before going on vacation—would require far deeper insight into the brain signals that encode language and other neural representations.

Unraveling the neural code is one of the most imposing challenges in neuroscience—and, to misappropriate Freud, would likely pave a royal road to an understanding of consciousness. Theorists have advanced many differing ideas to explain how the billions of neurons and trillions of synapses that connect them can ping meaningful messages to one another. The oldest is that the code corresponds to the rate of firing of the voltage spikes generated by a neuron.

Whereas the rate code may suffice for some stimuli, it might not be enough for booting a Marcel Proust or a Richard Feynman, supplying a mental screen capture of a madeleine cake or the conceptual abstraction of a textbook of differential equations. More recent work has focused on the precise timing of the intervals between each spike (temporal codes) and the constantly changing patterns of how neurons fire together (population codes).

Some help toward downloading to the brain might come from a decadelong endeavor to build an artificial hippocampus to help people with memory deficits, which may have the corollary benefit of helping researchers gain insights into the coding process. A collaboration between the University of Southern California and Wake Forest University has worked to fashion a replacement body part for this memory-forming brain structure. The hippocampus, seated deep within the brain’s temporal lobe, sustains damage in stroke or Alzheimer’s. An electronic bypass of a damaged hippocampus could restore the ability to create new memories. The project, funded by the National Science Foundation and the Defense Advanced Research Projects Agency, might eventually go further, enhancing normal memory or helping to deduce the particular codes needed for high-­level cognition.

The two groups—led by Theodore W. Berger at U.S.C. and Samuel Deadwyler at Wake Forest—are preparing a technical paper showing that an artificial hippocampus took over from the biological organ the task of consolidating a rat’s memory of pressing a lever to receive a drop of water. Normally the hippocampus emits signals that are relayed to cortical areas responsible for storing the long-term memory of an experience. For the experiment, a chemical temporarily incapacitated the hippocampus. When the rat pressed the correct bar, electrical input from sensory and other areas of the cortex were channeled through a microchip, which, the scientists say, dispatched the same signals the hippocampus would have sent. A demonstration that an artificial device mimicked hippocampal output would mark a step toward deducing the underlying code that could be used to create a memory in the motor cortex—and perhaps one day to unravel ciphers for even higher-level behaviors.

If the codes for the sentence “See Spot run”—or perhaps an entire technical manual—could be ascertained, it might, in theory, be possible to input them directly to an electrode array in the hippocampus (or cortical areas), evoking the scene in The Matrix in which instructions for flying a helicopter are downloaded by cell phone. Artificial hippocampus research postulates a scenario only slightly more prosaic. “The kinds of examples [the U.S. Department of Defense] likes to typically use are coded information for flying an F-15,” says Berger.

The seeming simplicity of the model of neural input envisaged by artificial hippocampus-related studies may raise more questions than it answers. Would such an implant overwrite existing memories? Would the code for the sentence “See Spot run” be the same for me as it is for you or, for that matter, a native Kurdish speaker? Would the hippocampal codes merge cleanly with other circuitry that provides the appropriate context, a semantic framework, for the sentence? Would “See Spot run” be misinterpreted as a laundry mishap instead of a trotting dog?

Some neuroscientists think the language of the brain may not be deciphered until understanding moves beyond the reading of mere voltage spikes. “Just getting a lot of signals and trying to understand what these signals mean and correlating them with particular behavior is not going to solve it,” notes Henry Markram, director of neuroscience and technology at the Swiss Federal Institute of Technology in Lausanne. A given input into a neuron or groups of neurons can produce a particular output—conversion of sensory inputs to long-term memory by the hippocampus, for instance—through many different pathways. “As long as there are lots of different ways to do it, you’re not even close,” he says.

The Blue Brain Project, which Markram heads, is an attempt that began in 2005 to use supercomputer-based simulations to reverse-engineer the brain at the molecular and cellular levels—modeling first the simpler rat organ and then the human version to unravel the underlying function of neural processes. The latter task awaits a computer that boasts a more than 1,000-fold improvement over the processing power of current supercomputers. The actual code, when it does emerge, may be structured very differently from what appears in today’s textbooks. “I think there will be a conceptual breakthrough that will have significant implications for how we think of reality,” Markram says. “It will be quite a profound thing. That’s probably why it’s such an intractable problem.”

The challenge involved in figuring out how to move information into the brain suggests a practical foreseeable limit for how far neurotechnology might be advanced. The task of forming the multitude of connections that make a memory is vastly different from magnetizing a set of bits on a hard disk. “Complex information like the contents of a book would require the interactions of a very large number of brain cells over a very large area of the nervous system,” observes neuroscientist John P. Donoghue of Brown University. “Therefore, you couldn’t address all of them, getting them to store in their connections the correct kind of information. So I would say based on current knowledge, it’s not possible.”

Writing to the brain may remain a dream lost in cyberspace. But the seeming impossibility does not make Donoghue less sanguine about ultimate expectations for feeding information the other way and developing brain-controlled prostheses for the severely disabled. He has been a leader in studies to implant an array of multiple electrodes into the brain that can furnish a direct line from the cortex to a prosthetic arm or even a wheelchair.

Donoghue predicts that in the next five years brain-machine interfaces will let a paralyzed person pick up a cup and take a drink of water and that, in some distant future, these systems might be further refined so that a person with an upper spinal cord injury might accomplish the unthinkable, perhaps even playing a game of basketball with prosthetics that would make a reality of The Six Million Dollar Man, the 1970s television series. Even without an information pipeline into the brain, disabled patients and basic researchers might still reap the benefits of lesser substitutes. Gert Pfurtscheller of the Graz University of Technology in Austria and his colleagues reported last year on a patient with a spinal cord injury who was able, merely by thinking, to traverse a virtual environment, moving from one end to the other of a simulated street. Duke University’s Miguel A. L. Nicolelis, another pioneer in brain-machine interfaces, has begun to explore how monkeys connected to brain-controlled prosthetic devices begin to develop a kinesthetic awareness, a sense of movement and touch, that is completely separate from sensory inputs into their biological bodies. “There’s some physiological evidence that during the experiment they feel more connected to the robots than to their own bodies,” he says.

The most important consequences of these investigations may be something other than neural implants and robotic arms. An understanding of central nervous system development acquired by the Blue Brain Project or another simulation may let educators understand the best ways to teach children and determine at what point a given pedagogical technique should be applied. “You can build an educational development program that is engineered to, in the shortest possible time, allow you to acquire certain capabilities,” Markram says. If he is right, research on neural implants and brain simulations will produce more meaningful practical benefits than dreams of the brain as a flash drive drawn from 20th-century science-fiction literature.

Note: This article was originally published with the title, “Jacking Into the Brain”.

November, 2008
Putting Thoughts into Action: Implants Tap the Thinking Brain

Researchers are decoding the brain to give a voice and a hand to the paralyzed—and to learn how it controls our movements
By Alan S. Brown

Key Concepts

- Surgeons have implanted a novel neural prosthesis into a paralyzed patient’s brain. The high-tech device enables the patient to communicate his thoughts to a computer, which translates them into spoken words.

- Nine people so far have received brain-implanted prostheses. In the past, patients have used these devices to spell words on a computer, pilot a wheelchair or flex a mechanical hand.

- One day implants may enable paralyzed people to move robotic arms or even bypass damaged parts of the nervous system to reanimate unresponsive limbs. In the meantime, the quest to develop implanted neural prostheses is revealing details of how the brain orchestrates movement.

Eight years ago, when Erik Ramsey was 16, a car accident triggered a brain stem stroke that left him paralyzed. Though fully conscious, Ramsey was completely paralyzed, essentially “locked in,” unable to move or talk. He could communicate only by moving his eyes up or down, thereby answering questions with a yes or a no.

Ramsey’s doctors recommended sending him to a nursing facility. Instead his parents brought him home. In 2004 they met neurologist Philip R. Kennedy, chief scientist at Neural Signals in Duluth, Ga. He offered Ramsey the chance to take part in an unusual experiment. Surgeons would implant a high-tech device called a neural prosthesis into Ramsey’s brain, enabling him to communicate his thoughts to a computer that would translate them into spoken words.

Today Ramsey sports a small metal electrode in his brain. Its thin wires penetrate a fraction of an inch into his motor cortex, the part of the brain that controls movement, including the motion of his vocal muscles. When Ramsey thinks of saying a sound, the implant captures the electrical firing of nearby neurons and transmits their impulses to a computer, which decodes them and produces the sounds. So far Ramsey can only say a few simple vowels, but Kennedy believes that he will recover his full range of speech by 2010.

Ramsey’s neural prosthesis ranks among the most sophisticated implanted devices that translate thoughts into actions. Such systems listen to the brain’s instructions for movement—even when actual movement is no longer possible—and decode the signals for use in operating a computer or moving a robot. The technology needed for such implants, including powerful microprocessors, improved filters and longer-lasting batteries, has advanced rapidly in the past few years. Funding for such projects has also grown. The U.S. Department of Defense, for example, sponsors research in prosthetics for wounded war veterans.

Only nine people, Ramsey included, have received brain-implanted prostheses. In the past, patients have used them to spell words on a computer, pilot a wheelchair or flex a mechanical hand. Monkeys have employed them to perform more complex tasks such as maneuvering mechanical arms to grab food or controlling a walking robot on a treadmill [see “Chips in Your Head,” by Frank W. Ohl and Henning Scheich; Scientific American Mind, April/May 2007]. Other experimental brain-computer interfaces read the brain’s output noninvasively, through electrodes attached to the human scalp [see “Thinking Out Loud,” by Nicola Neumann and Niels Birbaumer; Scientific American Mind, December 2004].

The technology promises to give thousands of victims of stroke, spinal cord injury and paralyzing illnesses the ability to, say, talk with a friend, flip through television channels or transport themselves by driving their own wheelchair. One day implants may enable paralyzed people to move robotic arms or even bypass damaged parts of the nervous system to reanimate unresponsive limbs. In the meantime, the quest to develop implanted neural prostheses is bringing with it revelations about how the brain manages motion and how it can remodel itself so that only a few neurons are needed to direct action through an implant.

Eavesdropping

Scientists have known for more than 220 years that electricity somehow controls muscle movement. In 1783 Italian physician Luigi Galvani, a contemporary of Benjamin Franklin, discovered that electric currents caused a severed pair of frog legs to twitch. By the 1860s German military doctors had discovered that small electric currents applied to the brain could cause certain muscles to contract. Over the following decades, dedicated researchers mapped which regions of the motor cortex control which groups of muscles in the body. But to discover how the brain actually orchestrates movement, scientists had to find a way to eavesdrop on the neural signals in the motor cortex while animals were awake and moving.

This task proved problematic until investigators figured out how to stably affix an electrode, a tiny sliver of conductive wire, to a neuron so they could register its weak, milliseconds-long pulses. When animals move, their brains shift slightly within their skulls, and the motions can rip an electrode from its anchor in the brain. In the late 1950s neurologists found that flooding the space between the skull and the brain with inert wax or neutral oil buffered the brain the way Styrofoam peanuts keep a box from moving inside a larger package. The buffer prevented a brain from shaking off its implant.

Despite this fix, no one could make sense at first of the chatter of individual neurons in the motor cortex. Researchers expected a one-to-one correspondence between the neurons that fired and the muscles that contracted during movements. But when they looked at individual neurons, they found the neurons would fire when a monkey moved its arm forward or backward or even when it kept the arm still.

In the late 1970s neurologist Apostolos Georgopoulos, now at the U.S. Department of Veterans Affairs and the University of Minnesota, had a brainstorm. The spinal cord exerts direct control over muscles, Georgopoulos realized. Thus, he supposed that the motor cortex might be directing movement at a somewhat higher level, specifying a trajectory rather than the muscles and joints needed to accomplish a movement.

To test his idea, Georgopoulos developed something called the center-out task, in which monkeys learn to move a joystick toward one of six targets arrayed in a semicircle. “Until then, all the research designs focused on very simple movements—forward, stop, back,” he explains. “In our experiment, the monkey was changing the position of its shoulder, elbow and wrist simultaneously.”

No one had looked at such complex motions before—or analyzed the data the way Georgo­poulos and his colleagues did. Instead of trying to correlate the firing of particular neurons with the contractions of certain muscles, he averaged the responses of small groups of neurons over thousands of experiments. From that average, he saw through the noise that neurons produce when they direct motion, engage in other tasks or just fire spuriously. Although individual neurons fired with every movement, each neuron had a preferred direction: when the monkey moved the joystick that way, its firing frequency peaked. Neighboring neurons with similar preferred directions also became more excited. The closer a monkey’s arm moved to a neuron’s preferred direction, the more rapidly it fired; the farther away the arm moved, the more slowly it fired.

“It’s a sort of democracy,” Georgopoulos explains. “A given cell will keep voting on the direction of the movement, whether it’s in the majority or the minority, but the majority always rules. And the majority vote is an excellent predictor of direction.” In this way, the motor cortex sets a strategy for a movement. It calculates the direction (and, as Georgopoulos and others later found, the acceleration) needed for the hand to reach a target. It then sends the information to the spinal cord, which implements that strategy by operating muscles. Those more general commands from the brain, researchers believed, might indeed be useful for controlling external devices.

Making a Move

But progress on developing a neural prosthesis that could translate thoughts into action was slow. At first the electrodes were unreliable, and the electrical connections were sometimes finicky. The neurons themselves would also act unpredictably.

“Brain cells don’t behave the same way every time. Perhaps the cells are changing, or maybe the patient is tense or tired,” says Brown University neuroscientist John Donoghue, the second scientist after Kennedy to develop a neural prosthesis for human implantation.

Researchers also despaired at the problem of gleaning useful information from a relatively small number of neurons. “Usually the brain uses millions of neurons to perform a motor task. Now we’re asking people with prostheses attached to maybe 50 neurons to do the same thing,” Donoghue says. Yet those few neurons proved surprisingly capable.

Implant pioneer Eberhard Fetz, a biophysicist at the University of Washington, recalls experiments conducted in the late 1970s and early 1980s in which a monkey learned to use an implant to move the dial on an electrical meter to receive a drop of applesauce. Fetz and his team did not train the monkey, but it quickly learned to control the needle by trial and error, just by thinking. “He learned that there was something he could do to drive the meter to the right and trigger the feeder,” Fetz recalls. “Once he got the hang of it, he could do it every time.”

Neuroscientists believe that once the monkey chanced on a successful pattern of neural impulses, continued successes triggered the rewiring of its brain to create a faster and more efficient mechanism for repeating that pattern. This process also underpins other types of motor learning, such as that required to manipulate a fork or chopsticks. That is, the monkey learned to work the dial as if it were an extension of the monkey’s own body—which, in many ways, it was.

The ability of the brain to rewire itself on the fly is called plasticity. Investigators see examples of it all the time. In 2002 neurobiologist Andrew Schwartz of the University of Pittsburgh and his colleagues reported brain plasticity in a monkey that was trained to hit a target in a 3-D virtual-reality game using a ball that it controlled with its thoughts. Once the monkey learned to hit the target every time, Schwartz altered the settings so that the ball veered a few degrees to the right. Within about five minutes the monkey had adapted to the adjustment and began hitting the target again. “The only way the monkey could correct the error was by changing the firing of the neurons that we were recording,” Schwartz explains.

This past June, Schwartz’s team reported teaching a monkey to manipulate a gripper
on a hinged double-jointed robotic arm to lift food off a hook. Ordinarily the brain uses millions of neurons to control such a multipart, intricate movement. The monkey learned to retrieve the food, at least some of the time, with an implant that read the signals from only a few dozen neurons.

Connecting with People

With time, researchers parlayed their monkey studies into pilot trials with paralyzed people. Early implants generally enabled patients to translate their thoughts into simple actions, such as moving a computer cursor in one or two dimensions rather than using the complex, three-dimensional actions of a robotic arm.

In 1996, for example, a group of surgeons working under Kennedy inserted the first neural prosthesis into the brain of a paralyzed former teacher and artist in the terminal stages of amyotrophic lateral sclerosis, a progressive paralysis also known as Lou Gehrig’s disease. In the two months after the surgery, the woman learned to use it to turn on and off lights on a computer screen. A few years later a second patient, a locked-in 53-year-old former drywall contractor named Johnny Ray, learned to use the implant to move a cursor to pick out computer icons, spell words and generate musical tones.

Since then, seven more patients have received implants. With each one, the technology became more versatile and reliable. The surgical procedures, too, have come a long way since experimenters had to stabilize electrodes with wax. Kennedy, for example, has developed a cone-shaped electrode that contains chemicals to encourage neuron growth. Surgeons make a small hole in the skull above the ear and over the motor cortex and secure the electrode to the bone. When nearby neurons grow into the cone, they begin transmitting electrical signals to the electrode, which transmits them to a wireless receiver attached to the top of the head.

Researchers have also tried to improve the fidelity of the signals they receive by tapping more neurons. Donoghue and his colleagues developed an electrode array capable of receiving signals from 96 individual neurons. In 2004 neurosurgeons implanted it into the brain of 24-year-old Matthew Nagle of Weymouth, Mass., who was paralyzed when he intervened in a fight and was knifed through the spinal cord. Within only minutes of calibrating the prosthesis, Nagle could move a cursor on a computer. Over the next three years, before he died from an unrelated infection, he learned to control a television, check e-mail, and open and close an artificial hand. He made some rudimentary attempts to draw, which requires fine-motor control. His first attempt to sketch a circle wandered all over the screen, his second try led to more pronounced curves and his third produced an oval.

As investigators accumulate experience with human prostheses, they have raised their sights. Donoghue, for example, is teaming up with biomedical engineer Hunter Peckham of Case Western Reserve University, who has developed an electrical device that stimulates nerves or muscles to enable some movement after a partial or lower-level spinal cord injury. But Peckham’s system alone allows only simple, preprogrammed motions, such as boosting a person from a wheelchair to a walker. By linking a neural prosthesis to the device, however, Donoghue and Peckham hope to create a system that gives users greater flexibility. “Our goal is that within five years we will have a brain-controlled system that lets a tetraplegic take a glass of water, lift it and bring it to the mouth,” Donoghue says.

Fetz hopes to eventually connect a brain prosthesis directly to the spinal cord to flexibly reanimate nerves and muscles after spinal cord injuries. Such a device would tap the cord’s natural ability to coordinate groups of muscles.

Neurologist Richard A. Andersen of the California Institute of Technology is taking a different tack. Instead of decoding the motor cortex, he wants to capture the brain’s intentions before they become motor commands. Andersen believes those commands originate in the posterior parietal cortex (PPC), an area near the top of the back of the head that transforms sensory stimuli into a movement blueprint. Unlike the motor cortex, which estimates the trajectory an arm must take to reach a target, neurons in the PPC produce “goal” signals that specify the target itself. Recently Andersen and his colleagues at the Massachusetts Institute of Technology and McGill University showed that the PPC also predicts and adjusts for changes in a target’s motion.

The PPC’s focus on the goal makes tapping it potentially more efficient than reading a brain area that plots trajectories, Andersen says. A prosthesis implanted in the PPC might enable a patient to rapidly pick out letters on a screen to spell out words—just as fast-touch typists do on a keyboard. Because of its flexibility, such a prosthesis might let a user operate a wider range of devices than a motor cortex implant designed to control specific movements would. Andersen is hoping to embed the appropriate electronics into a person’s parietal cortex within a year or two.

Finding a Voice

Kennedy’s speech prosthesis arguably poses the greatest challenge yet because he had almost no experimental data on which to base its operation. After all, monkeys do not speak, and Ramsey is the first person to receive an implant to produce speech. This means that Kennedy must find a way to separate speech signals from neural noise without animal research to guide him.

Ramsey’s implant connects with about 50 neurons in the part of his motor cortex that translates how he thinks a syllable should sound into the muscle commands to make the syllable. The implant captures the signals that control the coordinated motion of his mouth, lips and tongue to form sounds.

The link between Ramsey’s neural implant and speech is a sophisticated computer program called Directions into Velocities of Articulators (DIVA), developed by Frank H. Guenther, a cognitive neuroscientist at Boston University. DIVA is a mathematical description of how the brain controls speech, parsing the process into eight parts that represent different speech functions in the brain. Mathematical formulas define neural firing rates in each area and neuronal connections among areas. DIVA made it possible to build a neural decoder that can decipher the speech signals amid the neural noise coming out of Ramsey’s implant. The decoder translates the speech signals into sound data that it sends to a speech synthesizer, which generates human sounds.

Guenther built DIVA by scouring the research literature on the brain’s speech centers. His group continually refines the program through additional experiments. “If we want to investigate how the brain corrects speech, we’ll perturb a volunteer’s speech. They may say ‘bet,’ but they hear ‘bit.’ Our model might predict that four parts of the brain should light up when they hear the perturbed sound, and we’ll see how that compares with what happens on a [brain] image. If the image lights up in five places, then we update the model to reflect this new information.”

DIVA learns to speak from experience. Initially DIVA babbles like a human infant. As it “listens” to the resulting sounds and “senses” the position of its virtual muscles, it uses the feedback to modify its mathematical relationships to speak more clearly. “Then comes the imitation stage,” Guenther says. “We have a human say something, and the model tries to reproduce it. It will be wrong at first, but DIVA will use feedback to keep getting it closer. It usually takes about five or six attempts to get it right.”

Similarly, the neural decoder based on DIVA does not accurately translate Ramsey’s initial attempts to speak, in part because the computer program receives input from just a tiny fraction of the millions of neurons that are involved in speech. The program and Ramsey, however, get better with practice. Guenther starts this learning process by playing a sequence of vowel sounds on a computer—vowels are easier to pronounce than consonants—and Ramsey sings along in his mind. Ramsey and the decoder botched their first five attempts at each of the first three vowels. But then Ramsey adjusted his brain signals based on the feedback from the synthetic sounds the computer produced, and on the next five, he got three or more right.

“Ramsey was able to quickly improve his performance by adjusting the brain signals that were sent to the synthesis system,” Guenther recalls. “Most of this learning is subconscious motor learning, like learning to shoot baskets or whistle or ride a bike, rather than requiring a conscious attempt to change the way one communicates.” It is slow, arduous work. Ramsey has only enough energy for two or three weekly sessions that usually last no more than an hour or two.

Eventually Kennedy hopes to implant more electrodes in different parts of the brain’s speech motor region to provide richer neural input for the speech program. “We’d like to have several electrodes spread out over areas that control the tongue, mouth, jaw and facial muscles. If we had more implants, that would give us even better resolution.”

From such endeavors, the neurologist hopes to change the lives of tens of thousands of people. Those who are now entombed within their own bodies will once again be able to communicate and connect with friends, caretakers and family. People who cannot move from room to room or change a television on their own will find a new freedom. Wounded warriors returning from battle may receive artificial limbs that respond to their unspoken commands.

Erik Ramsey is just the beginning.

Note: This article was originally printed with the title, “Putting Thoughts into Action”.

Further Reading

Cognitive and Neural Systems Speech Lab at Boston University
Web site of Andrew Schwartz of the University of Pittsburgh

————————-

http://www.theconnection.org/shows/2005/06/20050628_b_main.asp

The Brain Chip

Guests:
John Donoghue, head of the neuroscience department at Brown University and co-founder of Cyberkinetics

Miguel Nicoleli, professor in Duke University’s neurobiology department.

Hosted by: Dick Gordon
Show Originally Aired: 6/28/2005
Download interview

http://www.news.com.au/story/0,10117,17013218-13762,00.html
US military experts are attempting to create an army of super-human soldiers who will be more intelligent and deadly thanks to a microchip implanted in their brains.
Scientists believe the implant will vastly improve the memory of troops so that they can recall every detail of their training and become more effective fighters.

Researchers at the University of Southern California’s bio-engineering department have created the chip, which acts in exactly the same way as the hippocampus – the part of the brain that deals with memory.

In experiments, the team removed that section of the brain of dead rats and inserted the chip in its place. The implant sent exactly the same electronic signals as the real thing.

The next stage of the project is to test the implant on live animals. If this work proves to be as successful, experiments could one day be carried out on soldiers.