Brain Signals to Text

Welcome back. There’s been some fascinating work on brain-computer interfaces (BCIs) aimed at restoring communication to those who have lost the ability to move or speak. While the primary focus has been on restoring gross motor skills--reaching and grasping or point-and-click typing with a computer cursor, a recently published pilot study by researchers affiliated primarily with Stanford and Brown universities described translating neural activity to text in real time. Got that? Converting thoughts about handwriting to computer screen text.

Cataloging the Study
The study report and media summaries appeared about a month ago. As you might expect, the research and development project actually began long before. 

BrainGate2 is NCT00912041 in the National Library of Medicine’s clinical trials database (www.clinicaltrials.gov/ct2/show/NCT00912041).

The U.S. National Library of Medicine, an institute of the National Institutes of Health, maintains a database of worldwide privately and publicly funded clinical studies. There you’ll find BrainGate2: Feasibility Study of an Intracortical Neural Interface System for Persons with Tetraplegia (i.e., paralysis in the upper and lower body). The study is described as an interventional clinical trial to identify the core methods and features for a medical device that could allow people with paralysis to recover a host of abilities that normally rely on the hands. The study began in 2009 with the estimated completion in 2022.

Study Procedures
The investigation focused on a single individual who lost nearly all movement below the neck after a spinal cord injury in 2007. Nine years later, the researchers implanted two BCI chips the size of baby aspirins on the surface of the left side of his brain. Each chip had 100 electrodes to receive electrical signals from neurons firing in the part of the brain’s motor cortex that controls hand and finger movement.

Brain’s motor cortex is in charge of planning, control and execution of voluntary movements (graphic from www.researchgate.net/journal/Frontiers-in-Human-Neuroscience-1662-5161).

The neural signals were transmitted via wires to a computer, where artificial-intelligence algorithms “learned” to decode the signals using a recurrent neural network approach to deduce the individual’s thoughts about handwriting motion.

The algorithm training had the participant mentally write letters of the alphabet on an imaginary legal pad with an imaginary pen, repeating each letter 10 times. Next, in many long sessions, the participant was shown groups of sentences and instructed to mentally handwrite each sentence. Over time, the algorithms improved in differentiating the neural signals of the different characters. 

Graphic of electrodes detecting neural signals of man mentally handwriting (from Howard Hughes Medical Institute video www.youtube.com/watch?v=pcApwQxbagg).

Wrap Up
Although the coupling of BCI with artificial-intelligence software to convert thoughts to screen-displayed text was only tested on one person, the proof-of-concept results were impressive. The participant was eventually able to generate about 18 words per minute with an accuracy of nearly 100% with a general purpose autocorrect. Healthy people of the same age reportedly text about 23 words per minute on a smartphone

When asked to write answers to open-ended questions, which required time for thought, the participant averaged close to 15 words per minute.

What the researchers found especially interesting was that, years after suffering the spinal cord injury and losing hand and finger movement, the neural activity associated with writing was not lost.

Thanks for stopping by.

P.S.
Study of brain to text handwriting in Nature journal: www.nature.com/articles/s41586-021-03506-2
Example articles on study
www.eurekalert.org/pub_releases/2021-05/sm-sss051021.php
www.sciencenews.org/article/brain-implants-translate-handwriting-text
www.axios.com/newsletters/axios-science-07aaba61-f030-4e22-a91c-357935245667.html?chunk=2&utm_term=emshare#story2
National Library of Medicine’s clinical trials database and BrainGate2 entry:
www.clinicaltrials.gov/
www.clinicaltrials.gov/ct2/show/NCT00912041