Thoughts-to-Text BCI
From brain signals to letters.
In 30 seconds
- Problem
- Turn the EEG of attempted handwriting into text, one character at a time, to help people who can no longer write.
- My part
- As lead researcher, I worked on decoding attempted handwriting from non-invasive EEG, one character at a time, with an autoencoder and t-SNE. I presented the work at NCUR, the National Conference on Undergraduate Research, and in two KSU symposium presentations in 2022, plus a related motor-imagery talk. I’m the first presenter listed on the Fall 2022 talk and the only presenter on the Spring 2022 poster.
- Result
- A method presented in 2022: an autoencoder compresses the EEG into a low-dimensional space, and t-SNE maps it to 2D to show how the characters separate.
- Proof
- 2 KSU symposium presentations, 2022, and a related talk
- Presented at NCUR and received a $2,000 Research Excellence stipend. Self-reported, no public record
What the study did
Earlier systems had people choose from a small, fixed set of words. This study aimed for a model that could type any word in English, so participants attempted to handwrite each character one at a time, following instructions on a computer screen.
The EEG was hard to work with: it is noisy, and trials of different characters look alike. To see whether the characters separate at all, an autoencoder compresses the data into a low-dimensional space, and t-SNE maps that space onto a 2D plane.
- Input EEG channels
- Compress Autoencoder
- Visualize t-SNE map in 2D
- Toward Text, one character at a time
- The EEG of each attempted character goes into an autoencoder.
- Its low-dimensional code is mapped onto a 2D plane with t-SNE.
- The 2D map shows how the characters separate.
Check the source Talk abstract · KSU
| Letter | Points |
|---|---|
| a | 24 |
| b | 24 |
| c | 24 |
| d | 24 |
| e | 24 |
Presented in 2022
-
Spring 2022 · Poster 255
Brain-to-text communication through an invasive BCI
26th Symposium of Student Scholars. I was the only presenter.
Record on KSU DigitalCommons -
Fall 2022 · Talk 136
Brain-to-text communication through a non-invasive BCI
Fall 2022 Symposium of Student Scholars, a 15-minute oral presentation with five presenters. I’m listed first.
Record on KSU DigitalCommons -
Fall 2022 · Talk 163 · Related project
Motor Imagery Detection Toward Non-Invasive Brainwave Based Typing
A related project, not the handwriting study, with seven presenters: deep neural networks detect imagined movement in EEG, with the aim of moving a cursor to write letters.
Record on KSU DigitalCommons
The two KSU records share one goal but cover different work: the Spring 2022 poster is titled for an invasive BCI and sets out the aim, while the Fall 2022 talk is titled for a non-invasive BCI and adds the EEG recordings and the autoencoder and t-SNE method shown on this page.
The principal investigator on all three was Dr. Md Abdullah Al Hafiz Khan.
One hard decision
Earlier studies had people use their neural signals to pick from a limited set of words. We chose the more general design: decode one character at a time, so the model could type any word in English, not only the words on a list.
One limit
EEG is noisy, and trials of different characters look alike, which makes the data hard to process. The presented work maps how the characters separate; it is not a finished real-time typing system.