The world of neurotech has seen many research institutes develop Brain Computer Interfaces (BCIs) to restore movement or speech like Chang Lab at the University of California, San Francisco. A team at the University of Washington created the first direct brain to brain BCI.
Rajesh Rao, Professor of Computer Science and Engineering and Electrical and Computer Engineering at the University of Washington and author of Brain Computer Interfacing: An Introduction, leads the Center for Neurotechnology and Neural Systems Lab where they conduct research at the intersection of the brain and AI. We interviewed Dr. Rao about his path to neurotech, BrainNet, and the future of neurotech.
Background: A Sci-Fi Inspiration
Similar to many, Dr. Rao spent his childhood Sundays watching and reading Science Fiction literature and TV shows like Star Trek. Showcasing many far-fetched technologies, Star Trek inspired Dr. Rao to ask the crucial question:
Is it really just science fiction or is it possible?
After excelling in high school science, Dr. Rao first experienced scientific research at the University of Maryland, College Park. He then accepted an academic scholarship to attend Angelo State University in Texas where he studied Computer Science, Mathematics, and Physics.
He would go on to complete his M.S. and Ph.D. at the University of Rochester where he began to combine his interests in computers and brain through Computer Vision; a field of computer science and AI that conducts video analysis through various machine learning algorithms. He would then go on to begin neuroscience work during his postdoc.
In 2000, Dr. Rao opened his lab in the University of Washington at the intersection of AI, machine learning, and computational neuroscience.
BrainNet: Pioneering Brain to Brain Communication
In 2013, Dr. Rao’s team started a series of experiments to communicate directly between brains. BCIs have been used extensively for control of devices and robots but Dr. Rao asked whether it was possible to send a control signal to a human brain.
Can two brains collborate directly, where one is sensing, the other is controlling?
The first study was set up with a sender wearing an EEG and a receiver under a TMS (transcranial magnetic stimulation) device on opposite sides of the University of Washington campus. The sender would use motor imagery, decoding the brain signals of imagining moving a hand, which would be relayed over the internet, across campus, to the TMS device which would stimulate the receiver’s region of motor cortex responsible for moving the hand, resulting in a keyboard press that was not directly controlled by the receiver.
Caption: Dr. Rao as the sender in the first iteration of BrainNet. He is wearing an EEG cap
The second iteration featured conscious decision making in the communication flow. A 20 Questions game was played, where the sender would think of an object and the receiver would ask questions (explicitly) and the sender would answer yes or no to that question through stimulation to the visual cortex.
We played a 20 Questions game where I think of an object and you repeatedly ask me yes or no questions. The person will eventually figure out that I was thinking of a truck only through brain to brain interaction.
Caption: The sender is pictured here with an EEG, who will send “yes” or “no” by looking at the respective flashing LED which evokes a steady-state visually evoked potential (SSVEP)
Then in 2019, the group increased the communication to three people — two senders and one receiver — to play a Tetris game. The idea was to simulate a social network, where one of the two senders is a 'bad actor' that sends unreliable information.
The receiver would be placed in front of the Tetris game, but without vision of the bottom layer. The senders saw the entire Tetris grid, but did not have direct control of the block, and thus would send a signal to “rotate” the block. One of the two sender’s signals would include noise that simulated the “bad actor.
Caption. Receiver does not see the bottom row where the block should fit (left column) while the sender sees the entire board but does not possess control of the block.
The series of studies was a proof of concept for collaboration through brain to brain communication.
Dr. Rao explained that the second and third iterations of the studies included a decision-making process since the first iteration would result in a “brain in a box” type of agent, where the receiver does not consciously process the information being conveyed.
So, why BrainNet?
With such a futuristic concept like direct brain to brain communication, it may not be intuitive what the application of this could be.
There are clearly limitations with the current technology; EEG is limited in bandwidth and TMS is a bulky device not suited for applications outside of the lab. However, in a future where the technology matures to a point where we can send and receive high resolution information between brains, what can we do?
We use language [to communicate], and it has its benefits and disadvantages. There's a finite number of words, and there's going to be limitations to the person's capacity to express their knowledge. How can we remove that bottleneck?
Language is inherently a bottleneck in communication requiring encoding (thought to language) and decoding (language to thought). BrainNet and direct brain to brain communication, given the signal resolution improves significantly, could reduce the friction of language.
Another application of brain to brain BCIs could be better abstract communication in settings like school, sports, and therapy.
If you can build a nice mapping between one person to another person's brain, you could have faster learning. A therapist could guide rehabilitation in another person's brain in a particular way.
Another futuristic application would be to transfer knowledge directly like in the Matrix.
Can I download Kung Fu into my brain? You cannot directly download it and expect it to work, you'll have to do a lot of practice after you've downloaded it, but it may be possible to download it and have a good initial starting point.
Caption: Interface from The Matrix (1999) where Neo “downloads” Kung Fu. Link to clip: https://www.youtube.com/watch?v=_UUFu8zjRxE
Finally, Dr. Rao points out that combining brains could improve human creativity by putting together multiple brains, each bringing a unique perspective to a problem.
Imagine if you can directly [collaborate] by harnessing brains direclty. The creativity of the human condition may be amplified and maybe we'll come up with solutions to some of the harder problems that a single mind may not be able to solve. If you put 100 minds together, perhaps that can solve a very hard problem that, because of our neural real estate limitations, we cannot solve.
This very futuristic perspective reflects Dr. Rao’s inspiration in science fiction. As a leader in this field, he could be the one to actuate brain to brain communication in the near future.
Dr. Rao’s lab is currently pursuing the idea of brain co-processors, where decoding of brain activity is coupled with encoding.
We are calling it a co-processor because it will help humans process information alongside the brain. You can think of it as artificial neural networks talking to biological neural networks.
This co-processing would potentially improve various brain capacities such as bridging brain regions after a stroke, or healing brain regions through adaptive stimulation. This relates to the statement Elon Musk made about a “tertiary [cortical] layer” regarding the possible use of Neuralink.
Dr. Rao also touched on the important idea of 'explainable AI'. Neural networks are being used in many applications of AI, but when the algorithm fails we will not understand why it failed. When dealing with highly-involved applications like self-driving cars and BCIs these mistakes can be detrimental.
If you don't understand the brain, at some point, you're not going to understand the behavior of both the brain and the interface.
This parallel work between AI, Neuroscience, and BCIs speaks to the state of AI and neuroscience.
Once you understand how the brain works, it makes it easier to also interface with the brain. Instead of using a blackbox AI interface of the brain, we can understand what happens when you stimulate the brain.
The future of BCIs: Speculation from a world-class researcher
We asked Dr. Rao how neurotech will advance both in terms of applications and technology.
He explained that neurotech will likely follow a similar trajectory as plastic surgery. In the 1950s and 1960s, plastic surgery was mainly used for medical treatment for cancer patients. However, as the technology developed and the risk of surgery dropped, plastic surgery has become a routine treatment for many.
I think in the same way, we are in the phase where a lot of the neurotechnology is being developed for medical purposes. And that's exactly where it needs to be.
Past the technologies being developed with a medicinal purpose, Dr. Rao explains that the main application will be in augmentation, similar to commercial plastic surgery.
Where the technology goes beyond medical uses in the future, we are going to see commercial applications that have to do with augmentation.
But what exactly does it mean to augment the human brain?
Dr. Rao explains that augmentation has been happening since the beginning of humanity where the first “cave people” picked up a rock to use as a tool. In more contemporary contexts, the smartphone and the internet are augmentations to communication and information retrieval.
If you think of that progression, neurotechnology is just a natural next step, where we are augmenting not just our physical capacity, but also our brain or mental capacities.
Augmentation could come in many forms including faster information retrieval and acquisition. Instead of accessing information through our sensory organs, the direct access through the brain would significantly increase speeds and maybe even how we interact with the world.
Dr. Rao also commented on the current state of commercial neurotech with advancements from Meta’s CTRL-Labs, Neuralink, and Kernel.
He mentions the prospects within neurotechnologies for accelerating learning and improving attention where some companies are looking into tDCS and alternate current stimulation devices to speed up knowledge acquisition.
You can almost draw parallels between some of these other kinds of augmentations. We've done it with drugs like caffeine, but this is a more direct interaction with the brain.
With other applications in the domain of gaming and sports, some companies are claiming that stimulation can improve reaction time or focus, but it remains debatable whether these effects are simply a placebo.
What will it take to get a ubiquitous BCI?
As mentioned in the interview with Ryan Field from Kernel, one of their big goals is to develop an everyday device for brain measurements. With the rise of creatively designed EEG devices, many companies are investing into the prospect of a ubiquitous BCI: the iPhone of BCI.
We asked Dr. Rao about his opinions on the developments around non-invasive recording modalities like EEG.
He mentions that the lab works with EEG, but in constrained lab settings where artifacts are controlled to a minimum. But when considering naturalistic settings where the user might be moving and talking, the EEG signal quality will likely fail.
There is an accuracy issue. The reliability is very questionable for EEG. For any kind of non-invasive technology, we have not seen a very reliable product in the market where it consistently performs to a high degree of accuracy and the recordings are really clean.
He also mentions that innovators like Kernel, who developed a non-invasive device using time-domain fNIRS technology, will push the capabilities of non-invasive BCIs further.
It's great that there are companies like Kernel and even Neuralink that are experimenting and coming up with new ways [of recording]. We need more contributions from material scientists, physicists, other fields that come up with new ways of imaging the brain.
Invasive or Non-Invasive
Finally, in the pursuit of a ubiquitous BCI, we asked Dr. Rao regarding his speculations of whether invasive or non-invasive technologies will ultimately provide the killer application.
That's a great question in terms of [whether invasive or non-invasive] will win. Ultimately there are going to be different applications [that] require different things.
With Dr. Rao’s explanation of risk-benefit, there may be a point where invasive implants become as ubiquitous as plastic surgery. With non-invasive technologies, rather than considering the risk, the question lies in the signal and whether it can provide any benefit.
To pick one, many people will probably bet on invasive technology being the one that will end up being the most reliable if they are able to bring down the cost and risk of surgery.
Conclusion
Huge thanks to Dr. Rajesh Rao of the University of Washington’s Neural Systems Lab and Center for Neurotechnology for his time. The team at NeurotechJP learned a ton from this conversation regarding his cutting-edge research, as well as his visions about the neurotech field.
Dr. Rao concluded the interview by reiterating the interdisciplinary nature of neurotech. Aspiring neuroscientists and engineers should embrace the diversity and breadth of knowledge required to research or develop neurotechnologies while keeping in mind that collaboration is key.
It's okay to specialize in one particular field, you don't have to learn everything about the different fields because it's impossible to learn all the fields. Learn as much as you can about the others and be comfortable collaborating!
Dr. Rao also authored 'Brain Computer Interfacing: An Introduction' which was introduced in 5 Featured University Labs/Facilities in BCI.
Along with BrainNet and brain-to-brain communication, we are eager to hear more from Dr. Rao and the Neural Systems Lab.
