A new artificial intelligence tool can reconstruct what someone is looking at by analyzing their brain scans, then generate that image with striking accuracy, according to research presented at the Cognitive Computational Neuroscience conference in New York last month. Developed by Michal Irani and colleagues at the Weizmann Institute of Science in Rehovot, Israel, the "mind-reading" system works in both directions—it can predict a person's brain activity from an image, or recreate an image from brain scan data. The researchers hope the technology will eventually allow scientists to reconstruct inner thoughts, mental imagery, and even the content of dreams.
The team built a "brain decoder" with two branches—one predicting image structure, such as where colors appear, and another predicting content, like bananas on a plate. They trained the model using publicly available data from eight volunteers who each viewed roughly 9,000 images while lying in high-resolution functional magnetic resonance imaging scanners. To overcome limited data availability, the researchers created an encoder that predicts brain activity from new images, then used both tools together to improve performance through repeated training cycles. Around 70% of the training data came from images never shown to people in fMRI scanners, according to Irani. The resulting "universal brain encoder" requires just one hour of fMRI data to work on a new person, compared to the 40 hours typically needed by other approaches—a crucial advantage given that imaging costs between $600 and $1,000 per hour.
In comparison testing, the tool significantly outperformed previously described systems at recreating what people saw. "All in all, really we outperformed the others by a significant margin," Irani stated. The decoder isn't flawless—it reconstructed a cake as three sandwiches and turned a dog in a bathtub into a similarly colored goat—but the research represents current state-of-the-art performance. Judy Illes, a neuroethicist and neurology professor at the University of British Columbia who wasn't involved in the work, called it "magnificent" and said the idea of using the approach therapeutically "is tremendously exciting." By combining data from multiple studies, the team identified brain regions that appear to serve shared functions across all individuals—one area responded to food images, while another lit up for sports pictures.
Irani plans to extend the technology beyond static images to video and audio, with the goal of reconstructing what people are thinking about, imagining, or experiencing in dreams. Such a tool could help "locked-in" patients who are completely paralyzed communicate through brain activity alone, or allow scientists to visualize what PTSD flashbacks look like, according to the research. However, the work raises serious concerns about mental privacy. Tommy Sprague, a neuroscientist at the University of California, Santa Barbara, noted that if there's a way to secretly extract information about what someone is thinking, "150 years of sci-fi can come true anytime, and that's worrisome in a lot of ways." The concern intensifies as researchers explore similar approaches using EEG—electrical brain activity measured through electrode caps or even headphones. Marcello Ienca, a neuroscientist and philosopher at the Technical University of Munich, warned that once calibrated to a user's brain, EEG devices could allow companies to extract information without consent, and some courts might accept mental image reconstructions as legal evidence. The capacity to decode visual thought without physical access to bulky scanning equipment could shift this from a laboratory curiosity into a privacy flashpoint that regulators and companies alike will need to address long before the science reaches commercial readiness.