Concept2Brain Generates Synthetic EEG Estimates for Images and Text

Concept2Brain was fitted to EEG archives from two earlier passive viewing experiments by members of the same group. The first involved 45 university students who received course credit and saw 120 naturalistic photos twice, one second each, on a black background. The second involved 43 students who saw 240 different photos once, two seconds each, on a gray background. Across both sets, 360 unique images were split in equal numbers among pleasant, neutral and unpleasant categories; screen layouts were varied so the model would learn picture content rather than borders. [1]

Signals came from 128 scalp electrodes plus a reference electrode, sampled 500 times per second. After excluding blinks and movement artifacts, 13,640 clean 1.5-second trials remained. Those details constrain what the model represents: its outputs are tied to that participant group, sensor layout and viewing protocol. [1]

The platform joins two deep networks. A conditional variational autoencoder compresses each 129-channel EEG record into a 256-number summary and attaches a participant identifier, preserving differences in waveform shape across people. After training, the decoder half can expand any such summary into a smoothed response representing an average over repeated trials, for any of the 88 simulated participants. [1]

A second network translates concepts. It uses OpenAI's CLIP, a model that places images and words in a common 512-dimensional space. A three-layer network maps those conceptual representations onto the 256-value brain-wave summary. On the public website, visitors can submit an image or text and download simulated responses. The authors also disclosed code, fitted weights and EEG datasets in public repositories. [1]

Making the model, weights and data public is intended to let other researchers inspect and reuse the approach. The authors also designed the framework so CLIP can be replaced by newer AI models in future versions. That modularity matters because the tool's conceptual stage depends on a third-party system whose filters and training choices are outside the study team's control. [1]

The authors ran ten rounds of cross-validation: 80 percent of photos for training, 10 percent to track overfitting, and 10 percent held out for prediction. They checked hallmark waves, rhythmic oscillations and individual traits, and compared outputs with independent public datasets and text prompts. For held-out photos, synthetic traces matched real recordings for most images from about 100 milliseconds after onset, when visual signals reach cortex, through roughly 700 milliseconds. [1]

One erotic photograph was an exception, showing a negative correlation with real data; the authors suggest CLIP's content-safety filters may have altered its representation. When the model received its 360 training images, simulated waveforms tracked empirical recordings across sensors and time points, reproducing early visual waves and the late positive potential, a slow parietal wave from 400 to 800 milliseconds tied to emotional engagement. The synthetic data also reproduced a drop in 10-hertz alpha activity after 300 milliseconds and matched each virtual participant most closely to that individual's real counterpart. [1]

On 1,078 emotional images from a standardized library outside training, the researchers observed a weak positive relationship between simulated late positive potential amplitude and human arousal ratings. Fearful versus neutral faces from 140 actors produced larger simulated waves around 170 to 200 milliseconds and larger later emotion-related waves for fearful faces, though facial-expression labels were absent from training. [1]

The team also examined generalization across laboratories. In one external set, four volunteers viewed 1,854 object concepts during magnetoencephalography, which records magnetic fields from neural activity, and three viewed 720 concepts during functional MRI. The model's synthetic waves matched early visual and later semantic phases in the magnetic recordings and correlated with fMRI activity near 75 milliseconds in early visual cortex and around 190 milliseconds in a face-processing region. A separate EEG dataset had 40 people view 40 faces and 40 cars; the model recovered the scalp pattern that distinguished the categories. The authors reported weaker associations in both external sets than in their own data. [1]

Because CLIP handles language too, the team tested 120 short descriptions classified as pleasant, neutral or unpleasant, along with prompts such as one asking for a picture of an alligator. Across the 88 virtual participants, pleasant and unpleasant descriptions elicited larger late waves from 400 to 800 milliseconds than neutral descriptions. In a two-sentence climate example, a statement about harmful consequences generated a stronger simulated response than one about mitigation, a pattern the authors describe as consistent with negativity bias. The paper reports the emotion, face and text comparisons descriptively, without formal statistical tests. [1]

The practical case is time and expense. EEG data collection is slow and costly, and the authors propose the platform for testing hypotheses or estimating sample sizes before human data collection. Lead author Alejandro Santos-Mayo, now at NeuroSpin and the University of Florida, called the system a proof of concept and a technical milestone. He invited readers to try the browser-based model. [1]

The study also signals a methodological emphasis. Much AI-brain work has centered on decoding what someone is experiencing; this project focuses on encoding, predicting electrical responses from a stimulus. The authors argue that conditional variational autoencoders can extract the non-linear latent structure of EEG. Santos-Mayo said he is applying related neural-network methods to other cognitive functions and building tools for researchers. [1]

Important limits bound the results. Although the platform accepts written prompts, it was trained only on brain recordings from people viewing photographs, not reading sentences. Text inputs therefore estimate a response tied to a related visual concept rather than simulating language comprehension; the authors caution that text-based predictions are less validated and may be noisier than picture-based outputs. Santos-Mayo cautioned that synthetic data cannot replace empirical human recordings or support definitive scientific conclusions. Its purpose, he said, is fast hypothesis testing or sample-size estimation before real data collection. [1]

The current model used 88 participants and 360 images, which the authors describe as establishing the architecture's viability rather than final fidelity. It produces smoothed waveforms resembling averages over repeated trials, so it cannot capture trial-to-trial variability in raw single EEG recordings or model cellular mechanisms. Its correspondence with real activity weakened in external datasets collected with different hardware and tasks. Santos-Mayo stressed that the model cannot replace a real brain or create a synthetic mind; such a system would require an effectively unattainable amount of data from one person, with no guaranteed success. Because the framework depends on CLIP, blind spots, biases or content filters in that third-party system can carry into generated waves, though CLIP can be swapped out. [1]

Several questions remain unanswered. How well would the approach generalize beyond 88 virtual brains to larger and more diverse samples? How faithfully could it handle actual language comprehension rather than photo-derived text estimates? Can its comparisons be tested with formal statistics instead of descriptive reports? How much do CLIP's filters and biases shape predictions for sensitive or unusual inputs? Can later versions capture single-trial variability or cellular mechanisms rather than smoothed averages? And how much can external-dataset performance improve across different hardware and tasks? These are open questions, not findings. [1]

The reported next steps are concrete but not outcomes. The researchers say they plan upgrades trained on larger datasets with more participants, more varied stimuli, additional imaging modalities and newer AI components, and they welcome collaboration. Santos-Mayo said he is developing new non-linear neural-network methods to decode brain activity and applying similar approaches to other cognitive functions. Whether those upgrades improve fidelity is a matter for future testing, not an established result. [1]

Sources

PsyPost: Relationships

Prepared with AI assistance from the cited sources.