You are here : Home > News > Predicting cognitive activities from fMRI images, it's possible

Scientific result | Brain | Cognition | MRI | Artificial intelligence

Predicting all cognitive activities from images of the functionning brain, it's possible

In this study, a NeuroSpin team succeeded in decoding a large set of human mental processes by training neural networks to predict cognitive labels from a cognitive atlas and by leveraging the data set of NeuroVault, a web-based database for collecting and sharing statistical maps of the human brain. A success that now makes it possible to infer a mental process from observed brain activity.

Published on 30 June 2022


To better understand the links between the cognitive (mental) processes responsible for the treatment of all the information we exchange with our environment and the global functional organization of the brain, scientists analyze "brain maps" established by functional magnetic resonance imaging (fMRI). The latter has opened up the possibility of studying, through statistical analyses, how and where our brain activity is modulated in the many mental processes of our daily lives. It is now common to introduce fMRI data into machine-learning models, in order to encode brain activity in response to a behavior, and then to be able to decode this activity, i.e. to draw conclusions about the function of a brain structure.


In this work, the researchers went much further by expanding the scope of the usual image-based studies : by using all statistical maps of fMRI imaging from the largest available data repository, NeuroVault, they trained machine learning models capable of decoding the cognitive concepts probed, including in novel studies that may involve previously unexplored experimental conditions. To do this, they labeled NeuroVault images with concepts from the Cognitive Atlas, an ontology of cognition, by tapping into the metadata associated with the atlas. They trained neural networks to predict these cognitive labels on tens of thousands of brain images. In this way, they were able to decode more than 50 classes of mental processes, while overcoming the heterogeneity, imbalance, and noise of the training data, without any prior knowledge of the experimental setting or the relevant concepts.

These results demonstrate that image-based meta-analyses can be undertaken on a large scale and with minimal manual data processing. They allow broad reverse inferences to be made, i.e., to conclude about mental processes from observed brain activity.

Contact : Bertrand Thirion (  
Website : 

NeuroVault is a public repository of unbounded fMRI statistical maps, segmentations and brain atlases.

Cognitive Atlas is a collaborative knowledge construction project for the development of a database that characterizes the state of current thinking in cognitive sciences.
From the statistical maps present in NeuroVault, the classification algorithm proposed by the team learns to decode the concepts of the Cognitive Atlas deduced from the available annotations. It is then able to successfully decode maps from new cognitive studies.

Top page