CRCNS Research Proposal: Collaborative Research: Evaluating Machine Learning Architectures Using a Massive Benchmark Dataset of Brain Responses to Natural Scenes

Machine learning technologies have the potential to radically transform the study of the human brain, but require far more data than is typically collected during conventional neuroscience experiments. The goal of this project is to drive the application of ML techniques to neuroscience research by generating a massive dataset of brain responses from the human visual system. The resulting dataset will be freely available to scientists, educators, and students. Through a yearly modeling competition, neuroscientists will gain experience in the application of advanced computational methods and ML researchers will gain a deeper understanding of the challenges and complexities of the human brain. Results of the modeling competition will be presented at an annual conference attended by both machine learning and neuroscience researchers and students, providing an opportunity for the two groups to interact and discuss approaches. This project will foster open collaboration between neuroscientists and artificial intelligence researchers and a culture of sharing data, ideas, and progress.

The long-term goal of this work is to generate data that will lead to the development of experimentally validated and computationally powerful models of the human visual system. The project leaders will use high-field (7 Tesla) functional magnetic resonance imaging (fMRI) to measure brain responses to a broad sampling of natural images in human observers. The specific objectives are as follows: (1) Acquire, pre-process, and distribute a massive, high-resolution fMRI dataset that exploits state-of-the-art imaging techniques. The dataset will include multiple samples of brain responses to roughly eighty thousand photographs drawn from an image collection that is widely used by the ML community. (2) Establish and host an annual competition for modeling this rich dataset at the conference on Cognitive Computational Neuroscience. (3) Bridge the gap between ML architectures and the human brain by testing new ML-inspired architectures as models of the visual system. The project leaders will focus specifically on recent developments in ML that suggest new hypotheses about the dorsal visual stream.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.