Deep Neural network models as digital twins for functional characterization of visual cortex

Abstract

Understanding how the brain processes visual information remains a fundamental challenge in neuroscience. Recent advances in deep learning have revolutionized our ability to model visual processing, enabling the development of "digital twins" - deep neural networks that accurately predict how individual neurons respond to arbitrary visual stimuli. This thesis explores how these models can advance our understanding of visual processing across different scales and species, through three projects combining computational prediction with biological validation. In the first project, we investigate the dynamic relationship between behavioral state and visual processing in mouse primary visual cortex. By extending digital twin models to incorporate both color processing and behavioral variables, we predict how neural responses change with behavioral state. Through systematic generation of maximally exciting inputs (MEIs) conditioned on behavioral measurements and subsequent closed-loop validation experiments, we uncover a novel mechanism where mice rapidly modulate their color processing based on pupil size to enhance detection of behaviorally relevant stimuli. The second project addresses a fundamental question in visual neuroscience: how is functional selectivity organized in higher visual areas? Using digital twins to characterize neuronal responses in macaque area V4, we identify distinct functional groups of neurons sharing preferences for complex visual features. Our key finding demonstrates that neurons within the same cortical column exhibit similar response preferences, providing evidence that columnar organization, previously established in early visual areas, extends to higher-order visual processing as a general principle of cortical computation. The third project establishes a standardized benchmark platform for evaluating digital twin models of mouse V1, addressing the growing need for systematic comparison of neural prediction models. We provide a comprehensive dataset of thousands of neurons responding to naturalistic stimuli, coupled with evaluation tools and metrics for rigorous model comparison. This framework promotes collaborative advancement through competitive model development and establishes clear criteria for measuring progress in neural response prediction. Taken together, this work establishes digital twins as powerful tools for investigating neural computation, enabling systematic exploration of hypotheses through a combination of computational modeling and targeted biological experiments. The development of standardized benchmarks ensures that progress in this field remains rigorous and reproducible. These advances provide a framework for investigating neural computation across scales, from single neurons to population dynamics, accelerating our understanding of visual processing in the brain.