Key perspectives
A patient study at Oslo University Hospital yields a unique dataset for investigating direct measurements of neural activity within the human brain. This study records neural activity in patients with epilepsy using hundreds of implanted electrodes, which provide superior signal-to-noise ratio and temporal resolution (~kHz) compared to other relevant methods such as fMRI. While recording rapid changes in neural activity, thousands of natural images are presented to the patients as part of a memory task. This patient study, led by Jørgen Sugar, bears promises to advance our understanding of visual processing and memory in the human brain. Our common project will combine this unique experimental data with the application of state-of-the-art signal processing and machine learning algorithms.
- Our objective is to understand human memory through data-driven analysis of neural activity. We will conduct the following research activities within the project:
- Signal processing: disentangling what type of signal features to extract and process, as well as how these should be encoded.
- Design, train, and evaluate models to predict (semantic categories of) images.
- Design, train, and evaluate models to predict (un)successful storage of stimulus into memory. Design, train, and evaluate models to predict what type of epilepsy activity (which does not cause seizures, so-called interictal activity) disrupts memory processes and visual processing.
- If successful, assess why the models succeed so that we can learn something about brain functions.
- Build general brain models from multiple participants with different spatial coverage of electrodes in the visual/memory system.
The project will narrow important knowledge gaps that the neuroscientific community has tried to fill for decades. We will contribute to a description of the mechanisms underlying visual processing and memory functions in humans and research how these systems are affected during memory disorders. Hence, we are addressing one of the main goals of neuroscience which is to understand how the brain uses sensory information to perform higher cognitive functions, and ultimately how these functions are disturbed in cognitive disorders.
The knowledge developed in this project will be important for:
(i) Our understanding of how the human brain perceives visual sensory stimuli and stores these into memory.
(ii) The scientific community: Models used to decode/predict visual stimuli from brain activity can be used in multiple other experimental setups like decoding content of videos from brain activity or to decode visual perception when verbal reports are not available (e.g., imagination, dreaming or in patients with an inability to communicate).
(iii) Patients: A major concern of patients with epilepsy (prevalence: ~1%) is daily life memory deficit.
Knowledge about pathological brain states causing sensory/memory deficits is crucial to increase the pathophysiological understanding of memory disorders, which will eventually improve diagnosis, guide the development of biomarkers, and help to refine treatments. A future extension of this project may target these pathological activity patterns by neuromodulation, pharmacological compounds to relief patients from their cognitive deficits.
PhD grant funded via SUURPh
SUURPh is a collaborative PhD program involving Simula, the University of Oslo (UiO), and the University of California, San Diego (UCSD): https://www.simula.no/education/research-exchanges/suurph/
Brainsight has a PhD research fellowship funded through SUURPh, with Amir Inaamullah Arfan kicking off in December 2023.