Marine Biological Laboratory Explores Human Memory With AI and Virtual Reality

The works of Plato state that when humans have an experience, some level of change occurs in their brain, which is powered by memory - specifically long-term memory.

This change is what Andre Fenton, professor of neural science at New York University, and Abhishek Kumar, assistant professor of cell and regenerative biology at the University of Wisconsin-Madison, are studying at the Marine Biological Laboratory (MBL) in Woods Hole, Massachusetts.

My life's work is to understand how minds operate, and especially to understand memory - not merely as a trace of the past in the brain but as an estimate of the future that the brain is afforded, Fenton said.

The researchers have upleveled their project by harnessing NVIDIA RTX GPUs and HP Z Workstations to visualize massive datasets and by integrating custom AI tools and syGlass, a virtual-reality (VR) platform for scientific exploration.

This project is additionally supported by grants from the National Institute of Mental Health and the Chan Zuckerberg Initiative.

A Neural Forest Uncovered Memory is the job of the brain's hippocampus. This C-shaped structure, resembling a seahorse, is the main focus of the MBL research group.

Fenton describes the cells within the hippocampus as a forest, where billions of neurons look like tiny tree trunks and the lines coming off the trunks look like leaves.

Projection image of neuronal cell nuclei (left) and dendrites (right) or branched extensions of a nerve cell. Images were acquired by Matthew Parent and Daryl Watkins. The team is studying a small portion of these leaves - representing protein markers: an incredibly tedious task due to their length, at about a micrometer each. A researcher must search through the forest of brain cells to find the correct protein markers, which make up only about 1% of all protein markers in the hippocampus.

The researchers were looking to ease the process of studying these proteins and what their varying structures may reveal about memory encoding.

Collecting and analyzing enough 3D volumetric data on protein markers was a bottleneck within the project until NVIDIA and HP technologies were introduced into the workflow.

This is a massive computational challenge, and the HP and NVIDIA technologies have enabled us to do the first step: capture, check and store the 3D image data, Fenton said.

Using these technologies, the MBL researchers captured 10 terabytes of volumetric data and then performed human visual-quality inspections.

Understanding Memory Could Prevent Neurological Diseases The team's ultimate goal of discovering the function of memory at a molecular level can boost research into the root causes of brain diseases tied to neurocognition, such as Alzheimer's and dementia.

People don't normally think of memory as part of their mental health, but almost all mental dysfunction depends on what your brain stores - the beliefs, the anticipations, the anxieties that you have and the things that you expect, said Fenton. These are all different aspects of what happens when you have a memory, so almost all neuropsychiatric illnesses and manipulation depend on this understanding.

As a step toward solving these large-scale problems, the researchers are looking at how memory is affected when proteins go to incorrect locations in the hippocampus.

The team is also examining the correlation between the structure and function of brain cells through high-resolution 3D images curated and stored using syGlass on the HP Z high performance workstation powered and supported with multiple NVIDIA RTX GPUs.

If we can understand how something is built, then if there's a problem, we can dissect that and get to the bottom of it, said Kumar. That's what we're trying to do: understand how we retain memory, so if a problem arises, we know how to fix it.

Enabling Virtual Reality and Student Exploration The use of syGlass on the HP Z6 desktop workstation, running on NVIDIA RTX GPUs, turned the researchers' endeavor from a time-consuming operation into an interactive scientific exploration - ideal for high-school-student engagement.

The HP-NVDIA-syGlass system lets us innovate by engaging three high-school interns, said Kumar. They had an abstract interest in our science, and we recognized that the syGlass virtual experience might enthrall them. We were right.

The researchers brought these three curious students into their lab this summer to analyze the memory proteins using the VR headsets, which allowed for 3D visuals of the data.

Their task was to find the specific proteins that were memory-related and label them as such. While this may sound like a simple task, the interns had to sift through a sea of billions of neurons to find only a few thousand protein markers that were relevant to the research.

High school intern using the syGlass VR headset to identify protein markers. Image taken by Andre Fenton. Due to the success of this pilot program, the team is now looking to expand high-school research opportunities for the project.

Why leave it at three students? said Fenton. Next year, it could be 10 at multiple locations helping us learn about brains while they learn about brains.

Learn more about NVIDIA-accelerated academic research.

LINK:	https://blogs.nvidia.com/blog/mbl-human-memory-ai-vr-rtx/...
	See more stories from nvidia

More from Nvidia