
The pace of technology innovation has accelerated in the past year, most dramatically in AI. And in 2024, there was no better place to be a part of creating those breakthroughs than NVIDIA Research.
NVIDIA Research is comprised of hundreds of extremely bright people pushing the frontiers of knowledge, not just in AI, but across many areas of technology.
In the past year, NVIDIA Research laid the groundwork for future improvements in GPU performance with major research discoveries in circuits, memory architecture and sparse arithmetic. The team's invention of novel graphics techniques continues to raise the bar for real-time rendering. And we developed new methods for improving the efficiency of AI - requiring less energy, taking fewer GPU cycles and delivering even better results.
But the most exciting developments of the year have been in generative AI.
We're now able to generate, not just images and text, but 3D models, music and sounds. We're also developing better control over what is generated: to generate realistic humanoid motion and to generate sequences of images with consistent subjects.
The application of generative AI to science has resulted in high-resolution weather forecasts that are more accurate than conventional numerical weather models. AI models have given us the ability to accurately predict how blood glucose levels respond to different foods. Embodied generative AI is being used to develop autonomous vehicles and robots.
And that was just this year. What follows is a deeper dive into some of NVIDIA Research's greatest generative AI work in 2024. Of course, we continue to develop new models and methods for AI, and expect even more exciting results next year.
ConsiStory: AI-Generated Images With Main Character Energy ConsiStory, a collaboration between researchers at NVIDIA and Tel Aviv University, makes it easier to generate multiple images with a consistent main character - an essential capability for storytelling use cases such as illustrating a comic strip or developing a storyboard.
The researchers' approach introduced a technique called subject-driven shared attention, which reduces the time it takes to generate consistent imagery from 13 minutes to around 30 seconds.
Read the ConsiStory paper.
ConsiStory is capable of generating a series of images featuring the same character. Edify 3D: Generative AI Enters a New Dimension NVIDIA Edify 3D is a foundation model that enables developers and content creators to quickly generate 3D objects that can be used to prototype ideas and populate virtual worlds.
Edify 3D helps creators quickly ideate, lay out and conceptualize immersive environments with AI-generated assets. Novice and experienced content creators can use text and image prompts to harness the model, which is now part of the NVIDIA Edify multimodal architecture for developing visual generative AI.
Read the Edify 3D paper and watch the video on YouTube.
Fugatto: Flexible AI Sound Machine for Music, Voices and More A team of NVIDIA researchers recently unveiled Fugatto, a foundational generative AI model that can create or transform any mix of music, voices and sounds based on text or audio prompts.
The model can, for example, create music snippets based on text prompts, add or remove instruments from existing songs, modify the accent or emotion in a voice recording, or generate completely novel sounds. It could be used by music producers, ad agencies, video game developers or creators of language learning tools.
Read the Fugatto paper.
GluFormer: AI Predicts Blood Sugar Levels Four Years Out Researchers from the Weizmann Institute of Science, Tel Aviv-based startup Pheno.AI and NVIDIA led the development of GluFormer, an AI model that can predict an individual's future glucose levels and other health metrics based on past glucose monitoring data.
The researchers showed that, after adding dietary intake data into the model, GluFormer can also predict how a person's glucose levels will respond to specific foods and dietary changes, enabling precision nutrition. The research team validated GluFormer across 15 other datasets and found it generalizes well to predict health outcomes for other groups, including those with prediabetes, type 1 and type 2 diabetes, gestational diabetes and obesity.
Read the GluFormer paper.
LATTE3D: Enabling Near-Instant Generation, From Text to 3D Shape Another 3D generator released by NVIDIA Research this year is LATTE3D, which converts text prompts into 3D representations within a second - like a speedy, virtual 3D printer. Crafted in a popular format used for standard rendering applications, the generated shapes can be easily served up in virtual environments for developing video games, ad campaigns, design projects or virtual training grounds for robotics.
Read the LATTE3D paper.
MaskedMimic: Reconstructing Realistic Movement for Humanoid Robots To advance the development of humanoid robots, NVIDIA researchers introduced MaskedMimic, an AI framework that applies inpainting - the process of reconstructing complete data from an incomplete, or masked, view - to descriptions of motion.
Given partial information, such as a text description of movement, or head and hand position data from a virtual reality headset, MaskedMimic can fill in the blanks to infer full-body motion. It's become part of NVIDIA Project GR00T, a research initiative to accelerate humanoid robot development.
Read the MaskedMimic paper.
StormCast: Boosting Weather Prediction, Climate Simulation In the field of climate science, NVIDIA Research announced StormCast, a generative AI model for emulating atmospheric dynamics. While other machine learning models trained on global data have a spatial resolution of about 30 kilometers and a temporal resolution of six hours, StormCast achieves a 3-kilometer, hourly scale.
More from Nvidia
07/02/2025
Every year, venomous snakes kill over 100,000 people and leave 300,000 more with devastating injuries - amputations, paralysis and permanent disabilities. The v...
06/02/2025
AI built for speech is now decoding the language of earthquakes.
A team of researchers from the Earth and environmental sciences division at Los Alamos Nationa...
06/02/2025
GeForce NOW celebrates its fifth anniversary this February with a lineup of five major releases. The month kicks off with Kingdom Come: Deliverance II. Prepare ...
05/02/2025
When non-technical users can create and deploy reliable AI workflows, organizations can do more to serve their clientele
Platforms for developing no- and low-c...
05/02/2025
The financial services industry is reaching an important milestone with AI, as organizations move beyond testing and experimentation to successful AI implementa...
05/02/2025
NVIDIA's GeForce RTX 5090 and 5080 GPUs - which are based on the groundbreaking NVIDIA Blackwell architecture -offer up to 8x faster frame rates with NVIDIA...
04/02/2025
AI reasoning models and agents are set to transform industries, but delivering their full potential at scale requires massive compute and optimized software. Th...
31/01/2025
The recently released DeepSeek-R1 model family has brought a new wave of excitement to the AI community, allowing enthusiasts and developers to run state-of-the...
30/01/2025
DeepSeek-R1 is an open model with state-of-the-art reasoning capabilities. Instead of offering direct responses, AI models like DeepSeek-R1 perform reasoning th...
30/01/2025
GeForce NOW turns five this February. Five incredible years of high-performance gaming have been made possible thanks to the members who've joined the cloud...
30/01/2025
New GeForce RTX 5090 and RTX 5080 GPUs - built on the NVIDIA Blackwell architect...
29/01/2025
AI agents with advanced perception and cognition capabilities are making digital experiences more dynamic and personalized across retail, finance, entertainment...
27/01/2025
Named after Greek mythology's goddess of the sea, France-based startup Amphi...
23/01/2025
Businesses across every industry are rolling out AI services this year. For Microsoft, Oracle, Perplexity, Snap and hundreds of other leading companies, using t...
23/01/2025
GeForce NOW is expanding mod support for hit game Baldur's Gate 3 in collaboration with Larian Studios and mod.io for Ultimate and Performance members.
Thi...
22/01/2025
Companies and organizations are increasingly using AI to protect their customers and thwart the efforts of fraudsters around the world.
Voice security company ...
22/01/2025
Editor's note: This post is part of Into the Omniverse, a series focused on ...
22/01/2025
AI agents - which can understand, adapt to and support each user's unique journey - are making online shopping and digital marketing more efficient and pers...
21/01/2025
More than 90 million new vehicles are introduced to roads across the globe every...
16/01/2025
Time to suit up, members. The multiverse is about to get a whole lot cloudier as GeForce NOW opens a portal to the first season of hit game Marvel Rivals from N...
16/01/2025
AI agents are poised to transform productivity for the world's billion knowledge workers with knowledge robots that can accomplish a variety of tasks. To ...
15/01/2025
Troves of unwatched surgical video footage are finding new life, fueling AI tools that help make surgery safer and enhance surgical education. The Surgical Data...
14/01/2025
AI is making inroads across the entire healthcare industry - from genomic research to drug discovery, clinical trial workflows and patient care.
In a fireside ...
14/01/2025
Quantum computing is one of the most exciting areas in computer science, promising progress in accelerated computing beyond what's considered possible today...
13/01/2025
For decades, leadership in computing and software ecosystems has been a cornerst...
13/01/2025
For decades, leadership in computing and software ecosystems has been a cornerst...
13/01/2025
IQVIA, the world's leading provider of clinical research services, commercial insights and healthcare intelligence, is working with NVIDIA to build custom f...
10/01/2025
Artificial intelligence is rapidly becoming the cornerstone of innovation in the...
09/01/2025
Driving the future of smart mobility, Hyundai Motor Group (the Group) is partnering with NVIDIA to develop the next generation of safe, secure mobility with AI ...
09/01/2025
This GFN Thursday recaps the latest cloud announcements from the CES trade show, including GeForce RTX gaming expansion across popular devices such as Steam Dec...
08/01/2025
Over the past year, generative AI has transformed the way people live, work and play, enhancing everything from writing and content creation to gaming, learning...
07/01/2025
Data is the fuel of AI applications, but the magnitude and scale of enterprise data often make it too expensive and time-consuming to use effectively.
Accordin...
07/01/2025
In the fast-evolving landscape of AI, it's becoming increasingly important to develop models that can accurately simulate and predict outcomes in physical, ...
06/01/2025
The next big moment in AI is in sight - literally.
Today, more than 1.5 billion enterprise level cameras deployed worldwide are generating roughly 7 trillion h...
06/01/2025
Generative AI and foundation models let autonomous machines generalize beyond th...
06/01/2025
According to Gartner, the worldwide end-user spending on all IT products for 202...
02/01/2025
Artificial intelligence and accelerated computing are being used to help solve the world's greatest challenges.
NVIDIA has reinvented the computing stack -...
02/01/2025
GeForce NOW is kicking off 2025 by delivering 14 games to the cloud this month, with two available to stream this week so members can get started on their New Y...
30/12/2024
The pace of technology innovation has accelerated in the past year, most dramati...
27/12/2024
NVIDIA's AI Podcast gives listeners the inside scoop on the ways AI is transforming nearly every industry. Since the show's debut in 2016, it's gar...
26/12/2024
This GFN Thursday wraps up another incredible year for cloud gaming. Take a look back at the top games and new features that made 2024 a standout for GeForce NO...
24/12/2024
Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, softwa...
19/12/2024
Shoppers pondering a new hairstyle can now try styles before committing to curls or a new color. An AI app by Ulta Beauty, the largest specialty beauty retailer...
19/12/2024
Stuck in a gaming rut? Get out of the loop this GFN Thursday with four new games...
18/12/2024
Editor's note: This post is part of the AI On blog series, which explores th...
18/12/2024
Imagine a future in which everyone is empowered to build and use their own AI agents. That future may not be far off, as new software is infused with intelligen...
18/12/2024
For more than two decades, the NVIDIA Graduate Fellowship Program has supported graduate students doing outstanding work relevant to NVIDIA technologies. Today,...
17/12/2024
In enterprise AI, understanding and working across multiple languages is no long...
17/12/2024
NVIDIA is taking the wraps off a new compact generative AI supercomputer, offering increased performance at a lower price with a software upgrade.
The new NVID...
16/12/2024
On Jan. 6 at 6:30 p.m. PT, NVIDIA founder and CEO Jensen Huang - with his trademark leather jacket and an unwavering vision - will step onto the CES 2025 stage....