Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, software, tools and accelerations for RTX PC and workstation users.
At Gamescom this week, NVIDIA announced that NVIDIA ACE - a suite of technologies for bringing digital humans to life with generative AI - now includes the company's first on-device small language model (SLM), powered locally by RTX AI.
The model, called Nemotron-4 4B Instruct, provides better role-play, retrieval-augmented generation and function-calling capabilities, so game characters can more intuitively comprehend player instructions, respond to gamers, and perform more accurate and relevant actions.
Available as an NVIDIA NIM microservice for cloud and on-device deployment by game developers, the model is optimized for low memory usage, offering faster response times and providing developers a way to take advantage of over 100 million GeForce RTX-powered PCs and laptops and NVIDIA RTX-powered workstations.
The SLM Advantage An AI model's accuracy and performance depends on the size and quality of the dataset used for training. Large language models are trained on vast amounts of data, but are typically general-purpose and contain excess information for most uses.
SLMs, on the other hand, focus on specific use cases. So even with less data, they're capable of delivering more accurate responses, more quickly - critical elements for conversing naturally with digital humans.
Nemotron-4 4B was first distilled from the larger Nemotron-4 15B LLM. This process requires the smaller model, called a student, to mimic the outputs of the larger model, appropriately called a teacher. During this process, noncritical outputs of the student model are pruned or removed to reduce the parameter size of the model. Then, the SLM is quantized, which reduces the precision of the model's weights.
With fewer parameters and less precision, Nemotron-4 4B has a lower memory footprint and faster time to first token - how quickly a response begins - than the larger Nemotron-4 LLM while still maintaining a high level of accuracy due to distillation. Its smaller memory footprint also means games and apps that integrate the NIM microservice can run locally on more of the GeForce RTX AI PCs and laptops and NVIDIA RTX AI workstations that consumers own today.
This new, optimized SLM is also purpose-built with instruction tuning, a technique for fine-tuning models on instructional prompts to better perform specific tasks. This can be seen in Mecha BREAK, a video game in which players can converse with a mechanic game character and instruct it to switch and customize mechs.
ACEs Up ACE NIM microservices allow developers to deploy state-of-the-art generative AI models through the cloud or on RTX AI PCs and workstations to bring AI to their games and applications. With ACE NIM microservices, non-playable characters (NPCs) can dynamically interact and converse with players in the game in real time.
ACE consists of key AI models for speech-to-text, language, text-to-speech and facial animation. It's also modular, allowing developers to choose the NIM microservice needed for each element in their particular process.
NVIDIA Riva automatic speech recognition (ASR) processes a user's spoken language and uses AI to deliver a highly accurate transcription in real time. The technology builds fully customizable conversational AI pipelines using GPU-accelerated multilingual speech and translation microservices. Other supported ASRs include OpenAI's Whisper, a open-source neural net that approaches human-level robustness and accuracy on English speech recognition.
Once translated to digital text, the transcription goes into an LLM - such as Google's Gemma, Meta's Llama 3 or now NVIDIA Nemotron-4 4B - to start generating a response to the user's original voice input.
Next, another piece of Riva technology - text-to-speech - generates an audio response. ElevenLabs' proprietary AI speech and voice technology is also supported and has been demoed as part of ACE, as seen in the above demo.
Finally, NVIDIA Audio2Face (A2F) generates facial expressions that can be synced to dialogue in many languages. With the microservice, digital avatars can display dynamic, realistic emotions streamed live or baked in during post-processing.
The AI network automatically animates face, eyes, mouth, tongue and head motions to match the selected emotional range and level of intensity. And A2F can automatically infer emotion directly from an audio clip.
Finally, the full character or digital human is animated in a renderer, like Unreal Engine or the NVIDIA Omniverse platform.
AI That's NIMble In addition to its modular support for various NVIDIA-powered and third-party AI models, ACE allows developers to run inference for each model in the cloud or locally on RTX AI PCs and workstations.
The NVIDIA AI Inference Manager software development kit allows for hybrid inference based on various needs such as experience, workload and costs. It streamlines AI model deployment and integration for PC application developers by preconfiguring the PC with the necessary AI models, engines and dependencies. Apps and games can then orchestrate inference seamlessly across a PC or workstation to the cloud.
ACE NIM microservices run locally on RTX AI PCs and workstations, as well as in the cloud. Current microservices running locally include Audio2Face, in the Covert Protocol tech demo, and the new Nemotron-4 4B Instruct and Whisper ASR in Mecha BREAK.
To Infinity and Beyond Digital humans go far beyond NPCs in games. At last month's SIGGRAPH conference, NVIDIA previewed James, an interactive digital human that can connect with people using emotions, humor and more. James is based on
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
04/07/2026
April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...
01/06/2026
January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026
Throughout the week, Dolby brings to life the latest innovatio...
08/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
08/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
08/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
08/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
08/05/2026
Liberty University on why broadcast technology isn't just a technical invest...
08/05/2026
COW Jobs: UGC, On-Camera Video Content
Brie Clayton May 7, 2026
0 Comments
UGC, On-Camera Video Content
April 17, 2026Documentary Editor - US,......
07/05/2026
Journalists reporting on Sudan are working in one of the most complex and fast-m...
07/05/2026
Multi-year partnership positions Victory+ as a free home for Dallas Cowboys orig...
07/05/2026
From sideline reporting to directing and producing, the talented sophomore is building a well-rounded foundation for a career in live sports production
In the ...
07/05/2026
Cobalt Digital has received two Future Best of Show awards at NAB Show 2026. The COBALT blueCORE platform was recognized by TV Tech, and the COBALT PACIFIC ULL-...
07/05/2026
ESPN analyst Mina Kimes will host the televised semifinals and finals of the 202...
07/05/2026
Bitmovin has announced that MUBI, the global film streaming platform, has selected Bitmovin as its cloud VOD encoding partner. Bitmovin's encoding infrastru...
07/05/2026
NBCUniversal Telemundo Enterprises and the U.S. Soccer Federation have announced...
07/05/2026
Angel City Football Club (ACFC) and Victory have announced a regional broadcast partnership bringing live match coverage to fans across the greater Los Angeles...
07/05/2026
Spiideo has announced the launch of AI Highlights inside Spiideo Play, its automated sports production platform. AI Highlights combines video, event data, audio...
07/05/2026
Leostream Corporation has announced a unified remote access ecosystem for high-p...
07/05/2026
The Atlanta Dream has partnered with Victory to stream all locally broadcast Dream games for free, expanding the team's digital distribution strategy and g...
07/05/2026
Riedel Communications has announced the appointment of Marc Engroff as Chief Fin...
07/05/2026
SES and ARD, Germany's largest public broadcasting network, have announced a long-term extension of their satellite distribution partnership through 2039. U...
07/05/2026
UpLight Technologies delivers a flexible video and lighting system for a new televised sportLaunching the Pro Cheer League required more than creating a compell...
07/05/2026
Full Day Productions and GSE Worldwide have announced Spikes Under the Lights, a...
07/05/2026
Seit dem Start im Jahr 2023 hat DJ (Beta) das personalisierte H rerlebnis von 94...
07/05/2026
Da quando stata presentata nel 2023, DJ (beta) ha aiutato a definire un'esperienza d'ascolto pi personalizzata per 94 milioni di utenti Spotify Premi...
07/05/2026
Since launching in 2023, DJ (beta) has helped shape a more personalized listenin...
07/05/2026
Desde o lan amento em 2023, o DJ (beta) j ajudou a deixar a experi ncia de ouvi...
07/05/2026
From our earliest days, Spotify has been built on a simple principle: Great audio should be easy to reach. It's what's driven us to expand from music to...
07/05/2026
Asian and Pacific Islander artists continue to shape the global soundscape, pushing creative boundaries and connecting with fans worldwide. This Asian & Pacific...
07/05/2026
Company announce long-requested choir instrument
In the latest expansion of their Symphonic Elements series, UJAM have introduced an all-new vocal instrumen...
07/05/2026
Modular-inspired sound generation with digital control
Buchla have introduced Ziggy, a new compact synthesizer built around the company's complex oscill...
07/05/2026
Combines analogue voices with digital synthesis and sequencing
Polyend have announced Drums, a new hybrid analogue and digital drum machine that combines sy...
07/05/2026
Control surface now officially supports four DAWs
When Nektar launched the Panorama CS12 control surface, it worked exclusively with Apple's Logic Pro, ...
07/05/2026
In the first quarter 2026, SGL Carbon generated consolidated sales of 184.5 million, which was 49.8 million, or 21.3%, lower than in the prior year (Q1 2025: ...
07/05/2026
Rohde & Schwarz and Greenerwave achieve precise and fast ESA antenna characteriz...
07/05/2026
Two multi-role L3Harris products - the Red Wolf launched effects vehicle and SK...
07/05/2026
L3Harris will be developing key features of a secure and resilient digital infra...
07/05/2026
Alysa Liu and Shohei Ohtani Help Drive Viewership as 91 of the Top 100 Broadcast...
07/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
07/05/2026
Cobalt Digital Wins Two Future Best of Show Awards at 2026 NAB Show
Manufacturer Recognized by TV Tech and TVBEurope for Innovation in signal processing
Cobal...
07/05/2026
Software and hardware platforms, AI power and user-friendliness on show...
07/05/2026
Intinor will demonstrate its latest technical enhancements for the Direkt series at BroadcastAsia 2026. With a continued focus on reliable contribution and remo...
07/05/2026
Bitmovin has announced that MUBI has chosen Bitmovin as its cloud VOD encoding partner, replacing MUBI's legacy on premises encoding setup to improve scalab...
07/05/2026
Meet Graduates from Berklees Class of 2026 Members of this years graduating class reflect on their proudest moments at Berklee and look ahead to whats next.
...
07/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
07/05/2026
At BroadcastAsia 2026, Interra Systems will demonstrate its latest innovations in automated quality control (QC), real-time monitoring, and captioning. The comp...
07/05/2026
At this year's Broadcast Asia, PlayBox Neo is set to unveil recent innovations across its PlayBox Neo Suite and integrated range of broadcast media solution...
Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, software, tools and accelerations for RTX PC and workstation users.
