Coming Up ACEs: Decoding the AI Technology That's Enhancing Games With Realistic Digital Humans
03/04/2024
Digital characters are leveling up.
Non-playable characters often play a crucial role in video game storytelling, but since they're usually designed with a fixed purpose, they can get repetitive and boring - especially in vast worlds where there are thousands.
Thanks in part to incredible advances in visual computing like ray tracing and DLSS, video games are more immersive and realistic than ever, making dry encounters with NPCs especially jarring.
Earlier this year, production microservices for the NVIDIA Avatar Cloud Engine launched, giving game developers and digital creators an ace up their sleeve when it comes to making lifelike NPCs. ACE microservices allow developers to integrate state-of-the-art generative AI models into digital avatars in games and applications. With ACE microservices, NPCs can dynamically interact and converse with players in-game and in real time.
Leading game developers, studios and startups are already incorporating ACE into their titles, bringing new levels of personality and engagement to NPCs and digital humans.
Bring Avatars to Life With NVIDIA ACE The process of creating NPCs starts with providing them a backstory and purpose, which helps guide the narrative and ensures contextually relevant dialogue. Then, ACE subcomponents work together to build avatar interactivity and enhance responsiveness.
NPCs tap up to four AI models to hear, process, generate dialogue and respond.
The player's voice first goes into NVIDIA Riva, a technology that builds fully customizable, real-time conversational AI pipelines and turns chatbots into engaging and expressive assistants using GPU-accelerated multilingual speech and translation microservices.
With ACE, Riva's automatic speech recognition (ASR) feature processes what was said and uses AI to deliver a highly accurate transcription in real time. Explore a Riva-powered demo of speech-to-text in a dozen languages.
The transcription then goes into an LLM - such as Google's Gemma, Meta's Llama 2 or Mistral - and taps Riva's neural machine translation to generate a natural language text response. Next, Riva's Text-to-Speech functionality generates an audio response.
Finally, NVIDIA Audio2Face (A2F) generates facial expressions that can be synced to dialogue in many languages. With the microservice, digital avatars can display dynamic, realistic emotions streamed live or baked in during post-processing.
The AI network automatically animates face, eyes, mouth, tongue and head motions to match the selected emotional range and level of intensity. And A2F can automatically infer emotion directly from an audio clip.
Each step happens in real time to ensure fluid dialogue between the player and the character. And the tools are customizable, giving developers the flexibility to build the types of characters they need for immersive storytelling or worldbuilding.
Born to Roll At GDC and GTC, developers and platform partners showcased demos leveraging NVIDIA ACE microservices - from interactive NPCs in gaming to powerful digital human nurses.
Ubisoft is exploring new types of interactive gameplay with dynamic NPCs. NEO NPCs, the product of its latest research and development project, are designed to interact in real time with players, their environment and other characters, opening up new possibilities for dynamic and emergent storytelling.
The capabilities of these NEO NPCs were showcased through demos, each focused on different aspects of NPC behaviors, including environmental and contextual awareness; real-time reactions and animations; and conversation memory, collaboration and strategic decision-making. Combined, the demos spotlighted the technology's potential to push the boundaries of game design and immersion.
Using Inworld AI technology, Ubisoft's narrative team created two NEO NPCs, Bloom and Iron, each with their own background story, knowledge base and unique conversational style. Inworld technology also provided the NEO NPCs with intrinsic knowledge of their surroundings, as well as interactive responses powered by Inworld's LLM. NVIDIA A2F provided facial animations and lip syncing for the two NPCs real time.
Inworld and NVIDIA set GDC abuzz with a new technology demo called Covert Protocol, which showcased NVIDIA ACE technologies and the Inworld Engine. In the demo, players controlled a private detective who completed objectives based on the outcome of conversations with NPCs on the scene. Covert Protocol unlocked social simulation game mechanics with AI-powered digital characters that acted as bearers of crucial information, presented challenges and catalyzed key narrative developments. This enhanced level of AI-driven interactivity and player agency is set to open up new possibilities for emergent, player-specific gameplay.
Built on Unreal Engine 5, Covert Protocol uses the Inworld Engine and NVIDIA ACE, including NVIDIA Riva ASR and A2F, to augment Inworld's speech and animation pipelines.
In the latest version of the NVIDIA Kairos tech demo built in collaboration with Convai, which was shown at CES, Riva ASR and A2F were used to significantly improve NPC interactivity. Convai's new framework allowed the NPCs to converse among themselves and gave them awareness of objects, enabling them to pick up and deliver items to desired areas. Furthermore, NPCs gained the ability to lead players to objectives and traverse worlds.
Digital Characters in the Real World The technology used to create NPCs is also being used to animate avatars and digital humans. Going beyond gaming, task-specific generative AI is moving into healthcare, customer service and
LINK: | https://blogs.nvidia.com/blog/ai-decoded-ace-microservices-digital-hum... |
See more stories from nvidia |
Most recent headlines
04/08/2024
Dalet Appoints Santiago Solanas as CEO to Lead Next Era of Growth and Innovation
Dalet, a leading technology and service provider for media-rich organizations, is excited to announce Santiago Solanas as its new Chief Executive Officer (CEO)....
03/06/2024
Dalet and Veritone Reach Agreement to Distribute, Transact and Monetize Media Archives
Dalet, a leading technology and service provider for media-rich organizations, a...
09/05/2024
Billie Eilish and Spotify Invite You Inside the Mysterious Universe of HIT ME HARD AND SOFT'
Billie Eilish fans have been waiting with bated breath for her third studio albu...
09/05/2024
12 Artists, 6 Countries, Endless Talent: Meet the RADAR Latin America Class of 2024
From reggaeton to M sica Mexicana, Latin artists are reshaping the rhythm of glo...
09/05/2024
12 Artistas, 6 pases, talento infinito: Conoce a la clase RADAR Latinoamrica del 2024
Desde el reggaet n hasta la M sica Mexicana, los artistas latinos est n dando fo...
09/05/2024
Join Costa Georgiadis, Melissa Leong & Samuel Johnson For New Ground-breaking Series: The Hospital: In The Deep End
Join Costa Georgiadis, Melissa Leong & Samuel Johnson For New Ground-breaking Se...
09/05/2024
Alone Australia audience continues to grow
Alone Australia audience continues to grow 9 May, 2024 Media releases The program continues to deliver for SBS with significant uplifts in digital BVOD vie...
09/05/2024
Factories of the Future: Aerojet Rocketdyne Completes Key Milestones in Expansion and Modernization Effort
Aerojet Rocketdyne is achieving key milestones executing the $215.6M Cooperative...
09/05/2024
Join Calrec at MPTS 2024
Join Calrec at MPTS 2024 | May 15 -16 | Booth A40 | Olympia, London We're looking forward to meeting up with customers and partners at this year's Media...
09/05/2024
NHL Draft to be First Live Televised Event from Vegas' Sphere
The 2024 Upper Deck NHL Draft will be held at Sphere in Las Vegas on Friday, June 28 and Saturday, June 29, marking the first time the event will be held in Las...
09/05/2024
WBD Nears 100M Streaming Subs, Streaming Profits Up
NEW YORK In its Q1 2024 earnings report, Warner Bros. Discovery continued to see growth in its streaming operations, adding 2 million streaming subs in Q1 2024 ...
09/05/2024
Custom Consoles Unveils M-Desk Refinements
LEIGHTON BUZZARD, U.K. Custom Consoles has announced refinements to its M-Desk Technical HA heavy duty height-adjustable desk workstation....
09/05/2024
Veritone Rolls Out AI Consulting, Services Group
DENVER Veritone has launched a consulting and services group focused on bridging the marketplace gap it says exists for seamless artificial intelligence (AI) im...
09/05/2024
How sound (and the technology behind it) can set the tone and enhance emotions in film and TV
Sefi Carmel, CEO and founder of SphereTrax, talks about the importance of the so...
09/05/2024
Yale University's Center for Collaborative Arts and Media (CCAM) Cultivates Creativity with Blackmagic Design
Yale University's Center for Collaborative Arts and Media (CCAM) Cultivates ...
09/05/2024
Blender: Awesome Rack Focus Rig
Blender: Awesome Rack Focus Rig Simon Ubsdell May 9, 2024 0 Comments In which we build a focusing rig in the Blender compositor to enable selective fo...
09/05/2024
Peacock Orders The Office' Offshoot
Peacock has picked up a series from Greg Daniels, who developed The Office for NBC, that is set in the same universe, according to the network, as the NBC hit...
09/05/2024
Sinclair Net Income Drops To $23 Million in First Quarter
Sinclair reported lower net income in the first quarter as distribution revenues rose and ad revenue excluding political spending dipped....
09/05/2024
Vizio Reports $12.1 Million Loss for First Quarter
Vizio Holding Corp., which has agreed to be acquired by Walmart, reported a loss in the first quarter despite growth in its advertising and data businesses....
09/05/2024
Kathleen Keefe, Broadcasting Exec, Dies at 68
Kathleen Keefe, who served as VP of sales for both Hearst Television and Post-Newsweek Stations, died peacefully in her sleep at her home in Roxbury, Connecticu...
09/05/2024
Future Today Adds Movies From Warner Bros Discovery to Fawesome
Streaming company Future Today said it made a deal with Warner Bros. Discovery that will bring 46 movies to Future Today's Fawesome channel....
09/05/2024
Warner Bros. Discovery Reports $966 Million First-Quarter Loss
Despite reporting a profit for its direct-to-consumer business, Warner Bros. Discovery reported a big loss as it took huge write-downs in the first quarter....
09/05/2024
Nexstar Reports Higher Net Income in First Quarter
Nexstar Media Group reported higher first-quarter earnings and said that it cut losses at The CW by $50 million year-over-year....
09/05/2024
Fubo Grabs Viewer Attention, According to TVision Study
Fubo, the sports-focused streaming service, scored high in getting its viewers attention, according to a study conducted by measurement and analytics company TV...
09/05/2024
SailGP Expanding LiveLineFX Graphics Platform to Other Sports
LONDON SailGP, which sponsors high-speed boat racing events worldwide says it is developing its LiveLineFX sports graphics platform for use in other sports....
09/05/2024
Disney, WBD to Offer Streaming Bundle Starting this Summer
BURBANK, Calif. Starting this summer, U.S. consumers will be able to purchase a new streaming bundle offered by Disney and Warner Bros. Discovery that includes ...
09/05/2024
Sony and Apollo plan to break up Paramount'
Companies are expected to sell off Paramounts linear channels as well as close Paramount Plus By Jenny Priestley Published: May 9, 2024 Companies are expe...
09/05/2024
Paramount Global Selects Amdocs Vubiquity to Provide Mana...
Amdocs (NASDAQ:DOX), a leading provider of software and services to communications and media companies, today announced that Vubiquity, an Amdocs company, has b...
09/05/2024
CueScript Awarded Best of Show for CueFlip at NAB 2024
CueScript, the leading developer of professional teleprompting solutions, has announced that its CueFlip screen-to-screen folding field prompter has won in this...
09/05/2024
Mother's Day Ideas from CBC
Capitol Broadcasting has a myriad of ways to celebrate Mom! In honor of Mother's Day, here are a few ideas CBC divisions have to help you honor, or remember...
09/05/2024
Scripps Appoints New President and CEO of the Scripps Howard Fund
CINCINNATI Meredith Delaney has been appointed president and CEO of the Scripps Howard Fund, the philanthropic arm of The E.W. Scripps Company....
09/05/2024
Walmart Launches the Onn. 4K Streaming Device
Following leaks earlier this month of Walmart's plans to launch a new 4K streaming box, Walmart has officially unveiled the Onn. 4K Pro, a hybrid streaming ...
09/05/2024
DirecTV Adds More CW Stations
DirecTV continues to add CW affiliates to its streaming line-up, with the addition of four stations owned by Gray Television....
09/05/2024
Takeshi Kitano's film KUBI graded with DaVinci Resolve
Takeshi Kitano's film KUBI graded with DaVinci Resolve Brie Clayton May 8, 2024 0 Comments Blackmagic Design today announced that DaVinci Resolve ...
09/05/2024
zweiB updates to latest version of R&S CLIPSTER
zweiB updates to latest version of R&S CLIPSTER Brie Clayton May 8, 2024 0 Comments Specialist in deliverables for cinema, events and trade shows Ro...
09/05/2024
Simplify Your Filmmaking with Portable Lights
Simplify Your Filmmaking with Portable Lights Sean Alami May 8, 2024 0 Comments Discover the sleek, mobile Zhiyun lights that enhance cinematic scenes...
09/05/2024
The Benefits of using an Audio Interface
The Benefits of using an Audio Interface Colin Smith May 8, 2024 0 Comments This tutorial takes you through the hardware and software benefits of usin...
09/05/2024
Mother's Day Idea from CBC
Capitol Broadcasting has a myriad of ways to celebrate Mom! In honor of Mother's Day, here are a few ideas CBC divisions have to help you honor, or remember...
09/05/2024
Lachlan Murdoch Sees Progress Toward Sports Streaming Product
Fox CEO Lachlan Murdoch said his company, The Walt Disney Co. and Warner Bros. Discovery are in a two-minute drill to get their streaming sports joint venture t...
09/05/2024
Dale Earnhardt Jr. Joins Prime Video's NASCAR Team
NASCAR legend Dale Earnhardt Jr. is joining the Prime Video broadcast booth when the streaming network begins NASCAR Cup Series coverage next year. Earnhardt Jr...
09/05/2024
Nexstar Pitches Sports on The CW, Extra Reach Via Stations (Upfronts)
If upfront advertisers are looking for reach, Nexstar Media Group believes its base of local stations gives it a unique way to achieve that goal....
09/05/2024
Pluto TV Debuts NBCUniversal Local Streaming News Channels
NBCUniversal Local's 15 NBC and Telemundo local and regional streaming news channels begin launching on Pluto TV this month, starting May 8 with five channe...
09/05/2024
Don't Tell Mom the Babysitter's Dead' Debuts on BET Plus May 16
BET Plus said it has set May 16 as the premiere date for its original movie Don't Tell Mom the Babysitter's Dead....
09/05/2024
Disney's Rita Ferro Is Banking on Sports, Streaming, Tech (Upfronts)
Heading into the upfront, the advertising market is noticeably healthier than it was a year ago, Rita Ferro, president of global advertising sales at The Walt D...
09/05/2024
Yahsat welcomes high-level delegation from South Africa led by HE Deputy President Paul Ma...
The visit discussed potential for Yahsat to expand its presence in South Africa ...
09/05/2024
VEON's Banglalink Leads 3G Phase-out, Deepening its Focus on 4G
09 May 2024 VEON's Banglalink Leads 3G Phase-out, Deepening its Focus on 4G Amsterdam, 09 May 2024: VEON Ltd. (NASDAQ: VEON, Euronext Amsterdam: VEON), a g...
09/05/2024
Steve Thrap, Longtime Staples Center VP of Broadcast Ops and Beloved Industry Stalwart, Dies at 72
Steve Thrap, Longtime Staples Center VP of Broadcast Ops and Beloved Industry St...
09/05/2024
SVG College Sports Media Awards: 2024 Finalists Unveiled; ACC Network's Roddy Jones to Host Ceremony on May 29
SVG College Sports Media Awards: 2024 Finalists Unveiled; ACC Network's Rodd...
09/05/2024
Into the Unknown: Teaser for Joko Anwar's Nightmares and Daydreams' Unveils a Sci-fi Supernatural Universe
Back to All News Into the Unknown: Teaser for Joko Anwar's Nightmares and ...
09/05/2024
Keep the Faith: The Believers' Returns for Season 2
Back to All News Keep the Faith: The Believers' Returns for Season 2 Entertainment 09 May 2024 GlobalThailand Link copied to clipboard Responding to ...