Sony Pixel Power calrec Sony

AI Decoded From GTC: The Latest Developer Tools and Apps Accelerating AI on PC and Workstation


Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and which showcases new hardware, software, tools and accelerations for RTX PC users.

NVIDIA's RTX AI platform includes tools and software development kits that help Windows developers create cutting-edge generative AI features to deliver the best performance on AI PCs and workstations.

At GTC - NVIDIA's annual technology conference - a dream team of industry luminaries, developers and researchers have come together to learn from one another, fueling what's next in AI and accelerated computing.

This special edition of AI Decoded from GTC spotlights the best AI tools currently available and looks at what's ahead for the 100 million RTX PC and workstation users and developers.

Chat with RTX, the tech demo and developer reference project that quickly and easily allows users to connect a powerful LLM to their own data, showcased new capabilities and new models in the GTC exhibit hall.

The winners of the Gen AI on RTX PCs contest were announced Monday. OutlookLLM, Rocket League BotChat and CLARA were highlighted in one of the AI Decoded talks in the generative AI theater and each are accelerated by NVIDIA TensorRT-LLM. Two other AI Decoded talks included using generative AI in content creation and a deep dive on Chat with RTX.

Developer frameworks and interfaces with TensorRT-LLM integration continue to grow as, Langchain, LlamaIndex and Oobabooga will all soon be accelerated - helping to grow the already more than 500 AI applications for RTX PCs and workstations.

NVIDIA NIM microservices are coming to RTX PCs and workstations. They provide pre-built containers, with industry standard APIs, enabling developers to accelerate deployment on RTX PCs and workstations. NVIDIA AI Workbench, an easy-to-use developer toolkit to manage AI model customization and optimization workflows, is now generally available for RTX developers.

These ecosystem integrations and tools will accelerate development of new Windows apps and features. And today's contest winners are an inspiring glimpse into what that content will look like.

Hear More, See More, Chat More Chat with RTX, or ChatRTX for short, uses retrieval-augmented generation, NVIDIA TensorRT-LLM software and NVIDIA RTX acceleration to bring local generative AI capabilities to RTX-powered Windows systems. Users can quickly and easily connect local files as a dataset to an open large language model like Mistral or Llama 2, enabling queries for quick, contextually relevant answers.

Moving beyond text, ChatRTX will soon add support for voice, images and new models.

Users will be able to talk to ChatRTX with Whisper - an automatic speech recognition system that uses AI to process spoken language. When the feature becomes available, ChatRTX will be able to understand spoken language, and provide text responses.

A future update will also add support for photos. By integrating OpenAI's CLIP - Contrastive Language-Image Pre-training - users will be able to search by words, terms or phrases to find photos in their private library.

In addition to Google's Gemma, ChatGLM will get support in a future update.

Developers can start with the latest version of the developer reference project on GitHub.

Generative AI for the Win The NVIDIA Generative AI on NVIDIA RTX developer contest prompted developers to build a Windows app or plug-in.

I found that playing against bots that react to game events with in-game messages in near real time adds a new level of entertainment to the game, and I'm excited to share my approach to incorporating AI into gaming as a participant in this developer contest. The target audience for my project is anyone who plays Rocket League with RTX hardware. - Brian Caffey, Rocket League BotChat developer

Submissions were judged on three criteria, including a short demo video posted to social media, relative impact and ease of use of the project, and how effectively NVIDIA's technology stack was used in the project. Each of the three winners received a pass to GTC, including a spot in the NVIDIA Deep Learning Institute GenAI/LLM courses, and a GeForce RTX 4090 GPU to power future development work.

OutlookLLM gives Outlook users generative AI features - such as email composition - securely and privately in their email client on RTX PCs and workstations. It uses a local LLM served via TensorRT-LLM.

Rocket League BotChat, for the popular Rocket League game, is a plug-in that allows bots to send contextual in-game chat messages based on a log of game events, such as scoring a goal or making a save. Designed to be used only in offline games against bot players, the plug-in is configurable in many ways via its settings menu.

CLARA (short for Command Line Assistant with RTX Acceleration) is designed to enhance the command line interface of PowerShell by translating plain English instructions into actionable commands. The extension runs locally, quickly and keeps users in their PowerShell context. Once it's enabled, users type their English instructions and press the tab button to invoke CLARA. Installation is straightforward, and there are options for both script-based and manual setup.

Congratulations to the #GenAIonRTX #DevContest winners:

CLARA Talk with computers in human language Matthew Yaeger

Rocket League BotChat Generate in-game chat messages Brian Caffey

Outlook LLM Compose emails securely with AI Francisco Gonzalez Blanch

NVIDIA AI Developer (@NVIDIAAIDev) March 19, 2024

From the Generative AI Theater GTC attendees can attend three AI Decoded talks on Wednesday, March 20 at the generative AI theater. These 15-minute sessions will guide the audience through ChatRTX and how developers can product
See more stories from nvidia

Most recent headlines


Dalet Appoints Santiago Solanas as CEO to Lead Next Era of Growth and Innovation

Dalet, a leading technology and service provider for media-rich organizations, is excited to announce Santiago Solanas as its new Chief Executive Officer (CEO)....


Astera Launches LunaBulb

Astera Launches LunaBulb posted: 13/06/2024 Astera LunaBulb PrepCaseKit Astera - a leading developer of innovative light solutions for film, event and TV...


Shaping the Future We Want to See: A Note From Shira Rockowitz and Kristin Feeley

This summer we're pulling back the curtain on Sundance Labs by sharing lette...


Give Me the Backstory: Get to Know Pedro Freire, the Writer-Director of Malu

By Lucy Spicer One of the most exciting things about the Sundance Film Festival is having a front-row seat for the bright future of independent filmmaking. Whi...


Spotify's 2024 Songs of Summer Predictions Are Ready to Bring the Heat

Can you hear the soundtrack to your life over the next few months? It's time for Spotify's Songs of Summer to take over your speakers as you hit the bea...


Billie Eilish and Spotify Bring the Mysterious Universe of HIT ME HARD AND SOFT' To Seoul

In May, Spotify and Billie Eilish welcomed U.K. fans inside the vast, mysterious...


Alexander Skarsgrd Spotlights Innovative Solutions to Our Planet's Biggest Challenges on the New Podcast How We Fix This'

Our planet is facing some serious issues. And How We Fix This-a new podcast host...


L3Harris Delivers Enhanced Airborne ISR Capability to Canada

Susan Harper, Consul General of Canada in Dallas, Texas, visits with the L3Harris Canadian Manned Airborne ISR (MAISR) team and U.S. Army Fixed Wing representat...


DMC Production Leverages Grass Valley AMPP for Revolutionary Remote Production of the 2024 RallyX Motorsports Championship

Montreal, Canada - June 12, 2024 - Grass Valley, a leader in live production sol...


Whisper extends EditShare to build new capabilities

Whisper Extends EditShare to Build New Capabilities Dynamic workflows designed by Tyrell will support major commissions Boston, MA, June 12, 2024 - EditShare...


Latest ATSC 3.0 Plugfest' Targets Transport, DRM, and More

COCKEYSVILLE, MD. The Sinclair Broadcast Group, along with the Pearl TV consortium, hosted yet another ATSC 3.0 device interoperability test at the ONE Media la...


Roku Unveils TV Streaming Ad Solution

SAN JOSE, Calif. Roku has announced Roku Exchange, a TV streaming-first advertising technology solution that connects ad inventory with advertiser demand....


DMC Production Reimagines Remote Race Workflow With Grass Valley AMPP

Scandinavian remote video production company DMC Production leveraged Grass Valley's AMPP for live coverage of the 2024 RallyX to live stream the motorsport...


Capitol Broadcasting Selects Triveni Digital to Simplify NextGen TV Operations

PRINCETON, N.J. Triveni Digital has announced that Capitol Broadcasting Company the owner and operator of WRAL-TV, WRAZ-TV, WILM-TV, and WNGT-TV in North Caroli...


Byron Allen Gets Fred Dressler Leadership Award at Mirror Awards in NYC

Byron Allen, founder, chairman and CEO of Allen Media Group, gets the Fred Dressler Leadership Award at the 2024 Mirror Awards June 13 in New York. Established ...


BOOK REVIEW: Here Now the News: An Inside Scoop Into New York's Best-Loved Anchors'

Jerry Barmash has published Here Now the News: An Inside Scoop into New York'...


Nate Burleson To Host Hollywood Squares'

Nate Burleson, co-host of CBS Mornings, will host the primetime game show Hollywood Squares on CBS. Drew Barrymore will inhabit the center square on the classic...


Taylor Tomlinson Gets Second Season for After Midnight' on CBS

After Midnight, the late-night show hosted by Taylor Tomlinson, has been renewed for a second season on CBS. The show is inspired by Comedy Central's @midni...


Shari Redstone's National Amusements Says Paramount-Skydance Deal is Dead

National Amusements Inc., the Redstone family holding company that owns a controlling interest in Paramount Global, said Tuesday it will not be doing a deal wit...


Netflix's Dancing for the Devil' Again Tops TVision Power Score Ranking

Netflix's Dancing for the Devil: The 7M TikTok Cult waltzed into the top slot in TVision's Power Score ranking of the biggest shows streaming on connect...


Blackmagic Design unveils world's first camera and editing system for Apple Vision Pro

The company has also introduced an updated version of DaVinci Resolve which incl...


No deal! Shari Redstone pulls plug on Skydance/Paramount Global merger

The company said it had not been able to reach mutually acceptable terms regarding the potential transaction with Skydance Media By Jenny Priestley Published...


IAB Tech Lab Releases Ad Creative ID Framework

NEW YORK In a notable development in the media industry efforts to improve the efficiency and cross-platform measurement of CTV ad campaigns, the IAB Tech Lab h...


NVISA and ATSC Deploy ATSC 3.0 Warning System at FEMA Facility

WASHINGTON, D.C. The NextGen Video Information Systems Alliance (NVISA) and the Advanced Television Systems Committee (ATSC) have announced a strategic partners...


Clear-Com Gen-IC Virtual Intercom Ships

ALAMEDA, Calif. Clear-Com has begun shipping its Gen-IC Virtual Intercom, a new system combining the benefits of cloud production with the system reliability fo...


DPA Microphones, Wisycom Join Forces to Expand U.S. Presence

LONGMONT, Colo. & CHANTILLY, Va. DPA Microphones and Wisycom have announced a strategic alliance to work together to expand their operations in the U.S....


TNT Sports Inks $650M Deal for French Open

With published reports indicating that Warner Bros. Discovery is fighting an last minute battle to retain some rights to air NBA games, WBD's TNT Sports has...


Nexstar Names Lisa Taylor SVP of Digital Ad Sales

IRVING, Texas Nexstar Media Group has appointed Lisa Taylor as senior vice president of digital advertising sales, where she will be responsible for acceleratin...


Paramount, Skydance Deal Collapses

The on-again, off-again, negotiations over a merger between Skydance Media and Paramount Global have collapsed, with the Wall Street Journal publishing a statem...


Revolutionizing Media Advertising: The Transformative Potential of AI in Campaign Planning

In today's dynamic and hyperconnected digital landscape, media advertising h...


SVG Sports Cloud Production Forum: Sessions Now Available To Watch on SVG PLAY

SVG Sports Cloud Production Forum: Sessions Now Available To Watch on SVG PLAY Catch up on the latest in live cloud-based production strategies and technology ...


InfoComm 2024, Part 1: Live-Sound Show Returns in Full Force

InfoComm 2024, Part 1: Live-Sound Show Returns in Full Force The event has evolved into the single-biggest installed-AV expo in the U.S. By Dan Daley, Audio Ed...


InfoComm 2024, Part 2: Gear for Big Sound in Stadium and Arena

InfoComm 2024, Part 2: Gear for Big Sound in Stadium and Arena Immersive live sound is part of sports entities' engagement strategies By Dan Daley, Audio E...


Netflix & Shondaland Announce the Song List and Soundtrack for 'Bridgerton' Season 3

Back to All News Netflix & Shondaland Announce the Song List and Soundtrack for...


Sweet Home' Season 3 Arrives July 19 in Fitting Finale to Epic Series

Back to All News Sweet Home' Season 3 Arrives July 19 in Fitting Finale to Epic Series Entertainment 12 June 2024 GlobalSouth Korea (Republic of Korea)...


Expo Pantalla. Mexico City. 1819 June 2024

Meet us at Expo Pantalla 2024. 18-19 June, 2024 WTC Ciudad de M xico To make an appointment, contact: [email protected] Find out more about Expo Pantalla ...


Arqiva x Wonderseekers - engaging children with science

Expanding a connection running for over 20 years, find out more about Arqiva and Wonderseekers More than 20 years ago, Arqiva donated unused land next to its M...


Scaling to New Heights: NVIDIA MLPerf Training Results Showcase Unprecedented Performance and Elasticity

The full-stack NVIDIA accelerated computing platform has once again demonstrated...


COCESNA and Thales signed two contracts to modernize its AMHS and AIM systems and navigation aids infrastructure

Facebook Twitter LinkedIn COCESNA and Thales signed two contracts to moder...


Footage of Cruelty and Abuse of Horses in Kildare revealed by RT Investigates

Hidden cameras capture horses being repeatedly hit and punched around the head Images show a horse too weak to stand being prodded with a pitchfork Major...


Nerding About NeRFs: How Neural Radiance Fields Transform 2D Images Into Hyperrealistic 3D Models

Let's talk about NeRFs - no, not the neon-colored foam dart blasters, but ne...


TOPS of the Class: Decoding AI Performance on RTX AI PCs and Workstations

Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, softwa...


Celebrate and Honor Pride With LGBTQIA+ Artists, Creators, and Curated Playlists on Spotify's GLOW Hub

From the rise of disco in the 1970s to the current wave of hyper pop, queer crea...


90 years of Dr. Schnabel GmbH - SGL's Limburg site has every reason to celebrate

On June 8, we celebrated the 90th anniversary of Dr. Schnabel GmbH at our Limbur...


Little J & Big Cuz returns for season four this July

Little J & Big Cuz returns for season four this July 11 June, 2024 Media releases *First look here* Award-winning animated children's series Little J ...



Tuesday 11 June 2024, Johannesburg. The National Film and Video Foundation (NFVF...


ST Engineering iDirect Joins WAVE Consortium as Board Member

Spearheading industry collaboration to transform the satcom industry into a fully interoperable ecosystem Herndon, VA., June 11, 2024 - ST Engineering iDirect,...


The Many Moving Parts of the Transition to NextGen TV

Fifteen years ago this month, U.S. broadcasters cut the cord on NTSC, shutting down analog TV and putting the finishing touches on a transition to digital tha...


Sally Wallington Joins Pebble To Lead Worldwide Sales

EPSOM, U.K. Sally Wallington has joined automation, content management and integrated channel specialist Pebble as senior vice president of sales with responsib...