Sony Pixel Power calrec Sony

Decoding How NVIDIA AI Workbench Powers App Development

19/06/2024

Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible and showcases new hardware, software, tools and accelerations for NVIDIA RTX PC and workstation users.

The demand for tools to simplify and optimize generative AI development is skyrocketing. Applications based on retrieval-augmented generation (RAG) - a technique for enhancing the accuracy and reliability of generative AI models with facts fetched from specified external sources - and customized models are enabling developers to tune AI models to their specific needs.

While such work may have required a complex setup in the past, new tools are making it easier than ever.

NVIDIA AI Workbench simplifies AI developer workflows by helping users build their own RAG projects, customize models and more. It's part of the RTX AI Toolkit - a suite of tools and software development kits for customizing, optimizing and deploying AI capabilities - launched at COMPUTEX earlier this month. AI Workbench removes the complexity of technical tasks that can derail experts and halt beginners.

What Is NVIDIA AI Workbench? Available for free, NVIDIA AI Workbench enables users to develop, experiment with, test and prototype AI applications across GPU systems of their choice - from laptops and workstations to data center and cloud. It offers a new approach for creating, using and sharing GPU-enabled development environments across people and systems.

A simple installation gets users up and running with AI Workbench on a local or remote machine in just minutes. Users can then start a new project or replicate one from the examples on GitHub. Everything works through GitHub or GitLab, so users can easily collaborate and distribute work. Learn more about getting started with AI Workbench.

How AI Workbench Helps Address AI Project Challenges Developing AI workloads can require manual, often complex processes, right from the start.

Setting up GPUs, updating drivers and managing versioning incompatibilities can be cumbersome. Reproducing projects across different systems can require replicating manual processes over and over. Inconsistencies when replicating projects, like issues with data fragmentation and version control, can hinder collaboration. Varied setup processes, moving credentials and secrets, and changes in the environment, data, models and file locations can all limit the portability of projects.

AI Workbench makes it easier for data scientists and developers to manage their work and collaborate across heterogeneous platforms. It integrates and automates various aspects of the development process, offering:

Ease of setup: AI Workbench streamlines the process of setting up a developer environment that's GPU-accelerated, even for users with limited technical knowledge.

Seamless collaboration: AI Workbench integrates with version-control and project-management tools like GitHub and GitLab, reducing friction when collaborating.

Consistency when scaling from local to cloud: AI Workbench ensures consistency across multiple environments, supporting scaling up or down from local workstations or PCs to data centers or the cloud.

RAG for Documents, Easier Than Ever NVIDIA offers sample development Workbench Projects to help users get started with AI Workbench. The hybrid RAG Workbench Project is one example: It runs a custom, text-based RAG web application with a user's documents on their local workstation, PC or remote system.

Every Workbench Project runs in a container - software that includes all the necessary components to run the AI application. The hybrid RAG sample pairs a Gradio chat interface frontend on the host machine with a containerized RAG server - the backend that services a user's request and routes queries to and from the vector database and the selected large language model.

This Workbench Project supports a wide variety of LLMs available on NVIDIA's GitHub page. Plus, the hybrid nature of the project lets users select where to run inference.

Workbench Projects let users version the development environment and code. Developers can run the embedding model on the host machine and run inference locally on a Hugging Face Text Generation Inference server, on target cloud resources using NVIDIA inference endpoints like the NVIDIA API catalog, or with self-hosting microservices such as NVIDIA NIM or third-party services.

The hybrid RAG Workbench Project also includes:

Performance metrics: Users can evaluate how RAG- and non-RAG-based user queries perform across each inference mode. Tracked metrics include Retrieval Time, Time to First Token (TTFT) and Token Velocity.

Retrieval transparency: A panel shows the exact snippets of text - retrieved from the most contextually relevant content in the vector database - that are being fed into the LLM and improving the response's relevance to a user's query.

Response customization: Responses can be tweaked with a variety of parameters, such as maximum tokens to generate, temperature and frequency penalty.

To get started with this project, simply install AI Workbench on a local system. The hybrid RAG Workbench Project can be brought from GitHub into the user's account and duplicated to the local system.

More resources are available in the AI Decoded user guide. In addition, community members provide helpful video tutorials, like the one from Joe Freeman below.

Customize, Optimize, Deploy Developers often seek to customize AI models for specific use cases. Fine-tuning, a technique that changes the model by training it with additional data, can be useful for style transfer or changing model behavior. AI Workbench helps with fine-tuning, as well.

The Llama-factory AI Workbench Project enables QLoRa, a fine-tuning method that minimizes memory requirements, for a variety of models, as well as
LINK: https://blogs.nvidia.com/blog/ai-decoded-workbench-hybrid-rag/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

31/08/2026

Pixotope Consolidates Virtual Production Tools into Unified Platform

Pixotope Technologies has announced a unified platform offering all Pixotope capabilities across three editions, available with the Pixotope 26.3 release in Q3 ...

31/08/2026

Elektron update Tonverk OS

Sampler gains Overbridge multitrack streaming Elektron's polyphonic sampler, the Tonverk, has just received an OS update that introduces multitrack stre...

31/08/2026

Thorus XT from UVI

Popular chorus plug-in gains second engine UVI's latest plug-in offers a fresh new take on their best-selling chorus effect, combining a pair of engines...

31/08/2026

Optus commits 3% of consumer brand spend to First Nations media in new landmark partnership with NITV

Optus commits 3% of consumer brand spend to First Nations media in new landmark ...

31/08/2026

FingerWorks Telestrators Expands European Presence Throug...

FingerWorks Telestrators, the industry leader in software-based telestration and live sports analysis solutions, announces its new partnership with Broadcast R...

31/08/2026

ARISDA Bridge Launches: DaVinci Resolve Timecode, Studio Clock, and Work-Time Tracking in One Windows App

ARISDA Bridge Launches: DaVinci Resolve Timecode, Studio Clock, and Work-Time Tr...

31/08/2026

Short Film All That's Left Shot with URSA Cine 12K LF

Short Film All That's Left Shot with URSA Cine 12K LF Brie Clayton August 31, 2026 0 Comments Intimate drama edited and graded in DaVinci Resolve ...

31/08/2026

Bridge Technologies Announces Support for Blackmagic IP10 Codec

Bridge Technologies Announces Support for Blackmagic IP10 Codec Brie Clayton August 31, 2026 0 Comments Both booths will showcase each other's tec...

31/08/2026

Creative COW Roundtable, August 23, 2026- AEAI

Creative COW Roundtable, August 23, 2026- AEAI Brie Clayton August 31, 2026 0 Comments COWs from around the world assemble to discuss Using AI to Beat...

31/08/2026

Homecoming Artist Spotlight: Susan Tedeschi

Homecoming Artist Spotlight: Susan Tedeschi The Grammy-winning singer and guitarist talks about finding her voice at Berklee ahead of her Homecoming performan...

31/08/2026

HC2 Meets with FCC Legal Advisor in Push for 5G Broadcasting

Share Copy link Facebook X Linkedin Bluesky Email...

31/08/2026

Tribeca Studios Releases First-Look Trailer For Tribeca 25, The New Documentary From Matt Tyrnauer

August 31st, 2026 Press Materials Here TRIBECA STUDIOS RELEASES FIRST-LOOK TRA...

31/08/2026

Digicel Pacific Selects Techtel for PNG Digital TV Head-End Transformation

Digicel Pacific Selects Techtel for PNG Digital TV Head-End Transformation 31 Aug Written By Mal Chandler Sydney, 31 August 2026 - Techtel has been appointe...

30/08/2026

Reason Studios launch Reason Free

Free collection includes 15 instruments Reason Studios have announced that they are making their popular Reason Rack plug-in free, and bundling it with a se...

29/08/2026

SSL to introduce O-Series V2.0

Odyssey & Oracle console software update incoming With a second console now joining their O-Series, SSL have announced the upcoming launch of O-Series V2.0,...

29/08/2026

Scripps Hires NBCU Sales Veteran Jen Carfley

Share Copy link Facebook X Linkedin Bluesky Email...

29/08/2026

Comcast's Universal Ads Taps Tracer to Upgrade Operations

Share Copy link Facebook X Linkedin Bluesky Email...

29/08/2026

PlayBox Technology to Showcase the Future of Media Orches...

PlayBox Technology is heading to IBC2026 with a clear message: the future of broadcast and streaming is not about replacing everything it is about making what...

29/08/2026

Berklee Announces Fall 2026 Signature Series

Berklee Announces Fall 2026 Signature Series This years concerts feature MonoNeon, Brandon Woody, Marcus Gilmore, and Caroline Davis, alongside Berklee Asia R...

28/08/2026

CBS L.A. Runs Two Hot Control Rooms, Dual Broadcasts for Rams-Chargers Preseason Game

KCBS takes the Chargers side, KCAL takes the Rams - with feeds out to 18 partner...

28/08/2026

Buddy Turns Childhood Comfort into Midnight Terror

(L-R) Topher Grace, Cristin Milioti, Delaney Quinn, Casper Kelly, and Keegan-Michael Key pose with the character Buddy at the Buddy Premiere during the 2026 S...

28/08/2026

IK Multimedia announce Sinphonica Maestosa Strings

The Italian orchestra, reimagined IK Multimedia have announced the upcoming launch of Sinphonica, a new family of Italian orchestral instruments designed f...

28/08/2026

Voxis from CEDAR Audio

Voice-isolation plug-in boasts sub-10ms latency Launched in 2025, CEDAR Audio's Icons range was created with the aim of bringing the company's renow...

28/08/2026

Charter Takes Scripps to Court Over Carriage of Las Vegas' KMCC

Share Copy link Facebook X Linkedin Bluesky Email...

28/08/2026

Backlight Expands Creative Operations Platform with AI-p...

Global media technology company Backlight will showcase its latest innovations across Iconik and Wildmoka live video creation and distribution at IBC2026 (booth...

28/08/2026

Emergent Brings Its AI-Enhanced Content Production Platfo...

Emergent, a developer of browser-based, AI-enhanced content production tools, will present its complete AI-enhanced content production platform at IBC2026, exhi...

28/08/2026

Joseph Bell Announces VFX & Animation World Atlas 2026

Joseph Bell Announces VFX & Animation World Atlas 2026 Joseph Bell August 28, 2026 1 Comment Joseph Bell's Visual Effects & Animation World Atlas ...

28/08/2026

Paris-Troyes en Champagne Goes Live on France.tv with AVEO

Paris-Troyes en Champagne Goes Live on France.tv with AVEO Brie Clayton August 28, 2026 0 Comments Regional audiovisual company builds broadcastgrade ...

28/08/2026

DHD to Introduce its Third-Generation Audio Mixing Console at IBC 2026

DHD to Introduce its Third-Generation Audio Mixing Console at IBC 2026 Brie Clayton August 28, 2026 0 Comments DHD.audio has chosen IBC 2026 as the la...

28/08/2026

Grass Valley AMPP Provides an All-Encompassing Approach t...

Encompass Digital Media and Grass Valley are expanding their collaboration with the launch of Altitude Playout Plus, a software-defined channel origination plat...

28/08/2026

Yospace releases new Ad Presentation solution

Yospace, the trusted leader in Dynamic Ad Insertion (DAI), has announced the release of its Ad Presentation solution, which enables broadcasters to apply non-li...

28/08/2026

TSL Showcases Reduced Complexity for Control and Monitori...

TSL Showcases Reduced Complexity for Control & Monitoring Across IP Workflows at IBC2026 New TSL Hummingbird capabilities including Dante control routing, toge...

28/08/2026

Lutz Rathmann to Become CEO of Riedel Communications

Share Copy link Facebook X Linkedin Bluesky Email...

28/08/2026

Spectrum Offers Lower-Income Internet Customers Free Amazon Prime

Share Copy link Facebook X Linkedin Bluesky Email...

28/08/2026

FCC Exempts Public Access Channels from Closed Captioning

Share Copy link Facebook X Linkedin Bluesky Email...

28/08/2026

Elon Musk's X Launches NFL Gametime on X

Share Copy link Facebook X Linkedin Bluesky Email...

28/08/2026

GlobalM and Lamhas Partner to Expand Managed IP Broadcast...

GlobalM and Lamhas Satellite Services today announced a new agreement that will see GlobalMs GMX technology deployed as part of Lamhas expanding managed IP broa...

28/08/2026

Boland Communications Introduces SMPTE ST 2110 Option for...

Integrated 25GbE Connectivity Delivers UHD Video Directly to Up to Two Monitors, Eliminates External Gateways, and Supports SMPTE ST 2022-7 Network-Path Protect...

28/08/2026

JT-DMF Invites IBC2026 Attendees to Help Shape the Future...

Interactive Ask Me Anything sessions will connect broadcasters and technology providers with the experts developing a practical, interoperable roadmap for sof...

28/08/2026

TAG Video Systems Cuts Live Production Latency Below Two...

At IBC2026, TAG will show sub-2-frame latency and 800 Gbps of aggregate scale for uncompressed ST 2110 monitoring in software, live MXL interoperability with pa...

28/08/2026

SportsBox BHVs Ultra-Compact 4K Remote Switcher Moves fro...

SportsBox, BHV's Ultra-Compact 4K Remote Switcher, Moves from Development to Production at IBC 2026 SportsBox brings flexibility without compromise to high...

28/08/2026

Riedel Evolves Leadership and Global Go-to-Market Organiz...

Lutz Rathmann named CEO of Riedel Communications Anthony Zuyderhoff to lead unified customer-facing organization as CCO Riedel Communications is strengthenin...

28/08/2026

Scripps Research establishes endowed fund honoring institute president and CEO Pete Schultz

LA JOLLA, CA-Scripps Research has established the Pete Schultz Fund for Innovati...

28/08/2026

Chemists develop simpler method to make molecular building block for drug development

LA JOLLA, CA-Chemical structures called cyclopropanols are among the most useful...

28/08/2026

Sky Secures Approval for Major Livingston Campus Investment, Reinforcing Long-Term Commitment to Scotland

Friday 28 August 2026 Sky Secures Approval for Major Livingston Campus Investme...

27/08/2026

ESPN Adds College Football Saturdays to Disney+ This Fall

ESPN and Disney are launching College Football Saturdays, adding two college football games per week to Disney for U.S. subscribers as part of their existing ...

27/08/2026

IBC 2026: Zixi to Showcase Live Video Platform and Broadcaster v19

Zixi will exhibit at IBC2026 (Hall 5, Stand 5.A76, RAI Amsterdam, September 11-14), demonstrating the Zixi Platform and the latest release of Zixi Broadcaster, ...