Sony Pixel Power calrec Sony

Decoding How NVIDIA AI Workbench Powers App Development

19/06/2024

Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible and showcases new hardware, software, tools and accelerations for NVIDIA RTX PC and workstation users.

The demand for tools to simplify and optimize generative AI development is skyrocketing. Applications based on retrieval-augmented generation (RAG) - a technique for enhancing the accuracy and reliability of generative AI models with facts fetched from specified external sources - and customized models are enabling developers to tune AI models to their specific needs.

While such work may have required a complex setup in the past, new tools are making it easier than ever.

NVIDIA AI Workbench simplifies AI developer workflows by helping users build their own RAG projects, customize models and more. It's part of the RTX AI Toolkit - a suite of tools and software development kits for customizing, optimizing and deploying AI capabilities - launched at COMPUTEX earlier this month. AI Workbench removes the complexity of technical tasks that can derail experts and halt beginners.

What Is NVIDIA AI Workbench? Available for free, NVIDIA AI Workbench enables users to develop, experiment with, test and prototype AI applications across GPU systems of their choice - from laptops and workstations to data center and cloud. It offers a new approach for creating, using and sharing GPU-enabled development environments across people and systems.

A simple installation gets users up and running with AI Workbench on a local or remote machine in just minutes. Users can then start a new project or replicate one from the examples on GitHub. Everything works through GitHub or GitLab, so users can easily collaborate and distribute work. Learn more about getting started with AI Workbench.

How AI Workbench Helps Address AI Project Challenges Developing AI workloads can require manual, often complex processes, right from the start.

Setting up GPUs, updating drivers and managing versioning incompatibilities can be cumbersome. Reproducing projects across different systems can require replicating manual processes over and over. Inconsistencies when replicating projects, like issues with data fragmentation and version control, can hinder collaboration. Varied setup processes, moving credentials and secrets, and changes in the environment, data, models and file locations can all limit the portability of projects.

AI Workbench makes it easier for data scientists and developers to manage their work and collaborate across heterogeneous platforms. It integrates and automates various aspects of the development process, offering:

Ease of setup: AI Workbench streamlines the process of setting up a developer environment that's GPU-accelerated, even for users with limited technical knowledge.

Seamless collaboration: AI Workbench integrates with version-control and project-management tools like GitHub and GitLab, reducing friction when collaborating.

Consistency when scaling from local to cloud: AI Workbench ensures consistency across multiple environments, supporting scaling up or down from local workstations or PCs to data centers or the cloud.

RAG for Documents, Easier Than Ever NVIDIA offers sample development Workbench Projects to help users get started with AI Workbench. The hybrid RAG Workbench Project is one example: It runs a custom, text-based RAG web application with a user's documents on their local workstation, PC or remote system.

Every Workbench Project runs in a container - software that includes all the necessary components to run the AI application. The hybrid RAG sample pairs a Gradio chat interface frontend on the host machine with a containerized RAG server - the backend that services a user's request and routes queries to and from the vector database and the selected large language model.

This Workbench Project supports a wide variety of LLMs available on NVIDIA's GitHub page. Plus, the hybrid nature of the project lets users select where to run inference.

Workbench Projects let users version the development environment and code. Developers can run the embedding model on the host machine and run inference locally on a Hugging Face Text Generation Inference server, on target cloud resources using NVIDIA inference endpoints like the NVIDIA API catalog, or with self-hosting microservices such as NVIDIA NIM or third-party services.

The hybrid RAG Workbench Project also includes:

Performance metrics: Users can evaluate how RAG- and non-RAG-based user queries perform across each inference mode. Tracked metrics include Retrieval Time, Time to First Token (TTFT) and Token Velocity.

Retrieval transparency: A panel shows the exact snippets of text - retrieved from the most contextually relevant content in the vector database - that are being fed into the LLM and improving the response's relevance to a user's query.

Response customization: Responses can be tweaked with a variety of parameters, such as maximum tokens to generate, temperature and frequency penalty.

To get started with this project, simply install AI Workbench on a local system. The hybrid RAG Workbench Project can be brought from GitHub into the user's account and duplicated to the local system.

More resources are available in the AI Decoded user guide. In addition, community members provide helpful video tutorials, like the one from Joe Freeman below.

Customize, Optimize, Deploy Developers often seek to customize AI models for specific use cases. Fine-tuning, a technique that changes the model by training it with additional data, can be useful for style transfer or changing model behavior. AI Workbench helps with fine-tuning, as well.

The Llama-factory AI Workbench Project enables QLoRa, a fine-tuning method that minimizes memory requirements, for a variety of models, as well as
LINK: https://blogs.nvidia.com/blog/ai-decoded-workbench-hybrid-rag/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

17/07/2026

Foundry Releases SmartRoto for Nuke

Foundry Releases SmartRoto for Nuke Brie Clayton July 17, 2026 0 Comments Spline-based AI powered plugin accelerates time-consuming rotoscoping, helping...

17/07/2026

A Short Documentary About a Giant Pencil Edited with DaVinci Resolve Studio

A Short Documentary About a Giant Pencil Edited with DaVinci Resolve Studio Brie Clayton July 17, 2026 0 Comments SBIFF jury award winner finished fro...

17/07/2026

Wheatstone to Feature Virtual Mixing Platform at IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

17/07/2026

Trump Threatens Networks Over Speech Coverage

Share Copy link Facebook X Linkedin Bluesky Email...

17/07/2026

Lightware Sustainability Report Details Product Efficienc...

Lightware has published its second voluntary Sustainability Report, showing how energy efficiency, product longevity and responsible material use are increasing...

17/07/2026

Foundry Unlocks Professional-Grade AI for VFX with Gripta...

Creative software developer Foundry today announced Griptape Enterprise, a new tier of the Griptape AI workflow orchestration platform, extended to meet the str...

17/07/2026

Foundry Releases SmartRoto for Nuke

Creative software developer Foundry today announced the availability of SmartRoto, a new AI-powered plugin upgrade for Nuke, NukeX, Nuke Studio, and Nuke Indie ...

17/07/2026

The Final Release of Blackmagic Fairlight Live is Now Available!

The Final Release of Blackmagic Fairlight Live is Now Available! Brie Clayton July 16, 2026 0 Comments Fairlight Live is a new audio mixer designed fo...

17/07/2026

Leveraging All of Entertainment's IP

Leveraging All of Entertainment's IP Andy Marken July 16, 2026 0 Comments Nobody ever wins the games. Period. There are survivors. There\s no win...

17/07/2026

RT Radio 1 continues its summer of stories, music, sport and conversation

New and returning series, live festival broadcasts, powerful documentaries and major sporting moments headline RT Radio 1's summer schedule RT Radio 1 is...

17/07/2026

NVIDIA Vera Rubin Maximizes Intelligence per Dollar for Post-Training Workloads - a Key Metric for Agentic AI

Think of a professional athlete. What separates elite performers is what happens...

17/07/2026

Dith, Jacqui and Louise bring hurling fans together ahead of the All-Ireland Finals

Up For The Match - Live From Croke Park this Saturday at 8pm on RT One and RT ...

17/07/2026

DiscoverIreland.ie announced as official show sponsor of The Traitors Ireland series 2

RT Commercial has today announced DiscoverIreland.ie, as sponsor of The Traitor...

16/07/2026

Business Innovation Synergizer: Grants of up to 30,000 for media organisations

Media organisations with tested plans for new products, services or revenue models can now apply for grants of up to 30,000 through the Business Innovation Syn...

16/07/2026

SVG Students To Watch: Kaden Sherman, Albany State University

This recent graduate and Georgia native has turned an early fascination with live production into a growing passion for camera work In the live-sports-video in...

16/07/2026

TNDV Marks First Year of Aspiration 35 Cinematic Production Truck

TNDV is marking the first anniversary of Aspiration 35, a mobile production truck built around ARRI Alexa 35 Live camera systems. Over its first year, the truck...

16/07/2026

ATSC Releases First Major Revision to A/85 Audio Loudness Recommended Practice Since 2013

ATSC has completed a major revision to its A/85 Recommended Practice: Techniques...

16/07/2026

FloSports Named Exclusive Global Media Partner for 2026 CrossFit Games

FloSports has announced an exclusive global media partnership with CrossFit for the 2026 CrossFit Games, presented by Air National Guard, beginning July 21 in S...

16/07/2026

Sennheiser Releases Spectera Firmware 1.4 and Command Button

Sennheiser has released firmware version 1.4 for its Spectera wireless system and announced the Spectera Command Button (Cat. No. 701014). The update adds comma...

16/07/2026

ESPN and Southwestern Athletic Conference Announce Multi-Year Media Rights Agreement Through 2030-31

ESPN and the Southwestern Athletic Conference (SWAC) have announced a multi-year...

16/07/2026

Audio-Technica Releases 20 Series Control Application for Elgato Stream Deck

Audio-Technica has announced the 20 Series Control Application, a Stream Deck plug-in for compatible Audio-Technica USB microphones. The application is compatib...

16/07/2026

SSL Live L550 Plus Consoles Power Audio Production for The Voice France Season 15

The fifteenth season of The Voice - La plus belle voix on TF1 used four Solid St...

16/07/2026

Gray Media and Atlanta Hawks Announce Georgia-Wide Broadcast Partnership for 2026-27 NBA Season

Gray Media and the Atlanta Hawks have announced a broadcast partnership that wil...

16/07/2026

SVG Sit-Down: Quarterback Showrunner Joe Zucco on Cinematography, Building Trust, Carrying On the NFL Films Legacy

Season 3 of Netflix's docuseries, which dropped this week, follows Jayden Da...

16/07/2026

Texas Rangers Switch Direct-to-Consumer Streaming Partner to BZZR

The Texas Rangers and Rangers Sports Network presented by Progressive (RSN) have announced BZZR as the new direct-to-consumer distributor for Rangers game broad...

16/07/2026

FIFA World Cup 2026: BBC Sport on Moving Across the Pond to Bring the Atmosphere of the Final Matches to Fans

With England's FIFA World Cup 2026 team getting ready to fly home after losi...

16/07/2026

With 36 Unilateral Cameras, NBC Sports Shapes 154th Open Championship Coverage for U.S. Viewers

In a return to Royal Birkdale, the team enjoys partnering with ETP and Gravity M...

16/07/2026

Altitude Sports & Entertainment's Tim De Lay on New SkyCam Digital Feed, Upgraded Studio Space

Along with a new studio at Ball Arena, the Colorado-based RSN is working with XR...

16/07/2026

FIFA World Cup 2026: Inside ORF's Austria-Based Production to Bring All the Emotions of the Tournament to Fans

Austria was back at the World Cup for the first time in 28 years, and the team d...

16/07/2026

Inside Gravity Media's Effort To Power the 154th Open Championship Production

With 42 RF cameras and high-power radio mics deployed, the spectrum environment ...

16/07/2026

Baby Audio release SubCulture

A new benchmark in bass enhancement Described as a new benchmark in bass enhancement , Baby Audio's latest release has been designed to augment the lo...

16/07/2026

Other Worlds Evolved from Zero G

Popular library rebuilt in new Orbita Engine Other Worlds Evolved is the latest of Zero G's sample libraries to be treated to a ground-up rebuild using ...

16/07/2026

Native Instruments announce Super*Saw

Created in collaboration with A. G. Cook The newest addition to Native Instruments' software collection introduces a new synth created by A. G. Cook, a ...

16/07/2026

NITV and SBS bring Australians to the heart of Garma

NITV and SBS bring Australians to the heart of Garma 16 July, 2026 Media releases The Garma Festival returns from 31 July-3 August, with NITV and SBS deliv...

16/07/2026

Rohde & Schwarz supplies conformance test systems to Marquistech for India's first GCF RTO and PTCRB ATL lab

Rohde & Schwarz supplies conformance test systems to Marquistech for India's...

16/07/2026

The Gauge: Poland | June 2026

June brought stabilization to the television market. Poles spent an average of 3 hours and 36 minutes a day in front of their TV screens exactly the same as i...

16/07/2026

New studio monitoring leads Glensound's 60th year line-up...

Glensound returns to IBC with a major new addition to its intelligent loudspeaker range, the Greater Divine Supernal studio monitor, alongside recent developmen...

16/07/2026

Complete spectrum of innovation from British exhibitors a...

33 leading exhibitors are taking part in the GREAT Britain and Northern Ireland Pavilions and linked locations across IBC2026 (Amsterdam RAI, 11 14 September)...

16/07/2026

Hitomi Broadcast to showcase HDR colour verification and...

Hitomi Broadcast will introduce Spectra, a new HDR colour verification solution measured in picture, not using VPID, at IBC2026 (Hall 10, Stand A40). Expanding ...

16/07/2026

SMPTE Kicks Off 110th Anniversary Campaign

Share Copy link Facebook X Linkedin Bluesky Email...

16/07/2026

Cox Media Group Taps Waymark for AI Ad Creation Tools

Share Copy link Facebook X Linkedin Bluesky Email...

16/07/2026

Parks: Cable Broadband Sub Losses Slow Down

Share Copy link Facebook X Linkedin Bluesky Email...

16/07/2026

CatchLight, Sony Launch Initiative to Help Local Newsrooms

Share Copy link Facebook X Linkedin Bluesky Email...

16/07/2026

XR Launches XR ONE Ad Ops Platform

Share Copy link Facebook X Linkedin Bluesky Email...

16/07/2026

Zixi and Comcast Technology Solutions Partner to Provide...

Zixi, a leader in live video delivery and workflow orchestration, and Comcast Technology Solutions (CTS) today announced a strategic new integration to deliver ...

16/07/2026

Blackmagic Design Cameras Capture Immersive Flight for New Cirrus App

Blackmagic Design Cameras Capture Immersive Flight for New Cirrus App Brie Clayton July 15, 2026 0 Comments Leader in Personal Aviation taps URSA Cin...