Sony Pixel Power calrec Sony

Decoding How NVIDIA AI Workbench Powers App Development

19/06/2024

Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible and showcases new hardware, software, tools and accelerations for NVIDIA RTX PC and workstation users.

The demand for tools to simplify and optimize generative AI development is skyrocketing. Applications based on retrieval-augmented generation (RAG) - a technique for enhancing the accuracy and reliability of generative AI models with facts fetched from specified external sources - and customized models are enabling developers to tune AI models to their specific needs.

While such work may have required a complex setup in the past, new tools are making it easier than ever.

NVIDIA AI Workbench simplifies AI developer workflows by helping users build their own RAG projects, customize models and more. It's part of the RTX AI Toolkit - a suite of tools and software development kits for customizing, optimizing and deploying AI capabilities - launched at COMPUTEX earlier this month. AI Workbench removes the complexity of technical tasks that can derail experts and halt beginners.

What Is NVIDIA AI Workbench? Available for free, NVIDIA AI Workbench enables users to develop, experiment with, test and prototype AI applications across GPU systems of their choice - from laptops and workstations to data center and cloud. It offers a new approach for creating, using and sharing GPU-enabled development environments across people and systems.

A simple installation gets users up and running with AI Workbench on a local or remote machine in just minutes. Users can then start a new project or replicate one from the examples on GitHub. Everything works through GitHub or GitLab, so users can easily collaborate and distribute work. Learn more about getting started with AI Workbench.

How AI Workbench Helps Address AI Project Challenges Developing AI workloads can require manual, often complex processes, right from the start.

Setting up GPUs, updating drivers and managing versioning incompatibilities can be cumbersome. Reproducing projects across different systems can require replicating manual processes over and over. Inconsistencies when replicating projects, like issues with data fragmentation and version control, can hinder collaboration. Varied setup processes, moving credentials and secrets, and changes in the environment, data, models and file locations can all limit the portability of projects.

AI Workbench makes it easier for data scientists and developers to manage their work and collaborate across heterogeneous platforms. It integrates and automates various aspects of the development process, offering:

Ease of setup: AI Workbench streamlines the process of setting up a developer environment that's GPU-accelerated, even for users with limited technical knowledge.

Seamless collaboration: AI Workbench integrates with version-control and project-management tools like GitHub and GitLab, reducing friction when collaborating.

Consistency when scaling from local to cloud: AI Workbench ensures consistency across multiple environments, supporting scaling up or down from local workstations or PCs to data centers or the cloud.

RAG for Documents, Easier Than Ever NVIDIA offers sample development Workbench Projects to help users get started with AI Workbench. The hybrid RAG Workbench Project is one example: It runs a custom, text-based RAG web application with a user's documents on their local workstation, PC or remote system.

Every Workbench Project runs in a container - software that includes all the necessary components to run the AI application. The hybrid RAG sample pairs a Gradio chat interface frontend on the host machine with a containerized RAG server - the backend that services a user's request and routes queries to and from the vector database and the selected large language model.

This Workbench Project supports a wide variety of LLMs available on NVIDIA's GitHub page. Plus, the hybrid nature of the project lets users select where to run inference.

Workbench Projects let users version the development environment and code. Developers can run the embedding model on the host machine and run inference locally on a Hugging Face Text Generation Inference server, on target cloud resources using NVIDIA inference endpoints like the NVIDIA API catalog, or with self-hosting microservices such as NVIDIA NIM or third-party services.

The hybrid RAG Workbench Project also includes:

Performance metrics: Users can evaluate how RAG- and non-RAG-based user queries perform across each inference mode. Tracked metrics include Retrieval Time, Time to First Token (TTFT) and Token Velocity.

Retrieval transparency: A panel shows the exact snippets of text - retrieved from the most contextually relevant content in the vector database - that are being fed into the LLM and improving the response's relevance to a user's query.

Response customization: Responses can be tweaked with a variety of parameters, such as maximum tokens to generate, temperature and frequency penalty.

To get started with this project, simply install AI Workbench on a local system. The hybrid RAG Workbench Project can be brought from GitHub into the user's account and duplicated to the local system.

More resources are available in the AI Decoded user guide. In addition, community members provide helpful video tutorials, like the one from Joe Freeman below.

Customize, Optimize, Deploy Developers often seek to customize AI models for specific use cases. Fine-tuning, a technique that changes the model by training it with additional data, can be useful for style transfer or changing model behavior. AI Workbench helps with fine-tuning, as well.

The Llama-factory AI Workbench Project enables QLoRa, a fine-tuning method that minimizes memory requirements, for a variety of models, as well as
LINK: https://blogs.nvidia.com/blog/ai-decoded-workbench-hybrid-rag/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

05/08/2026

Thomson toolkit added to virtual hub built to strengthen climate storytelling

Thomson's Climate Crisis Toolkit for Media in Tanzania is being included in a new virtual library designed to ensure accurate and engaging storytelling arou...

05/08/2026

SVG Regional Sports Production Summit 2026: All Sessions Now Available to Watch on SVG PLAY

RSNs, teams, leagues, and streamers explore the present and future of local spor...

05/08/2026

SVG New Sponsor Spotlight: Hitachi Vantara's Lenny Khaitov on Building Resilient Data Infrastructure for Sports Production

As sports-production workflows generate larger volumes of unstructured data and ...

05/08/2026

NESN's Women's Celebration Night Returns With Expanded Focus on the Next Generation

Three aspiring broadcasters will contribute on-camera and inside the production ...

05/08/2026

Union County Paints an Authentic Picture of Recovery and Community

Adam Meeks attends the premiere of his film Union County, an official selection of the 2026 Sundance Film Festival. (Photo by Jemal Countess/Sundance Institut...

05/08/2026

Steinberg celebrate VST & HALion anniversaries

Major discounts, updates & remakes of early VST plug-ins Steinberg are currently celebrating two significant anniversaries, with this week marking the 30th ...

05/08/2026

KRK introduce the V Series Five

Flagship monitor range gains wireless remote control KRK's flagship studio monitor range has just been upgraded, with three new fifth-generation models ...

05/08/2026

Rohde & Schwarz launches next phase of Airways New Zealand Air Traffic Control system upgrade

Rohde & Schwarz launches next phase of Airways New Zealand Air Traffic Control s...

05/08/2026

Rohde & Schwarz presents the R&SMP007, a man-portable tactical direction finding system for seamless signal detection from 9 MHz to 8 GHz

Rohde & Schwarz presents the R&S MP007, a man-portable tactical direction findin...

05/08/2026

Australia's retirement sector advertising rises more than 10% as millions plan for life after work according to Nielsen

Advertisers invested $53.6 million in the retirement sector in the last 12 month...

05/08/2026

New Nielsen data shows IKEA's New Zealand launch attracted more than 243,000 shoppers in the last month

Nielsen CMI and Ad Intel data reveals who's visiting IKEA, where else they s...

05/08/2026

Nielsen expands Four-Screen Ad Deduplication across CTV in Japan

Media buyers and sellers can now compare deduplicated campaign reach across computer, mobile, connected TV and linear TV across all measurable CTV publishers in...

05/08/2026

Stranger Things,' Bluey,' The Pitt' Top Nielsen's Streaming Charts Through First Half of 2026

Stranger Things' Leads All Streaming Titles with 23.3 Billion Minutes in Fir...

05/08/2026

Live Media Group, Gravity Media Strike Deal on U.S. OB Fleets

Share Copy link Facebook X Linkedin Bluesky Email...

05/08/2026

Beyond VVC: Call for Proposals for Next-Gen Video Coding

Share Copy link Facebook X Linkedin Bluesky Email...

05/08/2026

QuickLink Expands North American Team With Appointment of...

QuickLink, the leading provider of award-winning video production and remote guest integration solutions, announces the appointment of Kane Peterson to its Nort...

05/08/2026

Seven in 10 UK advertisers exceed production budgets desp...

XR research reveals marketers believe they have control over production costs, yet budget overruns, campaign delays and creative cutbacks remain commonplace Se...

05/08/2026

Game On Live Studio Uses Blackmagic Design to Stream Tropical 7s

Game On Live Studio Uses Blackmagic Design to Stream Tropical 7s Brie Clayton August 4, 2026 0 Comments Every match of world's largest youth rugby...

05/08/2026

Chris Fitzgerald to Head TDC's Experiential Growth

Chris Fitzgerald to Head TDC's Experiential Growth Brie Clayton August 4, 2026 0 Comments Technical Direction Company promotes Chris Fitzgerald to...

05/08/2026

Gravity Media, Live Media Group Form Strategic Partnership

Share Copy link Facebook X Linkedin Bluesky Email...

05/08/2026

Samsung Launches World's First HDR10+ Streaming on Prime Video

Share Copy link Facebook X Linkedin Bluesky Email...

05/08/2026

Profuz Digital unveils new advanced search capabilities a...

Profuz Digital is set to launch a powerful upgrade to its Profuz LAPIS digital media management platform at IBC this year at Stand 8.A88. France Galop, Canal+ F...

05/08/2026

KRK Brings Advanced Wireless Control to Professional Moni...

KRK, a Gibson brand and leader in professional studio monitoring for nearly four decades, launches its new V Series Five Studio Monitors, the latest evolution o...

05/08/2026

Stable performance in line with plan - Magyar Telekoms H1 and Q2 2026 results

Network expansion, continued customer growth and stable financial performance characterized Magyar Telekoms business results for the first half and second quart...

05/08/2026

The Sky Foundation launches new bursary programme to support the future of UK dance

Wednesday 5 August 2026 The Sky Foundation launches new bursary programme to su...

05/08/2026

Charlie Sheen Launches Mostly Sheenius in Partnership with Red Seat Ventures

Charlie Sheen Launches Mostly Sheenius in Partnership with Red Seat Ventures After Decades as One of the World's Most Recognizable and Interviewed Celebr...

04/08/2026

MLB's Jason Hedgcock on Using the Cloud To Produce World Baseball Classic Games for Netflix Japan

Live production in the cloud is a growing trend across the industry, and Major L...

04/08/2026

Major League Soccer Names Larry Berg as Next Commissioner; Will Take Helm in 2027

LAFC co-managing owner will succeed Don Garber on Jan. 1, 2027, as league prepar...

04/08/2026

Foresight: Driving Growth Through Broadcast Flexibility - A Calrec + PGA TOUR Case Study

In 2022, when the PGA TOUR was breaking ground on its new, state-of-the-art broa...

04/08/2026

Dealer Cam Debuts as Omaha Productions Takes World Series of Poker Main Event Final Table Live on ESPN

Provided by POVORA, the new camera angle offers a dealer's-eye view of the c...

04/08/2026

USL To Trial FIFA's Football Video Support System During 2026 USL Cup

Spiideo will provide the camera setup, with NEP handling broadcast integration and off-site video operations...

04/08/2026

Game On Live Studio Deploys Blackmagic Design Workflow Across 15 Fields at Tropical 7s

Production team streamed more than 750 youth rugby matches using ATEM switchers,...

04/08/2026

NDI, KTC Partner To Develop Network-Native Commercial Displays

Agreement gives KTC early access to NDI's Advanced SDK for integration across commercial displays, digital signage, and interactive panels...

04/08/2026

IBC2026: IBC Innovation Awards Finalists Highlight Advances in AI, Cloud, Open Standards, and Accessibility

Twelve projects across four categories recognize collaborative work in productio...

04/08/2026

Ekstraklasa, Live Park Select Hawk-Eye for Tracking, Offside, and Broadcast Technology

Multi-year agreement will introduce SkeleTRACK, semi-automated offside, automate...

04/08/2026

Radio France Deploys Redundant Lawo AoIP Workflow for 2026 Concert de Paris

Three mc 56 consoles, dual A__UHD Core engines, RAVENNA networking, and HOME management supported live sound, monitoring, and international broadcast production...

04/08/2026

IBC2026: Haivision to Debut Makito ONE, Falkon X4 in Europe

New contribution products support ultra-low-latency video transport across fixed, remote, and mobile production environments...

04/08/2026

IBC2026: Chyron to Highlight AI, Cloud, HTML Graphics, and Sports-Analysis Tools

Company will showcase updates across PRIME, Chyron LIVE, Paint, virtual production, newsroom workflows, and venue control...

04/08/2026

TAB Show 2026: MultiDyne to Showcase Fiber-Transport and IP Infrastructure Tools

Exhibit will feature SilverBack, VersaBrix, VersaFrame, VF-9100, and FiberSaver-10G platforms...

04/08/2026

IBC2026: TAG Video Systems Adds MXL Support, Containerized Deployment, Expanded QC Tools

Platform updates target cloud-based production, high-density ST 2110 monitoring,...

04/08/2026

Gravity Media, Live Media Group Partner To Expand U.S. Mobile Production Facility Operations

Live Media Group will take control of Gravity Media's U.S. mobile-unit fleet...

04/08/2026

HeadRush update Prime, Core & Flex Core firmware

Amp-modelling rigs get major upgrade HeadRush have recently released a major firmware update that brings some powerful new features to their Prime, Core and...

04/08/2026

Am I Audio launch I Am Choir

Generate a choir from a single voice track Following years of development, Am I Audio have just officially launched their debut plug-in. Created in collabor...

04/08/2026

Universal Audio's EMT 140 & 250 plug-ins go native

Official EMT-endorsed reverbs now available to all Two more of Universal Audio's renowned UAD plug-ins have just gained new native versions that can be ...

04/08/2026

SBS helps Australians understand why every Census counts

SBS helps Australians understand why every Census counts Media releases As Australia prepares for the 2026 Census on Tuesday 11 August, SBS is helping ensur...

04/08/2026

Canada invests in the future of HF communications with Rohde & Schwarz solutions

Canada invests in the future of HF communications with Rohde & Schwarz solutions After selection by the Canadian Department of National Defence (DND) as the p...

04/08/2026

Blendkit Expands Beyond Blender

Blendkit Expands Beyond Blender Brie Clayton August 4, 2026 0 Comments Popular 3D Asset Library Launches Alpha Plugins for Maya, Godot, and Rhino Blend...