Sony Pixel Power calrec Sony

Decoding How NVIDIA AI Workbench Powers App Development

19/06/2024

Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible and showcases new hardware, software, tools and accelerations for NVIDIA RTX PC and workstation users.

The demand for tools to simplify and optimize generative AI development is skyrocketing. Applications based on retrieval-augmented generation (RAG) - a technique for enhancing the accuracy and reliability of generative AI models with facts fetched from specified external sources - and customized models are enabling developers to tune AI models to their specific needs.

While such work may have required a complex setup in the past, new tools are making it easier than ever.

NVIDIA AI Workbench simplifies AI developer workflows by helping users build their own RAG projects, customize models and more. It's part of the RTX AI Toolkit - a suite of tools and software development kits for customizing, optimizing and deploying AI capabilities - launched at COMPUTEX earlier this month. AI Workbench removes the complexity of technical tasks that can derail experts and halt beginners.

What Is NVIDIA AI Workbench? Available for free, NVIDIA AI Workbench enables users to develop, experiment with, test and prototype AI applications across GPU systems of their choice - from laptops and workstations to data center and cloud. It offers a new approach for creating, using and sharing GPU-enabled development environments across people and systems.

A simple installation gets users up and running with AI Workbench on a local or remote machine in just minutes. Users can then start a new project or replicate one from the examples on GitHub. Everything works through GitHub or GitLab, so users can easily collaborate and distribute work. Learn more about getting started with AI Workbench.

How AI Workbench Helps Address AI Project Challenges Developing AI workloads can require manual, often complex processes, right from the start.

Setting up GPUs, updating drivers and managing versioning incompatibilities can be cumbersome. Reproducing projects across different systems can require replicating manual processes over and over. Inconsistencies when replicating projects, like issues with data fragmentation and version control, can hinder collaboration. Varied setup processes, moving credentials and secrets, and changes in the environment, data, models and file locations can all limit the portability of projects.

AI Workbench makes it easier for data scientists and developers to manage their work and collaborate across heterogeneous platforms. It integrates and automates various aspects of the development process, offering:

Ease of setup: AI Workbench streamlines the process of setting up a developer environment that's GPU-accelerated, even for users with limited technical knowledge.

Seamless collaboration: AI Workbench integrates with version-control and project-management tools like GitHub and GitLab, reducing friction when collaborating.

Consistency when scaling from local to cloud: AI Workbench ensures consistency across multiple environments, supporting scaling up or down from local workstations or PCs to data centers or the cloud.

RAG for Documents, Easier Than Ever NVIDIA offers sample development Workbench Projects to help users get started with AI Workbench. The hybrid RAG Workbench Project is one example: It runs a custom, text-based RAG web application with a user's documents on their local workstation, PC or remote system.

Every Workbench Project runs in a container - software that includes all the necessary components to run the AI application. The hybrid RAG sample pairs a Gradio chat interface frontend on the host machine with a containerized RAG server - the backend that services a user's request and routes queries to and from the vector database and the selected large language model.

This Workbench Project supports a wide variety of LLMs available on NVIDIA's GitHub page. Plus, the hybrid nature of the project lets users select where to run inference.

Workbench Projects let users version the development environment and code. Developers can run the embedding model on the host machine and run inference locally on a Hugging Face Text Generation Inference server, on target cloud resources using NVIDIA inference endpoints like the NVIDIA API catalog, or with self-hosting microservices such as NVIDIA NIM or third-party services.

The hybrid RAG Workbench Project also includes:

Performance metrics: Users can evaluate how RAG- and non-RAG-based user queries perform across each inference mode. Tracked metrics include Retrieval Time, Time to First Token (TTFT) and Token Velocity.

Retrieval transparency: A panel shows the exact snippets of text - retrieved from the most contextually relevant content in the vector database - that are being fed into the LLM and improving the response's relevance to a user's query.

Response customization: Responses can be tweaked with a variety of parameters, such as maximum tokens to generate, temperature and frequency penalty.

To get started with this project, simply install AI Workbench on a local system. The hybrid RAG Workbench Project can be brought from GitHub into the user's account and duplicated to the local system.

More resources are available in the AI Decoded user guide. In addition, community members provide helpful video tutorials, like the one from Joe Freeman below.

Customize, Optimize, Deploy Developers often seek to customize AI models for specific use cases. Fine-tuning, a technique that changes the model by training it with additional data, can be useful for style transfer or changing model behavior. AI Workbench helps with fine-tuning, as well.

The Llama-factory AI Workbench Project enables QLoRa, a fine-tuning method that minimizes memory requirements, for a variety of models, as well as
LINK: https://blogs.nvidia.com/blog/ai-decoded-workbench-hybrid-rag/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

06/09/2026

Dolby and MagentaTV Bring Fans Closer to the FIFA World Cup 2026 in Germany with Dolby Vision and Dolby Atmos

June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

29/07/2026

Central Christian Church Connects Remote Production Teams with Clear-Com's Gen-IC...

eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({...

29/07/2026

Streaming Embarks on Annual Summer Ascent in Nielsen's May 2026 Gauge Reports

Streaming Hits 48.6% of Total TV Usage in May YouTube Leads in Media Distributo...

29/07/2026

WVIT's Joseph DiMaggio Named SBE Flanders Engineer of the Year

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

Amagi Appoints Martin Wacker to Lead Sales for DACH -- St...

Industry veteran joins Amagi to drive growth in Europe's leading broadcast market and advance cloud-native broadcast, FAST, and AI-driven operations. LONDO...

29/07/2026

WebTransport opens new route for broadcast-quality media...

Work is ongoing to specify how to run RIST over WebTransport. This will deliver secure, stable, broadcast quality video playback and origination to any platform...

29/07/2026

Offline Files Makes the Drive Shelf Searchable on Mac

Offline Files Makes the Drive Shelf Searchable on Mac Brie Clayton July 29, 2026 0 Comments A local Mac app for finding files, photos, and footage acr...

29/07/2026

Blackmagic Design Announces New UltraStudio Mini 12G

Blackmagic Design Announces New UltraStudio Mini 12G Brie Clayton July 29, 2026 0 Comments New extremely portable Thunderbolt 4 capture and playback ...

29/07/2026

Brightline Lighting Introduces the MOD1

Brightline Lighting, a premier manufacturer of energy-efficient lighting systems for broadcast, videoconferencing, government, education and corporate applicati...

29/07/2026

Applications open for IFFI/WAVES Film Bazaar in Goa - Producer Delegation

Applications open for IFFI/WAVES Film Bazaar in Goa - Producer Delegation 28 July 2026 Image credit: Four Years Later Screen Australia has opened applications...

29/07/2026

Brightline Unveils Compact, Suspended MOD1 LED Light

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

NAB Meets With Trusty on Ownership Caps

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

QuickLink To Feature New StudioPro AnywhereTally At IBC 2026

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

HBO Max Launches Vertical Shorts to Improve Content Discovery

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

New Tool Identifies Sources Of Fake Videos

Share Copy link Facebook X Linkedin Bluesky Email...

29/07/2026

DNEG Migrates Editorial Workflows to Foundrys Hiero

Foundry today announced that Academy Award-winning creative studio DNEG has implemented Hiero to support its editorial workflows. Part of the Nuke product famil...

29/07/2026

QuickLink Unveils New StudioPro AnywhereTally Remote Tall...

The Latest Addition to the Brand's StudioPro Ecosystem Combines Simple Deployment, Cloud Control and Professional Grade Wireless Reliability QuickLink, a l...

29/07/2026

The Secret Master Adjustment Layer Hack in After Effects

The Secret Master Adjustment Layer Hack in After Effects Graham Quince July 28, 2026 0 Comments Trish Meyer (who literally beta-tested the original Af...

29/07/2026

Post Completed with Blackmagic Cloud for Shane Kosugi's Feature Film SEEK

Post Completed with Blackmagic Cloud for Shane Kosugi's Feature Film SEEK Brie Clayton July 28, 2026 0 Comments Cannes Indie International Film Fe...

29/07/2026

Isabeau Miller Named Chair of Berklee Songwriting Department

Isabeau Miller Named Chair of Berklee Songwriting Department The Berklee alumna, who previously served on the Songwriting faculty, will begin her new role on ...

29/07/2026

New RT true crime podcast launches today

RT launches Unsolved, a true crime podcast, presented by crime journalist Barry Cummins and presenter Katja Mia. Unsolved is a weekly podcast exploring unsol...

28/07/2026

LiveUs World Cup Record: 1,700 Units Support Wide Range of Transmission Needs

LiveU, the pioneer in IP-video transmission and cloud-based remote production, today detailed its historic performance records set during the 2026 FIFA World Cu...

28/07/2026

Case Study: How the Dallas Cowboys Removed Bottlenecks in Off-site Content Production

The NFL franchise rebuilt their off-site content operation for an era when teams...

28/07/2026

A New Way To Watch': How Formula 1 and Sky Sports Are Pushing Race Coverage Into a New Era

Longtime partners expand viewer choice, data-driven analysis, access, immersive ...

28/07/2026

Grass Valley Op Ed: Seven Paradoxes Shaping the Next Era of Media Production

There is a unique moment before every major production. The audience never sees it. The cameras are ready. The talent is preparing. Directors review the final...

28/07/2026

The CW Network, ESPN Ink Streaming Agreement With Launch on Aug. 4

The CW Network and ESPN have announced their highly anticipated streaming agreement to bring CW Sports' live events to the ESPN App will officially launch o...

28/07/2026

Lawo Delivers Audio Excellence for Europes Largest Classical Open-Air Concert on Bastille Day

As more than tens of thousands of spectators filled Paris' Champ de Mars and...

28/07/2026

Appear Assists Chile's First Large-Scale 4K FIFA World Cup Viewing Experience

Appear's X Platform played a central role in Chile's first large-scale c...

28/07/2026

IBC 2026: Solid State Logic Highlights Transition to Software-Defined Production

Responding to the broadcast industry's shift to embrace more flexible production platforms, Solid State Logic (SSL) is evolving its comprehensive System T p...

28/07/2026

TNT Sports Goes to Greater Heights With New DUNKMAN League

Production leverages innovative camera technologies, AI, and data to capture the action in four-week inaugural competition The first-ever professional dunking ...

28/07/2026

Rogers, Prime Video Ink 12-Year Sublicensing Deal for NHL Rights in Canada

Prime Video to exclusively broadcast Wednesday night national NHL regular-season games in Canada...

28/07/2026

CBS Sports Snags U.S. English-Language Rights for CONCACAF Champions Cup

Every match will stream live on Paramount+ beginning in February,...

28/07/2026

TK Audio unveil the TK-Lizer 3

Baxandall-inspired hardware EQ overhauled TK Audio's Baxandall-inspired hardware EQ unit has just made a return, bringing the popular unit back into the...

28/07/2026

Asparion reveal the D700 Rack

D700 system gains new live-focused rackmount surface Asparion have just announced an expansion of their D700 control surface range, introducing a new model ...

28/07/2026

Six NFVF Presidential Employment Stimulus Programme (PESP)-Funded Films Selected for the 47th Durban International Film Festival (DIFF)

27 July 2026, Johannesburg - The National Film and Video Foundation (NFVF), an a...

28/07/2026

Nielsen Launches Ad Intel AI: the Only Product of its Kind that Transforms Fragmented Ad Data into Actionable Intelligence

Ad Intel AI is Nielsen's first in a new generation of AI products built to t...

28/07/2026

aconnic is ramping up supply for the new ACCEED 1444 fiber migration

Contribution to Europe's technological autonomy in the key industry telecommunications aconnic AG (ISIN: DE000A0LBKW6), Munich, starts commercial shipments...

28/07/2026

The Tow Foundation Awards PMVP $400,000 Grant

Share Copy link Facebook X Linkedin Bluesky Email...

28/07/2026

LiveU's World Cup Support Breaks Company Records

Share Copy link Facebook X Linkedin Bluesky Email...

28/07/2026

Encompass Selected by Channel 4 to Deliver Managed Linear...

Encompass Digital Media has been selected by Channel 4, one of the UK's public service broadcasters, to deliver managed linear playout services as part of a...

28/07/2026

Cinegy partners with SAMA Digital Media Solutions to sati...

Cinegy GmbH, the premier provider of software-defined television technology, has confirmed a new partnership with leading Korean systems integrator and distribu...

28/07/2026

Limecraft Adds Multicam Player and Audio Fingerprint-Base...

Limecraft today announces version 2026.5 of its AI-powered production platform. Limecraft 2026.5 https://(www.limecraft.com/limecraft-2026-5-multicam-quad-view-...

28/07/2026

Appear powers first large-scale 4k FIFA World Cup viewing...

X Platform technology enabled a premium UHD viewing experience through a hybrid ground-to-cloud architecture developed with Chilevisi n, ClaroVTR and Pixop App...

28/07/2026

LiveU Delivers Record-Breaking LIQ and Cloud Service Depl...

LiveU, the pioneer in IP-video transmission and cloud-based remote production, today announced it has set historic performance records during the 2026 World Foo...

28/07/2026

DHD Releases New Firmware and Options for its Audio Produ...

dhd.audio announces that its latest generation firmware, Version 10.4, is now available to DHD customers. Version 10.4 introduces expanded configuration capabil...

28/07/2026

Study: TV Operating Systems Gain Influence Over Viewing

Share Copy link Facebook X Linkedin Bluesky Email...

28/07/2026

Maryland Public Television Promotes Ling Ling Sun to CTO

Share Copy link Facebook X Linkedin Bluesky Email...