Sony Pixel Power calrec Sony

Decoding How NVIDIA AI Workbench Powers App Development

19/06/2024

Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible and showcases new hardware, software, tools and accelerations for NVIDIA RTX PC and workstation users.

The demand for tools to simplify and optimize generative AI development is skyrocketing. Applications based on retrieval-augmented generation (RAG) - a technique for enhancing the accuracy and reliability of generative AI models with facts fetched from specified external sources - and customized models are enabling developers to tune AI models to their specific needs.

While such work may have required a complex setup in the past, new tools are making it easier than ever.

NVIDIA AI Workbench simplifies AI developer workflows by helping users build their own RAG projects, customize models and more. It's part of the RTX AI Toolkit - a suite of tools and software development kits for customizing, optimizing and deploying AI capabilities - launched at COMPUTEX earlier this month. AI Workbench removes the complexity of technical tasks that can derail experts and halt beginners.

What Is NVIDIA AI Workbench? Available for free, NVIDIA AI Workbench enables users to develop, experiment with, test and prototype AI applications across GPU systems of their choice - from laptops and workstations to data center and cloud. It offers a new approach for creating, using and sharing GPU-enabled development environments across people and systems.

A simple installation gets users up and running with AI Workbench on a local or remote machine in just minutes. Users can then start a new project or replicate one from the examples on GitHub. Everything works through GitHub or GitLab, so users can easily collaborate and distribute work. Learn more about getting started with AI Workbench.

How AI Workbench Helps Address AI Project Challenges Developing AI workloads can require manual, often complex processes, right from the start.

Setting up GPUs, updating drivers and managing versioning incompatibilities can be cumbersome. Reproducing projects across different systems can require replicating manual processes over and over. Inconsistencies when replicating projects, like issues with data fragmentation and version control, can hinder collaboration. Varied setup processes, moving credentials and secrets, and changes in the environment, data, models and file locations can all limit the portability of projects.

AI Workbench makes it easier for data scientists and developers to manage their work and collaborate across heterogeneous platforms. It integrates and automates various aspects of the development process, offering:

Ease of setup: AI Workbench streamlines the process of setting up a developer environment that's GPU-accelerated, even for users with limited technical knowledge.

Seamless collaboration: AI Workbench integrates with version-control and project-management tools like GitHub and GitLab, reducing friction when collaborating.

Consistency when scaling from local to cloud: AI Workbench ensures consistency across multiple environments, supporting scaling up or down from local workstations or PCs to data centers or the cloud.

RAG for Documents, Easier Than Ever NVIDIA offers sample development Workbench Projects to help users get started with AI Workbench. The hybrid RAG Workbench Project is one example: It runs a custom, text-based RAG web application with a user's documents on their local workstation, PC or remote system.

Every Workbench Project runs in a container - software that includes all the necessary components to run the AI application. The hybrid RAG sample pairs a Gradio chat interface frontend on the host machine with a containerized RAG server - the backend that services a user's request and routes queries to and from the vector database and the selected large language model.

This Workbench Project supports a wide variety of LLMs available on NVIDIA's GitHub page. Plus, the hybrid nature of the project lets users select where to run inference.

Workbench Projects let users version the development environment and code. Developers can run the embedding model on the host machine and run inference locally on a Hugging Face Text Generation Inference server, on target cloud resources using NVIDIA inference endpoints like the NVIDIA API catalog, or with self-hosting microservices such as NVIDIA NIM or third-party services.

The hybrid RAG Workbench Project also includes:

Performance metrics: Users can evaluate how RAG- and non-RAG-based user queries perform across each inference mode. Tracked metrics include Retrieval Time, Time to First Token (TTFT) and Token Velocity.

Retrieval transparency: A panel shows the exact snippets of text - retrieved from the most contextually relevant content in the vector database - that are being fed into the LLM and improving the response's relevance to a user's query.

Response customization: Responses can be tweaked with a variety of parameters, such as maximum tokens to generate, temperature and frequency penalty.

To get started with this project, simply install AI Workbench on a local system. The hybrid RAG Workbench Project can be brought from GitHub into the user's account and duplicated to the local system.

More resources are available in the AI Decoded user guide. In addition, community members provide helpful video tutorials, like the one from Joe Freeman below.

Customize, Optimize, Deploy Developers often seek to customize AI models for specific use cases. Fine-tuning, a technique that changes the model by training it with additional data, can be useful for style transfer or changing model behavior. AI Workbench helps with fine-tuning, as well.

The Llama-factory AI Workbench Project enables QLoRa, a fine-tuning method that minimizes memory requirements, for a variety of models, as well as
LINK: https://blogs.nvidia.com/blog/ai-decoded-workbench-hybrid-rag/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

11/04/2026

Federal Judge Extends Nexstar/Tegna TRO, Softens Some Provisions

Share Copy link Facebook X Linkedin Bluesky Email...

11/04/2026

Sling TV Launches $19.99 a Month Sling Essentials with ESPN

Share Copy link Facebook X Linkedin Bluesky Email...

11/04/2026

NAB Show Launches Content Creator VIP Program

Share Copy link Facebook X Linkedin Bluesky Email...

11/04/2026

FCC Announces Tentative Agenda for April Open Meeting

Share Copy link Facebook X Linkedin Bluesky Email...

11/04/2026

GARR and Cubbit launch the first geo-distributed storage...

Pilot phase begins for a new national infrastructure designed to safeguard academic and research data with full local data control, sovereignty, resilience, and...

11/04/2026

How to Stream Coachella 2026 at Home

How to Stream Coachella 2026 at Home Check this years stacked schedule for the annual music festivals full lineup, including when Berklee artists from Laufey ...

11/04/2026

Time Travel

Time Travel As Berklee on the Road programs in Puerto Rico and Italy mark decades-long anniversaries, we journey into the past and step into the future. Apri...

11/04/2026

April 10, 2026

Improving vaccine design for Ebola, HIV and more Scripps Research scientists and colleagues develop a nanodisc platform that offers a clearer view of how key vi...

10/04/2026

The Invisible OPEX Killer: Is Your Server Room Dragging You Down?

The Invisible OPEX Killer: Is Your Server Room Dragging You Down? In the broadcast world, we talk a lot about uptime. We talk about talent retention, latency...

10/04/2026

NAB 2026: Imagine Communications to Showcase Expanded Multiviewer Portfolio

Imagine Communications will showcase its multiviewer portfolio at NAB Show 2026 (April 19-22, Booth N1328, Las Vegas Convention Center), including Prismon and t...

10/04/2026

NAB 2026: Chyron Releases PRIME VSAR 2.3 with Updated Unreal Engine Integration

Chyron has released PRIME VSAR 2.3, an update to its virtual set and augmented reality solution for broadcast. The release adds compatibility with Unreal Engine...

10/04/2026

NAB 2026: Techex to Showcase New tx darwin Capabilities

Techex will exhibit at NAB Show 2026 (Booth W2267, April 19-23, Las Vegas Convention Center), demonstrating new tx darwin features including consumer multiview,...

10/04/2026

NAB 2026: NDI to Showcase Ecosystem and NDI 6.3

NDI will exhibit at NAB Show 2026, demonstrating its IP video ecosystem through live partner integrations, NDI 6.3 features, AI metadata workflows, and creator ...

10/04/2026

FOR-A Acquires Tamura Corporations Information Equipment Business

FOR-A has announced the acquisition of all shares of Tamu Radiance Corporation, a new company spun off from the Information Equipment Business of Tamura Corpora...

10/04/2026

NAB 2026: InSync Technology to Unveil New Video Processing and Frame Rate Conversion Products

InSync Technology will showcase new and updated video conversion products at NAB...

10/04/2026

TNT Sports and DAZN Announce Monthly Boxing Event Series in the United States

TNT Sports and DAZN have announced a partnership to air monthly boxing events in the United States under the brand The Fight. The series will be promoted in p...

10/04/2026

Panasonic Introduces SQ3 Series 4K LCD Displays for Professional Environments

Panasonic Projector and Display has announced the SQ3 Series of 4K LCD displays as part of its MEVIX professional display portfolio. All sizes will be available...

10/04/2026

Amagi Adds Agentic Capabilities to Its Media Operations Platform

Amagi has announced the addition of Agentic Media Operations to its Amagi NOW platform, integrating AI reasoning agents across its media supply chain workflows ...

10/04/2026

LTN Announces Network Enhancements Ahead of C-Band Spectrum Auction

LTN has announced enhancements to its global IP video network targeting broadcasters transitioning from satellite distribution. The updates come ahead of US fed...

10/04/2026

Daktronics Installs New LED Displays at Yankee Stadium

Daktronics has installed new LED displays at Yankee Stadium, upgrading the main centerfield board, two flanking boards, and two ribbon displays spanning the 200...

10/04/2026

NAB 2026: Harmonic Announces AI and Cloud Updates to Hybrid Streaming Solution

Harmonic has announced updates to its hybrid streaming solution, including Model Context Protocol (MCP) connectivity for AI applications, cloud-native deploymen...

10/04/2026

NAB 2026: MultiDyne to Debut FiberSaver-10G and VF-9100

MultiDyne Video and Fiber Optic Systems will introduce two new fiber transport products at NAB Show 2026 (Booth C4425, April 19-22): the FiberSaver-10G waveleng...

10/04/2026

NAB 2026: Telos Alliance and ip-studio to Demonstrate STUDIO ZERO

Telos Alliance and ip-studio will demonstrate STUDIO ZERO, a cloud-hosted virtual studio, at NAB Show 2026. First introduced at NAB Show 2023, STUDIO ZERO integ...

10/04/2026

Pixotope and d&b Solutions Announce Strategic Partnership for XR and Virtual Studio Production

d&b solutions, a London-based audio-visual, lighting, and media integration grou...

10/04/2026

ARRI and SmallHD Announce Lens Data Monitor Overlay License for Hi-5 and Hi-5 SX

ARRI and SmallHD have announced a new expansion license for ARRI's Hi-5 and Hi-5 SX hand units that displays lens data overlays on supported SmallHD monitor...

10/04/2026

Roku to Stream Exclusive Savannah Bananas Game Package on Roku Sports Channel

Roku and the Banana Ball Championship League (BBCL) have announced an exclusive streaming partnership to bring five BBCL games to the Roku Sports Channel in 202...

10/04/2026

Ratings Roundup: More Than 18 Million Fans Tune Into 2026 NCAA Mens March Madness on TNT and CBS Sports

Ratings Roundup is a rundown of recent rating news and is derived from press rel...

10/04/2026

No Other Land, Mr. Nobody Against Putin,and More Sundance Institute-Supported Films Nominated for Peabody Awards

The Peabody Awards don't just recognize great storytelling, they spotlight t...

10/04/2026

Fans Crown Winners at the First Spotify Podcast Awards in France

After launching the Spotify Podcast Awards in Mexico last year, we brought the fan-voted celebration to Paris this week for its first edition in France. Hosted ...

10/04/2026

Yamaha launch the DXR/DXS & CXR/CXS Mk3

Powered and unpowered live PA ranges upgraded Yamaha have just refreshed four of their hugely popular PA speaker ranges, delivering significant improvements...

10/04/2026

UJAM open Gorilla Engine to third-party developers

Underlying plug-in & VI technology now available to others UJAM's latest announcement sees the company open up' Gorilla Engine, the development pla...

10/04/2026

2026 NAB Show Exhibitor Insight: Bitcentral

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Bitcentral To Feature Connected Media Workflows At 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Bitcentral to Showcase Connected Media Workflows and Inte...

Bitcentral, a leading provider of professional media solutions for broadcast and digital video, will showcase its latest innovations at NAB Show 2026 (Booth W28...

10/04/2026

Ikegami to Introduce Expanded Range of Broadcast Production Solutions at NAB 2026

Ikegami to Introduce Expanded Range of Broadcast Production Solutions at NAB 202...

10/04/2026

AJA Debuts SMPTE ST 2110 and openGear Solutions Ahead of NAB 2026

AJA Debuts SMPTE ST 2110 and openGear Solutions Ahead of NAB 2026 Brie Clayton April 10, 2026 0 Comments New gear and updates address evolving hybrid ...

10/04/2026

Portland Fire+ Streaming Platform Launches

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Tod Musgrave Joins Proton as U.S. Sales & Marketing Director

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Proton Expands Minicam Portfolio With Proton Pro At 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

FCC To Vote on Changes to Audible Crawl Rule

Share Copy link Facebook X Linkedin Bluesky Email...

10/04/2026

Frequency Launches AI Platform for Streaming Television a...

Frequency, the engine behind the worlds leading streaming television channels, today launched its AI platform for Frequency Studio, powering the entire channel ...

10/04/2026

UKTV Highlights: Saturday April 25th - Friday May 1st 2026

What can I watch on UKTV and stream on U this week? This week on UKTV and the free streaming service U, viewers can watch a range of new and returning programm...

10/04/2026

Jnger Audio Joins EBU ADM Integration Group as Founding Member to Help Advance ADM/S-ADM Integration

J nger Audio Joins EBU ADM Integration Group as Founding Member to Help Advance...

10/04/2026

Multi-BAFTA-winning Chernobyl makes its free-to-air debut, marking 40 years since the disaster

Five-part Sky Original drama airs nightly on Sky Mix and Sky Atlantic from 20 Ap...