Sony Pixel Power calrec Sony

Decoding How NVIDIA AI Workbench Powers App Development

19/06/2024

Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible and showcases new hardware, software, tools and accelerations for NVIDIA RTX PC and workstation users.

The demand for tools to simplify and optimize generative AI development is skyrocketing. Applications based on retrieval-augmented generation (RAG) - a technique for enhancing the accuracy and reliability of generative AI models with facts fetched from specified external sources - and customized models are enabling developers to tune AI models to their specific needs.

While such work may have required a complex setup in the past, new tools are making it easier than ever.

NVIDIA AI Workbench simplifies AI developer workflows by helping users build their own RAG projects, customize models and more. It's part of the RTX AI Toolkit - a suite of tools and software development kits for customizing, optimizing and deploying AI capabilities - launched at COMPUTEX earlier this month. AI Workbench removes the complexity of technical tasks that can derail experts and halt beginners.

What Is NVIDIA AI Workbench? Available for free, NVIDIA AI Workbench enables users to develop, experiment with, test and prototype AI applications across GPU systems of their choice - from laptops and workstations to data center and cloud. It offers a new approach for creating, using and sharing GPU-enabled development environments across people and systems.

A simple installation gets users up and running with AI Workbench on a local or remote machine in just minutes. Users can then start a new project or replicate one from the examples on GitHub. Everything works through GitHub or GitLab, so users can easily collaborate and distribute work. Learn more about getting started with AI Workbench.

How AI Workbench Helps Address AI Project Challenges Developing AI workloads can require manual, often complex processes, right from the start.

Setting up GPUs, updating drivers and managing versioning incompatibilities can be cumbersome. Reproducing projects across different systems can require replicating manual processes over and over. Inconsistencies when replicating projects, like issues with data fragmentation and version control, can hinder collaboration. Varied setup processes, moving credentials and secrets, and changes in the environment, data, models and file locations can all limit the portability of projects.

AI Workbench makes it easier for data scientists and developers to manage their work and collaborate across heterogeneous platforms. It integrates and automates various aspects of the development process, offering:

Ease of setup: AI Workbench streamlines the process of setting up a developer environment that's GPU-accelerated, even for users with limited technical knowledge.

Seamless collaboration: AI Workbench integrates with version-control and project-management tools like GitHub and GitLab, reducing friction when collaborating.

Consistency when scaling from local to cloud: AI Workbench ensures consistency across multiple environments, supporting scaling up or down from local workstations or PCs to data centers or the cloud.

RAG for Documents, Easier Than Ever NVIDIA offers sample development Workbench Projects to help users get started with AI Workbench. The hybrid RAG Workbench Project is one example: It runs a custom, text-based RAG web application with a user's documents on their local workstation, PC or remote system.

Every Workbench Project runs in a container - software that includes all the necessary components to run the AI application. The hybrid RAG sample pairs a Gradio chat interface frontend on the host machine with a containerized RAG server - the backend that services a user's request and routes queries to and from the vector database and the selected large language model.

This Workbench Project supports a wide variety of LLMs available on NVIDIA's GitHub page. Plus, the hybrid nature of the project lets users select where to run inference.

Workbench Projects let users version the development environment and code. Developers can run the embedding model on the host machine and run inference locally on a Hugging Face Text Generation Inference server, on target cloud resources using NVIDIA inference endpoints like the NVIDIA API catalog, or with self-hosting microservices such as NVIDIA NIM or third-party services.

The hybrid RAG Workbench Project also includes:

Performance metrics: Users can evaluate how RAG- and non-RAG-based user queries perform across each inference mode. Tracked metrics include Retrieval Time, Time to First Token (TTFT) and Token Velocity.

Retrieval transparency: A panel shows the exact snippets of text - retrieved from the most contextually relevant content in the vector database - that are being fed into the LLM and improving the response's relevance to a user's query.

Response customization: Responses can be tweaked with a variety of parameters, such as maximum tokens to generate, temperature and frequency penalty.

To get started with this project, simply install AI Workbench on a local system. The hybrid RAG Workbench Project can be brought from GitHub into the user's account and duplicated to the local system.

More resources are available in the AI Decoded user guide. In addition, community members provide helpful video tutorials, like the one from Joe Freeman below.

Customize, Optimize, Deploy Developers often seek to customize AI models for specific use cases. Fine-tuning, a technique that changes the model by training it with additional data, can be useful for style transfer or changing model behavior. AI Workbench helps with fine-tuning, as well.

The Llama-factory AI Workbench Project enables QLoRa, a fine-tuning method that minimizes memory requirements, for a variety of models, as well as
LINK: https://blogs.nvidia.com/blog/ai-decoded-workbench-hybrid-rag/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

09/10/2026

Dolby Expands Dolby OptiView Platform with New Capabilities at IBC 2026

September 10 2026, 06:00 (PDT) Dolby Expands Dolby OptiView Platform with New Capabilities at IBC 2026 New Sports Intelligence helps providers better unders...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

12/09/2026

Colorfront And Qumulo Team-Up To Make Every Frame Of Footage Searchable For The Life Of A Production

September 11, 2026 BUDAPEST & SEATTLE, 10 September 2026 - Qumulo and Colorfron...

12/09/2026

New Type RT console and Apex M6 Modular I/O Platform

Calrec expands its IP ecosystem with new Type RT console and Apex M6 Modular I/O Platform at IBC 2026 Type RT is the centrepiece of Calrec's news at IBC 202...

12/09/2026

Calrec expands its IP ecosystem with new Type RT console...

At IBC 2026, Calrec has unveiled Type RT, a new 6, 12 and 18-motorised fader console range that condenses everything a broadcast audio desk needs into a compact...

12/09/2026

Matrox, NVIDIA Integrate ORIGIN, Holoscan For Media

Share Copy link Facebook X Linkedin Bluesky Email...

12/09/2026

Ross Video Debuts New Live Production Technology at IBC2026

Share Copy link Facebook X Linkedin Bluesky Email...

12/09/2026

FCC Nominee Thumann Severs to Testify in Senate Sept. 17

Share Copy link Facebook X Linkedin Bluesky Email...

12/09/2026

TV Skyline Becomes The World's First Ultrix FR30 Customer

Share Copy link Facebook X Linkedin Bluesky Email...

12/09/2026

Ikegami Introduces VN Versions of Four UNICAM XE Cameras At IBC2026

Share Copy link Facebook X Linkedin Bluesky Email...

12/09/2026

Riedel Technology Powers Record Breaking Esports World Cu...

WUPPERTAL, Germany Sept. 11, 2026 From July 6 to Aug. 23, 2026, Paris hosted the Esports World Cup (EWC), a seven-week live production spanning eight stages...

12/09/2026

Mediacom Deploys Riedel Easy5G for Wireless Camera Covera...

Riedel Communications today announced that Mediacom has purchased and deployed Easy5G , Riedel's Private 5G solution, for live broadcast coverage of La Vuel...

12/09/2026

A Decade of Innovation AMP VISUAL TV Builds on Ten Years...

AMP VISUAL TV is continuing the evolution of its production infrastructure with further investments in Riedel technology across its OB fleet and fixed productio...

12/09/2026

Touring Video Expands Flexible Production Workflows With...

Riedel Communications today announced that Touring Video, a premier entertainment mobile production service company, continues to expand its production workflow...

12/09/2026

Atomos Expands Accessory Ecosystem with New SHAPE and Sma...

Atomos (7.B19) today announced new third-party accessories for Sumo PRO-19 and Shinobi 7 II, developed by SHAPE and SmallRig to give production teams more ways ...

12/09/2026

Deity Microphones Brings PR 2ii, TC-2, and D4RX to IBC2026

Deity Microphones Brings PR 2ii, TC-2, and D4RX to IBC2026 Brie Clayton September 11, 2026 0 Comments The Deity PR-2ii arrives with production firmwar...

12/09/2026

Blackmagic Design Announces Special Pricing for Ultra HD Products

Blackmagic Design Announces Special Pricing for Ultra HD Products Brie Clayton September 11, 2026 0 Comments Now customers can afford to upgrade to Ul...

12/09/2026

Grass Valley EDIUS Future Assurance Program launched at IBC2026

Grass Valley EDIUS Future Assurance Program launched at IBC2026 Brie Clayton September 11, 2026 0 Comments New program gives EDIUS customers access to...

12/09/2026

Berklee Students Develop New Ideas for Live Entertainment and Travel in London

Berklee Students Develop New Ideas for Live Entertainment and Travel in London This summer, a group of music business students collaborated with Heathrow, Bri...

11/09/2026

IBC 2026: Calrec Launches Type RT Console and Apex M6 Modular I/O Platform

Calrec will introduce two new products at IBC2026 (Stand 8.C47, RAI Amsterdam, September 11-14): the Type RT broadcast production console and the Apex M6 Modula...

11/09/2026

IBC 2026: Grass Valley Launches AVORA All-in-One Live Production Platform

Grass Valley has announced AVORA, a standalone all-in-one live production solution launching at IBC2026 (Booth 9.A01 and IBC Future Tech Zone, Stand 14.C49, Sep...

11/09/2026

IBC 2026: CAMB.AI Launches Streaming SDK and Dashboard for Multilingual Live Broadcasting

CAMB.AI is introducing its Streaming SDK and Streaming Dashboard at IBC2026, wit...

11/09/2026

IBC 2026: EVS Launches XT-S Software-Defined Multi-Application Server

EVS has announced XT-S, a software-defined server platform that runs multiple live production applications on a common hardware foundation. The platform debuts ...

11/09/2026

IBC 2026: EKT Showcases Linux-Based ATSC 3.0 Converter Box

EKT is demonstrating a new Linux-based ATSC 3.0 set-top box at IBC2026. The second-generation product moves away from Android to reduce memory costs, with Linux...

11/09/2026

IBC 2026: Sennheiser Group to Showcase Wireless, Monitoring, and Broadcast Workflow

The Sennheiser Group will exhibit at IBC2026 (Hall 8, Booth D50, September 11-14...

11/09/2026

Ludacris Joins Chris Stapleton and Cindy Blackman Santana for Monday Night Footballs In the Air Tonight Open

ESPN's Monday Night Football will open its 2026-27 season with a new renditi...

11/09/2026

Dalet Integrates NVIDIA Synthetic Content Detection into Dalet Flex

Dalet has announced an integration of NVIDIA's Synthetic Video Detector AI model into Dalet Flex, enabling media organizations to identify potentially AI-ge...

11/09/2026

IBC 2026: MC-IF Demonstrates VVC Capabilities

The Media Coding Industry Forum (MC-IF) will demonstrate Versatile Video Coding (VVC) capabilities at IBC2026 (Hall 2, Stand 2.B13), as the codec moves toward c...

11/09/2026

IBC 2026: Sony to Demonstrate Live, News, Cinema, and Immersive Production

Sony will exhibit at IBC2026 (Hall 13, September 11-14), demonstrating products across live production, news, cinema, and immersive and spatial content workflow...

11/09/2026

Wave Central Named Recipient of 2026 Technology and Engineering Emmy for FPV Drone Systems

Wave Central, a Domo Broadcast Company, has been named a recipient of a 2026 Tec...

11/09/2026

IBC 2026: SNS Validates EVO Compatibility With Avid Media Composer 2026.8, Adds Browser-Based Upload to Outpost

Studio Network Solutions (SNS) is bringing a pair of workflow updates to IBC2026...

11/09/2026

IBC 2026: iodyne to Demo Pro Mini Smart Drive Across Four Partner Booths

iodyne will co-exhibit with Hedge at IBC2026 (Booth 7.A36, September 11-14) and appear across three additional partner booths, marking the first public demonstr...

11/09/2026

SVG GameDay, Ep. 30: Freelance Graphics Operator Corinne Yamada - Philly-Based Graphics Creation

In-venue and creative video staffers at the professional and collegiate level ha...

11/09/2026

CBS Sports Goes HDR Across Its Entire NFL Package, Heads to Rio in Week 3

The network will phase in HDR crew by crew while expanding Next Gen Stats integration, SkyDry availability, and onsite studio shows...

11/09/2026

IBC 2026: CAMB.AI Unveils the World's First Multilingual Broadcasting Agent Powered by NVIDIA

CAMB.AI is introducing its new Streaming SDK and Streaming Dashboard at IBC2026,...

11/09/2026

Netflix Goes Down Under With NBC Sports, NEP for NFL From Melbourne

Massive host operation at Melbourne Cricket Ground featured 54 cameras, 25,000 ST 2110 flows, REMI studio production in Stamford, and global distribution to mor...

11/09/2026

FOX Sports Homogenizes Coverage With Upscaled Game Tech, Seamless Integration With Studio Shows in L.A.

Production Anywhere Remote Kit (PARK) & AR-fueled Stage B are two key pieces to ...

11/09/2026

Get to Know This Fall and Winter's Filmmakers Through These 36 Sundance Institute-Supported Films

By Ramona Flume and Lucy Spicer There's a particular kind of magic that hap...

11/09/2026

Film Festival Watch: 8 Sundance Institute-Supported Films to See at the 2026 Toronto International Film Festival

By Jessica Herndon The fall film season is kicking off, and Sundance Institute-...

11/09/2026

Positive Grid announce the Spark Pedal

Spark series gains compact modelling stompbox Positive Grid's Spark amps have become a popular part of the company's line-up, offering a convenient ...

11/09/2026

Rapid Flow release omniGRID

New flagship sequencing plug-in revealed Rapid Flow have just launched a new 16-track sequencer plug-in which they say has been designed for total rhythmic...

11/09/2026

Rohde & Schwarz unveils the air-cooled R&STM1 medium-power UHF transmitter at IBC 2026

Rohde & Schwarz unveils the air-cooled R&S TM1 medium-power UHF transmitter at I...

11/09/2026

TV Usage Kicks Usual Summer Slowdown, Fueled by World Cup and Streaming in Nielsen's July Gauge Reports

FIFA World Cup 2026 and Peacock's Love Island USA' Power NBCU-Versant ...

11/09/2026

Atomos Debuts New HDR Monitors at IBC2026

Share Copy link Facebook X Linkedin Bluesky Email...

11/09/2026

Calrec to Unveil Type RT Compact Console at IBC2026

Share Copy link Facebook X Linkedin Bluesky Email...

11/09/2026

Lightware Launches LARA 2 with New Architecture for More...

Lightware has announced the launch of LARA (Lightware Advanced Room Automation) 2.0, a major upgrade to its built-in room automation platform, introducing impro...

11/09/2026

MusiMap launches latest technology unlocking new commerci...

10 September 2026 - MusiMap, the B2B music intelligence company that is transforming how companies can harness the emotional power of music to reach consumers a...

11/09/2026

Matrox Video Integrates Matrox ORIGIN with NVIDIA Holosca...

Matrox Video today announced an integration between Matrox ORIGIN and NVIDIA Holoscan for Media, enabling Matrox ORIGIN media services to be provisioned, config...

11/09/2026

Net Insight brings cloud-native live media orchestration...

Net Insight today announced that its cloud-native live media orchestration is now available through Oracle Cloud Marketplace, providing broadcasters, production...