Sony Pixel Power calrec Sony

Decoding How NVIDIA AI Workbench Powers App Development

19/06/2024

Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible and showcases new hardware, software, tools and accelerations for NVIDIA RTX PC and workstation users.

The demand for tools to simplify and optimize generative AI development is skyrocketing. Applications based on retrieval-augmented generation (RAG) - a technique for enhancing the accuracy and reliability of generative AI models with facts fetched from specified external sources - and customized models are enabling developers to tune AI models to their specific needs.

While such work may have required a complex setup in the past, new tools are making it easier than ever.

NVIDIA AI Workbench simplifies AI developer workflows by helping users build their own RAG projects, customize models and more. It's part of the RTX AI Toolkit - a suite of tools and software development kits for customizing, optimizing and deploying AI capabilities - launched at COMPUTEX earlier this month. AI Workbench removes the complexity of technical tasks that can derail experts and halt beginners.

What Is NVIDIA AI Workbench? Available for free, NVIDIA AI Workbench enables users to develop, experiment with, test and prototype AI applications across GPU systems of their choice - from laptops and workstations to data center and cloud. It offers a new approach for creating, using and sharing GPU-enabled development environments across people and systems.

A simple installation gets users up and running with AI Workbench on a local or remote machine in just minutes. Users can then start a new project or replicate one from the examples on GitHub. Everything works through GitHub or GitLab, so users can easily collaborate and distribute work. Learn more about getting started with AI Workbench.

How AI Workbench Helps Address AI Project Challenges Developing AI workloads can require manual, often complex processes, right from the start.

Setting up GPUs, updating drivers and managing versioning incompatibilities can be cumbersome. Reproducing projects across different systems can require replicating manual processes over and over. Inconsistencies when replicating projects, like issues with data fragmentation and version control, can hinder collaboration. Varied setup processes, moving credentials and secrets, and changes in the environment, data, models and file locations can all limit the portability of projects.

AI Workbench makes it easier for data scientists and developers to manage their work and collaborate across heterogeneous platforms. It integrates and automates various aspects of the development process, offering:

Ease of setup: AI Workbench streamlines the process of setting up a developer environment that's GPU-accelerated, even for users with limited technical knowledge.

Seamless collaboration: AI Workbench integrates with version-control and project-management tools like GitHub and GitLab, reducing friction when collaborating.

Consistency when scaling from local to cloud: AI Workbench ensures consistency across multiple environments, supporting scaling up or down from local workstations or PCs to data centers or the cloud.

RAG for Documents, Easier Than Ever NVIDIA offers sample development Workbench Projects to help users get started with AI Workbench. The hybrid RAG Workbench Project is one example: It runs a custom, text-based RAG web application with a user's documents on their local workstation, PC or remote system.

Every Workbench Project runs in a container - software that includes all the necessary components to run the AI application. The hybrid RAG sample pairs a Gradio chat interface frontend on the host machine with a containerized RAG server - the backend that services a user's request and routes queries to and from the vector database and the selected large language model.

This Workbench Project supports a wide variety of LLMs available on NVIDIA's GitHub page. Plus, the hybrid nature of the project lets users select where to run inference.

Workbench Projects let users version the development environment and code. Developers can run the embedding model on the host machine and run inference locally on a Hugging Face Text Generation Inference server, on target cloud resources using NVIDIA inference endpoints like the NVIDIA API catalog, or with self-hosting microservices such as NVIDIA NIM or third-party services.

The hybrid RAG Workbench Project also includes:

Performance metrics: Users can evaluate how RAG- and non-RAG-based user queries perform across each inference mode. Tracked metrics include Retrieval Time, Time to First Token (TTFT) and Token Velocity.

Retrieval transparency: A panel shows the exact snippets of text - retrieved from the most contextually relevant content in the vector database - that are being fed into the LLM and improving the response's relevance to a user's query.

Response customization: Responses can be tweaked with a variety of parameters, such as maximum tokens to generate, temperature and frequency penalty.

To get started with this project, simply install AI Workbench on a local system. The hybrid RAG Workbench Project can be brought from GitHub into the user's account and duplicated to the local system.

More resources are available in the AI Decoded user guide. In addition, community members provide helpful video tutorials, like the one from Joe Freeman below.

Customize, Optimize, Deploy Developers often seek to customize AI models for specific use cases. Fine-tuning, a technique that changes the model by training it with additional data, can be useful for style transfer or changing model behavior. AI Workbench helps with fine-tuning, as well.

The Llama-factory AI Workbench Project enables QLoRa, a fine-tuning method that minimizes memory requirements, for a variety of models, as well as
LINK: https://blogs.nvidia.com/blog/ai-decoded-workbench-hybrid-rag/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

07/10/2026

Dalet Flex LTS Delivers Smarter Media Operations from Ingest to Distribution

Dalet, a leading technology and service provider for media-rich organizations, today announced the latest Long-Term Supported (LTS) release of Dalet Flex. Build...

09/09/2026

IBC 2026: Sony Introduces SEL814G Fisheye Zoom G Lens

Sony Electronics has announced the SEL814G, the company's first fisheye zoom G Lens for full-frame Alpha E-mount cameras. The lens will be available in Octo...

09/09/2026

IBC 2026: Telestream Expands Practical AI Capabilities Across Vantage and UP Platform

Telestream has announced an expansion of its Practical AI capabilities across th...

09/09/2026

IBC 2026: Chyron and Haivision Announce Camera-to-Cloud Live Production Integration

Haivision and Chyron have announced a technology partnership integrating Haivisi...

09/09/2026

ManningCast Reveals 2026-27 Season Schedule with Heist Mini Movie

Omaha Productions has released ManningCast: Heist, a short film featuring Peyton and Eli Manning attempting to break into NFL Commissioner Roger Goodell's o...

09/09/2026

IBC 2026: Imagine Communications to Showcase XVR Updates

Imagine Communications will demonstrate the next evolution of XVR at IBC2026 (Stand 1.B73, RAI Amsterdam, September 11-14). XVR is Imagine's channel-in-a-b...

09/09/2026

IBC 2026: swXtch.io Launches AI Broadcast Orchestration Tool abbe

swXtch.io has announced abbe, an AI-based orchestration layer for live media workflows, making its world premiere at IBC2026 (Meeting Room 5.MR13, Hall 5, Septe...

09/09/2026

Inside the NFL Returns to X for Second Season, Marking 50th Anniversary

NFL Films has announced the second season of Inside the NFL on X as an X Original, premiering Wednesday, September 9. The series marks Inside the NFL's 50th...

09/09/2026

LiveU Expands AP Partnership with LU900Q Fleet and MAM Integration

LiveU has announced an expanded partnership with The Associated Press, deploying the largest fleet of LU900Q intelligent production units in the industry as par...

09/09/2026

Diversified to Consolidate New Jersey Operations into New Newark Facility

Diversified has announced plans to consolidate its four existing New Jersey locations into a new facility in Newark, expected to be fully operational by the end...

09/09/2026

IBC 2026: Wisycom Showcases MATF Antenna Matrix, PFL RFoF Box, and MPR60 Updates

Wisycom will exhibit new RF products at IBC2026 (Stand 8.D60, September 11-14), including the MATF Wideband Antenna Matrix, PFL Portable RF over Fiber Box, and ...

09/09/2026

IBC 2026: G&D and VuWall to Introduce vsNEO KVM-over-IP Platform

Guntermann & Drunck (G&D) and VuWall, both Panoptec companies, will introduce vsNEO at IBC2026 (Stand 8.B63, September 11, RAI Amsterdam). Pre-orders open in ea...

09/09/2026

Mo-Sys Completes Restructuring, Promotes Selin Kemal to Managing Director

Mo-Sys has completed a business restructuring and relocated to a new facility in Canada Water, London. As part of the reorganization, Selin Kemal has been promo...

09/09/2026

Aicox and TAG Video Systems Power Monitoring for Telefnicas TSAmediaHUB

Aicox has integrated TAG Video Systems' Core monitoring platform into Telef nica's TSAmediaHUB audiovisual contribution and distribution infrastructure....

09/09/2026

IBC 2026: Blackmagic Design Releases DaVinci Resolve 21.1

Blackmagic Design has released DaVinci Resolve 21.1, a free update available now through the Blackmagic Design website. The release adds new AI tools, workflow ...

09/09/2026

ESPN and Apple Music Collaborate on Monday Night Football Music Curation for 2026-27 Season

ESPN and Apple Music have announced a collaboration on music curation for Monday...

09/09/2026

Sacramento Production Services Deploys Wisycom RF System at Sutter Health Park

Sacramento Production Services has completed a wireless audio overhaul at Sutter Health Park, home of the Athletics during their three-year residency in Sacrame...

09/09/2026

NDI and NVIDIA Collaborate on AI-Powered Multilingual Content Workflows

NDI and NVIDIA have announced a collaboration to develop AI-powered workflows enabling broadcasters to create and distribute multilingual content from a single ...

09/09/2026

Sports Broadcasting Hall of Fame Announces the Class of 2026

Christopher Chip Adams, Troy Aikman, Jim Dove, Jim Eady, Bucky Gunts, Robert A. Iger, Charlie Jablonski, Robert Kraft, Tom Rinaldi, and Dick Stockton to be ho...

09/09/2026

NBC Sports Opens 21st SNF Season With Expanded Schedule, Traveling Studio Show, and Bulked Up Toolkit

ND2 returns as the primary unit, While FNIA gets SSQC and new flexible Rover...

09/09/2026

Celemony Tonalic gains additional Paul Kleber upright bass

New parts aimed at hip-hop and pop productions The latest update to Celemony's innovative virtual session musician expands the collection of built-in in...

09/09/2026

Sonarworks Autumn Sale

Save up to 40% until 21 September 2026 Sonarworks are currently running their Autumn Sale, with discounts of up to 40% currently available across their rang...

09/09/2026

Bjooks Beat Gems to begin shipping

Upcoming title officially launches in September 2026 Bjooks' celebrations for this year's 909 Day will be welcome news amongst drum machine lovers, ...

09/09/2026

NITV to bring First Nations netball to audiences nationwide

NITV to bring First Nations netball to audiences nationwide 9 September, 2026 Media releases Every match of the 2026 First Nations Tournament will be live ...

09/09/2026

NFVF CALL FOR PROPOSALS: YOUTH FILMMAKER PROJECT FACILITATING COMPANY 2026

The National Film and Video Foundation (NFVF) invites eligible Tier 1 and 2 South African production companies to submit proposals to serve as the Facilitating ...

09/09/2026

Nielsen launches CTV Ad spend data into its Italian Ad Intel product

Italy launch follows UK and Germany as Nielsen expands product across EMEA Milan, Italy, September 9, 2026 - Nielsen, a global leader in audience measurement, ...

09/09/2026

Gracenote Expands AI Product Roadmap with Sports MCP Server, Entertainment SLM and Agents

Planned solutions tackle accuracy, cost and latency barriers to deploying AI at ...

09/09/2026

Dalet Promotes Brian Doheny to CEO

Share Copy link Facebook X Linkedin Bluesky Email...

09/09/2026

Studio Technologies Expands StudioComm Monitoring With In...

Studio Technologies, a leading manufacturer of high-quality audio, video, and fiber-optic solutions, introduces its new Model 791 Control Console at IBC 2026 (N...

09/09/2026

Maxon Brings Exclusive Preview of Fall Innovations to IBC...

Maxon, maker of powerful, approachable software solutions for creators working in 2D and 3D design, motion graphics, visual effects and visualization, is headin...

09/09/2026

NUGEN Audio Leads the Dialog Intelligibility Conversation...

NUGEN Audio unveils DialogCheck v1.2, the latest evolution of its intelligent dialog intelligibility and compliance tool, at IBC 2026 (Pod P27). With a sharpene...

09/09/2026

Trusted Partner Network Launches Fix It or Risk It to Hel...

Trusted Partner Network Launches Fix It or Risk It to Help the Entertainment Industry Tackle Everyday Security Risks Industry-Wide Campaign Launches at IBC2...

09/09/2026

FCC to Vote on Opening Up Spectrum for Satellite, IoT

Share Copy link Facebook X Linkedin Bluesky Email...

09/09/2026

NDI, NVIDIA Collaborate On Real-Time AI Multilingual Content

Share Copy link Facebook X Linkedin Bluesky Email...

09/09/2026

83 percent of UK advertisers are using AI but speed and n...

AI has moved firmly beyond experimentation in UK advertising, with more than eight in ten marketers (83%) now actively using or piloting AI in production, accor...

09/09/2026

Pixitmedia by DataCore to showcase cloud and connected pr...

IBC2026 - Stand 7.D22 At IBC2026, Pixitmedia by DataCore will showcase its latest capabilities designed to simplify distributed production, accelerate workflow...

09/09/2026

Net Insight Showcases Open Live Media for Controlled Live...

Stockholm, Sweden September 9th, 2026 At IBC2026, on booth 1.B47, Net Insight will showcase Open Live Media, its architecture for delivering resilient live ...

09/09/2026

Net Insight and DMC expand repeatable remote production m...

Net Insight today announced that DMC Production Norway is continuing the use of its Open Live Media technology for premium European football, deploying Nimbra 5...

09/09/2026

Amagi to Showcase Four AI-Native Innovations Built to Pow...

Amagi, the media industry cloud platform for unified broadcast, streaming, and monetization, will showcase four new AI-native capabilities at IBC2026, Hall 5, S...

09/09/2026

Limecraft Brings Connected Automated and Intelligent Medi...

Limecraft today announces version 2026.6 of its AI-powered production platform, extending capabilities for connected media operations. An AI-assisted approach t...

09/09/2026

Telestream Advances Practical AI from Content Understandi...

Telestream, a global leader in media workflow technologies, today announced the next evolution of its Practical AI strategy, evolving its comprehensive portfoli...

09/09/2026

Dalet Appoints Brian Doheny as Chief Executive Officer

Dalet, a leading technology and service provider for media-rich organizations, today announced the appointment of Brian Doheny as Chief Executive Officer (CEO)....

09/09/2026

LiveU Offers Starlink for News and Sports Broadcast as Cl...

LiveU is now a Clarus Integrated Reseller, bringing Starlink's fast, reliable broadband directly into LiveU's cellular bonding EcoSystem through a singl...

09/09/2026

Study: New Streaming TV Series Orders Plumet

Share Copy link Facebook X Linkedin Bluesky Email...

09/09/2026

G&D, VuWall To Introduce vsNEO KVM Platform At IBC2026

Share Copy link Facebook X Linkedin Bluesky Email...

09/09/2026

FCC May Ramp Up Regulation of How Broadcasters Cover Polls

Share Copy link Facebook X Linkedin Bluesky Email...

09/09/2026

Avid to Demo New Avid Content Core Features at IBC2026

Share Copy link Facebook X Linkedin Bluesky Email...

09/09/2026

Wisycom Showcases Latest Innovations at IBC 2026

Wisycom is showcasing the latest additions to its ecosystem of professional RF technologies at IBC 2026 (Stand 8.D60). This includes its new MATF Wideband Anten...

09/09/2026

VisualOn and Cires21 Cut Delivery Bitrate Up to 76 Percen...

AMSTERDAM, IBC2026 VisualOn, Inc. and Cires21 today released an updated joint whitepaper at IBC 2026 demonstrating measurable, high-impact improvements in visu...