Project G-Assist Plug-In Builder Lets Anyone Customize AI on GeForce RTX AI PCs

AI is rapidly reshaping what's possible on a PC - whether for real-time image generation or voice-controlled workflows. As AI capabilities grow, so does their complexity. Tapping into the power of AI can entail navigating a maze of system settings, software and hardware configurations.

Enabling users to explore how on-device AI can simplify and enhance the PC experience, Project G-Assist - an AI assistant that helps tune, control and optimize GeForce RTX systems - is now available as an experimental feature in the NVIDIA app. Developers can try out AI-powered voice and text commands for tasks like monitoring performance, adjusting settings and interacting with supporting peripherals. Users can even summon other AIs powered by GeForce RTX AI PCs.

And it doesn't stop there. For those looking to expand Project G-Assist capabilities in creative ways, the AI supports custom plug-ins. With the new ChatGPT-based G-Assist Plug-In Builder, developers and enthusiasts can create and customize G-Assist's functionality, adding new commands, connecting external tools and building AI workflows tailored to specific needs. With the plug-in builder, users can generate properly formatted code with AI, then integrate the code into G-Assist - enabling quick, AI-assisted functionality that responds to text and voice commands.

Teaching PCs New Tricks: Plug-Ins and APIs Explained Plug-ins are lightweight add-ons that give software new capabilities. G-Assist plug-ins can control music, connect with large language models and much more.

Under the hood, these plug-ins tap into application programming interfaces (APIs), which allow different software and services to talk to each other. Developers can define functions in simple JSON formats, write logic in Python and quickly integrate new tools or features into G-Assist.

With the G-Assist Plug-In Builder, users can:

Use a responsive small language model running locally on GeForce RTX GPUs for fast, private inference.

Extend G-Assist's capabilities with custom functionality tailored to specific workflows, games and tools.

Interact with G-Assist directly from the NVIDIA overlay, without tabbing out of an application or workflow.

Invoke AI-powered GPU and system controls from applications using C++ and Python bindings.

Integrate with agentic frameworks using tools like Langflow, letting G-Assist function as a component in larger AI pipelines and multi-agent systems.

Built for Builders: Using Free APIs to Expand AI PC Capabilities NVIDIA's GitHub repository provides everything needed to get started on developing with G-Assist - including sample plug-ins, step-by-step instructions and documentation for building custom functionalities.

Developers can define functions in JSON and drop config files into a designated directory, where G-Assist can automatically load and interpret them. Users can even submit plug-ins for review and potential inclusion in the NVIDIA GitHub repository to make new capabilities available for others.

Hundreds of free, developer-friendly APIs are available today to extend G-Assist capabilities - from automating workflows to optimizing PC setups to boosting online shopping. For ideas, find searchable indices of free APIs for use across entertainment, productivity, smart home, hardware and more on publicapis.dev, free-apis.github.io, apilist.fun and APILayer.

Available sample plug-ins include Spotify, which enables hands-free music and volume control, and Google Gemini, which allows G-Assist to invoke a much larger cloud-based AI for more complex conversations, brainstorming and web searches using a free Google AI Studio API key.

In the clip below, G-Assist asks Gemini for advice on which Legend to pick in the hit game Apex Legends when solo queueing, as well as whether it's wise to jump into Nightmare mode for level 25 in Diablo IV:

And in the following clip, a developer uses the new plug-in builder to create a Twitch plug-in for G-Assist that checks if a streamer is live. After generating the necessary JSON manifest and Python files, the developer simply drops them into the G-Assist directory to enable voice commands like, Hey, Twitch, is [streamer] live?

In addition, users can customize G-Assist to control select peripherals and software applications with simple commands, such as to benchmark or adjust fan speeds, or to change lighting on supported Logitech G, Corsair, MSI and Nanoleaf devices.

Other examples include a Stock Checker plug-in that lets users quickly look up real-time stock prices and performance data, or a Weather plug-in allows users to ask G-Assist for current weather conditions in any city.

Details on how to build, share and load plug-ins are available on the NVIDIA GitHub repository.

Start Building Today With the G-Assist Plugin Builder and open API support, anyone can extend G-Assist to fit their exact needs. Explore the GitHub repository and submit features for review to help shape the next wave of AI-powered PC experiences.

Plug in to NVIDIA AI PC on Facebook, Instagram, TikTok and X - and stay informed by subscribing to the RTX AI PC newsletter.

Follow NVIDIA Workstation on LinkedIn and X.

See notice regarding software product information.

LINK:	https://blogs.nvidia.com/blog/rtx-ai-garage-g-assist-plugin-builder/...
	See more stories from nvidia

Most recent headlines