Sony Pixel Power calrec Sony

Run LLMs on AnythingLLM Faster With NVIDIA RTX AI PCs

29/05/2025

Large language models (LLMs), trained on datasets with billions of tokens, can generate high-quality content. They're the backbone for many of the most popular AI applications, including chatbots, assistants, code generators and much more.

One of today's most accessible ways to work with LLMs is with AnythingLLM, a desktop app built for enthusiasts who want an all-in-one, privacy-focused AI assistant directly on their PC.

With new support for NVIDIA NIM microservices on NVIDIA GeForce RTX and NVIDIA RTX PRO GPUs, AnythingLLM users can now get even faster performance for more responsive local AI workflows.

What Is AnythingLLM? AnythingLLM is an all-in-one AI application that lets users run local LLMs, retrieval-augmented generation (RAG) systems and agentic tools.

It acts as a bridge between a user's preferred LLMs and their data, and enables access to tools (called skills), making it easier and more efficient to use LLMs for specific tasks like:

Question answering: Getting answers to questions from top LLMs - like Llama and DeepSeek R1 - without incurring costs.

Personal data queries: Use RAG to query content privately, including PDFs, Word files, codebases and more.

Document summarization: Generating summaries of lengthy documents, like research papers.

Data analysis: Extracting data insights by loading files and querying it with LLMs.

Agentic actions: Dynamically researching content using local or remote resources, running generative tools and actions based on user prompts.

AnythingLLM can connect to a wide variety of open-source local LLMs, as well as larger LLMs in the cloud, including those provided by OpenAI, Microsoft and Anthropic. In addition, the application provides access to skills for extending its agentic AI capabilities via its community hub.

With a one-click install and the ability to launch as a standalone app or browser extension - wrapped in an intuitive experience with no complicated setup required - AnythingLLM is a great option for AI enthusiasts, especially those with GeForce RTX and NVIDIA RTX PRO GPU-equipped systems.

RTX Powers AnythingLLM Acceleration GeForce RTX and NVIDIA RTX PRO GPUs offer significant performance gains for running LLMs and agents in AnythingLLM - speeding up inference with Tensor Cores designed to accelerate AI.

AnythingLLM runs LLMs with Ollama for on-device execution accelerated through Llama.cpp and ggml tensor libraries for machine learning.

Ollama, Llama.cpp and GGML are optimized for NVIDIA RTX GPUs and the fifth-generation Tensor Cores. Performance on GeForce RTX 5090 is 2.4X compared to an Apple M3 Ultra.

GeForce RTX 5090 delivers 2.4x faster LLM inference in AnythingLLM than Apple M3 Ultra on both Llama 3.1 8B and DeepSeek R1 8B. As NVIDIA adds new NIM microservices and reference workflows - like its growing library of AI Blueprints - tools like AnythingLLM will unlock even more multimodal AI use cases.

AnythingLLM - Now With NVIDIA NIM AnythingLLM recently added support for NVIDIA NIM microservices - performance-optimized, prepackaged generative AI models that make it easy to get started with AI workflows on RTX AI PCs with a streamlined API.

NVIDIA NIMs are great for developers looking for a quick way to test a Generative AI model in a workflow. Instead of having to find the right model, download all the files and figure out how to connect everything, they provide a single container that has everything you need. And they can run both on Cloud and PC, making it easy to prototype locally and then deploy on the cloud.

By offering them within AnythingLLM's user-friendly UI, users have a quick way to test them and experiment with them. And then they can either connect them to their workflows with AnythingLLM, or leverage NVIDIA AI Blueprints and NIM documentation and sample code to plug them directly to their apps or projects.

Explore the wide variety of NIM microservices available to elevate AI-powered workflows, including language and image generation, computer vision and speech processing.

Each week, the RTX AI Garage blog series features community-driven AI innovations and content for those looking to learn more about NIM microservices and AI Blueprints, as well as building AI agents, creative workflows, digital humans, productivity apps and more on AI PCs and workstations.

Plug in to NVIDIA AI PC on Facebook, Instagram, TikTok and X - and stay informed by subscribing to the RTX AI PC newsletter.

Follow NVIDIA Workstation on LinkedIn and X. See notice regarding software product information.
LINK: https://blogs.nvidia.com/blog/rtx-ai-garage-anythingllm-nim/...
See more stories from nvidia

Most recent headlines

06/10/2025

France Tlvisions Wins Prestigious 2025 EBU Technology & Innovation Award in Groundbreaking Collaboration with Dalet

France T l visions, France's leading broadcaster, has received the 2025 EBU ...

04/09/2025

Monumental Sports & Entertainment and Dalet Win Prestigious 2025 NAB Show Project of the Year Award

Monumental Sports & Entertainment (MSE), in collaboration with Dalet, has been a...

07/08/2025

Tata Motors & Dolby Bring Dolby Atmos to Harrier.ev, Redefining In-Car Entertainment Experience

July 8 2025, 22:30 (PDT) Tata Motors & Dolby Bring Dolby Atmos to Harrier.ev, R...

19/07/2025

FCC Releases Details of Its Plans to Review Emergency Alert System

WASHINGTON In the run-up to its August open meeting, the Federal Communications Commission has released details of its previously announced plans to embark on a...

19/07/2025

Lawo Mixing Consoles Take Center Stage at Concert de Paris

PARIS Lawo has announced that it once again provided the IP-based audio infrastructure for the live sound and broadcast requirements of the French public broadc...

18/07/2025

Realm of Satan Disturbs the Comfortable at 2024 Sundance Film Festival

Blanche Barton, Peter Gilmore, Scott Cummings, Sundance Institute Director of Programming Kim Yutani, and Peggy Nadramia throwing horns. (Stephen Lovekin/Shutte...

18/07/2025

Give Me the Backstory: Get to Know Heidi Ewing and Rachel Grady, the Directors of FOLKTALES

By Jessica Herndon One of the most exciting things about the Sundance Film Fest...

18/07/2025

NESN Unveils Enhanced NESN 360 App for iOS, Android Devices

BOSTON Sports network NESN today relaunched the NESN 360 app with a redesigned user experience and upgrades....

18/07/2025

Berklee Singers Wow the Competition on Netflixs Building the Band

Berklee Singers Wow the Competition on Netflixs Building the Band Netflixs new music competition show features two Berklee artists looking to make their pop b...

18/07/2025

Trump Expected to Sign Bill Defunding CPB After House Approves $1B Cuts

As the House of Representatives voted early Friday morning to eliminate federal funding for public broadcasting, advocates warned that the elimination of $1 bil...

18/07/2025

Imagine Communications Sharpens Focus on Sports, Live Events Market

DENVER Imagine Communications has announced that it is deepening its commitment to the North American sports and live events market with investments in its prod...

18/07/2025

Netflix Beats Revenue, Profit Expectations

LOS GATOS, Calif. Netflix once again reported strong financial growth, with revenue increasing 16% year-over-year to $11.079 billion in Q2 2025 as net income s...

18/07/2025

Live from the 153rd Open Championship: SMT Enhances Video Board Experience; Enables Practice Scheduling for Players

Live from the 153rd Open Championship: SMT Enhances Video Board Experience; Enab...

18/07/2025

ShotTracker's Davyeon Ross on Revolutionizing Basketball On the Court and Beyond Through Tracking Tech

ShotTracker's Davyeon Ross on Revolutionizing Basketball On the Court and ...

18/07/2025

Home Pitch: How the Danish League Brought Its Live Match Production In-House

Home Pitch: How the Danish League Brought Its Live Match Production In-House Matchday Production produced 800+ matches across Denmark in its first year By Geor...

18/07/2025

Live From the 153rd Open Championship: EMG Debuts Spidercam for Golf, Expands Support for NBC Sports

Live From the 153rd Open Championship: EMG Debuts Spidercam for Golf, Expands Su...

18/07/2025

Usyk vs Dubois II: Bringing the Fight to Life with State of the Art Cameras and Graphics from DAZN and Gravity Media

Usyk vs Dubois II: Bringing the fight to life with state of the art cameras and ...

18/07/2025

Usyk vs Dubois II: DAZN's Head of Boxing Chris Glanville Talks Helicopters, a Ringwalk Spectacular and Directing the Action

Usyk vs Dubois II: DAZN's head of boxing Chris Glanville talks helicopters, ...

18/07/2025

Live From the 153rd Open Championship: NBC Expands Unilateral Effort; Shared Studio Gives Live From,' NBC Sports a Great View

Live From the 153rd Open Championship: NBC Expands Unilateral Effort; Shared Stu...

18/07/2025

Warner Bros. Discovery's Anne Graham on Training New Professionals to Ensure the Future of the Business

Warner Bros. Discovery's Anne Graham on Training New Professionals to Ensure...

18/07/2025

Live From the 153rd Open Championship: How R&A Spidercam Became a Reality for Golf Coverage

Live From the 153rd Open Championship: How R&A Spidercam Became a Reality for Go...

18/07/2025

ESPN Bulks Up Onsite Studios, Cameras, Player Mics for WNBA All-Star Weekend

ESPN Bulks Up Onsite Studios, Cameras, Player Mics for WNBA All-Star Weekend Besides the game, four studio shows will be produced onsite By Jason Dachman, Edit...

18/07/2025

The Sky Arts Awards Are Back, Live From The Roundhouse This September

Friday 18 July 2025 Credit: Marc Sethi The Sky Arts Awards will return on 16 September, once again taking place at the iconic Roundhouse in London. Produced b...

18/07/2025

2025-07-18

This month, players can dive into special crossover events featuring SpongeBob SquarePants in Snake.io+ and Crossy Road Castle, available exclusively on Apple A...

18/07/2025

Hurling Fans Flock Together for a Night of Craic agus Ceol on RT's Up For The Match this Saturday night

Up for the Match brings the fun, the passion-and a bit of Riverdance- tomorrow n...

17/07/2025

Realm of SatanDisturbs the Comfortable at 2024 Sundance Film Festival

Blanche Barton, Peter Gilmore, Scott Cummings, Sundance Institute Director of Programming Kim Yutani, and Peggy Nadramia throwing horns. (Stephen Lovekin/Shutte...

17/07/2025

Audiobooks+ Brings More Choice and Flexibility to Spotify Premium Subscribers

Spotify's goal with audiobooks is to help grow the publishing industry and create the best possible experience for booklovers, which is why we're launch...

17/07/2025

L3Harris to Expand Solid Rocket Motor Production in Arkansas

An advanced large solid rocket motor hot fire test conducted at L3Harris' Camden, Arkansas, site....

17/07/2025

L3Harris Has the Future of PNT Ready Now

L3Harris payloads and components have been on board every U.S. GPS satellite-more than 70 missions since the 1970s. Our technology is at the core of GPS availab...

17/07/2025

L3Harris Introduces Launched Effects Vehicles to Increase US Multi-Domain Superiority

L3Harris' pack of launched effects is comprised of multi-role vehicles that ...

17/07/2025

Nielsen taps Ryan Moore as Chief Business Officer of Gracenote

NEW YORK - July 17, 2025 - Gracenote, the content data business unit of Nielsen, has announced the appointment of Ryan Moore to the new role of Chief Business O...

17/07/2025

David Cohen Joins Miri Technologies as VP, Sales and Business Development

READING, Pa. Network technology startup Miri Technologies has added David Cohen as vice president, business development. He'll be tasked with expanding the ...

17/07/2025

Berklee Wins CASE Award for Standout Digital Campaign

Berklee Wins CASE Award for Standout Digital Campaign The Alumni Affairs team's Variations Signature Series earned recognition for offering an authentic p...

17/07/2025

Senate Votes to Strip Federal Funds for Public Broadcasting

Early Thursday morning, the U.S. Senate voted to cut federal funding for PBS and NPR in a bill that also included cuts for foreign aid. In a 51-48 vote, the act...

17/07/2025

British Innovation Leads the Way at IBC2025

GREAT Britain and Northern Ireland Pavilions across IBC2025 (12 15 September, RAI Amsterdam) feature a remarkable 34 companies, each providing unique, innovativ...

17/07/2025

SMPTE Forms Study Group on Content Provenance and Authenticity In Media

WHITE PLAINS, N.Y. The Society of Motion Picture and Television Engineers today introduced its Content Provenance and Authenticity (CPA) in Media Study Group (S...

17/07/2025

NBCU Reports Highest Sales Volume in History' for 2025-26 Upfronts

NEW YORK Despite widespread concerns about a soft advertising market, NBCUniversal said it closed its 2025-26 upfront negotiation cycle with a record ad sales v...

17/07/2025

Study: TV Ads Are Top Influencer of Older Adults Buying Retirement Homes

NEW YORK Television advertising remains an effective means to market and influence younger baby boomer and Generation X consumers approaching retirement age, ac...

17/07/2025

Service Electric Cable TV Deploys OpenVault PMN

ALLENTOWN, Pa. & JERSEY CITY, N.J. Service Electric Cable TV and OpenVault have announced that the operator is deploying the OpenVault Proactive Network Manage...

17/07/2025

FCC Chair Outlines Busy Summer Agenda That Includes Major EAS Vote

WASHINGTON Following up on his first major policy speech just two ago, Federal Communications Commission Chair Brendan Carr is promising to buck the usual summ...

17/07/2025

SES Receives All Required Regulatory Approvals to Complete Intelsat Acquisition

Luxembourg, 14 July 2025 - SES received the final regulatory approvals for the SES-Intelsat transaction, including the US Federal Communications Commission. O...

17/07/2025

Berklee Alumni Nominated for 2025 Emmy Awards

Berklee Alumni Nominated for 2025 Emmy Awards Twenty-seven Berklee alumni received nominations across categories in sound mixing, sound editing, and compositi...

17/07/2025

What We Watched the First Half of 2025

Back to All News What We Watched the First Half of 2025 Entertainment 17 July 2025 Global Link copied to clipboard Today, we're sharing our latest Eng...

17/07/2025

Vote for Ivory 3 American Concert D in the MIDI Innovation Awards!

Cast your vote in 2 categories for Ivory 3 American Concert DIvory 3 American Concert D is in the 2025 MIDI Innovation Awards and we are honored to be nominated...

17/07/2025

SVG Sit-Down: OpenDrives' Sean Lee, Michael Wilsker, and Jason Matousek on Evolving Data Management

SVG Sit-Down: OpenDrives' Sean Lee, Michael Wilsker, and Jason Matousek on E...

17/07/2025

Oracle's Geoff Tognetti on Staying Ahead of the AI Curve Through Further Adoption

Oracle's Geoff Tognetti on Staying Ahead of the AI Curve Through Further Ado...

17/07/2025

NBA's Summer League Is a Hot Bed of AI, Immersive Innovation Testing

NBA's Summer League Is a Hot Bed of AI, Immersive Innovation Testing Audio mixing, intelligent camera framing are among AI-driven efforts By Ken Kerschbaum...

17/07/2025

SVG's TranSPORT Conference Returns to NYC on Oct. 21; Register Today!

SVG's TranSPORT Conference Returns to NYC on Oct. 21; Register Today! By SVG Staff Thursday, July 17, 2025 - 8:29 am Print This Story | Subscribe St...

17/07/2025

Gravity Media Chosen as Group Name for Merged OB Business

Gravity Media chosen as group name for merged OB business By George Bevir Wednesday, July 16, 2025 - 11:30 Print This Story Gravity Media executive chairm...

17/07/2025

SES Closes Deal to Acquire Intelsat, Creating New Giant in Satellite Connectivity

SES Closes Deal to Acquire Intelsat, Creating New Giant in Satellite Connectivit...