
Large language models (LLMs), trained on datasets with billions of tokens, can generate high-quality content. They're the backbone for many of the most popular AI applications, including chatbots, assistants, code generators and much more.
One of today's most accessible ways to work with LLMs is with AnythingLLM, a desktop app built for enthusiasts who want an all-in-one, privacy-focused AI assistant directly on their PC.
With new support for NVIDIA NIM microservices on NVIDIA GeForce RTX and NVIDIA RTX PRO GPUs, AnythingLLM users can now get even faster performance for more responsive local AI workflows.
What Is AnythingLLM? AnythingLLM is an all-in-one AI application that lets users run local LLMs, retrieval-augmented generation (RAG) systems and agentic tools.
It acts as a bridge between a user's preferred LLMs and their data, and enables access to tools (called skills), making it easier and more efficient to use LLMs for specific tasks like:
Question answering: Getting answers to questions from top LLMs - like Llama and DeepSeek R1 - without incurring costs.
Personal data queries: Use RAG to query content privately, including PDFs, Word files, codebases and more.
Document summarization: Generating summaries of lengthy documents, like research papers.
Data analysis: Extracting data insights by loading files and querying it with LLMs.
Agentic actions: Dynamically researching content using local or remote resources, running generative tools and actions based on user prompts.
AnythingLLM can connect to a wide variety of open-source local LLMs, as well as larger LLMs in the cloud, including those provided by OpenAI, Microsoft and Anthropic. In addition, the application provides access to skills for extending its agentic AI capabilities via its community hub.
With a one-click install and the ability to launch as a standalone app or browser extension - wrapped in an intuitive experience with no complicated setup required - AnythingLLM is a great option for AI enthusiasts, especially those with GeForce RTX and NVIDIA RTX PRO GPU-equipped systems.
RTX Powers AnythingLLM Acceleration GeForce RTX and NVIDIA RTX PRO GPUs offer significant performance gains for running LLMs and agents in AnythingLLM - speeding up inference with Tensor Cores designed to accelerate AI.
AnythingLLM runs LLMs with Ollama for on-device execution accelerated through Llama.cpp and ggml tensor libraries for machine learning.
Ollama, Llama.cpp and GGML are optimized for NVIDIA RTX GPUs and the fifth-generation Tensor Cores. Performance on GeForce RTX 5090 is 2.4X compared to an Apple M3 Ultra.
GeForce RTX 5090 delivers 2.4x faster LLM inference in AnythingLLM than Apple M3 Ultra on both Llama 3.1 8B and DeepSeek R1 8B. As NVIDIA adds new NIM microservices and reference workflows - like its growing library of AI Blueprints - tools like AnythingLLM will unlock even more multimodal AI use cases.
AnythingLLM - Now With NVIDIA NIM AnythingLLM recently added support for NVIDIA NIM microservices - performance-optimized, prepackaged generative AI models that make it easy to get started with AI workflows on RTX AI PCs with a streamlined API.
NVIDIA NIMs are great for developers looking for a quick way to test a Generative AI model in a workflow. Instead of having to find the right model, download all the files and figure out how to connect everything, they provide a single container that has everything you need. And they can run both on Cloud and PC, making it easy to prototype locally and then deploy on the cloud.
By offering them within AnythingLLM's user-friendly UI, users have a quick way to test them and experiment with them. And then they can either connect them to their workflows with AnythingLLM, or leverage NVIDIA AI Blueprints and NIM documentation and sample code to plug them directly to their apps or projects.
Explore the wide variety of NIM microservices available to elevate AI-powered workflows, including language and image generation, computer vision and speech processing.
Each week, the RTX AI Garage blog series features community-driven AI innovations and content for those looking to learn more about NIM microservices and AI Blueprints, as well as building AI agents, creative workflows, digital humans, productivity apps and more on AI PCs and workstations.
Plug in to NVIDIA AI PC on Facebook, Instagram, TikTok and X - and stay informed by subscribing to the RTX AI PC newsletter.
Follow NVIDIA Workstation on LinkedIn and X. See notice regarding software product information.
Most recent headlines
06/10/2025
France T l visions, France's leading broadcaster, has received the 2025 EBU ...
04/09/2025
Monumental Sports & Entertainment (MSE), in collaboration with Dalet, has been a...
07/08/2025
July 8 2025, 22:30 (PDT) Tata Motors & Dolby Bring Dolby Atmos to Harrier.ev, R...
19/07/2025
WASHINGTON In the run-up to its August open meeting, the Federal Communications Commission has released details of its previously announced plans to embark on a...
19/07/2025
PARIS Lawo has announced that it once again provided the IP-based audio infrastructure for the live sound and broadcast requirements of the French public broadc...
18/07/2025
Blanche Barton, Peter Gilmore, Scott Cummings, Sundance Institute Director of Programming Kim Yutani, and Peggy Nadramia throwing horns. (Stephen Lovekin/Shutte...
18/07/2025
By Jessica Herndon
One of the most exciting things about the Sundance Film Fest...
18/07/2025
BOSTON Sports network NESN today relaunched the NESN 360 app with a redesigned user experience and upgrades....
18/07/2025
Berklee Singers Wow the Competition on Netflixs Building the Band Netflixs new music competition show features two Berklee artists looking to make their pop b...
18/07/2025
As the House of Representatives voted early Friday morning to eliminate federal funding for public broadcasting, advocates warned that the elimination of $1 bil...
18/07/2025
DENVER Imagine Communications has announced that it is deepening its commitment to the North American sports and live events market with investments in its prod...
18/07/2025
LOS GATOS, Calif. Netflix once again reported strong financial growth, with revenue increasing 16% year-over-year to $11.079 billion in Q2 2025 as net income s...
18/07/2025
Live from the 153rd Open Championship: SMT Enhances Video Board Experience; Enab...
18/07/2025
ShotTracker's Davyeon Ross on Revolutionizing Basketball On the Court and ...
18/07/2025
Home Pitch: How the Danish League Brought Its Live Match Production In-House Matchday Production produced 800+ matches across Denmark in its first year By Geor...
18/07/2025
Live From the 153rd Open Championship: EMG Debuts Spidercam for Golf, Expands Su...
18/07/2025
Usyk vs Dubois II: Bringing the fight to life with state of the art cameras and ...
18/07/2025
Usyk vs Dubois II: DAZN's head of boxing Chris Glanville talks helicopters, ...
18/07/2025
Live From the 153rd Open Championship: NBC Expands Unilateral Effort; Shared Stu...
18/07/2025
Warner Bros. Discovery's Anne Graham on Training New Professionals to Ensure...
18/07/2025
Live From the 153rd Open Championship: How R&A Spidercam Became a Reality for Go...
18/07/2025
ESPN Bulks Up Onsite Studios, Cameras, Player Mics for WNBA All-Star Weekend Besides the game, four studio shows will be produced onsite By Jason Dachman, Edit...
18/07/2025
Friday 18 July 2025
Credit: Marc Sethi
The Sky Arts Awards will return on 16 September, once again taking place at the iconic Roundhouse in London. Produced b...
18/07/2025
This month, players can dive into special crossover events featuring SpongeBob SquarePants in Snake.io+ and Crossy Road Castle, available exclusively on Apple A...
18/07/2025
Up for the Match brings the fun, the passion-and a bit of Riverdance- tomorrow n...
17/07/2025
Blanche Barton, Peter Gilmore, Scott Cummings, Sundance Institute Director of Programming Kim Yutani, and Peggy Nadramia throwing horns. (Stephen Lovekin/Shutte...
17/07/2025
Spotify's goal with audiobooks is to help grow the publishing industry and create the best possible experience for booklovers, which is why we're launch...
17/07/2025
An advanced large solid rocket motor hot fire test conducted at L3Harris' Camden, Arkansas, site....
17/07/2025
L3Harris payloads and components have been on board every U.S. GPS satellite-more than 70 missions since the 1970s. Our technology is at the core of GPS availab...
17/07/2025
L3Harris' pack of launched effects is comprised of multi-role vehicles that ...
17/07/2025
NEW YORK - July 17, 2025 - Gracenote, the content data business unit of Nielsen, has announced the appointment of Ryan Moore to the new role of Chief Business O...
17/07/2025
READING, Pa. Network technology startup Miri Technologies has added David Cohen as vice president, business development. He'll be tasked with expanding the ...
17/07/2025
Berklee Wins CASE Award for Standout Digital Campaign The Alumni Affairs team's Variations Signature Series earned recognition for offering an authentic p...
17/07/2025
Early Thursday morning, the U.S. Senate voted to cut federal funding for PBS and NPR in a bill that also included cuts for foreign aid. In a 51-48 vote, the act...
17/07/2025
GREAT Britain and Northern Ireland Pavilions across IBC2025 (12 15 September, RAI Amsterdam) feature a remarkable 34 companies, each providing unique, innovativ...
17/07/2025
WHITE PLAINS, N.Y. The Society of Motion Picture and Television Engineers today introduced its Content Provenance and Authenticity (CPA) in Media Study Group (S...
17/07/2025
NEW YORK Despite widespread concerns about a soft advertising market, NBCUniversal said it closed its 2025-26 upfront negotiation cycle with a record ad sales v...
17/07/2025
NEW YORK Television advertising remains an effective means to market and influence younger baby boomer and Generation X consumers approaching retirement age, ac...
17/07/2025
ALLENTOWN, Pa. & JERSEY CITY, N.J. Service Electric Cable TV and OpenVault have announced that the operator is deploying the OpenVault Proactive Network Manage...
17/07/2025
WASHINGTON Following up on his first major policy speech just two ago, Federal Communications Commission Chair Brendan Carr is promising to buck the usual summ...
17/07/2025
Luxembourg, 14 July 2025 - SES received the final regulatory approvals for the SES-Intelsat transaction, including the US Federal Communications Commission.
O...
17/07/2025
Berklee Alumni Nominated for 2025 Emmy Awards Twenty-seven Berklee alumni received nominations across categories in sound mixing, sound editing, and compositi...
17/07/2025
Back to All News
What We Watched the First Half of 2025
Entertainment
17 July 2025
Global
Link copied to clipboard
Today, we're sharing our latest Eng...
17/07/2025
Cast your vote in 2 categories for Ivory 3 American Concert DIvory 3 American Concert D is in the 2025 MIDI Innovation Awards and we are honored to be nominated...
17/07/2025
SVG Sit-Down: OpenDrives' Sean Lee, Michael Wilsker, and Jason Matousek on E...
17/07/2025
Oracle's Geoff Tognetti on Staying Ahead of the AI Curve Through Further Ado...
17/07/2025
NBA's Summer League Is a Hot Bed of AI, Immersive Innovation Testing Audio mixing, intelligent camera framing are among AI-driven efforts By Ken Kerschbaum...
17/07/2025
SVG's TranSPORT Conference Returns to NYC on Oct. 21; Register Today! By SVG Staff
Thursday, July 17, 2025 - 8:29 am
Print This Story | Subscribe
St...
17/07/2025
Gravity Media chosen as group name for merged OB business By George Bevir
Wednesday, July 16, 2025 - 11:30
Print This Story
Gravity Media executive chairm...
17/07/2025
SES Closes Deal to Acquire Intelsat, Creating New Giant in Satellite Connectivit...