Large language models (LLMs), trained on datasets with billions of tokens, can generate high-quality content. They're the backbone for many of the most popular AI applications, including chatbots, assistants, code generators and much more.
One of today's most accessible ways to work with LLMs is with AnythingLLM, a desktop app built for enthusiasts who want an all-in-one, privacy-focused AI assistant directly on their PC.
With new support for NVIDIA NIM microservices on NVIDIA GeForce RTX and NVIDIA RTX PRO GPUs, AnythingLLM users can now get even faster performance for more responsive local AI workflows.
What Is AnythingLLM? AnythingLLM is an all-in-one AI application that lets users run local LLMs, retrieval-augmented generation (RAG) systems and agentic tools.
It acts as a bridge between a user's preferred LLMs and their data, and enables access to tools (called skills), making it easier and more efficient to use LLMs for specific tasks like:
Question answering: Getting answers to questions from top LLMs - like Llama and DeepSeek R1 - without incurring costs.
Personal data queries: Use RAG to query content privately, including PDFs, Word files, codebases and more.
Document summarization: Generating summaries of lengthy documents, like research papers.
Data analysis: Extracting data insights by loading files and querying it with LLMs.
Agentic actions: Dynamically researching content using local or remote resources, running generative tools and actions based on user prompts.
AnythingLLM can connect to a wide variety of open-source local LLMs, as well as larger LLMs in the cloud, including those provided by OpenAI, Microsoft and Anthropic. In addition, the application provides access to skills for extending its agentic AI capabilities via its community hub.
With a one-click install and the ability to launch as a standalone app or browser extension - wrapped in an intuitive experience with no complicated setup required - AnythingLLM is a great option for AI enthusiasts, especially those with GeForce RTX and NVIDIA RTX PRO GPU-equipped systems.
RTX Powers AnythingLLM Acceleration GeForce RTX and NVIDIA RTX PRO GPUs offer significant performance gains for running LLMs and agents in AnythingLLM - speeding up inference with Tensor Cores designed to accelerate AI.
AnythingLLM runs LLMs with Ollama for on-device execution accelerated through Llama.cpp and ggml tensor libraries for machine learning.
Ollama, Llama.cpp and GGML are optimized for NVIDIA RTX GPUs and the fifth-generation Tensor Cores. Performance on GeForce RTX 5090 is 2.4X compared to an Apple M3 Ultra.
GeForce RTX 5090 delivers 2.4x faster LLM inference in AnythingLLM than Apple M3 Ultra on both Llama 3.1 8B and DeepSeek R1 8B. As NVIDIA adds new NIM microservices and reference workflows - like its growing library of AI Blueprints - tools like AnythingLLM will unlock even more multimodal AI use cases.
AnythingLLM - Now With NVIDIA NIM AnythingLLM recently added support for NVIDIA NIM microservices - performance-optimized, prepackaged generative AI models that make it easy to get started with AI workflows on RTX AI PCs with a streamlined API.
NVIDIA NIMs are great for developers looking for a quick way to test a Generative AI model in a workflow. Instead of having to find the right model, download all the files and figure out how to connect everything, they provide a single container that has everything you need. And they can run both on Cloud and PC, making it easy to prototype locally and then deploy on the cloud.
By offering them within AnythingLLM's user-friendly UI, users have a quick way to test them and experiment with them. And then they can either connect them to their workflows with AnythingLLM, or leverage NVIDIA AI Blueprints and NIM documentation and sample code to plug them directly to their apps or projects.
Explore the wide variety of NIM microservices available to elevate AI-powered workflows, including language and image generation, computer vision and speech processing.
Each week, the RTX AI Garage blog series features community-driven AI innovations and content for those looking to learn more about NIM microservices and AI Blueprints, as well as building AI agents, creative workflows, digital humans, productivity apps and more on AI PCs and workstations.
Plug in to NVIDIA AI PC on Facebook, Instagram, TikTok and X - and stay informed by subscribing to the RTX AI PC newsletter.
Follow NVIDIA Workstation on LinkedIn and X. See notice regarding software product information.
Most recent headlines
04/09/2025
Monumental Sports & Entertainment (MSE), in collaboration with Dalet, has been a...
30/05/2025
American Underground Returns to American Tobacco Campus A full-circle homecoming for Durham's leading coworking community
Durham, N.C. (May 29, 2025) - In ...
30/05/2025
COLLEGE STATION, Texas Texas A&M University here has selected Grass Valley's IP-enabled live production solutions to upgrade its athletics coverage workflow...
30/05/2025
PHOENIX Trinity Capital has announced a commitment to provide $62.7 million in capital to Atmosphere TV, a streaming platform that offers more than 30 original ...
30/05/2025
How Musicians Can (and Should) Use AI-According to Berklee Experts At Berklee's first AI and Music Innovation event, artists and educators explored how ar...
30/05/2025
The Most Underrated Hip-Hop Producers, According to J. Rawls The producer whos worked with Mos Def, Talib Kweli, the Roots, and Common, joined a panel on camp...
30/05/2025
May 30th, 2025 Tribeca Festival 2025 Announces Creator Economy Programming Lineup
Tribeca Festival Elevates Digital Creators with Red Carpet, Big-Screen Debut...
30/05/2025
17th-Annual SVG College Summit Unites Video-Production Community as Industry Und...
30/05/2025
UEFA Champions League Final 2025: CBS Sports To Produce Pitchside Studio Show, ...
30/05/2025
University of Wyoming's Dennis Trapani on a Busy Year of Live Event Producti...
30/05/2025
Women's College World Series: ESPN Enhances Softball Coverage With Expanded ...
30/05/2025
Back to All News
Netflix and the Brazilian Tourism Board Announce Cooperation A...
30/05/2025
AI pinpoints new anti-aging drug candidates More than 70% of the drugs identified by artificial intelligence extended the lifespan of C. elegans worms.
May 29,...
29/05/2025
Director Erica Tremblay on the set of Little Chief' during the 2018 Native Filmmakers Lab. Photo by Tytianna Harris for Sundance Institute...
29/05/2025
Japan's music scene is entering an exciting new chapter. Long known for its ...
29/05/2025
Rest in Power, a True Legend of Our Screens
Johannesburg, South Africa 27 May...
29/05/2025
Johannesburg, South Africa 28 May 2025 - The National Film and Video Foundation (NFVF), Chief Executive Officer (CEO), Mr. Vincent Blennies, has resigned from...
29/05/2025
The media and entertainment landscape is changing rapidly. Emerging technologies like AI and cloud workflows are transforming post production, while economic sh...
29/05/2025
Don't Sell Out the Power of Sound
George Lucas famously said that sound and music are 50 percent of the entertainment in a movie and while the exact figure...
29/05/2025
L3Harris receives a contract to develop a next-generation security processor to ...
29/05/2025
Renewal Spans all 18 Local Fox Markets and Includes Streaming Measurement of Loc...
29/05/2025
Leading streamers expanded TV, movie and sports titles by 5% in Q2; Netflix led ...
29/05/2025
CARSON, Calif. Rep. Nanette Barrag n (D-Calif.) joined Federal Communications Commission member Anna Gomez and Carson City Mayor Lula Davis-Holmes on Tuesday to...
29/05/2025
NEW YORK Nielsen said it struck a multiyear renewal agreement with Fox Television Stations for measurement across 18 of Fox's owned-and-operated local stati...
29/05/2025
MIAMI NBCUniversal Telemundo Enterprises has launched the Caso Cerrado FAST channel, a 24/7 streaming destination featuring more than 800 hours of the iconic ...
29/05/2025
Pebble, the leading automation, content management and integrated channel specialist, has completed a large installation in Kuala Lumpur, Malaysia, as Media Pri...
29/05/2025
Tintri , a DDN subsidiary, and leading provider of the world's only workload-aware, AI-powered data management solutions, today announced that its high-per...
29/05/2025
MALAGA, Spain Media integration and asset management solutions provider Tedial has formed a strategic partnership with Moments Lab, a provider of AI-based video...
29/05/2025
Sony announced the latest addition to its Cinema Line family, the FX2. The camera, designed as an entry-level product in the broader Cinema Line range, will be ...
29/05/2025
RESTON, Va. Comscore has announced that its AI-powered ID-free audiences offering has been integrated into Yahoo DSP, adding to the existing suite of targeting ...
29/05/2025
SINGAPORE Atomos introduced its StudioSonic shotgun mic for filmmakers, journalists, YouTubers and production professionals at Broadcast Asia here, May 27-29 at...
29/05/2025
UEFA Champions League Final 2025: DAZN on the goals for its unilateral platinum ...
29/05/2025
UEFA Champions League Final 2025: Host broadcaster DAZN reveals the plan for thi...
29/05/2025
Back to All News
Alex de la Iglesia, Carmen Maura and Blanca Suarez Together in...
29/05/2025
Webinar and Report: State Broadband Rate Regulation
May 29, 2025
Network Economics
News
Cartesian presents key findings from our latest report for...
29/05/2025
We're pleased to be working with Nectar360 (Sainsbury's retail media business) and Albert Heijn, who are in the latter stages of the compliance audit an...
29/05/2025
May 29 2025, 23:30 (PDT) Mahindra Thar ROXX brings Dolby Atmos for an elevated entertainment experience
First to the world: 4-channel immersive audio with D...
29/05/2025
Ready for a front-row seat to the next scientific revolution?
That's the idea behind Doudna - a groundbreaking supercomputer announced today at Lawrence Be...
29/05/2025
LONDON - 29th May 2025: Fifty years ago, The Good Life made its debut. Now, the ...
29/05/2025
Roland-Garros 2025: Bringing the passion and drama with new cameras for storytel...
29/05/2025
Roland-Garros 2025: The FFT and host broadcaster Whisper target a younger audien...
29/05/2025
Roland-Garros 2025: Whisper on getting into the psyche of the FFT as the new hos...
29/05/2025
2025 Sports Emmys: Riot Games' Leanne McGowan on Valorant Champions 2024 By Jason Dachman, Editorial Director, U.S.
Thursday, May 29, 2025 - 10:04 am
P...
29/05/2025
Back to All News
Literary Suspense Comes Alive in Korean Series Notes from the...
29/05/2025
Moments Lab, a leader in AI video discovery, and LucidLink, the pioneer in real-...
29/05/2025
Comscore Adds Consumer AI Tool Usage Data to Its Industry-Leading Suite of Repor...
29/05/2025
Apple today introduced new Logic Pro updates for Mac and iPad, supercharging beat making and producing. The innovative Stem Splitter feature now offers even gre...
29/05/2025
Dragon Hearts- RT documentary follows cancer survivors united through the power...
29/05/2025
2FM is hitting the road this summer, broadcasting live from Ireland's biggest music festivals, starting with Forbidden Fruit this June Bank Holiday weekend....
Large language models (LLMs), trained on datasets with billions of tokens, can generate high-quality content. They're the backbone for many of the most popular AI applications, including chatbots, assistants, code generators and much more.
