
As AI use cases continue to expand - from document summarization to custom software agents - developers and enthusiasts are seeking faster, more flexible ways to run large language models (LLMs).
Running models locally on PCs with NVIDIA GeForce RTX GPUs enables high-performance inference, enhanced data privacy and full control over AI deployment and integration. Tools like LM Studio - free to try - make this possible, giving users an easy way to explore and build with LLMs on their own hardware.
LM Studio has become one of the most widely adopted tools for local LLM inference. Built on the high-performance llama.cpp runtime, the app allows models to run entirely offline and can also serve as OpenAI-compatible application programming interface (API) endpoints for integration into custom workflows.
The release of LM Studio 0.3.15 brings improved performance for RTX GPUs thanks to CUDA 12.8, significantly improving model load and response times. The update also introduces new developer-focused features, including enhanced tool use via the tool_choice parameter and a redesigned system prompt editor.
The latest improvements to LM Studio improve its performance and usability - delivering the highest throughput yet on RTX AI PCs. This means faster responses, snappier interactions and better tools for building and integrating AI locally.
Where Everyday Apps Meet AI Acceleration LM Studio is built for flexibility - suited for both casual experimentation or full integration into custom workflows. Users can interact with models through a desktop chat interface or enable developer mode to serve OpenAI-compatible API endpoints. This makes it easy to connect local LLMs to workflows in apps like VS Code or bespoke desktop agents.
For example, LM Studio can be integrated with Obsidian, a popular markdown-based knowledge management app. Using community-developed plug-ins like Text Generator and Smart Connections, users can generate content, summarize research and query their own notes - all powered by local LLMs running through LM Studio. These plug-ins connect directly to LM Studio's local server, enabling fast, private AI interactions without relying on the cloud.
Example of using LM Studio to generate notes accelerated by RTX. The 0.3.15 update adds new developer capabilities, including more granular control over tool use via the tool_choice parameter and an upgraded system prompt editor for handling longer or more complex prompts.
The tool_choice parameter lets developers control how models engage with external tools - whether by forcing a tool call, disabling it entirely or allowing the model to decide dynamically. This added flexibility is especially valuable for building structured interactions, retrieval-augmented generation (RAG) workflows or agent pipelines. Together, these updates enhance both experimentation and production use cases for developers building with LLMs.
LM Studio supports a broad range of open models - including Gemma, Llama 3, Mistral and Orca - and a variety of quantization formats, from 4-bit to full precision.
Common use cases span RAG, multi-turn chat with long context windows, document-based Q&A and local agent pipelines. And by using local inference servers powered by the NVIDIA RTX-accelerated llama.cpp software library, users on RTX AI PCs can integrate local LLMs with ease.
Whether optimizing for efficiency on a compact RTX-powered system or maximizing throughput on a high-performance desktop, LM Studio delivers full control, speed and privacy - all on RTX.
Experience Maximum Throughput on RTX GPUs At the core of LM Studio's acceleration is llama.cpp - an open-source runtime designed for efficient inference on consumer hardware. NVIDIA partnered with the LM Studio and llama.cpp communities to integrate several enhancements to maximize RTX GPU performance.
Key optimizations include:
CUDA graph enablement: Groups multiple GPU operations into a single CPU call, reducing CPU overhead and improving model throughput by up to 35%.
Flash attention CUDA kernels: Boosts throughput by up to 15% by improving how LLMs process attention - a critical operation in transformer models. This optimization enables longer context windows without increasing memory or compute requirements.
Support for the latest RTX architectures: LM Studio's update to CUDA 12.8 ensures compatibility with the full range of RTX AI PCs - from GeForce RTX 20 Series to NVIDIA Blackwell-class GPUs, giving users the flexibility to scale their local AI workflows from laptops to high-end desktops.
Data measured using different versions of LM Studio and CUDA backends on GeForce RTX 5080 on DeepSeek-R1-Distill-Llama-8B model. All configurations measured using Q4_K_M GGUF (Int4) quantization at BS=1, ISL=4000, OSL=200, with Flash Attention ON. Graph showcases 27% speedup with the latest version of LM Studio due to NVIDIA contributions to the llama.cpp inference backend. With a compatible driver, LM Studio automatically upgrades to the CUDA 12.8 runtime, enabling significantly faster model load times and higher overall performance.
These enhancements deliver smoother inference and faster response times across the full range of RTX AI PCs - from thin, light laptops to high-performance desktops and workstations.
Get Started With LM Studio LM Studio is free to download and runs on Windows, macOS and Linux. With the latest 0.3.15 release and ongoing optimizations, users can expect continued improvements in performance, customization and usability - making local AI faster, more flexible and more accessible.
Users can load a model through the desktop chat interface or enable developer mode to expose an OpenAI-compatible API.
To quickly get started, download the latest version of LM Studio and open up the application.
Click the magnifying glass icon on the left panel to open up the Discover
More from Nvidia
26/06/2025
As of today, NVIDIA now supports the general availability of Gemma 3n on NVIDIA RTX and Jetson. Gemma, previewed by Google DeepMind at Google I/O last month, in...
26/06/2025
Editor's note: This blog is a part of Into the Omniverse, a series focused o...
26/06/2025
Mark Theriault founded the startup FITY envisioning a line of clever cooling products: cold drink holders that come with freezable pucks to keep beverages cold ...
26/06/2025
This GFN Thursday rolls out a new reward and games for GeForce NOW members. Whether hunting for hot new releases or rediscovering timeless classics, members can...
24/06/2025
To get the most out of AI, optimizations are critical. When developers think about optimizing AI models for inference, model compression techniques-such as quan...
24/06/2025
To speed up AI adoption across industries, HPE and NVIDIA today launched new AI factory offerings at HPE Discover in Las Vegas.
The new lineup includes everyth...
24/06/2025
From the heart of Germany's automotive sector to manufacturing hubs across F...
19/06/2025
GeForce NOW is throwing open the vault doors to welcome the legendary Borderland series to the cloud.
Whether a seasoned Vault Hunter or new to the mayhem of P...
18/06/2025
Project G-Assist - available through the NVIDIA App - is an experimental AI assistant that helps tune, control and optimize NVIDIA GeForce RTX systems.
NVIDIA&...
17/06/2025
As a global labor shortage leaves 50 million positions unfilled across industrie...
13/06/2025
Industrial AI isn't slowing down. Germany is ready.
Following London Tech Week and GTC Paris at VivaTech, NVIDIA founder and CEO Jensen Huang's Europea...
12/06/2025
Generative AI has reshaped how people create, imagine and interact with digital ...
12/06/2025
Level up GeForce NOW experiences this summer with 40% off Performance Day Passes. Enjoy 24 hours of premium cloud gaming with RTX ON, delivering low latency and...
11/06/2025
NVIDIA is launching a comprehensive, industry-defining autonomous vehicle (AV) software platform to accelerate large-scale deployment of safe, intelligent trans...
11/06/2025
NVIDIA Research has developed an AI light switch for videos that can turn daytim...
11/06/2025
Using NVIDIA platforms, tools and libraries, European telecommunications institu...
11/06/2025
NVIDIA was today named an Autonomous Grand Challenge winner at the Computer Visi...
11/06/2025
In the face of growing labor shortages and need for sustainability, European man...
11/06/2025
AI is packing and shipping efficiency for the retail and consumer packaged goods (CPG) industries, with a majority of surveyed companies in the space reporting ...
11/06/2025
Urban populations are expected to double by 2050, which means around 2.5 billion...
11/06/2025
Telecom companies last year spent nearly $295 billion in capital expenditures an...
11/06/2025
In a new effort to advance sovereign AI for European public service media, NVIDI...
11/06/2025
At GTC Paris - held alongside VivaTech, Europe's largest tech event - NVIDIA founder and CEO Jensen Huang delivered a clear message: Europe isn't just a...
10/06/2025
Germany's Leibniz Supercomputing Centre, LRZ, is gaining a new supercomputer...
10/06/2025
With a more detailed simulation of the Earth's climate, scientists and resea...
10/06/2025
Cisco and NVIDIA are helping set a new standard for secure, scalable and high-performance enterprise AI.
Announced today at the Cisco Live conference in San Di...
09/06/2025
AI isn't waiting. And this week, neither is Europe.
At London's Olympia, under a ceiling of steel beams and enveloped by the thrum of startup pitches, ...
08/06/2025
U.K. Prime Minister Keir Starmer's ambition for Britain to be an AI maker, not an AI taker, is becoming a reality at London Tech Week.
With NVIDIA's ...
05/06/2025
GeForce NOW is a gamer's ticket to an unforgettable summer of gaming. With 25 titles coming this month and endless ways to play, the summer is going to be e...
04/06/2025
NVIDIA is working with companies worldwide to build out AI factories - speeding ...
04/06/2025
Humans learn the norms, values and behaviors of society from each other - and Bernt B rnich, founder and CEO of 1X Technologies, thinks robots should learn like...
04/06/2025
4:2:2 cameras - capable of capturing double the color information compared with most standard cameras - are becoming widely available for consumers. At the same...
02/06/2025
Editor's note: This blog, originally published on October 28, 2024, has been...
02/06/2025
Since a 7.8-magnitude earthquake hit Syria and T rkiye two years ago - leaving 5...
29/05/2025
Ready for a front-row seat to the next scientific revolution?
That's the idea behind Doudna - a groundbreaking supercomputer announced today at Lawrence Be...
29/05/2025
Large language models (LLMs), trained on datasets with billions of tokens, can generate high-quality content. They're the backbone for many of the most popu...
29/05/2025
GeForce NOW is supercharging Valve's Steam Deck with a new native app - delivering the high-quality GeForce RTX-powered gameplay members are used to on a po...
28/05/2025
Building effective agentic AI systems requires rethinking how technology interac...
27/05/2025
Over a century ago, Henry Ford pioneered the mass production of cars and engines...
27/05/2025
NVIDIA and Google share a long-standing relationship rooted in advancing AI inno...
22/05/2025
GeForce NOW is turning up the heat this summer with a hot new deal. For a limited time, save 40% on six-month Performance memberships and enjoy premium GeForce ...
21/05/2025
As robots increasingly make their way to the largest enterprises' manufacturing plants and warehouses, the need for access to critical business and operatio...
20/05/2025
Industrial AI is transforming how factories operate, innovate and scale.
The convergence of AI, simulation and digital twins is poised to unlock new levels of ...
19/05/2025
Agentic AI is redefining scientific discovery and unlocking research breakthroughs and innovations across industries. Through deepened collaboration, NVIDIA and...
19/05/2025
Across robot training and development, NVIDIA Research is uncovering breakthroughs in areas such as multimodal generative AI and synthetic data generation.
The...
19/05/2025
Generative AI is transforming PC software into breakthrough experiences - from digital humans to writing assistants, intelligent agents and creative tools.
NVI...
18/05/2025
Electricity. The Internet. Now it's time for another major technology, AI, to sweep the globe.
NVIDIA founder and CEO Jensen Huang took the stage at a pack...
18/05/2025
Empowering engineering teams with more tools for building AI factories, NVIDIA t...
18/05/2025
The age of video analytics AI agents is here.
Video is one of the defining feat...
18/05/2025
TSMC, Cadence, KLA, Siemens and Synopsys are advancing semiconductor manufacturi...