How to Fine-Tune an LLM on NVIDIA GPUs With Unsloth

Modern workflows showcase the endless possibilities of generative and agentic AI on PCs.

Of many, some examples include tuning a chatbot to handle product-support questions or building a personal assistant for managing one's schedule. A challenge remains, however, in getting a small language model to respond consistently with high accuracy for specialized agentic tasks.

That's where fine-tuning comes in.

Unsloth, one of the world's most widely used open-source frameworks for fine-tuning LLMs, provides an approachable way to customize models. It's optimized for efficient, low-memory training on NVIDIA GPUs - from GeForce RTX desktops and laptops to RTX PRO workstations and DGX Spark, the world's smallest AI supercomputer.

Another powerful starting point for fine-tuning is the just-announced NVIDIA Nemotron 3 family of open models, data and libraries. Nemotron 3 introduces the most efficient family of open models, ideal for agentic AI fine-tuning.

Teaching AI New Tricks Fine-tuning is like giving an AI model a focused training session. With examples tied to a specific topic or workflow, the model improves its accuracy by learning new patterns and adapting to the task at hand.

Choosing a fine-tuning method for a model depends on how much of the original model the developer wants to adjust. Based on their goals, developers can use one of three main fine-tuning methods:

Parameter-efficient fine-tuning (such as LoRA or QLoRA):

How it works: Updates only a small portion of the model for faster, lower-cost training. It's a smarter and efficient way to enhance a model without altering it drastically.

Target use case: Useful across nearly all scenarios where full fine-tuning would traditionally be applied - including adding domain knowledge, improving coding accuracy, adapting the model for legal or scientific tasks, refining reasoning, or aligning tone and behavior.

Requirements: Small- to medium-sized dataset (100-1,000 prompt-sample pairs).

Full fine-tuning:

How it works: Updates all of the model's parameters - useful for teaching the model to follow specific formats or styles.

Target use case: Advanced use cases, such as building AI agents and chatbots that must provide assistance about a specific topic, stay within a certain set of guardrails and respond in a particular manner.

Requirements: Large dataset (1,000+ prompt-sample pairs).

Reinforcement learning:

How it works: Adjusts the behavior of the model using feedback or preference signals. The model learns by interacting with its environment and uses the feedback to improve itself over time. This is a complex, advanced technique that interweaves training and inference - and can be used in tandem with parameter-efficient fine-tuning and full fine-tuning techniques. See Unsloth's Reinforcement Learning Guide for details.

Target use case: Improving the accuracy of a model in a particular domain - such as law or medicine - or building autonomous agents that can orchestrate actions on a user's behalf.

Requirements: A process that contains an action model, a reward model and an environment for the model to learn from.

Another factor to consider is the VRAM required per each method. The chart below provides an overview of the requirements to run each type of fine-tuning method on Unsloth.

Fine-tuning requirements on Unsloth. Unsloth: A Fast Path to Fine-Tuning on NVIDIA GPUs LLM fine-tuning is a memory- and compute-intensive workload that involves billions of matrix multiplications to update model weights at every training step. This type of heavy parallel workload requires the power of NVIDIA GPUs to complete the process quickly and efficiently.

Unsloth shines at this workload, translating complex mathematical operations into efficient, custom GPU kernels to accelerate AI training.

Unsloth helps boost the performance of the Hugging Face transformers library by 2.5x on NVIDIA GPUs. These GPU-specific optimizations, combined with Unsloth's ease of use, make fine-tuning accessible to a broader community of AI enthusiasts and developers.

The framework is built and optimized for NVIDIA hardware - from GeForce RTX laptops to RTX PRO workstations and DGX Spark - providing peak performance while reducing VRAM consumption.

Unsloth provides helpful guides on how to get started and manage different LLM configurations, hyperparameters and options, along with example notebooks and step-by-step workflows.

Check out some of these Unsloth guides:

Fine-Tuning LLMs With NVIDIA RTX 50 Series GPUs and Unsloth

Fine-Tuning LLMs With NVIDIA DGX Spark and Unsloth

Learn how to install Unsloth on NVIDIA DGX Spark. Read the NVIDIA technical blog for a deep dive of fine-tuning and reinforcement learning on the NVIDIA Blackwell platform.

For a hands-on local fine-tuning walkthrough, watch Matthew Berman showing reinforcement learning running on a NVIDIA GeForce RTX 5090 using Unsloth in the video below.

Available Now: NVIDIA Nemotron 3 Family of Open Models The new Nemotron 3 family of open models - in Nano, Super, and Ultra sizes - built on a new hybrid latent Mixture-of-Experts (MoE) architecture, introduces the most efficient family of open models with leading accuracy, ideal for building agentic AI applications.

Nemotron 3 Nano 30B-A3B, available now, is the most compute-efficient model in the lineup. It's optimized for tasks such as software debugging, content summarization, AI assistant workflows and information retrieval at low inference costs. Its hybrid MoE design delivers:

Up to 60% fewer reasoning tokens, significantly reducing inference cost.

A 1 million-token context window, allowing the model to retain far more information for long, multistep tasks.

Nemotron 3 Super is a high-accuracy reasoning model for multi-agent applications, while Nemotron 3 Ultra is for complex AI applications

LINK:	https://blogs.nvidia.com/blog/rtx-ai-garage-fine-tuning-unsloth-dgx-sp...
	See more stories from nvidia

More from Nvidia