
Under the hood of every AI application are algorithms that churn through data in their own language, one based on a vocabulary of tokens.
Tokens are tiny units of data that come from breaking down bigger chunks of information. AI models process tokens to learn the relationships between them and unlock capabilities including prediction, generation and reasoning. The faster tokens can be processed, the faster models can learn and respond.
AI factories - a new class of data centers designed to accelerate AI workloads - efficiently crunch through tokens, converting them from the language of AI to the currency of AI, which is intelligence.
With AI factories, enterprises can take advantage of the latest full-stack computing solutions to process more tokens at lower computational cost, creating additional value for customers. In one case, integrating software optimizations and adopting the latest generation NVIDIA GPUs reduced cost per token by 20x compared to unoptimized processes on previous-generation GPUs - delivering 25x more revenue in just four weeks.
By efficiently processing tokens, AI factories are manufacturing intelligence - the most valuable asset in the new industrial revolution powered by AI.
What Is Tokenization? Whether a transformer AI model is processing text, images, audio clips, videos or another modality, it will translate the data into tokens. This process is known as tokenization.
Efficient tokenization helps reduce the amount of computing power required for training and inference. There are numerous tokenization methods - and tokenizers tailored for specific data types and use cases can require a smaller vocabulary, meaning there are fewer tokens to process.
For large language models (LLMs), short words may be represented with a single token, while longer words may be split into two or more tokens.
The word darkness, for example, would be split into two tokens, dark and ness, with each token bearing a numerical representation, such as 217 and 655. The opposite word, brightness, would similarly be split into bright and ness, with corresponding numerical representations of 491 and 655.
In this example, the shared numerical value associated with ness can help the AI model understand that the words may have something in common. In other situations, a tokenizer may assign different numerical representations for the same word depending on its meaning in context.
For example, the word lie could refer to a resting position or to saying something untruthful. During training, the model would learn the distinction between these two meanings and assign them different token numbers.
For visual AI models that process images, video or sensor data, a tokenizer can help map visual inputs like pixels or voxels into a series of discrete tokens.
Models that process audio may turn short clips into spectrograms - visual depictions of sound waves over time that can then be processed as images. Other audio applications may instead focus on capturing the meaning of a sound clip containing speech, and use another kind of tokenizer that captures semantic tokens, which represent language or context data instead of simply acoustic information.
How Are Tokens Used During AI Training? Training an AI model starts with the tokenization of the training dataset.
Based on the size of the training data, the number of tokens can number in the billions or trillions - and, per the pretraining scaling law, the more tokens used for training, the better the quality of the AI model.
As an AI model is pretrained, it's tested by being shown a sample set of tokens and asked to predict the next token. Based on whether or not its prediction is correct, the model updates itself to improve its next guess. This process is repeated until the model learns from its mistakes and reaches a target level of accuracy, known as model convergence.
After pretraining, models are further improved by post-training, where they continue to learn on a subset of tokens relevant to the use case where they'll be deployed. These could be tokens with domain-specific information for an application in law, medicine or business - or tokens that help tailor the model to a specific task, like reasoning, chat or translation. The goal is a model that generates the right tokens to deliver a correct response based on a user's query - a skill better known as inference.
How Are Tokens Used During AI Inference and Reasoning? During inference, an AI receives a prompt - which, depending on the model, may be text, image, audio clip, video, sensor data or even gene sequence - that it translates into a series of tokens. The model processes these input tokens, generates its response as tokens and then translates it to the user's expected format.
Input and output languages can be different, such as in a model that translates English to Japanese, or one that converts text prompts into images.
To understand a complete prompt, AI models must be able to process multiple tokens at once. Many models have a specified limit, referred to as a context window - and different use cases require different context window sizes.
A model that can process a few thousand tokens at once might be able to process a single high-resolution image or a few pages of text. With a context length of tens of thousands of tokens, another model might be able to summarize a whole novel or an hourlong podcast episode. Some models even provide context lengths of a million or more tokens, allowing users to input massive data sources for the AI to analyze.
Reasoning AI models, the latest advancement in LLMs, can tackle more complex queries by treating tokens differently than before. Here, in addition to input and output tokens, the model generates a host of reasoning tokens over minutes or hours as it thinks about how to solve a given problem.
These
More from Nvidia
24/04/2025
Advancing AI requires a full-stack approach, with a powerful foundation of computing infrastructure - including accelerated processors and networking technologi...
24/04/2025
Get the controllers ready and clear the calendar - it's a jam-packed GFN Thu...
23/04/2025
As AI models evolve and adoption grows, enterprises must perform a delicate balancing act to achieve maximum value.
That's because inference - the process ...
23/04/2025
Financial services has long been at the forefront of adopting technological innovations. Today, generative AI and agentic systems are redefining the industry, f...
23/04/2025
AI is rapidly reshaping what's possible on a PC - whether for real-time image generation or voice-controlled workflows. As AI capabilities grow, so does the...
23/04/2025
An AI agent is only as accurate, relevant and timely as the data that powers it....
22/04/2025
Whether at sea, land or in the sky - even outer space - NVIDIA technology is helping research scientists and developers alike explore and understand oceans, wil...
22/04/2025
Traditionally, data centers have relied on air cooling - where mechanical chillers circulate chilled air to absorb heat from servers, helping them maintain opti...
22/04/2025
About 15% of the world's population - over a billion people - are affected by neurological disorders, from commonly known diseases like Alzheimer's and ...
17/04/2025
Gold prospecting in Venezuela has led to a malaria resurgence, but researchers h...
17/04/2025
As the days grow longer and the flowers bloom, GFN Thursday brings a fresh lineup of games to brighten the week.
Dive into thrilling hunts and dark fantasy adv...
16/04/2025
Isomorphic Labs is reimagining the drug discovery process with an AI-first approach. At the heart of this work is a new way of thinking about biology.
Max Jade...
16/04/2025
Editor's note: This post is part of Into the Omniverse, a series focused on how developers, 3D practitioners, and enterprises can transform their workflows ...
15/04/2025
The final days of the AI Mathematical Olympiad's latest competition were a transcontinental relay for team NVIDIA.
Every evening, two team members on oppos...
15/04/2025
Every company and country wants to grow and create economic opportunity - but they need virtually limitless intelligence to do so. Working with its ecosystem pa...
15/04/2025
CoreWeave today became one of the first cloud providers to bring NVIDIA GB200 NV...
14/04/2025
NVIDIA is working with its manufacturing partners to design and build factories that, for the first time, will produce NVIDIA AI supercomputers entirely in the ...
11/04/2025
As a teenager, Bradley Rothenberg was obsessed with CAD: computer-aided design software.
Before he turned 30, Rothenberg channeled that interest into building ...
10/04/2025
NVIDIA this week recognized the contributions of partners in Europe, the Middle ...
10/04/2025
As AI-powered tools continue to evolve, NVIDIA GeForce RTX 50 Series and NVIDIA ...
10/04/2025
Get ready to explore the Deep South. South of Midnight, the action-adventure gam...
09/04/2025
Necessity is the mother of invention. And sometimes, what a person really needs ...
09/04/2025
NVIDIA is collaborating with Google Cloud to bring agentic AI to enterprises seeking to locally harness the Google Gemini family of AI models using the NVIDIA B...
06/04/2025
This National Robotics Week, NVIDIA highlighted the pioneering technologies that...
03/04/2025
The Nintendo Switch 2, unveiled April 2, takes performance to the next level, powered by a custom NVIDIA processor featuring an NVIDIA GPU with dedicated RT Cor...
03/04/2025
Real-time AI is unlocking new possibilities in media and entertainment, improving viewer engagement and advancing intelligent content creation.
At NAB Show, a...
03/04/2025
Editor's note: This post is part of the AI On blog series, which explores the latest techniques and real-world applications of agentic AI, chatbots and copi...
03/04/2025
GeForce NOW isn't fooling around.
This month, 21 games are joining the cloud gaming library of over 2,000 titles. Whether chasing epic adventures, testing ...
02/04/2025
In the latest MLPerf Inference V5.0 benchmarks, which reflect some of the most c...
02/04/2025
AI is rapidly transforming how organizations solve complex challenges.
The early stages of enterprise AI adoption focused on using large language models to cre...
02/04/2025
Video editing workflows are getting a lot more colorful.
Adobe recently announc...
31/03/2025
Advances in physical AI are enabling organizations to embrace embodied AI across...
27/03/2025
A new resident is moving into the cloud - KRAFTON's inZOI joins the 2,000+ games in the GeForce NOW cloud gaming library.
Plus, members can get ready for a...
26/03/2025
The reliability of the electric grid is critical.
From handling demand surges and evolving power needs to preventing infrastructure failures that can cause wil...
25/03/2025
Generative AI is unlocking new capabilities for PCs and workstations, including ...
20/03/2025
The power and utilities sector keeps the lights on for the world's populatio...
20/03/2025
Time to sharpen the blade. GeForce NOW brings a legendary addition to the cloud: Ubisoft's highly anticipated Assassin's Creed Shadows is now available ...
19/03/2025
The roots of many of NVIDIA's landmark innovations - the foundational techno...
19/03/2025
AI has been shaping the media and entertainment industry for decades, from early recommendation engines to AI-driven editing and visual effects automation. Real...
19/03/2025
NVIDIA this week recognized 14 partners leading the way across the Americas for their work advancing agentic and physical AI across industries.
The 2025 Americ...
18/03/2025
The quick-service restaurant industry is a marvel of modern logistics, where spe...
18/03/2025
Global telecommunications networks can support millions of user connections per day, generating more than 3,800 terabytes of data per minute on average.
That m...
18/03/2025
The telecom industry is increasingly embracing AI to deliver seamless connections - even in conditions of poor signal strength - while maximizing sustainability...
18/03/2025
AI agents are transforming work, delivering time and cost savings by helping peo...
18/03/2025
AI deployments thrive on speed, data and scale. That's why NVIDIA is expandi...
18/03/2025
As quantum computers continue to develop, they will integrate with AI supercomputers to form accelerated quantum supercomputers capable of solving some of the w...
18/03/2025
AI is fueling a new industrial revolution - one driven by AI factories.
Unlike traditional data centers, AI factories do more than store and process data - the...
18/03/2025
As generative AI capabilities expand, NVIDIA is equipping developers with the to...
17/03/2025
Under the hood of every AI application are algorithms that churn through data in their own language, one based on a vocabulary of tokens.
Tokens are tiny units...
13/03/2025
What's next in AI is at GTC 2025. Not only the technology, but the people and ideas that are pushing AI forward - creating new opportunities, novel solution...