If you want to ride the next big wave in AI, grab a transformer.
They're not the shape-shifting toy robots on TV or the trash-can-sized tubs on telephone poles.
So, What's a Transformer Model? A transformer model is a neural network that learns context and thus meaning by tracking relationships in sequential data like the words in this sentence.
Transformer models apply an evolving set of mathematical techniques, called attention or self-attention, to detect subtle ways even distant data elements in a series influence and depend on each other.
First described in a 2017 paper from Google, transformers are among the newest and one of the most powerful classes of models invented to date. They're driving a wave of advances in machine learning some have dubbed transformer AI.
Stanford researchers called transformers foundation models in an August 2021 paper because they see them driving a paradigm shift in AI. The sheer scale and scope of foundation models over the last few years have stretched our imagination of what is possible, they wrote.
What Can Transformer Models Do? Transformers are translating text and speech in near real-time, opening meetings and classrooms to diverse and hearing-impaired attendees.
They're helping researchers understand the chains of genes in DNA and amino acids in proteins in ways that can speed drug design.
Transformers, sometimes called foundation models, are already being used with many data sources for a host of applications. Transformers can detect trends and anomalies to prevent fraud, streamline manufacturing, make online recommendations or improve healthcare.
People use transformers every time they search on Google or Microsoft Bing.
The Virtuous Cycle of Transformer AI Any application using sequential text, image or video data is a candidate for transformer models.
That enables these models to ride a virtuous cycle in transformer AI. Created with large datasets, transformers make accurate predictions that drive their wider use, generating more data that can be used to create even better models.
Stanford researchers say transformers mark the next stage of AI's development, what some call the era of transformer AI. Transformers made self-supervised learning possible, and AI jumped to warp speed, said NVIDIA founder and CEO Jensen Huang in his keynote address this week at GTC.
Transformers Replace CNNs, RNNs Transformers are in many cases replacing convolutional and recurrent neural networks (CNNs and RNNs), the most popular types of deep learning models just five years ago.
Indeed, 70 percent of arXiv papers on AI posted in the last two years mention transformers. That's a radical shift from a 2017 IEEE study that reported RNNs and CNNs were the most popular models for pattern recognition.
No Labels, More Performance Before transformers arrived, users had to train neural networks with large, labeled datasets that were costly and time-consuming to produce. By finding patterns between elements mathematically, transformers eliminate that need, making available the trillions of images and petabytes of text data on the web and in corporate databases.
In addition, the math that transformers use lends itself to parallel processing, so these models can run fast.
Transformers now dominate popular performance leaderboards like SuperGLUE, a benchmark developed in 2019 for language-processing systems.
How Transformers Pay Attention Like most neural networks, transformer models are basically large encoder/decoder blocks that process data.
Small but strategic additions to these blocks (shown in the diagram below) make transformers uniquely powerful.
A look under the hood from a presentation by Aidan Gomez, one of eight co-authors of the 2017 paper that defined transformers. Transformers use positional encoders to tag data elements coming in and out of the network. Attention units follow these tags, calculating a kind of algebraic map of how each element relates to the others.
Attention queries are typically executed in parallel by calculating a matrix of equations in what's called multi-headed attention.
With these tools, computers can see the same patterns humans see.
Self-Attention Finds Meaning For example, in the sentence:
She poured water from the pitcher to the cup until it was full.
We know it refers to the cup, while in the sentence:
She poured water from the pitcher to the cup until it was empty.
We know it refers to the pitcher.
Meaning is a result of relationships between things, and self-attention is a general way of learning relationships, said Ashish Vaswani, a former senior staff research scientist at Google Brain who led work on the seminal 2017 paper.
Machine translation was a good vehicle to validate self-attention because you needed short- and long-distance relationships among words, said Vaswani.
Now we see self-attention is a powerful, flexible tool for learning, he added.
How Transformers Got Their Name Attention is so key to transformers the Google researchers almost used the term as the name for their 2017 model. Almost.
Attention Net didn't sound very exciting, said Vaswani, who started working with neural nets in 2011.
.Jakob Uszkoreit, a senior software engineer on the team, came up with the name Transformer.
I argued we were transforming representations, but that was just playing semantics, Vaswani said.
The Birth of Transformers In the paper for the 2017 NeurIPS conference, the Google team described their transformer and the accuracy records it set for machine translation.
Thanks to a basket of techniques, they trained their model in just 3.5 days on eight NVIDIA GPUs, a small fraction of the time and cost of training prior models. They trained it on datasets with up to a billion pairs of words.
It was an intense three-month sprint to the paper submissio
Most recent headlines
09/11/2025
Dalet today announced a transformative leap forward for media operations: Agentic Artificial Intelligence (AI) that unifies the Dalet ecosystem under one natura...
15/10/2025
Across the world, early-career reporters are interrogating systems that shape daily life, from mortuaries and classrooms to the corridors of legal education and...
15/10/2025
St. Louis Blues Pay Tribute to Ozzy Osbourne in New Pregame Open VideoThe NHL franchise worked with production firms 2WEI and 4th Floor CreativeBy Mark J Burns,...
15/10/2025
Cobalt Digital, G3DVu, Proxima Vision, and Telos Alliance Renew Premier Sponsors...
15/10/2025
By Paige Bethmann
Before my grandmother passed away in 2019, I went to visit her in the hospital where she'd been for a few weeks. When I arrived, the nurs...
15/10/2025
Two years ago, Spotify set out to grow the entire publishing industry by making ...
15/10/2025
People turn to DJ for a listening experience that feels personal, dynamic, and j...
15/10/2025
Las personas recurren a DJ para vivir una experiencia de escucha personal, din m...
15/10/2025
Rock legend Bruce Springsteen and luminary actor Jeremy Allen White came togethe...
15/10/2025
At Spotify, we're all about supporting rising talent and helping them connec...
15/10/2025
New SBS Documentary Series THE CANCER KILLERS Premieres 4 November on SBS & SBS ...
15/10/2025
SBS celebrates the Festivals of Lights with bold, illuminating stories across th...
15/10/2025
One of humanity's most profound questions continues: Is there any place in our galaxy suitable for life beyond Earth? NASA intends to find out, with the hel...
15/10/2025
eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({ content_source:......
15/10/2025
Scality, a global leader in cyber-resilient storage software for the AI era, today announced the advancement of its comprehensive AI ecosystem certification pro...
15/10/2025
The Hollywood Professional Association (HPA) today unveiled key highlights of the 2026 HPA Tech Retreat, scheduled for Feb. 15 19 at the Westin Rancho Mirage Go...
15/10/2025
At NAB Show New York 2025 (Stand 244), Interra Systems will showcase the future of media QC, monitoring, and captioning, highlighting its award-winning, AI- and...
15/10/2025
Grass Valley today announced that RMC BFM, part of the CMA-CGM Group and France's third-largest private media group, has chosen Grass Valley's productio...
15/10/2025
Appear, the global leader in live production technology, will demonstrate how its powerful hardware and software innovations are shaping the future of hybrid, s...
15/10/2025
Rakim to Join Berklee's Hip-Hop Hall of Fame at Signature Series Concert The rap icon becomes the latest inductee, following luminaries such as Roxanne Sh...
15/10/2025
Berklee Brings Live Music to the Head of the Charles Student artists will perform on multiple riverfront stages during the 60th anniversary of the world's...
15/10/2025
CobbTV, the government access television channel for Cobb County, Ga. recently acquired an automated playout system from Pebble, a global provider of automation...
15/10/2025
OSLO, Norway Appear, the global leader in live production technology, will demonstrate how its powerful hardware and software innovations are shaping the future...
15/10/2025
NEW YORK The NBA is making major changes to the NBA App and NBA TV as it takes control of them from TNT Sports, which has long managed the league's digital ...
15/10/2025
SAN MATEO, Calif. In what promises to be a major expansion of interactive features and personalized content on the DirecTV platform, the operator and Glance hav...
15/10/2025
SAN JOSE, Calif. Roku has launched changes to its user interface (UI) that the streaming platform says will better showcase original programming on the platform...
15/10/2025
LOS ANGELES Software-defined data storage and data services provider OpenDrives has elevated Alex Dunfey to chief technology officer, responsible for driving th...
15/10/2025
Abu Dhabi, UAE October 15, 2025: Space42 (ADX: SPACE42), the UAE-based AI-powe...
15/10/2025
Sheldon Nichols and Will Trickett to find and prepare more classic cars for budget conscious would-be owners from Wiser Films
UKTV have recommissioned the clas...
15/10/2025
Wednesday 15 October 2025
Nick Cave
JPEG (275 KB)
Explore the creative univer...
15/10/2025
Wednesday 15 October 2025
From breaking up with your partner to quitting your job - new polling shows some messages are just too scary to send
Over a fifth (...
15/10/2025
Rohde & Schwarz unveils compact MXO 3 oscilloscopes with 4 and 8 channels: Advan...
15/10/2025
Back to All News
Revenge Series The Resurrected' Captivates Audiences Across Asia
Entertainment
15 October 2025
GlobalTaiwan
Link copied to clipboard
...
15/10/2025
Back to All News
Bringing the Best in VFX and Virtual Production Together as Eyeline
Jeffrey Shapiro
CEO, Eyeline
Business
15 October 2025
GlobalCanadaInd...
15/10/2025
Deployed by Astound Business Solutions, Harmonic's Primary Distribution Solution Ensures Outstanding Video Quality and Seamless Ad Insertion at the Edge
SA...
15/10/2025
Collaboration extends a trusted relationship as RMC BFM invests in scalable, fut...
15/10/2025
Series coming in 2026 stars Tom Vaughan-Lawlor, Justine Mitchell and Jason O'Mara released today
RT today released first look images of new comedy-drama ...
14/10/2025
SVG Europe Summit 2025: All Sessions Now Available to Watch on SVG PLAYNetworking event that preceded IBC2025 shone a light on elite live sports innovation acro...
14/10/2025
SVG Sit-Down: Author Rich Podolsky on Writing Madden & Summerall: How They Revo...
14/10/2025
SVG All-Stars: Michael Reiners, Coordinating Producer, FloRacingThe Illinois State grad steers a vast schedule of motorsports events at tracks across the countr...
14/10/2025
Content protection: Getting the right management for your DRM By Neal Romanek
Friday, October 10, 2025 - 10:11
Print This Story
Eluvio power the EPCR'...
14/10/2025
As League Takes Over Ops, NBA TV and NBA App Add 60 Games, Weekday Studio Show, ...
14/10/2025
Time and effort: World's largest student-led broadcast prepares to go On Air...
14/10/2025
(L-R) Guest, Kimberly Robinson Jones, Geeta Gandbhir, Pamela Dias, and Takema Ro...
14/10/2025
Lossless ist jetzt mit Spotify Premium verf gbar.
Verlustfreies Audio war eine...
14/10/2025
La qualit Lossless est disponible sur Spotify Premium.
Le format sans perte de...
14/10/2025
For the seventh edition of Spotify and FC Barcelona's artist jersey series, ...
14/10/2025
Spotify is committed to bringing the best listening experience to all our users, and that includes parents and families. That's why we're expanding mana...
14/10/2025
Since its debut, the Spotify Original podcast Caso 63 has been more than just a story; it's been a cultural sensation. The science fiction thriller captivat...
If you want to ride the next big wave in AI, grab a transformer.
