Of GTC's 900+ sessions, the most wildly popular was a conversation hosted by NVIDIA founder and CEO Jensen Huang with seven of the authors of the legendary research paper that introduced the aptly named transformer - a neural network architecture that went on to change the deep learning landscape and enable today's era of generative AI.
Everything that we're enjoying today can be traced back to that moment, Huang said to a packed room with hundreds of attendees, who heard him speak with the authors of Attention Is All You Need.
Sharing the stage for the first time, the research luminaries reflected on the factors that led to their original paper, which has been cited more than 100,000 times since it was first published and presented at the NeurIPS AI conference. They also discussed their latest projects and offered insights into future directions for the field of generative AI.
While they started as Google researchers, the collaborators are now spread across the industry, most as founders of their own AI companies.
We have a whole industry that is grateful for the work that you guys did, Huang said.
From L to R: Lukasz Kaiser, Noam Shazeer, Aidan Gomez, Jensen Huang, Llion Jones, Jakob Uszkoreit, Ashish Vaswani and Illia Polosukhin. Origins of the Transformer Model The research team initially sought to overcome the limitations of recurrent neural networks, or RNNs, which were then the state of the art for processing language data.
Noam Shazeer, cofounder and CEO of Character.AI, compared RNNs to the steam engine and transformers to the improved efficiency of internal combustion.
We could have done the industrial revolution on the steam engine, but it would just have been a pain, he said. Things went way, way better with internal combustion.
Now we're just waiting for the fusion, quipped Illia Polosukhin, cofounder of blockchain company NEAR Protocol.
The paper's title came from a realization that attention mechanisms - an element of neural networks that enable them to determine the relationship between different parts of input data - were the most critical component of their model's performance.
We had very recently started throwing bits of the model away, just to see how much worse it would get. And to our surprise it started getting better, said Llion Jones, cofounder and chief technology officer at Sakana AI.
Having a name as general as transformers spoke to the team's ambitions to build AI models that could process and transform every data type - including text, images, audio, tensors and biological data.
That North Star, it was there on day zero, and so it's been really exciting and gratifying to watch that come to fruition, said Aidan Gomez, cofounder and CEO of Cohere. We're actually seeing it happen now.
Packed house at the San Jose Convention Center. Envisioning the Road Ahead Adaptive computation, where a model adjusts how much computing power is used based on the complexity of a given problem, is a key factor the researchers see improving in future AI models.
It's really about spending the right amount of effort and ultimately energy on a given problem, said Jakob Uszkoreit, cofounder and CEO of biological software company Inceptive. You don't want to spend too much on a problem that's easy or too little on a problem that's hard.
A math problem like two plus two, for example, shouldn't be run through a trillion-parameter transformer model - it should run on a basic calculator, the group agreed.
They're also looking forward to the next generation of AI models.
I think the world needs something better than the transformer, said Gomez. I think all of us here hope it gets succeeded by something that will carry us to a new plateau of performance.
You don't want to miss these next 10 years, Huang said. Unbelievable new capabilities will be invented.
The conversation concluded with Huang presenting each researcher with a framed cover plate of the NVIDIA DGX-1 AI supercomputer, signed with the message, You transformed the world.
Jensen presents lead author Ashish Vaswani with a signed DGX-1 cover. There's still time to catch the session replay by registering for a virtual GTC pass - it's free.
To discover the latest in generative AI, watch Huang's GTC keynote address:
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
04/07/2026
April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...
01/06/2026
January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026
Throughout the week, Dolby brings to life the latest innovatio...
13/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
13/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
13/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
13/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
13/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
13/05/2026
Creative software developer Foundry today announced the latest developments on Nuke Stage. A purpose-built application for end-to-end virtual production and in-...
12/05/2026
Beyond the Hype: A Strategic Post-Hoc Analysis of NAB 2026 If NAB Show 2026 had an underlying theme, it was a quiet, industry-wide pivot from the high-energy sp...
12/05/2026
Guntermann and Drunck (G&D), a Panoptec Technologies Group company, and CT Square, led by Chandresh Shah, have announced a joint venture to distribute G&D and V...
12/05/2026
With 30 days until the start of the FIFA World Cup 2026, Telemundo, the exclusive Spanish-language home of the tournament in the United States, has announced th...
12/05/2026
The NHL has announced the return of Stanley Pup for its third consecutive year, a 90-minute special featuring adoptable rescue dogs competing on a miniature rin...
12/05/2026
NBCUniversal presented its 2026 Upfront to advertisers at Radio City Music Hall, detailing upcoming programming across NBC, Peacock, Bravo, and Versant properti...
12/05/2026
FOX Sports has announced funding for the Fandom and Social Connection Initiative at Harvard Kennedy School's Shorenstein Center on Media, Politics, and Publ...
12/05/2026
TNDV and Live Media, both divisions of Live Media Group, supported live broadcast coverage around NCAA Final Four weekend in Indianapolis, including the March M...
12/05/2026
The European Football Alliance (EFA) has announced a content distribution agreement with Fubo Sports Network, the free ad-supported streaming TV (FAST) channel ...
12/05/2026
For the first time, Spanish-speaking fans in the U.S. will have two separate tel...
12/05/2026
CP Communications led a comprehensive spectrum management initiative on behalf of Churchill Downs during Kentucky Derby week, coordinating RF assets across the ...
12/05/2026
LiveU has announced a strategic partnership with DRONERESPONDERS, a 501(c)3 non-...
12/05/2026
Open Broadcast Systems has announced that BMC TV, a specialist in IP transport of broadcast content, has selected the Open Broadcast Systems 5G Flyaway solution...
12/05/2026
NEP Europe, part of NEP Group, has announced it will deliver broadcast solutions...
12/05/2026
Grass Valley has announced continued collaboration with Ravensbourne University ...
12/05/2026
Stats Perform has announced the launch of Opta Pulse, an AI-assisted video creation and distribution platform for leagues, rights holders, and broadcasters. The...
12/05/2026
FOX Sports has announced a collaboration with Sesame Workshop to integrate Sesame Street characters into FOX Sports' FIFA World Cup 2026 programming. Conten...
12/05/2026
To date, NHL Productions has produced 19 broadcasts with commentary in American Sign Language
NHL in ASL (American Sign Language) may be just one show, but the...
12/05/2026
Google's Brian Albert: creators, athletes, highlights, nostalgia, second-scr...
12/05/2026
A still from Past Lives by Celine Song, an official selection of the Premieres program at the 2023 Sundance Film Festival. (Courtesy of Sundance Institute | p...
12/05/2026
Spotify is where fans and artists come together, turning discovery into somethin...
12/05/2026
Features patented Marco-MMC clocking technology
Black Lion Audio's latest release combines the company's expertise in clocking with their renowned p...
12/05/2026
New Track Panel, sequencer upgrades & more
Following their recent public beta release, Reason Studios have announced the full release of Reason 14. With the...
12/05/2026
One month to go! SBS reveals expansive FIFA World Cup 2026 lineup beyond the pi...
12/05/2026
Rohde & Schwarz presents its advanced solutions for power electronics testing at...
12/05/2026
aconnic AG (ISIN: DE000A0LBKW6), Munich, has developed a modified fund raising p...
12/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/05/2026
Tyrell Corporation, specialists in high-end live sports and entertainment broadcasts, was tasked with delivering compelling broadcast coverage of premier equest...
12/05/2026
Registration is now open for IBC2026 as the global media, entertainment and technology community prepares to converge on the RAI Amsterdam from 11 14 September ...
12/05/2026
Ross Video, a global leader in live video production technology, will present its latest innovations and integrated production workflows at BroadcastAsia 2026, ...
12/05/2026
500 selected leaders from around the world across start-ups, corporates, and venture capital. Over 50bn in assets under management among attending investors, a...
12/05/2026
CVP, one of Europe's leading suppliers of professional video and broadcast solutions, has appointed Randy Koeman as Sales Manager, Netherlands, marking anot...
12/05/2026
Taiwan Television has expanded its HD content ingest workflow capabilities by investing in the PlayBox Neo Suite - a leading multi-channel multi-server UHD/HD/S...
12/05/2026
Jigsaw24 is strengthening its media team with the appointment of Tom Laker as Client Director and Ashish Chauhan as Project Manager. These strategic hires bring...
12/05/2026
Broadcast playout leader showcases hybrid IP infrastructure and workflow automation with Videoland deployment as regional benchmark...
12/05/2026
dzjinius, the all-in-one production management platform for the media and entertainment industry, will showcase its centralised approach to production workflows...
12/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
Of GTC's 900+ sessions, the most wildly popular was a conversation hosted by NVIDIA founder and CEO Jensen Huang with seven of the authors of the legendary research paper that introduced the aptly named transformer - a neural network architecture that went on to change the deep learning landscape and enable today's era of generative AI.
