You Transformed the World,' NVIDIA CEO Tells Researchers Behind Landmark AI Paper

21/03/2024

Of GTC's 900+ sessions, the most wildly popular was a conversation hosted by NVIDIA founder and CEO Jensen Huang with seven of the authors of the legendary research paper that introduced the aptly named transformer - a neural network architecture that went on to change the deep learning landscape and enable today's era of generative AI.

Everything that we're enjoying today can be traced back to that moment, Huang said to a packed room with hundreds of attendees, who heard him speak with the authors of Attention Is All You Need.

Sharing the stage for the first time, the research luminaries reflected on the factors that led to their original paper, which has been cited more than 100,000 times since it was first published and presented at the NeurIPS AI conference. They also discussed their latest projects and offered insights into future directions for the field of generative AI.

While they started as Google researchers, the collaborators are now spread across the industry, most as founders of their own AI companies.

We have a whole industry that is grateful for the work that you guys did, Huang said.

From L to R: Lukasz Kaiser, Noam Shazeer, Aidan Gomez, Jensen Huang, Llion Jones, Jakob Uszkoreit, Ashish Vaswani and Illia Polosukhin. Origins of the Transformer Model The research team initially sought to overcome the limitations of recurrent neural networks, or RNNs, which were then the state of the art for processing language data.

Noam Shazeer, cofounder and CEO of Character.AI, compared RNNs to the steam engine and transformers to the improved efficiency of internal combustion.

We could have done the industrial revolution on the steam engine, but it would just have been a pain, he said. Things went way, way better with internal combustion.

Now we're just waiting for the fusion, quipped Illia Polosukhin, cofounder of blockchain company NEAR Protocol.

The paper's title came from a realization that attention mechanisms - an element of neural networks that enable them to determine the relationship between different parts of input data - were the most critical component of their model's performance.

We had very recently started throwing bits of the model away, just to see how much worse it would get. And to our surprise it started getting better, said Llion Jones, cofounder and chief technology officer at Sakana AI.

Having a name as general as transformers spoke to the team's ambitions to build AI models that could process and transform every data type - including text, images, audio, tensors and biological data.

That North Star, it was there on day zero, and so it's been really exciting and gratifying to watch that come to fruition, said Aidan Gomez, cofounder and CEO of Cohere. We're actually seeing it happen now.

Packed house at the San Jose Convention Center. Envisioning the Road Ahead Adaptive computation, where a model adjusts how much computing power is used based on the complexity of a given problem, is a key factor the researchers see improving in future AI models.

It's really about spending the right amount of effort and ultimately energy on a given problem, said Jakob Uszkoreit, cofounder and CEO of biological software company Inceptive. You don't want to spend too much on a problem that's easy or too little on a problem that's hard.

A math problem like two plus two, for example, shouldn't be run through a trillion-parameter transformer model - it should run on a basic calculator, the group agreed.

They're also looking forward to the next generation of AI models.

I think the world needs something better than the transformer, said Gomez. I think all of us here hope it gets succeeded by something that will carry us to a new plateau of performance.

You don't want to miss these next 10 years, Huang said. Unbelievable new capabilities will be invented.

The conversation concluded with Huang presenting each researcher with a framed cover plate of the NVIDIA DGX-1 AI supercomputer, signed with the message, You transformed the world.

Jensen presents lead author Ashish Vaswani with a signed DGX-1 cover. There's still time to catch the session replay by registering for a virtual GTC pass - it's free.

To discover the latest in generative AI, watch Huang's GTC keynote address:

LINK:	https://blogs.nvidia.com/blog/gtc-2024-transformer-ai-research-panel-j...
	See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

01/04/2026

DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION

January 4 2026, 18:00 (PST) DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION Douyin Users Can Now Create And Share Videos With Stun...

18/03/2026

Net Insight Names Larissa GrnerMeeus CPO

Newly named chief product officer (CPO), Larissa G rner Meeus will return to Net Insight on May 4. Larissa G rner Meeus will become CPO of Net Insight on May 4...

18/03/2026

SVG Europe's The Football Summit 2026: Select Sessions Now Available to Watch on SVG PLAY

SVG Europe's The Football Summit 2026 explored how the sports broadcasting i...

18/03/2026

SVG's SportsTech@NABShow Blog Goes Live as Countdown to Vegas Heats Up

The 2026 NAB Show kicks off one month from today and SVG is once again set to cover the show from every angle. SVG's SportsTech@NAB Show Blog is now live - ...

18/03/2026

The Premiere of Dead Lover Showcases Grave Robbing and Sex With a Giant Finger

(L-R) The cast and crew of Dead Lover at The Ray Theater for its premiere at the 2025 Sundance Film Festival. (Photo by Robin Marshall/Shutterstock for Sundan...

18/03/2026

Music Row Piano from Wiltone Productions

Yamaha C7 captured in Nashville Wiltone Productions have announced the release of Music Row Piano, a deeply sampled Yamaha C7 piano library that's been ...

18/03/2026

Techivation introduce T-Warmer Mk2

Bass-enhancement plug-in upgraded Along with a steady stream of new releases, Techivation have recently been revisiting some of the older plug-ins in their ...

18/03/2026

Tonal Balance Control 3 from iZotope

Mix-reference plug-in overhauled iZotope's powerful mix-referencing plug-in has just reached its third major version, and now boasts a new capture proce...

18/03/2026

Toontrack Transistor Organ EKX

Latest EZKeys 2 expansion arrives Toontrack's staggering collection of EZKeys 2 expansions has grown once again, and the latest instalment delivers a on...

18/03/2026

Jess Ho explores the politics of food in new SBS Audio podcast For The Culture

Jess Ho explores the politics of food in new SBS Audio podcast For The Culture 18 March, 2026 Media releases New SBS Audio podcast For The Culture, hosted ...

18/03/2026

Future-ready broadcasts with ATSC 3.0 and enhanced services from Rohde & Schwarz at NAB 2026

Future-ready broadcasts with ATSC 3.0 and enhanced services from Rohde & Schwarz...

18/03/2026

LCTWS 2026

London Calling - When The Industry Convened to Help Streaming Find its MoJo In this blog, Laura Rognoni reflects on key discussions from the Connected TV World...

18/03/2026

SMPTE Unveils 2026 NAB Show Educational Presentations

SMPTE Unveils 2026 NAB Show Educational Presentations Brie Clayton March 18, 2026 0 Comments SMPTE , the home of media professionals, technologists, a...

18/03/2026

Auditel Ad Campaign Shot on Blackmagic PYXIS 12K

Auditel Ad Campaign Shot on Blackmagic PYXIS 12K Brie Clayton March 18, 2026 0 Comments LED wall virtual production blends 12K open gate acquisition w...

18/03/2026