Sony Pixel Power calrec Sony

You Transformed the World,' NVIDIA CEO Tells Researchers Behind Landmark AI Paper

21/03/2024

Of GTC's 900+ sessions, the most wildly popular was a conversation hosted by NVIDIA founder and CEO Jensen Huang with seven of the authors of the legendary research paper that introduced the aptly named transformer - a neural network architecture that went on to change the deep learning landscape and enable today's era of generative AI.

Everything that we're enjoying today can be traced back to that moment, Huang said to a packed room with hundreds of attendees, who heard him speak with the authors of Attention Is All You Need.

Sharing the stage for the first time, the research luminaries reflected on the factors that led to their original paper, which has been cited more than 100,000 times since it was first published and presented at the NeurIPS AI conference. They also discussed their latest projects and offered insights into future directions for the field of generative AI.

While they started as Google researchers, the collaborators are now spread across the industry, most as founders of their own AI companies.

We have a whole industry that is grateful for the work that you guys did, Huang said.

From L to R: Lukasz Kaiser, Noam Shazeer, Aidan Gomez, Jensen Huang, Llion Jones, Jakob Uszkoreit, Ashish Vaswani and Illia Polosukhin. Origins of the Transformer Model The research team initially sought to overcome the limitations of recurrent neural networks, or RNNs, which were then the state of the art for processing language data.

Noam Shazeer, cofounder and CEO of Character.AI, compared RNNs to the steam engine and transformers to the improved efficiency of internal combustion.

We could have done the industrial revolution on the steam engine, but it would just have been a pain, he said. Things went way, way better with internal combustion.

Now we're just waiting for the fusion, quipped Illia Polosukhin, cofounder of blockchain company NEAR Protocol.

The paper's title came from a realization that attention mechanisms - an element of neural networks that enable them to determine the relationship between different parts of input data - were the most critical component of their model's performance.

We had very recently started throwing bits of the model away, just to see how much worse it would get. And to our surprise it started getting better, said Llion Jones, cofounder and chief technology officer at Sakana AI.

Having a name as general as transformers spoke to the team's ambitions to build AI models that could process and transform every data type - including text, images, audio, tensors and biological data.

That North Star, it was there on day zero, and so it's been really exciting and gratifying to watch that come to fruition, said Aidan Gomez, cofounder and CEO of Cohere. We're actually seeing it happen now.

Packed house at the San Jose Convention Center. Envisioning the Road Ahead Adaptive computation, where a model adjusts how much computing power is used based on the complexity of a given problem, is a key factor the researchers see improving in future AI models.

It's really about spending the right amount of effort and ultimately energy on a given problem, said Jakob Uszkoreit, cofounder and CEO of biological software company Inceptive. You don't want to spend too much on a problem that's easy or too little on a problem that's hard.

A math problem like two plus two, for example, shouldn't be run through a trillion-parameter transformer model - it should run on a basic calculator, the group agreed.

They're also looking forward to the next generation of AI models.

I think the world needs something better than the transformer, said Gomez. I think all of us here hope it gets succeeded by something that will carry us to a new plateau of performance.

You don't want to miss these next 10 years, Huang said. Unbelievable new capabilities will be invented.

The conversation concluded with Huang presenting each researcher with a framed cover plate of the NVIDIA DGX-1 AI supercomputer, signed with the message, You transformed the world.

Jensen presents lead author Ashish Vaswani with a signed DGX-1 cover. There's still time to catch the session replay by registering for a virtual GTC pass - it's free.

To discover the latest in generative AI, watch Huang's GTC keynote address:
LINK: https://blogs.nvidia.com/blog/gtc-2024-transformer-ai-research-panel-j...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

07/05/2026

IAB Tech Lab Introduces Standardized Signals For CTV Ad Portfolio

Share Copy link Facebook X Linkedin Bluesky Email...

07/05/2026

Cobalt Digital Wins Two Future Best of Show Awards at 202...

Cobalt Digital Wins Two Future Best of Show Awards at 2026 NAB Show Manufacturer Recognized by TV Tech and TVBEurope for Innovation in signal processing Cobal...

07/05/2026

FOR-A delivers creativity practicality and technical exce...

Software and hardware platforms, AI power and user-friendliness on show...

07/05/2026

Intinor brings advanced IP transport diagnostics and HDR...

Intinor will demonstrate its latest technical enhancements for the Direkt series at BroadcastAsia 2026. With a continued focus on reliable contribution and remo...

07/05/2026

Bitmovin Powers Global Film Streaming Platform MUBI

Bitmovin has announced that MUBI has chosen Bitmovin as its cloud VOD encoding partner, replacing MUBI's legacy on premises encoding setup to improve scalab...

07/05/2026

Meet Graduates from Berklees Class of 2026

Meet Graduates from Berklees Class of 2026 Members of this years graduating class reflect on their proudest moments at Berklee and look ahead to whats next. ...

07/05/2026

Convergence is Now Established Reality,' Says NAB 2026 Report

Share Copy link Facebook X Linkedin Bluesky Email...

07/05/2026

Interra Systems to Showcase Best in Class QC Monitoring a...

At BroadcastAsia 2026, Interra Systems will demonstrate its latest innovations in automated quality control (QC), real-time monitoring, and captioning. The comp...

07/05/2026

PlayBox Neo shines a spotlight on delivering Secure Scala...

At this year's Broadcast Asia, PlayBox Neo is set to unveil recent innovations across its PlayBox Neo Suite and integrated range of broadcast media solution...

07/05/2026

BFBS selects Synamedia to deliver New Digital Platform fo...

Strategic three-year partnership marks delivery phase of BFBS' next-generation media vision Media organisation and military charity BFBS (British Forces Br...

07/05/2026

Amagi Unlocks New Opportunities for CTV Performance with...

Amagi, the agentic industry cloud platform for unified broadcast, streaming, and monetization, today announced the availability of its In-Content Ads offering v...

07/05/2026

Lightware Introduces Rack Taurus For Clean Centralized AV...

Lightware Visual Engineering has developed Rack Taurus for rack-based environments such as education and corporate training spaces. It enables clean and simple ...

07/05/2026

Jigsaw24 Showcases End-to-End Media Workflows at MPTS 202...

Jigsaw24, the UK's leading media equipment supplier and systems integrator, will showcase its expanded end-to-end workflow capabilities at MPTS 2026, signal...

07/05/2026

Ease Live Granted US Patent for Technology Powering Perfe...

Ease Live, an Evertz company, today announced that it has been granted a US Patent for its proprietary graphical overlay technology, the foundation for interact...

07/05/2026

Leader prepares advanced Test and Measurement showcase fo...

Test & measurement innovator Leader Electronics of Europe will present an extensive range of Leader, PHABRIX and LeaderPhabrix test and measurement solutions at...

07/05/2026

LiveU Signals a New Era of Live Broadcasting at Broadcast...

At Broadcast Asia 2026 (Stand 5H3 8), LiveU is redefining live production with its most powerful IP-video EcoSystem to date built to help broadcasters and con...

07/05/2026

Colorfront Streaming Server Awarded Trusted Partner Network Gold Logo Certification

January 8, 2024 Colorfront (colorfront.com), a leader in high-performance, on-s...

07/05/2026

Colorfront Unleashes New Opportunities for Content Owners Plus Other Ground-Breaking Visual Experiences at Nab 2024

March 20, 2024 NAB 2024, Las Vegas - Colorfront (colorfront.com), the multi-awa...

07/05/2026

Colorfront Delivers Innovations for HDR Cinema at Cinemacon 2025

April 1, 2025 CINEMACON, APRIL 1, 2025 - Colorfront (colorfront.com), the multi-award-winning developer of high-performance dailies/transcoding/streaming syste...

07/05/2026

Colorfront to Showcase Advanced HDR Cinema Mastering Solutions at CineEurope 2025

June 15, 2025 Colorfront (colorfront.com), an Academy and Emmy Award-winning de...

07/05/2026

Colorfront at ICTA Barcelona Cinema Technology Summit

June 15, 2025 Colorfront participated in the ICTA Barcelona Cinema Technology Summit on Sunday, June 15, 2025. Held at The Phenomena Experience, the event feat...

07/05/2026

Colorfront Does IBC 2024 Live From Stockholm!

July 1, 2025 Colorfront (colorfront.com), the multi-award-winning developer of high-performance dailies/transcoding/streaming systems for motion pictures, OTT,...

07/05/2026

Colorfront Transkoder Delivers a Masterful Performance at Annapurna Studios

July 1, 2025 Passion and dedication will take you places. Come with us on a short trip to the heart of India, where Annapurna Studios is living-up to the inspi...

07/05/2026

Colorfront Opens a New Chapter in Color Tools For Cinema & Television

July 3, 2025 Colorfront (colorfront.com), the multi-award-winning developer of high-performance dailies/transcoding/streaming systems for motion pictures, OTT,...

07/05/2026

Colorfront at IBC 2025: Advanced Tools Make Easy Work of Critical Mastering Tasks and More!

September 1, 2025 IBC 2025, Amsterdam - Colorfront (colorfront.com) - the Acade...

07/05/2026

Colorfront Introduces Colorfront Immersive Utility, a New Mac App for Creating Apple Immersive Video

April 17, 2026 LOS ANGELES - April 17, 2026 - Colorfront today announced Colorf...

07/05/2026

Colorfront Delivers Even More AI Automation Power and Extends Technology Partnerships with Dolby Apple

April 23, 2026 NAB 2026, Las Vegas - the Academy and Emmy Award-winning develop...

07/05/2026

CNN Founder Ted Turner Dies at 87

Share Copy link Facebook X Linkedin Bluesky Email...

07/05/2026

FCC Urges Appeals Court to Toss Challenges to Nexstar-Tegna Deal

Share Copy link Facebook X Linkedin Bluesky Email...

07/05/2026

Carr Announces FCC Staff Promotions

Share Copy link Facebook X Linkedin Bluesky Email...

07/05/2026

Recreating the 1974 Doctor Who Time Tunnel in After Effects

Recreating the 1974 Doctor Who Time Tunnel in After Effects Graham Quince May 6, 2026 0 Comments The Time Tunnel from Doctor Who titles is one of th...

07/05/2026

Well known Irish faces pushed to the edge in new series of Uncharted with Ray Goggins

Well known Irish faces pushed to the edge in new series of Uncharted with Ray G...

07/05/2026

Powerful four generation story A Traveller Family airs on RT One and RT Player this Monday night

RT will premiere A Traveller Family, a compelling new documentary exploring the...

07/05/2026

Linked and Loaded: Gaijin Single Sign-On Now Available on GeForce NOW

Less typing, more tanking. Faster logins mean more time in the gaming action - and this week provides GeForce NOW members with a smoother path straight into th...

06/05/2026

Wisycom RF Solutions Support Gravity Medias Live Cycling and Marathon Broadcasts

Gravity Media Chief RF Communications Engineer Glenn Willems uses Wisycom RF over Fiber and wireless solutions across major cycling events and international mar...

06/05/2026

Sennheiser Spectera Module Now Available in Bitfocus Companion and Buttons

A Sennheiser Spectera module is now available in Bitfocus Companion and Buttons, enabling direct integration of Spectera with the two software platforms. The mo...

06/05/2026

Ted Turner, Cable Television Pioneer, Sports Broadcasting Hall of Famer, Dead at 87

Ted Turner, the visionary media entrepreneur whose appetite for disruption helpe...

06/05/2026

FIFA World Cup 2026: Peacock Launches Visin de Campo (aka Pitchside Live), Will Stream All 104 Matches in Spanish

Peacock is going all-in on the beautiful game - streaming all 104 FIFA World Cup...

06/05/2026

NoiseWorks Audio launch VoiceAssist Basic, Standard & Advanced

New pricing tiers for vocal/dialogue restoration tool NoiseWorks Audio's AI-powered vocal and dialogue processing plug-in is now available in three diff...

06/05/2026

RME TotalMix FX 2 now available

Popular mixing & routing software overhauled Following a recent public beta test, RME have launched the final release version of the powerful mixing and rou...

06/05/2026

Focusrite: Designing The ISA C8X Audio Interface

New SOS Video Feature Focusrites ISA C8X is a milestone product that brings together the companys analogue heritage and their expertise in digital audio. Yo...

06/05/2026

L3Harris to Boost Polish Navy Combat Power with Advanced Ship System

Polands Miecznik-class frigates are part of the largest contract in Polish shipbuilding history. (Image Credit: PGZ Stocznia Wojenna)...

06/05/2026

FCC's Anna Gomez Urges Rigorous Review of Paramount-WBD Merger

Share Copy link Facebook X Linkedin Bluesky Email...

06/05/2026

Riedel Ups Marc Engroff to CFO, Shifts Frank Eischet to Group COO

Share Copy link Facebook X Linkedin Bluesky Email...

06/05/2026

Amagi Launches In-Content Ads' to Attract More CTV Advertisers

Share Copy link Facebook X Linkedin Bluesky Email...