Sony Pixel Power calrec Sony

You Transformed the World,' NVIDIA CEO Tells Researchers Behind Landmark AI Paper

21/03/2024

Of GTC's 900+ sessions, the most wildly popular was a conversation hosted by NVIDIA founder and CEO Jensen Huang with seven of the authors of the legendary research paper that introduced the aptly named transformer - a neural network architecture that went on to change the deep learning landscape and enable today's era of generative AI.

Everything that we're enjoying today can be traced back to that moment, Huang said to a packed room with hundreds of attendees, who heard him speak with the authors of Attention Is All You Need.

Sharing the stage for the first time, the research luminaries reflected on the factors that led to their original paper, which has been cited more than 100,000 times since it was first published and presented at the NeurIPS AI conference. They also discussed their latest projects and offered insights into future directions for the field of generative AI.

While they started as Google researchers, the collaborators are now spread across the industry, most as founders of their own AI companies.

We have a whole industry that is grateful for the work that you guys did, Huang said.

From L to R: Lukasz Kaiser, Noam Shazeer, Aidan Gomez, Jensen Huang, Llion Jones, Jakob Uszkoreit, Ashish Vaswani and Illia Polosukhin. Origins of the Transformer Model The research team initially sought to overcome the limitations of recurrent neural networks, or RNNs, which were then the state of the art for processing language data.

Noam Shazeer, cofounder and CEO of Character.AI, compared RNNs to the steam engine and transformers to the improved efficiency of internal combustion.

We could have done the industrial revolution on the steam engine, but it would just have been a pain, he said. Things went way, way better with internal combustion.

Now we're just waiting for the fusion, quipped Illia Polosukhin, cofounder of blockchain company NEAR Protocol.

The paper's title came from a realization that attention mechanisms - an element of neural networks that enable them to determine the relationship between different parts of input data - were the most critical component of their model's performance.

We had very recently started throwing bits of the model away, just to see how much worse it would get. And to our surprise it started getting better, said Llion Jones, cofounder and chief technology officer at Sakana AI.

Having a name as general as transformers spoke to the team's ambitions to build AI models that could process and transform every data type - including text, images, audio, tensors and biological data.

That North Star, it was there on day zero, and so it's been really exciting and gratifying to watch that come to fruition, said Aidan Gomez, cofounder and CEO of Cohere. We're actually seeing it happen now.

Packed house at the San Jose Convention Center. Envisioning the Road Ahead Adaptive computation, where a model adjusts how much computing power is used based on the complexity of a given problem, is a key factor the researchers see improving in future AI models.

It's really about spending the right amount of effort and ultimately energy on a given problem, said Jakob Uszkoreit, cofounder and CEO of biological software company Inceptive. You don't want to spend too much on a problem that's easy or too little on a problem that's hard.

A math problem like two plus two, for example, shouldn't be run through a trillion-parameter transformer model - it should run on a basic calculator, the group agreed.

They're also looking forward to the next generation of AI models.

I think the world needs something better than the transformer, said Gomez. I think all of us here hope it gets succeeded by something that will carry us to a new plateau of performance.

You don't want to miss these next 10 years, Huang said. Unbelievable new capabilities will be invented.

The conversation concluded with Huang presenting each researcher with a framed cover plate of the NVIDIA DGX-1 AI supercomputer, signed with the message, You transformed the world.

Jensen presents lead author Ashish Vaswani with a signed DGX-1 cover. There's still time to catch the session replay by registering for a virtual GTC pass - it's free.

To discover the latest in generative AI, watch Huang's GTC keynote address:
LINK: https://blogs.nvidia.com/blog/gtc-2024-transformer-ai-research-panel-j...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

01/04/2026

DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION

January 4 2026, 18:00 (PST) DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION Douyin Users Can Now Create And Share Videos With Stun...

30/03/2026

NAB 2026: Manifold to Demonstrate 400GbE COTS FPGA Support

Manifold Technologies, a Germany-based provider of cloud infrastructure for live broadcast production, will demonstrate support for 400GbE COTS FPGA accelerator...

30/03/2026

NAB 2026: Boland Communications Introduces QD-OLED Series Monitors

Boland Communications will introduce its QD4K315HDR10, a 31.5-inch QD-OLED monitor, at NAB Show 2026 (Booth C3519, April 18-22). The company is also introducing...

30/03/2026

NAB 2026: PTZOptics to Showcase Move 4K and Horizon Platform

PTZOptics will demonstrate its Move 4K PTZ cameras and Horizon web-based control platform at NAB Show 2026 (Booth N1902). Move 4K with Horizon is now available...

30/03/2026

NAB 2026: Net Insight to Showcase Updated Nimbra Edge

Net Insight will demonstrate the next version of Nimbra Edge, its orchestration and control layer for live media services across multi-domain environments, at N...

30/03/2026

NAB 2026: Appear to Showcase Live Production Processing

Appear ASA will exhibit at NAB Show 2026 (Booth W1531, April 19-22, Las Vegas). The company completed an IPO in November 2025. Our customer-first approach is ...

30/03/2026

NAB 2026: Harmonic Announces New Live Sports Streaming Capabilities

Harmonic has announced new capabilities for its sports streaming platform, covering multiview, programmatic advertising, in-stream advertising, and content wate...

30/03/2026

NAB 2026: Ateme to Showcase GenAI, Agentic AI, and Streaming

Ateme (Booth W1723) will demonstrate broadcast, streaming, and AI-driven media workflow solutions at NAB Show 2026. GenAI and Agentic AI Ateme will demonstrat...

30/03/2026

NAB 2026: Bitmovin's Player Web X Adds Advertising Support, Vertical Video, and Proprietary ABR Algorithm

Bitmovin has announced new capabilities for Player Web X, its web video player, ...

30/03/2026

NAB 2026: Brazil's Minister of Communications and FCC Commissioner To Speak

The 2026 NAB Show (April 18-22, exhibits April 19-22, Las Vegas Convention Center) will host Brazil's Minister of Communications, Frederico de Siqueira Filh...

30/03/2026

NAB 2026: EVS To Showcase Expanded Live Production Ecosystem

EVS will exhibit at NAB Show 2026 (Booth N1841), highlighting new products and updates across its live production portfolio, including the debut of T-Motion med...

30/03/2026

NAB 2026: Solid State Logic To Demonstrate Expanded Virtual System T Platform

Solid State Logic will demonstrate its virtualized System T platform at NAB Show 2026 (Booth C6907). Demonstrations will include the VTE1 virtual DSP engine, ne...

30/03/2026

NAB 2026: Globecast To Showcase Managed Media Services Approach

Globecast will exhibit at NAB Show 2026 (Booth W3335), highlighting its hybrid service model spanning satellite, IP, fiber, and cloud. The company will demonst...

30/03/2026

NAB 2026: IP Showcase Returns as IPMX Moves to Deployment

The Alliance for IP Media Solutions (AIMS), Advanced Media Workflow Association (AMWA), and the Video Services Forum (VSF) have announced that the IP Showcase w...

30/03/2026

NAB 2026: BBright To Demonstrate Single-Stream ST 2110 Playout

At NAB Show 2026 BBright will present a demonstration of its One Stream for the World concept, showing how a single ST 2110 playout stream can simultaneously ...

30/03/2026

NAB 2026: OpenDrives To Demonstrate New Storage and Edge Products

OpenDrives will demonstrate new products at NAB Show 2026, with two locations in the West Hall: a pod (W3443-E) in the Sports Business Hub and a cabana at W1158...

30/03/2026

Behind the Mic: Amazon Prime Hosts 90th Master Tournament With Host Terry Gannon

Behind The Mic provides a roundup of recent news regarding on-air talent, including new deals, departures, and assignments compiled from press releases and repo...

30/03/2026

Op-Ed: Preparing for Agentic AI in Live Sports

The economics of live sports streaming have changed. New rights models, cloud production tools, and lower-cost distribution have made it possible for high schoo...

30/03/2026

Movimento Strings from Sonora Cinematic

MPE-capable chamber strings library announced Alongside their collection of Kontakt instruments, Sonora Cinematic have been steadily introducing a series of...

30/03/2026

UJAM release Groovemate Latigo

Latin-inspired percussion instrument announced Built on a newly developed engine and interface, UJAM's latest instrument has been designed to create Lat...

30/03/2026

Best Service launch Desert Winds

Latest Eduardo Tarilonte collaboration announced The latest library to join Best Service's ever-growing range includes four solo wind instruments that c...

30/03/2026

SOS Music Creators Survey 2026

We want to hear from you! Complete our SOS Quick Survey and enter the prize draw for a chance to win one of three $50 Amazon vouchers! Sound On Sound carri...

30/03/2026

Government of Canada Selects MAS for Strategic Tanker Fleet Sustainment

CC-330 Husky. 2024 Eric Desbiens Photography. Used with permission for the announcement and related communications. No residual rights....

30/03/2026

L3Harris Included in MDA Space Solution for RCN ISTAR Program

L3Harris Technologies will provide WESCAM CMX -8 sensor systems for integration on new Uncrewed Aircraft Systems from MDA Space, enhancing the Royal Canadian Na...

30/03/2026

EVS to Debut T-Motion Robotics at 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

30/03/2026

SDVI To Feature New Rally Media Supply Chain Management Enhancements

Share Copy link Facebook X Linkedin Bluesky Email...

30/03/2026

Boland Communications Introduces QD4K315HDR10 QD-OLED Series Monitors

Share Copy link Facebook X Linkedin Bluesky Email...

30/03/2026

Mileto Tecnologia accelerates streaming growth with Synam...

Synamedia today announced that Mileto Tecnologia, one of Brazil's largest pay-TV operators, has chosen the Synamedia Go platform to support its rapid OTT ex...

30/03/2026

FOR-A's Software-Defined, AI-Powered Development Advances with Nippon TV and NVIDIA Technology

FOR-A's Software-Defined, AI-Powered Development Advances with Nippon TV and...

30/03/2026

Give Your Astrophotography REAL Depth - After Effects Tutorial

Give Your Astrophotography REAL Depth - After Effects Tutorial Graham Quince March 30, 2026 0 Comments In this tutorial, I talk you through the full w...

30/03/2026

Alfalite returns to NAB Show alongside FOR-A, showcasing LED solutions for broadcast and mission-critical environments

Alfalite returns to NAB Show alongside FOR-A, showcasing LED solutions for broad...

30/03/2026

WideOrbit Announces New Name, New Features for Flagship Radio Automation Software

Introducing WO Aurora WideOrbit is pleased to introduce WO Aurora, a new name fo...

30/03/2026

Sky announces changes to its Diversity Advisory Council

Sky welcomes Karen Blackett CBE to its DAC and thanks Baroness Prashar and Ndidi Okezie as they step down after five yearsMonday 30 March 2026 Sky announces ch...

30/03/2026

Netflix Announces the Reunion for Love is Blind: Sweden Season 3 - Premiering April 2

Back to All News Netflix Announces the Reunion for Love is Blind: Sweden Season...

30/03/2026

Netflix unveils new images from the second season of 'Gangs of Galicia'

Back to All News Netflix unveils new images from the second season of Gangs of Galicia Entertainment 30 March 2026 GlobalSpain Link copied to clipboard Do...

30/03/2026

The Latest on Netflix Anime, Unveiled at AnimeJapan 2026

Back to All News The Latest on Netflix Anime, Unveiled at AnimeJapan 2026 Entertainment 30 March 2026 GlobalJapan Link copied to clipboard From romance an...

30/03/2026

KBRO Leverages Harmonic's Fiber-on-Demand Solution for Network Upgrades

Leading Taiwan Broadband Operator Drives Fiber Deeper with Harmonic SAN JOSE, Calif. - March 30, 2026 - Harmonic (NASDAQ: HLIT) today announced that KBRO, a lea...

30/03/2026

Top 10 Reasons Government Meetings Need Transcriptions (and Why It Matters More Than Ever)

Tyngsboro, Mass., March 30, 2026 - City councils, county commissions, school boa...

29/03/2026

Victory+ Turns to Creator Economy, Bringing In Popular Women's Sports Influencer Coach Jackie J to Host Live NWSL Alt-Cast

Cloud-based production, real-time engagement, and creator-driven storytelling ai...

28/03/2026

Harrison launch LiveTrax 3

Now features DiGiCo console integration Harrison's live recording and virtual soundcheck software has just reached its third major version, which among ...

28/03/2026

Sonora Cinematic launch Movimento Strings

MPE-capable chamber strings library announced Alongside their collection of Kontakt instruments, Sonora Cinematic have been steadily introducing a series of...

28/03/2026

Globecast Reimagines Managed Media Services for a Hybrid...

Globecast, the leading provider of broadcast, media and entertainment managed services, will showcase its reimagined approach to media operations at the 2026 NA...

28/03/2026

Fubo Inks Deals for More Baseball RSNs

Share Copy link Facebook X Linkedin Bluesky Email...

27/03/2026

SVG GameDay, Ep. 9: Chicago Cubs' Chris Simonson - Flying the W at Wrigley Field

In-venue and creative video staffers at the professional and collegiate level ha...

27/03/2026

Comcast Business Powers 2026 THE PLAYERS Championship Network and Broadcast Infrastructure

Comcast Business deployed network infrastructure for the 2026 PLAYERS Championsh...

27/03/2026

CS live Equips New OB Van With Riedel MediorNet, hi Control System, and Artist Intercom

Czech production company CS live has equipped its newest outside broadcast van w...