Sony Pixel Power calrec Sony

You Transformed the World,' NVIDIA CEO Tells Researchers Behind Landmark AI Paper

21/03/2024

Of GTC's 900+ sessions, the most wildly popular was a conversation hosted by NVIDIA founder and CEO Jensen Huang with seven of the authors of the legendary research paper that introduced the aptly named transformer - a neural network architecture that went on to change the deep learning landscape and enable today's era of generative AI.

Everything that we're enjoying today can be traced back to that moment, Huang said to a packed room with hundreds of attendees, who heard him speak with the authors of Attention Is All You Need.

Sharing the stage for the first time, the research luminaries reflected on the factors that led to their original paper, which has been cited more than 100,000 times since it was first published and presented at the NeurIPS AI conference. They also discussed their latest projects and offered insights into future directions for the field of generative AI.

While they started as Google researchers, the collaborators are now spread across the industry, most as founders of their own AI companies.

We have a whole industry that is grateful for the work that you guys did, Huang said.

From L to R: Lukasz Kaiser, Noam Shazeer, Aidan Gomez, Jensen Huang, Llion Jones, Jakob Uszkoreit, Ashish Vaswani and Illia Polosukhin. Origins of the Transformer Model The research team initially sought to overcome the limitations of recurrent neural networks, or RNNs, which were then the state of the art for processing language data.

Noam Shazeer, cofounder and CEO of Character.AI, compared RNNs to the steam engine and transformers to the improved efficiency of internal combustion.

We could have done the industrial revolution on the steam engine, but it would just have been a pain, he said. Things went way, way better with internal combustion.

Now we're just waiting for the fusion, quipped Illia Polosukhin, cofounder of blockchain company NEAR Protocol.

The paper's title came from a realization that attention mechanisms - an element of neural networks that enable them to determine the relationship between different parts of input data - were the most critical component of their model's performance.

We had very recently started throwing bits of the model away, just to see how much worse it would get. And to our surprise it started getting better, said Llion Jones, cofounder and chief technology officer at Sakana AI.

Having a name as general as transformers spoke to the team's ambitions to build AI models that could process and transform every data type - including text, images, audio, tensors and biological data.

That North Star, it was there on day zero, and so it's been really exciting and gratifying to watch that come to fruition, said Aidan Gomez, cofounder and CEO of Cohere. We're actually seeing it happen now.

Packed house at the San Jose Convention Center. Envisioning the Road Ahead Adaptive computation, where a model adjusts how much computing power is used based on the complexity of a given problem, is a key factor the researchers see improving in future AI models.

It's really about spending the right amount of effort and ultimately energy on a given problem, said Jakob Uszkoreit, cofounder and CEO of biological software company Inceptive. You don't want to spend too much on a problem that's easy or too little on a problem that's hard.

A math problem like two plus two, for example, shouldn't be run through a trillion-parameter transformer model - it should run on a basic calculator, the group agreed.

They're also looking forward to the next generation of AI models.

I think the world needs something better than the transformer, said Gomez. I think all of us here hope it gets succeeded by something that will carry us to a new plateau of performance.

You don't want to miss these next 10 years, Huang said. Unbelievable new capabilities will be invented.

The conversation concluded with Huang presenting each researcher with a framed cover plate of the NVIDIA DGX-1 AI supercomputer, signed with the message, You transformed the world.

Jensen presents lead author Ashish Vaswani with a signed DGX-1 cover. There's still time to catch the session replay by registering for a virtual GTC pass - it's free.

To discover the latest in generative AI, watch Huang's GTC keynote address:
LINK: https://blogs.nvidia.com/blog/gtc-2024-transformer-ai-research-panel-j...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

06/05/2026

NAB Launches Weekly Podcast on Local Broadcast Policy

Share Copy link Facebook X Linkedin Bluesky Email...

06/05/2026

Mavis Launches Mavis Studio iPad For Media Production

Share Copy link Facebook X Linkedin Bluesky Email...

06/05/2026

Narrative Entertainment partners with Encompass to provid...

Narrative Entertainment has partnered with Encompass to deliver high-quality subtitling of its Great! network content using the Altitude Intelligence AI assiste...

06/05/2026

SipRadius extends its seamless creation and connectivity...

SipRadius, widely recognized for making content processing and connectivity secure and seamless, is proud to launch a dramatic new approach to AI content creati...

06/05/2026

Big Blue Marble at ANGA COM - TV as a Service in the spot...

When the broadband and media industry gathers at ANGA COM in Cologne from May 19 to 21, Big Blue Marble will be at the forefront. The international broadcast an...

06/05/2026

Cinegy makes its MPTS debut with software-defined televis...

Cinegy GmbH, a leading developer of software-defined television technology, is proud to exhibit at MPTS for the first time. Visitors to the stand will discover ...

06/05/2026

Val Jeanty Receives 2026 Doris Duke Artist Award

Val Jeanty Receives 2026 Doris Duke Artist Award Jeanty, a composer, percussionist, and turntablist, is the fourth Berklee recipient of the prestigious award ...

06/05/2026

Zeiss Launches CinCraft LensCore

Share Copy link Facebook X Linkedin Bluesky Email...

06/05/2026

Gomez Urges Rigorous FCC Review of Paramount-WBD Merger

Share Copy link Facebook X Linkedin Bluesky Email...

06/05/2026

Wisycom Solves Extreme RF Challenges Across Miles of Live...

When live cycling races and international marathons stretch for miles across cities and countryside, there is no margin for RF failure in live broadcast. As Chi...

06/05/2026

ZEISS CinCraft LensCore - Cinema Lens Looks for Compositi...

Oberkochen/Germany, May 5, 2026 ZEISS announces the launch of CinCraft LensCore, a novel solution for creating physically based cinematic lens looks for visual...

06/05/2026

NVIDIA Spectrum-X - the Open, AI-Native Ethernet Fabric - Sets the Standard for Gigascale AI, Now With MRC

The race to build the world's most powerful AI factories demands networking ...

06/05/2026

May 05, 2026

How changes to proteins can alter drug interactions for new precision therapies Scripps Research team maps how chemical modifications to proteins affect drug bi...

05/05/2026

Lessons from fragile contexts on responding to disinformation

Experts from the world of academia, tech, business, politics and media convened for a Thomson Talks at the Cambridge Disinformation Summit in April. It's th...

05/05/2026

Samsung Galaxy S26 Ultra Phone Cameras Bring New Excitement to Street League Skateboarding

Three phones were hardwired for power and transmission to the truck; camera feat...

05/05/2026

Case Study: How Zaki Rose Rebuilt Its Production Infrastructure, and What It Means for Sports Content Creators

The creative studio behind campaigns for the NBA, Fanatics Sportsbook & Casino, ...

05/05/2026

Nielsen Co-Viewing Pilot Shows Average 4% Viewership Increase for February Live Events

Nielsen has announced results from a co-viewing pilot program covering February&...

05/05/2026

Nippon TV and FOR-A Win NAB Product of the Year and Future Best of Show Awards for viztrick AiDi

viztrick AiDi, an on-device AI solution developed by Nippon TV, delivered global...

05/05/2026

ARRI Introduces Omnibar LED Linear Fixture for Film, Live Entertainment, and Content Creation

ARRI has announced Omnibar, a battery-powered, IP65-rated multi-color LED linear...

05/05/2026

France Tlvisions Becomes First Broadcaster to Deploy Imagine Communications SNP-XS

Imagine Communications has announced that France T l visions is the first broadc...

05/05/2026

WNBA Announces Historic Canadian Media Rights Agreement with Bell Media

The Women's National Basketball Association (WNBA) and Bell Media today announced a multiyear agreement to broadcast and stream WNBA games in Canada beginni...

05/05/2026

Save the Date: SVG Remote Production Forum Heads to WBD's Techwood Studios in Atlanta on Sept. 23-24

SVG is proud to announce Warner Bros. Discovery's Techwood Studios in Atlant...

05/05/2026

Look Who's Talking: ESPN Integrates New Automated Commentator-ID Technology Into Scorebar Graphic for UFL Coverage

With no operator required, AutoMic workflow automates talent identification on U...

05/05/2026

Return Flight: How Live Broadcast Drones Died - and Were Reborn - on the Ski Slopes of Northern Italy

A crash in 2015 set the industry back, but this winter proved that drones are he...

05/05/2026

RADAR Spotlights the Next Generation of Asian Artists, From Indonesia to Taiwan

Another year, and more proof that Asia continues to shape some of the world's most exciting new sounds. This year's RADAR artists draw from deep local r...

05/05/2026

Spotify and ACL Music Fest Team Up to Give Fans a Personalized Experience for 2026

The Austin City Limits Music Fest 2026 lineup just dropped, and this year, Spoti...

05/05/2026

Bjooks to launch Beat Gems Kickstarter

New drum machine book campaign incoming Bjooks have announced that during Superbooth 2026, they will be launching a Kickstarter campaign to fund the product...

05/05/2026

Native Instruments release Komplete 26

Flagship all-in-one production bundle updated The latest version of Native Instruments' flagship virtual instrument and plug-in bundle has just been ann...

05/05/2026

Rohde & Schwarz to host RF Testing Innovations Forum 2026, helping design engineers elevate their RF expertise

Rohde & Schwarz to host RF Testing Innovations Forum 2026, helping design engine...

05/05/2026

L3Harris Provides Key Technologies for Newly Commissioned Navy Submarines

L3Harris provides communications, electronic warfare, sensors and mission systems that enable Virginia-class submarine crews to operate with confidence in conte...

05/05/2026

AgileTV consolidates its strength in 2025: EBITDA and cash conversion increase thanks to revenue growth and operational efficiency

The company grew by 7.6% in net revenue and 16.3% in EBITDA, achieving a 33% inc...

05/05/2026

Gray Media Closes Purchase of 10 Allen Media Group Stations

Share Copy link Facebook X Linkedin Bluesky Email...

05/05/2026

Dang Ly Joins Operative as Chief Product Officer

Share Copy link Facebook X Linkedin Bluesky Email...

05/05/2026

CIMM, TVB Release Local TV Currency Measurement Guidelines

Share Copy link Facebook X Linkedin Bluesky Email...

05/05/2026

ARRI Introduces Omnibar LED Linear Fixture

Share Copy link Facebook X Linkedin Bluesky Email...

05/05/2026

France Televisions Continues ST 2110 Migration With Imagi...

Project Marks First Major Broadcast Deployment of Latest Addition to SNP Lineup Imagine Communications today announced that France T l visions is the first br...

05/05/2026

Shotoku Broadcast Systems Wins 2026 NAB Show Product of t...

Shotoku Broadcast Systems Wins 2026 NAB Show Product of the Year Award Shotoku Broadcast Systems announced today that its Swoop range of robotic cranes has be...

05/05/2026

DigitalGlues creativespace Intelligence Wins Futures Best...

DigitalGlue's creative.space Intelligence Wins Future's Best of Show Award, Presented by TV Tech creative.space Intelligence (CSI), part of the creativ...

05/05/2026

Zixi Showcases Next-Generation Live Video Workflows and M...

Zixi, a leader in live video delivery and workflow orchestration, will showcase next-generation broadcast workflows at the Media Production and Technology Show ...

05/05/2026

Stingr marks its launch with a new approach to second-screen interactivity

Stingr marks its launch with a new approach to second-screen interactivity Brie Clayton May 5, 2026 0 Comments Huge leap forward in revenues and engag...

05/05/2026

Shotoku Broadcast Systems Wins 2026 NAB Show Product of the Year Award

Shotoku Broadcast Systems Wins 2026 NAB Show Product of the Year Award Brie Clayton May 5, 2026 0 Comments Shotoku Broadcast Systems announced today tha...

05/05/2026

DHD to Promote Latest Advances in Audio Production at MPT...

Following a successful NAB Show in Las Vegas, DHD will promote examples from its wide range of broadcast-quality audio production equipment at the May 13th-14th...

05/05/2026

LucidLink Redefines Cloud Media Workflows at MPTS 2026

LucidLink today announced its programme for MPTS 2026, where it will exhibit at Stand M59 at Olympia London, 13 to 14 May. The company will showcase its latest ...

05/05/2026

Limecraft Announces Version 2026-3 of its Cloud-Based Tel...

Limecraft today announces the release of Limecraft 2026.3, the third platform update in its 2026 release cycle. Limecraft is an AI-powered production platform t...

05/05/2026

Stingr marks its launch with a new approach to second-scr...

Huge leap forward in revenues and engagement...

05/05/2026

Broadcast Solutions strengthens CTO Office for technical...

Broadcast Solutions, a leading system integrator and provider of innovative solutions for the broadcast media industry, has taken another significant step in st...