You Transformed the World,' NVIDIA CEO Tells Researchers Behind Landmark AI Paper
21/03/2024
Everything that we're enjoying today can be traced back to that moment, Huang said to a packed room with hundreds of attendees, who heard him speak with the authors of Attention Is All You Need.
Sharing the stage for the first time, the research luminaries reflected on the factors that led to their original paper, which has been cited more than 100,000 times since it was first published and presented at the NeurIPS AI conference. They also discussed their latest projects and offered insights into future directions for the field of generative AI.
While they started as Google researchers, the collaborators are now spread across the industry, most as founders of their own AI companies.
We have a whole industry that is grateful for the work that you guys did, Huang said.
From L to R: Lukasz Kaiser, Noam Shazeer, Aidan Gomez, Jensen Huang, Llion Jones, Jakob Uszkoreit, Ashish Vaswani and Illia Polosukhin. Origins of the Transformer Model The research team initially sought to overcome the limitations of recurrent neural networks, or RNNs, which were then the state of the art for processing language data.
Noam Shazeer, cofounder and CEO of Character.AI, compared RNNs to the steam engine and transformers to the improved efficiency of internal combustion.
We could have done the industrial revolution on the steam engine, but it would just have been a pain, he said. Things went way, way better with internal combustion.
Now we're just waiting for the fusion, quipped Illia Polosukhin, cofounder of blockchain company NEAR Protocol.
The paper's title came from a realization that attention mechanisms - an element of neural networks that enable them to determine the relationship between different parts of input data - were the most critical component of their model's performance.
We had very recently started throwing bits of the model away, just to see how much worse it would get. And to our surprise it started getting better, said Llion Jones, cofounder and chief technology officer at Sakana AI.
Having a name as general as transformers spoke to the team's ambitions to build AI models that could process and transform every data type - including text, images, audio, tensors and biological data.
That North Star, it was there on day zero, and so it's been really exciting and gratifying to watch that come to fruition, said Aidan Gomez, cofounder and CEO of Cohere. We're actually seeing it happen now.
Packed house at the San Jose Convention Center. Envisioning the Road Ahead Adaptive computation, where a model adjusts how much computing power is used based on the complexity of a given problem, is a key factor the researchers see improving in future AI models.
It's really about spending the right amount of effort and ultimately energy on a given problem, said Jakob Uszkoreit, cofounder and CEO of biological software company Inceptive. You don't want to spend too much on a problem that's easy or too little on a problem that's hard.
A math problem like two plus two, for example, shouldn't be run through a trillion-parameter transformer model - it should run on a basic calculator, the group agreed.
They're also looking forward to the next generation of AI models.
I think the world needs something better than the transformer, said Gomez. I think all of us here hope it gets succeeded by something that will carry us to a new plateau of performance.
You don't want to miss these next 10 years, Huang said. Unbelievable new capabilities will be invented.
The conversation concluded with Huang presenting each researcher with a framed cover plate of the NVIDIA DGX-1 AI supercomputer, signed with the message, You transformed the world.
Jensen presents lead author Ashish Vaswani with a signed DGX-1 cover. There's still time to catch the session replay by registering for a virtual GTC pass - it's free.
To discover the latest in generative AI, watch Huang's GTC keynote address:
LINK: | https://blogs.nvidia.com/blog/gtc-2024-transformer-ai-research-panel-j... |
See more stories from nvidia |
Most recent headlines
04/08/2024
Dalet Appoints Santiago Solanas as CEO to Lead Next Era of Growth and Innovation
Dalet, a leading technology and service provider for media-rich organizations, is excited to announce Santiago Solanas as its new Chief Executive Officer (CEO)....
03/06/2024
Dalet and Veritone Reach Agreement to Distribute, Transact and Monetize Media Archives
Dalet, a leading technology and service provider for media-rich organizations, a...
21/05/2024
Comcast Unveils $15 Price Tag for StreamSaver Bundle
PHILADELPHIA Comcast has released pricing and launch details of its recently announced StreamSaver bundle of Apple TV+, Netflix and Peacock services with the ne...
21/05/2024
TiVo Launches TiVo One Cross-Screen Ad Platform
SAN JOSE, Calif. Xperi's TiVo subsidiary continues to expand the capabilities and reach of its independent media platform with the debut of the TiVo One cr...
21/05/2024
COW Job Listing: Expert Viral Editor Viral content
COW Job Listing: Expert Viral Editor Viral content Brie Clayton May 21, 2024 0 Comments Expert Viral Editor Viral content March 6, 2024COW Job Listi...
21/05/2024
Bulls Face Off Against Cardinals Triple A-Affiliate with Two Fireworks Shows, First Responders Appreciation, Wool E. Bull's Birthday and More
First Tacos & Tallboys Tuesday with Tune Tuesday, return of Winning Wednesday & ...
21/05/2024
Avid partners with Post Super on post production training course
The programme aims to provide those working in post with a comprehensive understanding of their role within the broader post production ecosystem By Jenny Prie...
21/05/2024
Stephen van Rooyen appointed VodafoneZiggo CEO
He joins VodafoneZiggo following the retirement of Jeroen Hoencamp and will be based at the companys offices in Utrecht By Jenny Priestley Published: May 21,...
21/05/2024
TVBEurope May/June 2024 issue out now
Our latest issue hears from France T l visions about their plans for Paris 2024, we also celebrate the winners of our NAB 2024 Best of Show Awards and explore c...
21/05/2024
AMG Chooses Brightline For Frame' Ad Format
LOS ANGELES Allen Media Group (AMG) has launched an instream Frame ad for advertisers on its The Weather Channel, Local Now and HBCU Go connected TV (CTV) app...
21/05/2024
Wisconsin TV Stations Change Call Signs, Channel Numbers
MILWAUKEE Low power TV (LPTV) station operator Roseland Broadcasting has changed the call sign and the operating channel number of its two Wisconsin stations in...
21/05/2024
U.S. FAST Channels Hit Record Numbers
NEW YORK Despite rapid growth in recent years and worries that the FAST channel market may be reaching saturation, a new report from FASTMaster shows that the t...
21/05/2024
Cineverse to Offer Remastered HD, 4K Episodes of the Bob Rosss 'Joy of Painting
LOS ANGELES Hoping to build on the success of The Bob Ross Channel, Cineverse ha...
21/05/2024
DLT Launches 80's Sitcom Flashback Fast Channel
NEW YORK DLT Entertainment has announced the launch of 80's Sitcom Flashback, a FAST Channel exclusively curated to celebrate the comedies that dominated pr...
21/05/2024
SPL Media House Selects Grass Valley's AMPP
MONTREAL Grass Valley has announced that SPL Media House (SPL MH) in Riyadh, Saudi Arabia has deployed Grass Valley's AMPP SaaS Platform deployment in the r...
21/05/2024
Christ Church Deploys Pliant Technologies Intercoms at West Monroe Location
WEST MONROE, La. Pliant Technologies has announced that Christ Church, which has campuses in West Monroe, Ruston, and Sterlington, has deployed its CrewCom Wire...
21/05/2024
Hollyland Announces Pyro S, a New Wireless 4K Video Monitoring System for Filmmakers
Hollyland Announces Pyro S, a New Wireless 4K Video Monitoring System for Filmma...
21/05/2024
Avid Unveils Revolutionary Post-Production Micro-Certifications with New Learning Partner, Post Super
Avid Unveils Revolutionary Post-Production Micro-Certifications with New Learnin...
21/05/2024
MIX's Carpool Casanova Announces Spirit Award Finalists for Season 3
MIX 101.5's Carpool Casanova will wrap season three this Friday, May 24, 2024, but his final location has yet to be determined. The ultimate Spirit Award W...
21/05/2024
Dr. Ray Seol Awarded Grant for Creative Individuals from Mass Cultural Council
Dr. Ray Seol Awarded Grant for Creative Individuals from Mass Cultural Council The grant will expand his Seu Aprendiz project designed to help individuals bui...
21/05/2024
2024-05-21
PARIS Apple and le-de-France Mobilit s today introduced an easy, secure, and private way for customers to add a new Navigo card to Apple Wallet and purchase pa...
21/05/2024
Skeem Saam: Monday's episode, 20 May 2024 [video]
Skeem Saam: Monday's episode, 20 May 2024 [video]Missed an episode of Skeem Saam? No problem! Watch the latest episode of your favourite South African soapi...
21/05/2024
Popular children's TV shows you may have forgotten
Popular children's TV shows you may have forgottenSouth African television channels have entertained us over the years. Here is looking back at some popular...
21/05/2024
Actor Dumisani Dlamini discusses his role in Isitha: The Enemy'
Actor Dumisani Dlamini discusses his role in Isitha: The Enemy'Legendary actor Dumisani Dlamini has landed the character of Nsimbi in e.tv's popular te...
21/05/2024
Tonight on Smoke and Mirrors: General's romantic gesture to Lulu meets with hesitation
Tonight on Smoke and Mirrors: General's romantic gesture to Lulu meets with ...
21/05/2024
New Performance Optimizations Supercharge NVIDIA RTX AI PCs for Gamers, Creators and Developers
NVIDIA today announced at Microsoft Build new AI performance optimizations and i...
21/05/2024
NVIDIA Expands Collaboration With Microsoft to Help Developers Build, Deploy AI Applications Faster
If optimized AI workflows are like a perfectly tuned orchestra - where each comp...
21/05/2024
RT 2FM announces Drive It with The 2 Johnnies' is to come to an end
RT 2FM today announced that the Drive It with The 2 Johnnies is to come to an end on 31 May. Head of RT 2Fm, Dan Healy said: After two very successful years...
21/05/2024
RT launches a series of Animated Shorts on the theme of Home
HOME IS WHERE THE STORY BEGINS RT launches a series of Animated Shorts on the theme of Home Watch: rte.ie/player/kids Among these short animations is Envelo...
21/05/2024
A Superbloom of Updates in the May Studio Driver Gives Fresh Life to Content Creation
Editor's note: This post is part of our In the NVIDIA Studio series, which c...
21/05/2024
May 20, 2024
New method to reveal what drives brain diseases Scripps Research scientists develop CRISPR screen technology to determine disease mechanism from tissues with ac...
20/05/2024
Masters of Reinvention commissioned for Yesterday & UKTV Play
22nd May 2024 UKTV has commissioned Masters of Reinvention (6x60) for its leading factual channel Yesterday and free streaming service UKTV Play, to be produced...
20/05/2024
The Tuba Thieves Asks What It Means to Listen
PARK CITY, UTAH - JANUARY 22: The cast and crew of The Tuba Thieves pose during the 2023 Sundance Film Festival The Tuba Thieves premiere at Prospector Squa...
20/05/2024
Spotify CLASSICS: The 100 Greatest Hip-Hop Songs of the Streaming Era
Reviews by Carl Chery, Kemet High, and Adrian Covert In February, we launched Spotify CLASSICS-our first-ever program to celebrate catalog music. Our inaugur...
20/05/2024
Hamish Blake to host Alone Australia Season 2: The Reunion on 29 May
Hamish Blake to host Alone Australia Season 2: The Reunion on 29 May 20 May, 2024 Media releases As the season nears its end, SBS's hit series cements ...
20/05/2024
Poolside Gossip
Created by showrunner Abe Sylvia, the Apple TV+ comedic drama Palm Royale navigates the tale of one womans ambitious journey to make it amongst the upper crust,...
20/05/2024
SPL Media House selects Grass Valley AMPP SaaS Platform to Distribute Live Saudi Pro League Football Matches Globally
AMPP's cloud enabled rapid deployment of live production workflows with dist...
20/05/2024
EditShare celebrates anniversaries at CABSAT with innovations, integrations and implementations
EditShare Celebrates Anniversaries at CABSAT with Innovations, Integrations and ...
20/05/2024
Sony-Apollo's Bid for Paramount Gets Serious
A bid from Sony Pictures Entertainment and Apollo Global Management to acquire Paramount gained steam late last week, according to a report in the New York Time...
20/05/2024
Vizrt Unveils Viz Libro 8.3 With New Social Media Cropping Function
BERGEN, Norway Vizrt has released Viz Libero 8.3 with the announcement that its latest software version enables users to crop video to various aspect ratios fro...
20/05/2024
The Vampire Next Door Created with Pocket Cinema Camera 6K and DaVinci Resolve
The Vampire Next Door Created with Pocket Cinema Camera 6K and DaVinci Resolve Brie Clayton May 20, 2024 0 Comments Blackmagic Design today announced ...
20/05/2024
Tom Wills To Retire After Nearly 50 Years at WJXT Jacksonville
Tom Wills, long-running anchor at WJXT Jacksonville, Florida, has announced his retirement. He started at WJXT in 1975 and has anchored early-evening news for o...
20/05/2024
AMC Networks Promotes Emily Gotto to Senior VP at Shudder
AMC Networks said it promoted Emily Gotto to senior VP of acquisition and production for Shudder, the company's horror-themed streaming service....
20/05/2024
Paramount Names Jonathan Bingaman President, Domestic Multi-Platform Licensing & Distribution
Jonathan Bingaman has been named president, domestic multi-platform licensing & ...
20/05/2024
DLT Entertainment Gets FAST Laughs With 80's Sitcom Flashback Channel
Producer and distributor DLT Entertainment said it launched a free ad-supported streaming television (FAST) channel called 80's Sitcom Flashback....
20/05/2024
Zeam Media Names Sandie Trombert CMO
Zeam Media said it hired Sandie Trombert as chief marketing officer....
20/05/2024
Meet the VFX producer
Angus Berryman, VFX producer at UNIT studio tells TVBEurope how perseverance paid off as his career progressed By Matthew Corrigan Published: May 20, 2024 ...
20/05/2024
Sony signs NDA with Paramount Global but is reticent about earlier plan
The New York Times said the move was a significant step forward in their effort to court Paramount By Matthew Corrigan Published: May 20, 2024 Updated: May...
20/05/2024
CBC HR Team Getting Ballpark Staff Ready to Play Ball
A pair of stalwart members of the Capitol Broadcasting's dream team have been getting the company's two Coastal Plain League teams ready to launch their...
20/05/2024
Tonight on Skeem Saam: Ntswaki sneaks Toby into the house, but sadly for her, Leeto arrives unannounced
Tonight on Skeem Saam: Ntswaki sneaks Toby into the house but sadly for her, Lee...