You Transformed the World,' NVIDIA CEO Tells Researchers Behind Landmark AI Paper
21/03/2024
Everything that we're enjoying today can be traced back to that moment, Huang said to a packed room with hundreds of attendees, who heard him speak with the authors of Attention Is All You Need.
Sharing the stage for the first time, the research luminaries reflected on the factors that led to their original paper, which has been cited more than 100,000 times since it was first published and presented at the NeurIPS AI conference. They also discussed their latest projects and offered insights into future directions for the field of generative AI.
While they started as Google researchers, the collaborators are now spread across the industry, most as founders of their own AI companies.
We have a whole industry that is grateful for the work that you guys did, Huang said.
From L to R: Lukasz Kaiser, Noam Shazeer, Aidan Gomez, Jensen Huang, Llion Jones, Jakob Uszkoreit, Ashish Vaswani and Illia Polosukhin. Origins of the Transformer Model The research team initially sought to overcome the limitations of recurrent neural networks, or RNNs, which were then the state of the art for processing language data.
Noam Shazeer, cofounder and CEO of Character.AI, compared RNNs to the steam engine and transformers to the improved efficiency of internal combustion.
We could have done the industrial revolution on the steam engine, but it would just have been a pain, he said. Things went way, way better with internal combustion.
Now we're just waiting for the fusion, quipped Illia Polosukhin, cofounder of blockchain company NEAR Protocol.
The paper's title came from a realization that attention mechanisms - an element of neural networks that enable them to determine the relationship between different parts of input data - were the most critical component of their model's performance.
We had very recently started throwing bits of the model away, just to see how much worse it would get. And to our surprise it started getting better, said Llion Jones, cofounder and chief technology officer at Sakana AI.
Having a name as general as transformers spoke to the team's ambitions to build AI models that could process and transform every data type - including text, images, audio, tensors and biological data.
That North Star, it was there on day zero, and so it's been really exciting and gratifying to watch that come to fruition, said Aidan Gomez, cofounder and CEO of Cohere. We're actually seeing it happen now.
Packed house at the San Jose Convention Center. Envisioning the Road Ahead Adaptive computation, where a model adjusts how much computing power is used based on the complexity of a given problem, is a key factor the researchers see improving in future AI models.
It's really about spending the right amount of effort and ultimately energy on a given problem, said Jakob Uszkoreit, cofounder and CEO of biological software company Inceptive. You don't want to spend too much on a problem that's easy or too little on a problem that's hard.
A math problem like two plus two, for example, shouldn't be run through a trillion-parameter transformer model - it should run on a basic calculator, the group agreed.
They're also looking forward to the next generation of AI models.
I think the world needs something better than the transformer, said Gomez. I think all of us here hope it gets succeeded by something that will carry us to a new plateau of performance.
You don't want to miss these next 10 years, Huang said. Unbelievable new capabilities will be invented.
The conversation concluded with Huang presenting each researcher with a framed cover plate of the NVIDIA DGX-1 AI supercomputer, signed with the message, You transformed the world.
Jensen presents lead author Ashish Vaswani with a signed DGX-1 cover. There's still time to catch the session replay by registering for a virtual GTC pass - it's free.
To discover the latest in generative AI, watch Huang's GTC keynote address:
LINK: | https://blogs.nvidia.com/blog/gtc-2024-transformer-ai-research-panel-j... |
See more stories from nvidia |
Most recent headlines
04/08/2024
Dalet Appoints Santiago Solanas as CEO to Lead Next Era of Growth and Innovation
Dalet, a leading technology and service provider for media-rich organizations, is excited to announce Santiago Solanas as its new Chief Executive Officer (CEO)....
03/06/2024
Dalet and Veritone Reach Agreement to Distribute, Transact and Monetize Media Archives
Dalet, a leading technology and service provider for media-rich organizations, a...
09/05/2024
SailGP Expanding LiveLineFX Graphics Platform to Other Sports
LONDON SailGP, which sponsors high-speed boat racing events worldwide says it is developing its LiveLineFX sports graphics platform for use in other sports....
09/05/2024
Disney, WBD to Offer Streaming Bundle Starting this Summer
BURBANK, Calif. Starting this summer, U.S. consumers will be able to purchase a new streaming bundle offered by Disney and Warner Bros. Discovery that includes ...
09/05/2024
Sony and Apollo plan to break up Paramount'
Companies are expected to sell off Paramounts linear channels as well as close Paramount Plus By Jenny Priestley Published: May 9, 2024 Companies are expe...
09/05/2024
Paramount Global Selects Amdocs Vubiquity to Provide Mana...
Amdocs (NASDAQ:DOX), a leading provider of software and services to communications and media companies, today announced that Vubiquity, an Amdocs company, has b...
09/05/2024
CueScript Awarded Best of Show for CueFlip at NAB 2024
CueScript, the leading developer of professional teleprompting solutions, has announced that its CueFlip screen-to-screen folding field prompter has won in this...
09/05/2024
Mother's Day Ideas from CBC
Capitol Broadcasting has a myriad of ways to celebrate Mom! In honor of Mother's Day, here are a few ideas CBC divisions have to help you honor, or remember...
09/05/2024
Scripps Appoints New President and CEO of the Scripps Howard Fund
CINCINNATI Meredith Delaney has been appointed president and CEO of the Scripps Howard Fund, the philanthropic arm of The E.W. Scripps Company....
09/05/2024
Walmart Launches the Onn. 4K Streaming Device
Following leaks earlier this month of Walmart's plans to launch a new 4K streaming box, Walmart has officially unveiled the Onn. 4K Pro, a hybrid streaming ...
09/05/2024
DirecTV Adds More CW Stations
DirecTV continues to add CW affiliates to its streaming line-up, with the addition of four stations owned by Gray Television....
09/05/2024
Takeshi Kitano's film KUBI graded with DaVinci Resolve
Takeshi Kitano's film KUBI graded with DaVinci Resolve Brie Clayton May 8, 2024 0 Comments Blackmagic Design today announced that DaVinci Resolve ...
09/05/2024
zweiB updates to latest version of R&S CLIPSTER
zweiB updates to latest version of R&S CLIPSTER Brie Clayton May 8, 2024 0 Comments Specialist in deliverables for cinema, events and trade shows Ro...
09/05/2024
Simplify Your Filmmaking with Portable Lights
Simplify Your Filmmaking with Portable Lights Sean Alami May 8, 2024 0 Comments Discover the sleek, mobile Zhiyun lights that enhance cinematic scenes...
09/05/2024
The Benefits of using an Audio Interface
The Benefits of using an Audio Interface Colin Smith May 8, 2024 0 Comments This tutorial takes you through the hardware and software benefits of usin...
09/05/2024
Mother's Day Idea from CBC
Capitol Broadcasting has a myriad of ways to celebrate Mom! In honor of Mother's Day, here are a few ideas CBC divisions have to help you honor, or remember...
09/05/2024
Lachlan Murdoch Sees Progress Toward Sports Streaming Product
Fox CEO Lachlan Murdoch said his company, The Walt Disney Co. and Warner Bros. Discovery are in a two-minute drill to get their streaming sports joint venture t...
09/05/2024
Dale Earnhardt Jr. Joins Prime Video's NASCAR Team
NASCAR legend Dale Earnhardt Jr. is joining the Prime Video broadcast booth when the streaming network begins NASCAR Cup Series coverage next year. Earnhardt Jr...
09/05/2024
Nexstar Pitches Sports on The CW, Extra Reach Via Stations (Upfronts)
If upfront advertisers are looking for reach, Nexstar Media Group believes its base of local stations gives it a unique way to achieve that goal....
09/05/2024
Pluto TV Debuts NBCUniversal Local Streaming News Channels
NBCUniversal Local's 15 NBC and Telemundo local and regional streaming news channels begin launching on Pluto TV this month, starting May 8 with five channe...
09/05/2024
Don't Tell Mom the Babysitter's Dead' Debuts on BET Plus May 16
BET Plus said it has set May 16 as the premiere date for its original movie Don't Tell Mom the Babysitter's Dead....
09/05/2024
Disney's Rita Ferro Is Banking on Sports, Streaming, Tech (Upfronts)
Heading into the upfront, the advertising market is noticeably healthier than it was a year ago, Rita Ferro, president of global advertising sales at The Walt D...
09/05/2024
Rohde & Schwarz showcases operational efficiency at MPTS
Rohde & Schwarz showcases operational efficiency at MPTS Uncompromised quality and reliability coupled with simplicity and efficiency Rohde & Schwarz will ...
08/05/2024
Saved! and Rudo y Cursi Among Films With May Anniversaries
Diego Luna's character Beto aims to play professional soccer in Carlos Cuar n's film Rudo y Cursi....
08/05/2024
SGL Carbon: Focus on global growth trends pays off
SGL Carbon had a solid start to the first quarter of 2024. Despite the slight decline in sales of 3.9% to 272.6 million (Q1 2023: 283.7 million), adjusted EBI...
08/05/2024
The results are in from Malm Arena as Semi Final 1 participants move through to the Grand Final of this year's Eurovision Song Contest
The results are in from Malm Arena as Semi Final 1 participants move through to...
08/05/2024
Powerful NITV documentary Kindred to premiere during Reconciliation Week
Powerful NITV documentary Kindred to premiere during Reconciliation Week Media releases A journey of family, friendship and home *Watch the trailer here* ...
08/05/2024
Aerojet Rocketdyne Selected by DoD to Demonstrate Powder-in, Engine-out Hypersonic Propulsion Manufacturing
Aerojet Rocketdyne delivers a broad range of capabilities to support hypersonic ...
08/05/2024
L3Harris is Advancing the Future Fleet by Delivering Integrated Solutions
Selected by Fincantieri Marinette Marine, L3Harris is the shipboard design integrator and a key partner on the Fincantieri team for delivering critical subsyste...
08/05/2024
EditShare Goes Beyond Storage at MPTS
EditShare Goes Beyond Storage at MPTS Workflow, server and delivery solutions from storyboard to screen Stand D40, London Olympia, May 15-16, 2024 - EditShar...
08/05/2024
Tegna, Indiana Fever Expand Broadcast Distribution to 11 Additional Markets
TYSONS, Va. Tegna Inc. and the Indiana Fever have announced they have expanded the broadcast distribution of 17 Fever WNBA games to 11 additional markets....
08/05/2024
Comcast Business Boosts Internet Speeds
PHILADELPHIA Comcast Business has increased Internet speeds for customers, nationwide, at no additional cost and has introduced its fastest Internet plans yet ...
08/05/2024
Clear-Com to Showcase Cutting-Edge Communication Solution...
Clear-Com , a renowned provider of professional real-time communication solutions, is thrilled to announce its participation at InfoComm 2024, the largest profe...
08/05/2024
MRMC Achieves its Second Kings Award for Enterprise
Mark Roberts Motion Control (MRMC), a Nikon company, is one of the 252 organisations recognised with a prestigious King's Award for Enterprise. Announced to...
08/05/2024
EditShare goes beyond storage at MPTS
EditShare, the technology leader that enables storytellers to create and manage collaborative workflows at every stage from storyboard to screen, is exhibiting ...
08/05/2024
Magnifi and Linius Fuse Live and Archive Content to Redef...
Magnifi by VideoVerse an AI-driven video technology company and powerful video-editing SaaS platform is pleased to announce a strategic partnership with Linius ...
08/05/2024
NVIDIA RTX GPU Connects DaVinci Resolve to Power
NVIDIA RTX GPU Connects DaVinci Resolve to Power Brie Clayton May 8, 2024 0 Comments Two of the most important questions that the Creative COW audienc...
08/05/2024
Q&A with Visiting Artist Lei Liang
Q&A with Visiting Artist Lei Liang The acclaimed composer discusses his collaboration with Boston Conservatory, his work as a research artist, and what it mea...
08/05/2024
More Stations in San Antonio Launch NextGen TV Broadcasts
A second group of stations has begun broadcasting NextGen TV signals in San Antonio, Texas....
08/05/2024
NBC Orders More Deal or No Deal Island'
NBC has ordered a second season of unscripted Deal or No Deal Island. Joe Manganiello hosts, and executive produces alongside Howie Mandel. Manganiello will ret...
08/05/2024
Tegna Names Greg Retsinas GM At KGW, Portland, Oregon
Tegna said it named Greg Retsinas as president and general manager of KGW, the NBC affiliate in Portland, Oregon, effective June 3....
08/05/2024
Measurement Company Comscore Posts $5.2 Million Loss in 1st Quarter
Comscore lost money in the first quarter as weak linear ad sales impacted its cross-platform measurement business....
08/05/2024
Funny Business as NBC Orders Happy's Place,' Renews Lopez vs. Lopez'
NBC said it bolstered its comedy slate by renewing Lopez vs. Lopez and ordering Happy's Place, starring Reba McEntire....
08/05/2024
Knuckles' Knocks Fallout' From Top of TVision Power Score Rankings
Paramount Plus's Knuckles knocked Amazon Prime Video's Fallout out of the top spot in TVision's Power Score ranking of the top show on connected TV ...
08/05/2024
Tegna Adds 11 Markets To Lineup for Indiana Fever Games
Tegna, which made a deal to broadcast 17 Indiana Fever games featuring top WNBA draft pick Caitlin Clark, said it made deals that will put the games on stations...
08/05/2024
EchoStar Loses 348,000 Pay TV Subs in 1st Quarter
EchoStar said it lost another 348,000 pay TV subscribers so far this year, closing the first quarter with 8.18 million subscribers. That total includes 6.26 mil...
08/05/2024
Fox Has Net Income of $666 Million in 3rd Quarter
Fox reported a profit for its fiscal third quarter....
08/05/2024
IPG's Kinesso Unit Adds Senior Executives to Lineup
Kinesso, the centralized tech and data unit at media agency IPG Mediabrands, named Tom Amies-Cull as global chief operating officer and Amie Owen as global chie...
08/05/2024
Post production houses adopt Cleanfeed Cinema solution
The solutions in-browser stream focuses on low latency, making it suited to low bandwidth scenarios By Matthew Corrigan Published: May 8, 2024 The solutio...
08/05/2024
TAG unveils subtitling language detection feature
Driven by algorithms, the solution performs a quality analysis informed by language-specific dictionaries By Matthew Corrigan Published: May 8, 2024 Drive...