Sony Pixel Power calrec Sony

Pinterest Boosts Home Feed Engagement 16% With Switch to GPU Acceleration of Recommenders

04/08/2022

Pinterest has engineered a way to serve its photo-sharing community more of the images they love.

The social-image service, with more than 400 million monthly active users, has trained bigger recommender models for improved accuracy at predicting people's interests.

Pinterest handles hundreds of millions of user requests an hour on any given day. And it must also narrow down relevant images from roughly 300 billion images on the site to roughly 50 for each person.

The last step - ranking the most relevant and engaging content for everyone using Pinterest - required a leap in acceleration to run heftier models, with minimal latency, for better predictions.

Pinterest has improved the accuracy of its recommender models powering people's home feeds and other areas, increasing engagement by as much as 16%.

The leap was enabled by switching from CPUs to NVIDIA GPUs, which could easily be applied next to other areas, including advertising images, according to Pinterest.

Normally we would be happy with a 2% increase, and 16% is just a beginning for home feeds. We see additional gains - it opens a lot of doors for opportunities, said Pong Eksombatchai, a software engineer at Pinterest.

Transformer models capable of better predictions are shaking up industries from retail to entertainment and advertising. But their leaps in performance gains of the past few years have come with a need to serve models that are some 100x bigger as their number of model parameters and computations skyrockets.

Huge Inference Gains, Same Infrastructure Cost Like many, Pinterest engineers wanted to tap into state-of-the-art recommender models to increase engagement. But serving these massive models on CPUs presented a 100x increase in cost and latency. That wasn't going to maintain its magical user experience - fresh and more appealing images - occurring within a fraction of a second.

If that latency happened, then obviously our users wouldn't like that very much because they would have to wait forever, said Eksombatchai. We are pretty close to the limit of what we can do on CPU basically.

The challenge was to serve these hundredfold larger recommender models within the same cost and latency constraints.

Working with NVIDIA, Pinterest engineers began architectural changes to optimize their inference pipeline and recommender models to enable the transition from CPU to GPU cloud instances. The technology transition began late last year and required major changes to how the company manages workloads. The result is a 100x gain in inference efficiency on the same IT budget, meeting their goals.

We are starting to use really, really big models now. And that is where the GPU comes in - to help make these models possible, Eksombatchai said.

Tapping Into cuCollections Switching from CPUs to GPUs required rethinking its inference systems architecture. Among other issues, engineers had to change how they send workloads to their inference servers. Fortunately, there are tools to assist in making the transition easier.

The Pinterest inference server built for CPUs had to be altered because it was set up to send smaller batch sizes to its servers. GPUs can handle much larger workloads, so it's necessary to set up larger batch requests to increase efficiency.

One area where this comes into play is with its embedding table lookup module. Embedding tables are used to track interactions between various context-specific features and interests of user profiles. They can track where you navigate, and what people Pin on Pinterest, share or numerous other actions, helping refine predictions on what users might like to click on next.

They are used to incrementally learn user preference based on context in order to make better content recommendations to those using Pinterest. Its embedding table lookup module required two computation steps repeated hundreds of times because of the number of features tracked.

Pinterest engineers greatly reduced this number of operations using a GPU-accelerated concurrent hash table from NVIDIA cuCollections. And they set up a custom consolidated embedding lookup module so they could merge requests into a single lookup. Better results were seen immediately.

Using cuCollections helped us to remove bottlenecks, said Eksombatchai.

Enlisting CUDA Graphs Pinterest relied on CUDA Graphs to eliminate what was remaining of the small batch operations, further optimizing its inference models.

CUDA Graphs helps reduce the CPU interactions when launching on GPUs. They're designed to enable workloads to be defined as graphs rather than single operations. They provide a mechanism to launch multiple GPU operations through a single CPU operation, reducing CPU overheads.

Pinterest enlisted CUDA Graphs to represent the model inference process as a static graph of operation instead of as those individually scheduled. This enabled the computation to be handled as a single unit without any kernel launching overhead.

The company now supports CUDA Graph as a new backend of its model server. When a model is first loaded, the model server runs the model inference once to build the graph instance. This graph can then be run repeatedly in inference to show content on its app or site.

Implementing CUDA Graphs helped Pinterest to significantly reduce inference latency of its recommender models, according to its engineers.

GPUs have enabled Pinterest to do something that was impossible with CPUs on the same budget, and by doing this they can make changes that have a direct impact on various business metrics.

Learn about Pinterest's GPU-driven inference and optimizations at its GTC session, Serving 100x Bigger Recommender Models, and in the Pinterest Engineering blog.

Register for GTC, running Sept. 19-22, for free to attend sessions with NVIDIA and dozens of industry leaders.
LINK: https://blogs.nvidia.com/blog/2022/08/04/pinterest-gpu-acceleration-re...
See more stories from nvidia

Most recent headlines

07/12/2022

Dalet Flex Takes a Leap Forward

Dalet, a leading technology and service provider for media-rich organizations, today announced key new capabilities available in Dalet Flex that support importa...

04/11/2022

Game better with DTS Headphone:X

The best gamers know DTS Headphone:X technology ups their play. Now major magazines have caught on, too. DTS Headphone:X technology creates immersive soundsc...

08/10/2022

Dalet Next-Gen Solutions at IBC 2022

Dalet, a leading technology and service provider for media-rich organizations, is pleased to announce its return to the IBC Show, held in Amsterdam from Septemb...

12/08/2022

El rapero mexicano Santa Fe Klan presenta Mundo, su lbum ms personal, con contenido nico para sus fans en Spotify

Con apenas 22 a os, ngel Jair Quezada, quien se hace llamar Santa Fe Klan, es u...

12/08/2022

In Conversation: Winning Time

The HBO original series Winning Time: The Rise of the Lakers Dynasty re-creates the Los Angeles Lakers' historic climb to prominence over the rest of the NB...

12/08/2022

SAFTAs16 open call for in memorial segment.

The National Film and Video Foundation (NFVF), an agency of the Department of Sport, Arts, and Culture and custodian of the SAFTAs is calling all South Africans...

12/08/2022

Boston Conservatory at Berklee Dancers to Perform at KCON Los Angeles

Boston Conservatory at Berklee Dancers to Perform at KCON Los Angeles Thirteen dance students will perform four times throughout the convention, including ope...

12/08/2022

Cloud cover for Spanish theatre festival

A 4K cloud workflow from TVU Networks has enabled the broadcast of Spain's Almagro Classical Theater Festival for RTVE, who deployed a multi-camera producti...

12/08/2022

My IBC: Andrew Starks, AIMS board member

In a new series of features, Future speaks to a selection of industry stakeholders about their typical IBC routines, and what theyre looking forward to at this ...

12/08/2022

Disneys DTC Subs Surpass Netflix

BURBANK, Calif. As Disney announced significant subscription price increases (opens in new tab) for its streaming services and launch dates for a lower-cost ad-...

12/08/2022

RTVE Deploys TVU Networks Cloud Technology

ALMAGRO, Spain Spanish national public broadcaster Radiotelevisi n Espa ola (RTVE) in July used cloud-based 4K and 5G live production technology from TVU Networ...

12/08/2022

Olympic Broadcasting Services Lays Out Initial Plans for Paris 2024

PARIS With just under two years to go until the start of the Paris 2024 Olympic Games, Olympic Broadcasting Services (OBS) has revealed its initial plans for th...

12/08/2022

Producer Evan Shapiro to Keynote NAB Show New York Opening

WASHINGTON Award-winning film producer Evan Shapiro will keynote the 2022 NAB Show New York opening event. In his presentation, Bringing Media Into the Current...

12/08/2022

FTC Eyes Rules Cracking Down on 'Commercial Surveillance'

WASHINGTON D.C. The Federal Trade Commission has announced it is exploring rules to crack down on harmful commercial surveillance and lax data security....

12/08/2022

SDVI to Highlight Rally Enhancements at IBC2022

AMSTERDAM At the 2022 IBC Show in Amsterdam, Sept 9-13, SDVI will showcase the latest developments in media supply chain optimization, including net-zero emissi...

12/08/2022

A+E to Launch Crime+Investigation Play' DTC Service in U.K.

LONDON A+E Networks UK & EMEA has announced that it will launch the direct-to-consumer true crime streaming service Crime+Investigation Play on the 5th of Septe...

12/08/2022

Sky Studios Elstree Selects Xytech Systems Resource Management Software

BOREHAMWOOD, U.K. Sky Studios Elstree, a joint venture of Sky and NBCUniversal, has selected Xytech Systems resource management software for its complete operat...

12/08/2022

Chicken Soup Closes Redbox Deal

COS COB, Conn. & OAKBROOK TERRACE, Ill. Chicken Soup for the Soul Entertainment has announced that the company has completed the acquisition of Redbox Entertain...

12/08/2022

HBO Max Creates 'Our Flag Means Death' Using Blackmagic Design

FREMONT, Calif. Blackmagic Design has announced that the HBO Max series Our Flag Means Death used extensive Blackmagic Design products to create the stunning ...

12/08/2022

NDI: Connection without complication

TVBEurope sits down for a Q&A with Tarif Sayed, president and GM of NDI Global, to discuss his background, his path to NDI, and what makes it a great fit for co...

12/08/2022

The Guys from IPnema

As open-standards-based IP has come of age as a network protocol for the broadcast, AV and corporate sectors, the emphasis has moved from making essence transpo...

12/08/2022

Canon Releases New PTZ Protocols for CR-N500 and CR-N300 4K PTZ Cameras

Canon Releases New PTZ Protocols for CR-N500 and CR-N300 4K PTZ Cameras Brie Clayton August 11, 2022 0 Comments Canon CR-N500 and CR-N300 Firmware Upd...

12/08/2022

PHABRIX launches QxP - The portable QxL

PHABRIX launches QxP - The portable QxL Brie Clayton August 11, 2022 0 Comments PHABRIX, a broadcast test & measurement innovator, has announced the la...

12/08/2022

VEGAS Creative Software Launches VEGAS Pro 20

VEGAS Creative Software Launches VEGAS Pro 20 Brie Clayton August 11, 2022 0 Comments VEGAS Pro 20 Enhances Cloud Collaboration with New VEGAS Hub Feat...

12/08/2022

ATOMOS CONNECT and SHOGUN CONNECT new firmware enables Frame.io Camera to Cloud workflows

ATOMOS CONNECT and SHOGUN CONNECT new firmware enables Frame.io Camera to Cloud ...

12/08/2022

The Russell Howard Hour returns to Sky Max on Thursday 8th September at 10pm

Russell Howard is back to share his hilarious takes on the spiralling headlines with the 6th series of The Russell Howard Hour, coming to Sky MAX and streaming ...

12/08/2022

We picked' a great day for it!

Posted on 12 Aug 2022 ICG joined local volunteers and Coastal Conservationist and Marine Medic, Tracey Hope for the daily beach clean on the Fylde coast at St ...

12/08/2022

Metal Festival Feels Full Force of CODA Audio

Described by Winston McCall of Australian metalcore band, Parkway Drive, as the most metal place on earth! the Full Force festival in L bnitz, Germany has bee...

12/08/2022

Celebrities Embark on The Journey of a Lifetime in Meter TV's Adventure Series

Banijay today announces Swedish label Meter TV, part of Banijay Nordic, will pro...

12/08/2022

The Isley Brothers reimagine a 1975 classic with Beyonc [listen]

The Isley Brothers reimagine a 1975 classic with Beyonc [listen] R&B legends The Isley Brothers have teamed up with Beyonc on a duet of their 1975 classic, M...

12/08/2022

Actress Nomzamo Mbatha researched Nandi while filming Isibaya'

Actress Nomzamo Mbatha researched Nandi while filming Isibaya' Shaka Ilembe' producer and actress Nomzamo Mbatha says she's always wanted to play Q...

12/08/2022

Skeem Saam' September Spoilers: Kwaito leaves, Alfred is bribed

Skeem Saam' September Spoilers: Kwaito leaves, Alfred is bribedThe Skeem Saam' teasers reveal that Clement Maosa, who plays Kwaito Seakamela-Maputla wi...

12/08/2022

WATCH: Skeem Saam - Tonight's teaser E30 S11

WATCH: Skeem Saam - Tonight's teaser E30 S11Find out what's going on tonight on Skeem Saam! by SA TV and Soapies 12-08-2022 16:00 in SkeemSaam Fri...

12/08/2022

WATCH: Generations - The Legacy, tonight's teaser E190 S8

WATCH: Generations - The Legacy, tonight's teaser E190 S8Keen to know what's happening on the latest episode of Generations: The Legacy? by SA TV and ...

12/08/2022

WATCH: Uzalo- Tonight's teaser E115 S8

WATCH: Uzalo- Tonight's teaser E115 S8Don't get left behind, catch up on Uzalo now! by SA TV and Soapies 12-08-2022 16:05 in Soapies Friday's ...

12/08/2022

WATCH: Imbewu - The Seed, tonight's teaser E87 S5

WATCH: Imbewu - The Seed, tonight's teaser E87 S5Catch up on what's going on to night on Imbewu by SA TV and Soapies 12-08-2022 16:08 in Soapies F...

12/08/2022

WATCH: Diep City - Tonight's teaser E95 S2

WATCH: Diep City - Tonight's teaser E95 S2We've got you covered on what's happening tonight. by SA TV and Soapies 12-08-2022 16:10 in Soapies ...

12/08/2022

WATCH: The River - Tonight's teaser E135 S5

WATCH: The River - Tonight's teaser E135 S5Watch tonight's episode of The River and catch the latest episode every weekday at 20:00 on 1 Magic, don'...

12/08/2022

WATCH: Scandal - Tonight's teaser E96 S14

WATCH: Scandal - Tonight's teaser E96 S14Get to know what's happening on tonight's episode! by SA TV and Soapies 12-08-2022 16:15 in Scandal F...

12/08/2022

WATCH: Gomora - Tonight's teaser E80 S3

WATCH: Gomora - Tonight's teaser E80 S3Don't miss out on what's happening on tonight's episode of Gomora, and watch the latest episode every wee...

12/08/2022

WATCH: Muvhango - Tonight's teaser E419 S22

WATCH: Muvhango - Tonight's teaser E419 S22Don't miss out on the latest drama in Muvhango! by SA TV and Soapies 12-08-2022 16:20 in Muvhango Frida...

12/08/2022

WATCH: 7de Laan - Tonight's teaser E425 S22

WATCH: 7de Laan - Tonight's teaser E425 S22Don't miss out on what's happening on tonight's episode of 7de Laan and Catch the latest episode ever...

12/08/2022

Belinda the Bok anthem singer': Who is SA's Whitney Houston?

Belinda the Bok anthem singer': Who is SA's Whitney Houston?Belinda Davids has been tasked with singing the national anthem at the Springboks vs All Bla...

12/08/2022

Ross elevates its production switcher lineup with significant upgrades to the Carbonite & Acuity product lines

Ross Video's comprehensive production switcher lineup grows more robust and ...

12/08/2022

AJA Releases Mini-Config v2.26.4

AJA Releases Mini-Config v2.26.4 posted: 12/08/2022 Feature-rich software update streamlines conversion workflows, delivers enhanced flexibility AJA Vid...

12/08/2022

Daily devotion: Your bible verse and prayer for 12 August 2022

Daily devotion: Your bible verse and prayer for 12 August 2022GOOD MORNING! Here's your daily bible verse and prayer to get you ready and excited about your...

12/08/2022

Actor Arthur Masoma discusses playing Phuthi in Skeem Saam'

Actor Arthur Masoma discusses playing Phuthi in Skeem Saam' Actor Arthur Masoma recently joined Skeem Saam' after auditioning for the role of Phuthi L...

12/08/2022

From the Garden of Eden to the promised land: IT IS WRITTEN

From the Garden of Eden to the promised land: IT IS WRITTENOur Father is preparing a new heaven and a new earth, built for His bride, beautifully dressed for he...

12/08/2022

Beats n Bars festival moves to American Tobacco in Durham

The Beats n Bars Festival is relocating to the American Tobacco Campus in September. The festival kicks off Friday, Sept. 16, with a free hip hop show at Bouri...

11/08/2022

Celebrate the 25th Birthday of The Full Monty, Plus 4 More Sundance Films With August Anniversaries

By Vanessa Zimmer Twenty-five years ago this month, many of us learned the mean...