Sony Pixel Power calrec Sony

Ray Shines with NVIDIA AI: Anyscale Collaboration to Help Developers Build, Tune, Train and Scale Production LLMs

18/09/2023

Large language model development is about to reach supersonic speed thanks to a collaboration between NVIDIA and Anyscale.

At its annual Ray Summit developers conference, Anyscale - the company behind the fast growing open-source unified compute framework for scalable computing - announced today that it is bringing NVIDIA AI to Ray open source and the Anyscale Platform. It will also be integrated into Anyscale Endpoints, a new service announced today that makes it easy for application developers to cost-effectively embed LLMs in their applications using the most popular open source models.

These integrations can dramatically speed generative AI development and efficiency while boosting security for production AI, from proprietary LLMs to open models such as Code Llama, Falcon, Llama 2, SDXL and more.

Developers will have the flexibility to deploy open-source NVIDIA software with Ray or opt for NVIDIA AI Enterprise software running on the Anyscale Platform for a fully supported and secure production deployment.

Ray and the Anyscale Platform are widely used by developers building advanced LLMs for generative AI applications capable of powering intelligent chatbots, coding copilots and powerful search and summarization tools.

NVIDIA and Anyscale Deliver Speed, Savings and Efficiency Generative AI applications are captivating the attention of businesses around the globe. Fine-tuning, augmenting and running LLMs requires significant investment and expertise. Together, NVIDIA and Anyscale can help reduce costs and complexity for generative AI development and deployment with a number of application integrations.

NVIDIA TensorRT-LLM, new open-source software announced last week, will support Anyscale offerings to supercharge LLM performance and efficiency to deliver cost savings. Also supported in the NVIDIA AI Enterprise software platform, Tensor-RT LLM automatically scales inference to run models in parallel over multiple GPUs, which can provide up to 8x higher performance when running on NVIDIA H100 Tensor Core GPUs, compared to prior-generation GPUs.

TensorRT-LLM automatically scales inference to run models in parallel over multiple GPUs and includes custom GPU kernels and optimizations for a wide range of popular LLM models. It also implements the new FP8 numerical format available in the NVIDIA H100 Tensor Core GPU Transformer Engine and offers an easy-to-use and customizable Python interface.

NVIDIA Triton Inference Server software supports inference across cloud, data center, edge and embedded devices on GPUs, CPUs and other processors. Its integration can enable Ray developers to boost efficiency when deploying AI models from multiple deep learning and machine learning frameworks, including TensorRT, TensorFlow, PyTorch, ONNX, OpenVINO, Python, RAPIDS XGBoost and more.

With the NVIDIA NeMo framework, Ray users will be able to easily fine-tune and customize LLMs with business data, paving the way for LLMs that understand the unique offerings of individual businesses.

NeMo is an end-to-end, cloud-native framework to build, customize and deploy generative AI models anywhere. It features training and inferencing frameworks, guardrailing toolkits, data curation tools and pretrained models, offering enterprises an easy, cost-effective and fast way to adopt generative AI.

Options for Open-Source or Fully Supported Production AI Ray open source and the Anyscale Platform enable developers to effortlessly move from open source to deploying production AI at scale in the cloud.

The Anyscale Platform provides fully managed, enterprise-ready unified computing that makes it easy to build, deploy and manage scalable AI and Python applications using Ray, helping customers bring AI products to market faster at significantly lower cost.

Whether developers use Ray open source or the supported Anyscale Platform, Anyscale's core functionality helps them easily orchestrate LLM workloads. The NVIDIA AI integration can help developers build, train, tune and scale AI with even greater efficiency.

Ray and the Anyscale Platform run on accelerated computing from leading clouds, with the option to run on hybrid or multi-cloud computing. This helps developers easily scale up as they need more computing to power a successful LLM deployment.

The collaboration will also enable developers to begin building models on their workstations through NVIDIA AI Workbench and scale them easily across hybrid or multi-cloud accelerated computing once it's time to move to production.

NVIDIA AI integrations with Anyscale are in development and expected to be available by the end of the year.

Developers can sign up to get the latest news on this integration as well as a free 90-day evaluation of NVIDIA AI Enterprise.

To learn more, attend the Ray Summit in San Francisco this week or watch the demo video below.

See this notice regarding NVIDIA's software roadmap.
LINK: https://blogs.nvidia.com/blog/2023/09/18/llm-anyscale-nvaie/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NAB Hires FCC Staffer Ben Arden as SVP, Deputy General Counsel

Share Copy link Facebook X Linkedin Bluesky Email...

01/05/2026

ARRI Introduces Compact New cforce MAX Lens Motor

Share Copy link Facebook X Linkedin Bluesky Email...

01/05/2026

CPI Media Deploys QuickLink StudioCall

Share Copy link Facebook X Linkedin Bluesky Email...

01/05/2026

FCC Proposes to Amend Audible Crawl Rule to Preserve Accessibility

Share Copy link Facebook X Linkedin Bluesky Email...

01/05/2026

Republican AGs Join Nexstar/Tegna Antitrust Suit

Share Copy link Facebook X Linkedin Bluesky Email...

01/05/2026

Dan Johnson Elevates Precision Editing With NUGEN Audio D...

LONDON, APRIL 30, 2026 The Post Republic London's Re-recording Mixer and Dialogue Editor Dan Johnson has built a reputation for clean, emotionally resonan...

01/05/2026

Adobe Unveils Powerful New Innovations in Photoshop & Lightroom

Adobe Unveils Powerful New Innovations in Photoshop & Lightroom Deepa Subramaniam April 30, 2026 0 Comments Your most tedious creative tasks just got ea...

01/05/2026

Berklee Partners with Santander US to Establish Global Opportunity Fund

Berklee Partners with Santander US to Establish Global Opportunity Fund The $400,000 grant offers students access to experiential learning opportunities withi...

01/05/2026

Student Spotlight: Keziah Thomas

Student Spotlight: Keziah Thomas The Indian composer, who was named the 2026 student commencement speaker for Berklee College of Music, talks about how shes p...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

30/04/2026

PWHL Reports Record Growth in Third Regular Season as Playoffs Begin April 30

The Professional Women's Hockey League (PWHL) concluded its third regular season on Saturday, reporting growth across attendance, viewership, digital engage...

30/04/2026

NBC Sports Launches National Sunday MLB Coverage Beginning May 3

NBC Sports will air national MLB coverage on Sundays beginning May 3, with MLB Sunday Leadoff on Peacock and NBCSN at 12:30 p.m. ET, followed by the debut of th...

30/04/2026

Clear-Com Appoints Brian Grahn and Ben Turnwell to New Roles

Clear-Com has appointed Brian Grahn as Market Outreach Manager of the Americas and Ben Turnwell as Business Development Manager for EMEA live. Grahn joined Cle...

30/04/2026

ARRI Introduces cforce MAX Lens Motor for Hi-5 Lens Control System

ARRI has introduced the cforce MAX, a new lens motor for the Hi-5 lens control system. The cforce MAX is twice as fast as the cforce plus motor it replaces whil...

30/04/2026

Knuerr, Voxtronic, and IHSE to Present Integrated Control Room Solution at Airspace World

Knuerr, Voxtronic, and IHSE will jointly present an integrated control room solu...

30/04/2026

The CW Network and ESPN to Stream CW Sports Live Events on ESPN App

The CW Network and ESPN have announced an agreement to make the ESPN App the exclusive streaming home for all CW Sports live events. CW Sports will continue to ...

30/04/2026

Sennheiser Spectera Deployed on Ed Sheerans The Loop Global Stadium Tour

Ed Sheeran's The Loop' tour launched in Auckland in January 2026 before moving on to Australia, with South America and the United States to follow late...

30/04/2026

Audinate Launches Dante Preset Creator for Offline Network Configuration

Audinate has announced Dante Preset Creator, a free online tool for configuring Dante network settings before hardware is available on site. Presets created in ...

30/04/2026

Yahoo Sports Appoints Jarrod Schwarz as General Manager

Yahoo Sports has announced the appointment of Jarrod Schwarz as General Manager of Yahoo Sports. Schwarz will oversee product, design, and technology; revenue a...

30/04/2026

Nielsen: U.S. Viewers Spent 79.8 Billion Minutes Watching Soccer in 2025

Nielsen has released a new report, Get Ready with Media Intelligence: 2026 FIFA World Cup Edition, examining U.S. soccer viewership trends, fan engagement, and ...

30/04/2026

USA Lacrosse Names SportsEngine Preferred Youth Sports Management Platform Partner

USA Lacrosse and SportsEngine have announced an expanded partnership, naming Spo...

30/04/2026

Telos Alliance To Appear at MPTS 2026 Across Multiple Partner Booths

Telos Alliance will participate in the 2026 Media Production and Technology Show (MPTS), taking place May 13-14 at Olympia London. Rather than exhibiting from a...

30/04/2026

DAZN Bolsters U.S. Ambitions With ViewLift Acquisition, Targets Evolving Regional Sports Landscape

The global streamer buys the U.S. DTC platform solutions provider for a reported...

30/04/2026

Tigo Sports Upgrades Video Infrastructure with Ateme Technology

Tigo Sports, Paraguay's leading sports broadcaster, has upgraded its video infrastructure with Ateme solutions for live encoding, multiplexing, and signal c...

30/04/2026

World Rugby and IMG Announce Long-Term Media Rights Partnership

World Rugby and IMG have announced a long-term media rights partnership focused on growing rugby in the United States ahead of the Men's and Women's Rug...

30/04/2026

NWSL & Overtime Re-Up Gen-Z Focused Content Partnership

For the second year in a row, Overtime and the National Women's Soccer League (NWSL) are teaming up through a renewed content partnership to bring fans even...

30/04/2026

ESPN Executive Vice President David Roberts Set To Retire

The 22-year ESPN vet's responsibilities will reportedly be taken over by SVP Mike Foss...

30/04/2026

SVG GameDay, Ep. 13: Anaheim Ducks' Scott Fausneaucht - Skating with the Ducks of Orange County

In-venue and creative video staffers at the professional and collegiate level ha...

30/04/2026

Prime Video Announces Multiyear Agreement with Duke Mens Basketball

Amazon and Duke University have announced a multiyear agreement for Prime Video to present exclusive coverage of three Duke Blue Devils men's basketball neu...

30/04/2026

Ratings Roundup: NBA Playoffs Return to NBC Sports up 38%; UFL Viewership up Midway Through Regular Season

Ratings Roundup is a rundown of recent rating news and is derived from press rel...

30/04/2026

Introducing Verified by Spotify, a Signal of Authenticity and Trust for the Artists Behind the Music

Music is evolving, and so are the ways you discover and connect with artists. In...

30/04/2026

First-Ever Stockholm Music Week Celebrates the Industry's Present and Future

Between April 22-29, the first inaugural Stockholm Music Week brought together thought leaders and partners across industries including music, tech, government,...

30/04/2026

API launch the Vision+ console

Iconic large-format console upgraded API's iconic Vision console has just been treated to an overhaul that aims to meet the demands of today's profe...

30/04/2026

Lewitt introduce the LCT 440 Pure stereo pair

Comes complete with miking accessories The LCT 440 Pure has proven to be a popular member of Lewitt's mic line-up, offering impressive technical perform...

30/04/2026

SynthFest UK 2026 Announced

24 October 2026 at The Octagon, Sheffield Now in its eighth year, SynthFest UK is the largest event of its kind in the UK, bringing together the top keyboar...

30/04/2026

SBS & NITV LEAD NATIONAL RECONCILIATION WEEK 2026 WITH LANDMARK GULPILIL DOCUMENTARY

SBS & NITV LEAD NATIONAL RECONCILIATION WEEK 2026 WITH LANDMARK GULPILIL DOCUMEN...

30/04/2026

Rohde & Schwarz equips new Terminal 3 at Frankfurt Airport with security scanners

Rohde & Schwarz equips new Terminal 3 at Frankfurt Airport with security scanner...

30/04/2026

Jennifer Ehle and Alex Hassell join the cast of ITV's propulsive lunar thriller First Woman

Jennifer Ehle (Contagion, Zero Dark Thirty) and Alex Hassell (Rivals, Wasteman, ...

30/04/2026

L3Harris Technologies Reports Strong First Quarter 2026 Results

MELBOURNE, Fla., April 30, 2026 - L3Harris Technologies (NYSE: LHX) reports first quarter 2026 results. Highlights Orders of $7.8 billion; book-to-bill of 1....

30/04/2026

Golf Craft Interview: A1s Florian Brown and James Deason

Behind the Broadcast: The Sound of Elite Golf Golf is gaining popularity; the 2025 Ryder Cup achieved record-breaking viewing figures in the UK specifically, wi...

30/04/2026

CPI Media Leverages QuickLink StudioCall for Integrating...

STA VENERA, MALTA, APRIL 29, 2026 CPI Media, a voluntary organization within the Missionary Society of St Paul (MSSP) and a leading media house dedicated to p...

30/04/2026

Hitomi Broadcast Presents UK Debut of MatchBox Panorama a...

New software platform delivers comprehensive timing measurement across production workflows...

30/04/2026

Cutting edge British innovation leads at BroadcastAsia 20...

Once again, the UK Pavilion in Hall 5 of BroadcastAsia 2026 will feature the latest and best in technology specifically developed and tailored for modern media ...