Sony Pixel Power calrec Sony

Ray Shines with NVIDIA AI: Anyscale Collaboration to Help Developers Build, Tune, Train and Scale Production LLMs

18/09/2023

Large language model development is about to reach supersonic speed thanks to a collaboration between NVIDIA and Anyscale.

At its annual Ray Summit developers conference, Anyscale - the company behind the fast growing open-source unified compute framework for scalable computing - announced today that it is bringing NVIDIA AI to Ray open source and the Anyscale Platform. It will also be integrated into Anyscale Endpoints, a new service announced today that makes it easy for application developers to cost-effectively embed LLMs in their applications using the most popular open source models.

These integrations can dramatically speed generative AI development and efficiency while boosting security for production AI, from proprietary LLMs to open models such as Code Llama, Falcon, Llama 2, SDXL and more.

Developers will have the flexibility to deploy open-source NVIDIA software with Ray or opt for NVIDIA AI Enterprise software running on the Anyscale Platform for a fully supported and secure production deployment.

Ray and the Anyscale Platform are widely used by developers building advanced LLMs for generative AI applications capable of powering intelligent chatbots, coding copilots and powerful search and summarization tools.

NVIDIA and Anyscale Deliver Speed, Savings and Efficiency Generative AI applications are captivating the attention of businesses around the globe. Fine-tuning, augmenting and running LLMs requires significant investment and expertise. Together, NVIDIA and Anyscale can help reduce costs and complexity for generative AI development and deployment with a number of application integrations.

NVIDIA TensorRT-LLM, new open-source software announced last week, will support Anyscale offerings to supercharge LLM performance and efficiency to deliver cost savings. Also supported in the NVIDIA AI Enterprise software platform, Tensor-RT LLM automatically scales inference to run models in parallel over multiple GPUs, which can provide up to 8x higher performance when running on NVIDIA H100 Tensor Core GPUs, compared to prior-generation GPUs.

TensorRT-LLM automatically scales inference to run models in parallel over multiple GPUs and includes custom GPU kernels and optimizations for a wide range of popular LLM models. It also implements the new FP8 numerical format available in the NVIDIA H100 Tensor Core GPU Transformer Engine and offers an easy-to-use and customizable Python interface.

NVIDIA Triton Inference Server software supports inference across cloud, data center, edge and embedded devices on GPUs, CPUs and other processors. Its integration can enable Ray developers to boost efficiency when deploying AI models from multiple deep learning and machine learning frameworks, including TensorRT, TensorFlow, PyTorch, ONNX, OpenVINO, Python, RAPIDS XGBoost and more.

With the NVIDIA NeMo framework, Ray users will be able to easily fine-tune and customize LLMs with business data, paving the way for LLMs that understand the unique offerings of individual businesses.

NeMo is an end-to-end, cloud-native framework to build, customize and deploy generative AI models anywhere. It features training and inferencing frameworks, guardrailing toolkits, data curation tools and pretrained models, offering enterprises an easy, cost-effective and fast way to adopt generative AI.

Options for Open-Source or Fully Supported Production AI Ray open source and the Anyscale Platform enable developers to effortlessly move from open source to deploying production AI at scale in the cloud.

The Anyscale Platform provides fully managed, enterprise-ready unified computing that makes it easy to build, deploy and manage scalable AI and Python applications using Ray, helping customers bring AI products to market faster at significantly lower cost.

Whether developers use Ray open source or the supported Anyscale Platform, Anyscale's core functionality helps them easily orchestrate LLM workloads. The NVIDIA AI integration can help developers build, train, tune and scale AI with even greater efficiency.

Ray and the Anyscale Platform run on accelerated computing from leading clouds, with the option to run on hybrid or multi-cloud computing. This helps developers easily scale up as they need more computing to power a successful LLM deployment.

The collaboration will also enable developers to begin building models on their workstations through NVIDIA AI Workbench and scale them easily across hybrid or multi-cloud accelerated computing once it's time to move to production.

NVIDIA AI integrations with Anyscale are in development and expected to be available by the end of the year.

Developers can sign up to get the latest news on this integration as well as a free 90-day evaluation of NVIDIA AI Enterprise.

To learn more, attend the Ray Summit in San Francisco this week or watch the demo video below.

See this notice regarding NVIDIA's software roadmap.
LINK: https://blogs.nvidia.com/blog/2023/09/18/llm-anyscale-nvaie/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

04/08/2026

Dalet Announces Commercial Availability of Dalia, Bringing Media-Aware Agentic AI to Enterprise Productions

Dalet, a leading technology and service provider for media-rich organizations, t...

04/07/2026

Detective Conan: Fallen Angel of the Highway Opens in Dolby Cinemas Across Japan, Presented in Dolby Atmos and Dolby ...

April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...

01/06/2026

Dolby Sets the New Standard for Premium Entertainment at CES 2026

January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026 Throughout the week, Dolby brings to life the latest innovatio...

02/05/2026

Dalet Flex LTS Delivers Smarter Search, Faster Editing, and an AI-Ready Foundation for Modern Media

Dalet, a leading technology and service provider for media-rich organizations, t...

01/05/2026

NBCUniversal's Peacock to Be First Streamer to Integrate Dolby's Full Suite of Premium Picture and Sound Innovations

January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...

14/04/2026

Sinclair's WTOV Taps Brightline for Lighting Upgrade

Share Copy link Facebook X Linkedin Bluesky Email...

14/04/2026

Grass Valley Showcases Alliance Ecosystem at 2026 NAB Show

Share Copy link Facebook X Linkedin Bluesky Email...

14/04/2026

FCC Selects New Lead Administrator for U.S. Cyber Trust Mark Program

Share Copy link Facebook X Linkedin Bluesky Email...

14/04/2026

Gray Media Names Jim Hays GM of WTHI

Share Copy link Facebook X Linkedin Bluesky Email...

14/04/2026

NAB Blasts CTA in FCC Sports Probe Comments

Share Copy link Facebook X Linkedin Bluesky Email...

14/04/2026

Wowza to Showcase AI-Powered Video Workflows and Emerging...

Wowza will return to NAB Show 2026 with a set of live demonstrations focused on how video infrastructure is evolving for a new generation of AI-powered and oper...

14/04/2026

Stegawave Debuts Real-Time Forensic Watermarking to Tackle Piracy in Live Sports Streaming

Stegawave Debuts Real-Time Forensic Watermarking to Tackle Piracy in Live Sports...

14/04/2026

Living in Boston: A Guide for Incoming Boston Conservatory Students

Living in Boston: A Guide for Incoming Boston Conservatory Students From navigating the T to balancing school with professional gigs, a current student shar...

14/04/2026

Just What Is Genre These Days, Anyway?

Just What Is Genre These Days, Anyway? Understanding the business and art of genre-bending in 2026. April 10, 2026 By Bryan Parys Illustration by Jack Fla...

14/04/2026

Lenora Helm Hammonds Is Turning Passion Into Plan A

Lenora Helm Hammonds Is Turning Passion Into Plan A The dean of the Professional Education Division has seen the industry from all sides. Now shes bringing it...

14/04/2026

How Michelle Zalabak Found Her Dream Career in Music and Finance

How Michelle Zalabak Found Her Dream Career in Music and Finance The Warner Music Group deal analysis manager helps determine what artist catalogs are worth a...

13/04/2026

ToolsOnAir Composition Builder 2026 Boilerplate

ToolsOnAir Composition Builder 2026 Boilerplate More Details: The Composition Builder 2026 application for macOS enables TV stations and Live Event broadcast...

13/04/2026

ToolsOnAr just:live pro 2026 Boilerplate

ToolsOnAr just:live pro 2026 Boilerplate More Details: just:live pro 2026 is a Multi-Channel Live Production Playout solution for video and static or real-ti...

13/04/2026

ToolsOnAr just:play pro 2026 Boilerplate

ToolsOnAr just:play pro 2026 Boilerplate More Details: just:play pro 2026 is a Multi-Channel automated 24/7 Master Control playout solution with SD, HD and U...

13/04/2026

ToolsOnAr live:cut 2026 Boilerplate

ToolsOnAr live:cut 2026 Boilerplate More Details: live:cut is an option to just:in mac pro 2025 and enables multicamera production workflows for up to 16 cam...

13/04/2026

ToolsOnAir Just In Mac Lite NDI 2026 Boilerplate

ToolsOnAir Just In Mac Lite NDI 2026 Boilerplate More Details: The Just In Mac Lite NDI application is a streamlined media capture solution designed specific...

13/04/2026

ToolsOnAir Just In Mac Lite 2026 Boilerplate

ToolsOnAir Just In Mac Lite 2026 Boilerplate More Details: The Just In Mac Lite application is a streamlined media capture solution designed specifically for...

13/04/2026

ToolsOnAir just:in mac pro 2026 Boilerplate

ToolsOnAir just:in mac pro 2026 Boilerplate More Details: just:in mac pro is a macOS-based client-server multichannel capture solution to record SDI, HDMI, N...

13/04/2026

Jnger Audio Joins EBU ADM Implementers Group as Founding Member

Telos Alliance has announced that J nger Audio has joined the EBU ADM Implementers Group (ADM-IG) as a founding member. The group is focused on advancing ADM an...

13/04/2026

NAB 2026: Grass Valley to Showcase Alliance Partner Ecosystem

Grass Valley will demonstrate its Alliance Partner ecosystem at NAB Show 2026 (Booth C2408, Central Hall, April 19-22), showing AMPP integrations across live pr...

13/04/2026

NAB 2026: Media Links to Demonstrate IP Transport Solutions

Media Links will exhibit at NAB Show 2026 (Booth W2033), demonstrating IP transport solutions for live production including hitless protection technology, Xscen...

13/04/2026

NBC Sports Partners with Overtime for OT7 Football League and Navy All-American Bowl

NBC Sports has announced a programming, distribution, and sales partnership with...

13/04/2026

FloSports Promotes Jayar Donlan from COO to President

FloSports has promoted Chief Operating Officer Jayar Donlan to President, effective immediately. In his new role, Donlan will lead the company's commercial,...

13/04/2026

MASV Case Study: PanCam Pictures Uses MASV for Remote Post-Production at Senior Bowl 2026

PanCam Pictures, the documentary production company founded by Paul Camarata, us...

13/04/2026

NAB 2026: Mimir to Showcase Cloud Production Platform

Mimir will exhibit at NAB Show 2026 (North Hall, Booth N2850), demonstrating its cloud-native media production platform with new capabilities including Mimir Cu...

13/04/2026

NAB 2026: BBright Adds RIST Protocol Support to IP Gateway

BBright has announced that its IP Gateway now supports the Reliable Internet Stream Transport (RIST) protocol. The addition will be introduced at NAB Show 2026 ...

13/04/2026

Net Insight Awarded ESA NAVISP Development Project for PNT Technology

Net Insight has been awarded a development project through the European Space Agency's Navigation Innovation and Support Program (NAVISP), with co-funding f...

13/04/2026

NAB 2026: intoPIX to Showcase JPEG XS, IPMX, and SMPTE 2110 Solutions

intoPIX will exhibit at NAB Show 2026, marking the company's 20th anniversary. The company will demonstrate its JPEG XS compression portfolio and IPMX-appro...

13/04/2026

Inside the Launch of BravesVision: How Braves, Raycom Sports Pulled Off One of the Most Ambitious Efforts in Regional-Sports-Media History

Starting from scratch, the team built an in-house content platform comprising ga...

13/04/2026

NAB 2026: AI Will Make Its Presence Felt in Audio Offerings, Presentations

Here's a look at some of the new products and updates, along with audio-centric conferences, that attendees will find next week at the show When the 2026 N...

13/04/2026

NAB 2026: Avid to Demonstrate Integrated Newsroom Capabilities

Avid will launch new integrated newsroom capabilities for Avid for News at NAB Show 2026 (Booth N2226, April 18-22), demonstrating how Avid Content Core connect...

13/04/2026

NAB 2026: Synamedia Launches Cloud-Controlled Edge Playout Version of Quortex PowerVu

Synamedia has announced a new version of Quortex PowerVu, an IP-native, software...

13/04/2026

NAB 2026: Mediaproxy Adds AI Brand and Advertisement Tracking to LogServer

Mediaproxy has developed a suite of AI-powered tools for brand and advertisement tracking, integrated into its LogServer compliance logging and analysis platfor...

13/04/2026

NAB 2026: Disguise to Demonstrate Media Server and Software Integrations

Disguise will demonstrate its media servers and software at NAB Show 2026, appearing across five partner booths in Central Hall: MRMC, B&H, Planar, CarbonBlack,...

13/04/2026

NAB 2026: OpenDrives Introduces Edge Hybrid Cloud-Edge Performance Accelerator

OpenDrives is introducing OpenDrives Edge at NAB Show 2026, a hybrid cloud-edge performance accelerator for distributed video and rich media workflows. The prod...

13/04/2026

ESPN Returns to The Shed for 2026 WNBA Draft, Expanding Camera Arsenal and Deepening Fan Coverage

The show will deploy 18 cameras across two sets and the draft floor, including a...

13/04/2026

Musik Hack update SweetEQ

Intuitive EQ plug-in gets an upgrade Following its official launch back in February 2026, Musik Hack's intuitive EQ plug-in has been treated to its firs...

13/04/2026

Vintage Vault 5 from UVI

Flagship soft synth collection expanded The latest version of UVI's flagship vintage-inspired soft synth collection has just arrived, expanding the suit...

13/04/2026

Sonuscore launch Lux Orchestral Strings Elements

Free version of innovative string library arrives Released in October 2025, Lux Orchestral Strings was said to be Sonuscore's most ambitious library to ...

13/04/2026

Girls' Research Camp at SGL Carbon in Meitingen inspires young women to pursue STEM careers

The Girls' Research Camp is part of the Technology - Future in Bavaria edu...

13/04/2026

Rohde & Schwarz transforms submarine communications for realtime underwater dominance at UDT 2026

Rohde & Schwarz transforms submarine communications for real time underwater dom...

13/04/2026

Rohde & Schwarz enables Pulsar signal simulation to support next-generation navigation devices

Rohde & Schwarz enables Pulsar signal simulation to support next-generation navi...

13/04/2026

When Missiles Move at 5X the Speed of Sound, Timing Is Everything

L3Harris is accelerating the development of infrared payloads for Space Development Agency's Tranche 2 Tracking Layer, to help meet urgent national defense ...

13/04/2026

US Army Selects L3Harris for Next-Generation Night-Vision System

By leveraging cutting-edge unfilmed Gen III image intensifier technology, NOVA delivers unmatched clarity, range, and reliability in low-light environments - en...