Large language model development is about to reach supersonic speed thanks to a collaboration between NVIDIA and Anyscale.
At its annual Ray Summit developers conference, Anyscale - the company behind the fast growing open-source unified compute framework for scalable computing - announced today that it is bringing NVIDIA AI to Ray open source and the Anyscale Platform. It will also be integrated into Anyscale Endpoints, a new service announced today that makes it easy for application developers to cost-effectively embed LLMs in their applications using the most popular open source models.
These integrations can dramatically speed generative AI development and efficiency while boosting security for production AI, from proprietary LLMs to open models such as Code Llama, Falcon, Llama 2, SDXL and more.
Developers will have the flexibility to deploy open-source NVIDIA software with Ray or opt for NVIDIA AI Enterprise software running on the Anyscale Platform for a fully supported and secure production deployment.
Ray and the Anyscale Platform are widely used by developers building advanced LLMs for generative AI applications capable of powering intelligent chatbots, coding copilots and powerful search and summarization tools.
NVIDIA and Anyscale Deliver Speed, Savings and Efficiency Generative AI applications are captivating the attention of businesses around the globe. Fine-tuning, augmenting and running LLMs requires significant investment and expertise. Together, NVIDIA and Anyscale can help reduce costs and complexity for generative AI development and deployment with a number of application integrations.
NVIDIA TensorRT-LLM, new open-source software announced last week, will support Anyscale offerings to supercharge LLM performance and efficiency to deliver cost savings. Also supported in the NVIDIA AI Enterprise software platform, Tensor-RT LLM automatically scales inference to run models in parallel over multiple GPUs, which can provide up to 8x higher performance when running on NVIDIA H100 Tensor Core GPUs, compared to prior-generation GPUs.
TensorRT-LLM automatically scales inference to run models in parallel over multiple GPUs and includes custom GPU kernels and optimizations for a wide range of popular LLM models. It also implements the new FP8 numerical format available in the NVIDIA H100 Tensor Core GPU Transformer Engine and offers an easy-to-use and customizable Python interface.
NVIDIA Triton Inference Server software supports inference across cloud, data center, edge and embedded devices on GPUs, CPUs and other processors. Its integration can enable Ray developers to boost efficiency when deploying AI models from multiple deep learning and machine learning frameworks, including TensorRT, TensorFlow, PyTorch, ONNX, OpenVINO, Python, RAPIDS XGBoost and more.
With the NVIDIA NeMo framework, Ray users will be able to easily fine-tune and customize LLMs with business data, paving the way for LLMs that understand the unique offerings of individual businesses.
NeMo is an end-to-end, cloud-native framework to build, customize and deploy generative AI models anywhere. It features training and inferencing frameworks, guardrailing toolkits, data curation tools and pretrained models, offering enterprises an easy, cost-effective and fast way to adopt generative AI.
Options for Open-Source or Fully Supported Production AI Ray open source and the Anyscale Platform enable developers to effortlessly move from open source to deploying production AI at scale in the cloud.
The Anyscale Platform provides fully managed, enterprise-ready unified computing that makes it easy to build, deploy and manage scalable AI and Python applications using Ray, helping customers bring AI products to market faster at significantly lower cost.
Whether developers use Ray open source or the supported Anyscale Platform, Anyscale's core functionality helps them easily orchestrate LLM workloads. The NVIDIA AI integration can help developers build, train, tune and scale AI with even greater efficiency.
Ray and the Anyscale Platform run on accelerated computing from leading clouds, with the option to run on hybrid or multi-cloud computing. This helps developers easily scale up as they need more computing to power a successful LLM deployment.
The collaboration will also enable developers to begin building models on their workstations through NVIDIA AI Workbench and scale them easily across hybrid or multi-cloud accelerated computing once it's time to move to production.
NVIDIA AI integrations with Anyscale are in development and expected to be available by the end of the year.
Developers can sign up to get the latest news on this integration as well as a free 90-day evaluation of NVIDIA AI Enterprise.
To learn more, attend the Ray Summit in San Francisco this week or watch the demo video below.
See this notice regarding NVIDIA's software roadmap.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
06/09/2026
June 9 2026, 23:00 (PDT) Dolby and MagentaTV Bring Fans Closer to the FIFA Worl...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
04/07/2026
April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...
12/06/2026
YES Network and The Gotham Sports App will air seven Athletes Unlimited Softball...
12/06/2026
The United Football League will host its FAST Innovation Suite at the 2026 United Bowl presented by Credit One Bank on Saturday, June 13 at 3:00 p.m. ET at Audi...
12/06/2026
PTZOptics and LayerJot will present live demonstrations at InfoComm 2026 showing how natural-language AI prompting, robotic camera control, and on-device comput...
12/06/2026
MultiDyne Video and Fiber Optic Systems will exhibit at InfoComm 2026, featuring...
12/06/2026
Ateme has announced that Eurovision Services is using Ateme's software-based frame-rate conversion technology for international live event workflows. The de...
12/06/2026
Bitmovin and Simplestream have announced a partnership with Xperi to simplify the launch of OTT streaming services on TiVo OS smart TVs and devices. The collabo...
12/06/2026
Net Insight has announced that a multinational technology company is deploying a...
12/06/2026
MLB Players Inc., the business arm of the MLB Players Association, has announced a partnership with Athletes First to develop and sell brand partnerships across...
12/06/2026
Guntermann and Drunck (G&D) and VuWall have announced the CommandKeyboard-Advanc...
12/06/2026
Comcast Smart Solutions announces a new smart technology deployment with Major L...
12/06/2026
Elevation Worship completed the initial leg of its Elevation Nights 2026 tour ...
12/06/2026
AJA Video Systems has announced KONA IP25 support for Colorfront Transkoder and ...
12/06/2026
Audinate Group Limited (ASX: AD8) will exhibit at InfoComm 2026 (Booth C7321, Ce...
12/06/2026
Pac-12 Commissioner Teresa Gould has announced the appointment of Scott Adametz as Chief Technology Officer. The Pac-12 describes the hire as the first CTO appo...
12/06/2026
Grass Valley has announced AMPP Edge Live, a production system combining Grass Valley hardware, NVIDIA Blackwell GPU acceleration, and AMPP OS in a single platf...
12/06/2026
At one time a trailblazer with the launch of the Longhorn Network, the Universit...
12/06/2026
Ratings Roundup is a rundown of recent rating news and is derived from press rel...
12/06/2026
Chyron has announced PAINT 10.4, an update to its illustrated replay and sports ...
12/06/2026
SVP, Production Mark Gross: With the new schedule, with not having every Sunday night, it's given us an opportunity to take a step back and reimagine what ...
12/06/2026
For Televisa Technical Engineering Manager Roberto N nez Ibarra and the small team of 12 technicians and two production personnel at the IBC things are already ...
12/06/2026
Simple Steps to Better Acoustics - Taming The Small Room
Most of us mix in spare rooms and small spaces, where the acoustics fight us at every turn. At Gear...
12/06/2026
Latest addition expands vintage-inspired effects palette
Meris' Ottobit pedal range draws its inspiration from vintage gaming consoles, and the latest a...
12/06/2026
Soundbox-based chamber strings series expanded
Sonora Cinematic have just announced the launch of the second instalment in their Soundbox-based chamber stri...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
AJA Announces KONA IP25 Integration with Colorfront Software
Brie Clayton June 12, 2026
0 Comments
Collaboration enables uncompressed SMPTE ST 2110 I/O ...
12/06/2026
URSA Cine 12K LF Used to Create Visuals for STUTS' K-Arena Concert
Brie Clayton June 12, 2026
0 Comments
Organic visuals projected on a giant scre...
12/06/2026
MTI FILM Acquires Mango New Edit, Expanding its Global Post-Production Services ...
12/06/2026
AI Point Tracking Speeds Up Complex VFX Tracks in Mocha Pro
Jessie Electa Petrov June 12, 2026
0 Comments
The 2026.5 release adds automatic point trac...
12/06/2026
Bitmovin, a provider of video streaming solutions, has partnered with Simplestream, a provider of OTT and broadcast solutions, and technology provider Xperi, to...
12/06/2026
Leostream Corporation, creator of the world-leading Leostream Remote Desktop Access Platform, today announced Jigsaw24, a leading B2B IT solutions provider wit...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
12/06/2026
How Aussie indie games and screen are levelling up with IP 11 June 2026
Ari Harrison, Pro Jank Footy
Head of Games Joey Egger and Ari Harrison of Umbrella sha...
12/06/2026
Back to All News
Netflix Unveils First-Look Images from Villaflor, Santiago Mitre's New Film
Entertainment
12 June 2026
GlobalArgentina
Link copied to ...
12/06/2026
Back to All News
Cross the Boundaries of the Real World into the Spirit Realm i...
12/06/2026
With John Daro, DI Colourist at Warner Bros. Water Tower Color Tuesday 23 June, 10am-1pm or 3-6pm
Restar Corporation
Tokyo
Register here
John Daro is the ...
12/06/2026
Meet The Grumpy Onion Ireland's newest online sensation, all he wants is to ...
12/06/2026
RT stays Up All Night with brand new daily 2026 FIFA World Cup Vodcast
Adding to the fun around 2026 FIFA World Cup tournament, RT has launched a brand new d...
12/06/2026
A fentanyl countermeasure that adapts to combat future black-market drugs Scripps Research scientists developed a vaccine that teaches the immune system to rapi...
11/06/2026
The immense size of the tourney and its Atlantic-spanning operation also disting...
11/06/2026
Nielsen has released a new soccer fandom consumer research report, The Fans Behind The Game: FIFA World Cup 2026 Edition, examining the soccer audience in the...
Large language model development is about to reach supersonic speed thanks to a collaboration between NVIDIA and Anyscale.
