
Large language model development is about to reach supersonic speed thanks to a collaboration between NVIDIA and Anyscale.
At its annual Ray Summit developers conference, Anyscale - the company behind the fast growing open-source unified compute framework for scalable computing - announced today that it is bringing NVIDIA AI to Ray open source and the Anyscale Platform. It will also be integrated into Anyscale Endpoints, a new service announced today that makes it easy for application developers to cost-effectively embed LLMs in their applications using the most popular open source models.
These integrations can dramatically speed generative AI development and efficiency while boosting security for production AI, from proprietary LLMs to open models such as Code Llama, Falcon, Llama 2, SDXL and more.
Developers will have the flexibility to deploy open-source NVIDIA software with Ray or opt for NVIDIA AI Enterprise software running on the Anyscale Platform for a fully supported and secure production deployment.
Ray and the Anyscale Platform are widely used by developers building advanced LLMs for generative AI applications capable of powering intelligent chatbots, coding copilots and powerful search and summarization tools.
NVIDIA and Anyscale Deliver Speed, Savings and Efficiency Generative AI applications are captivating the attention of businesses around the globe. Fine-tuning, augmenting and running LLMs requires significant investment and expertise. Together, NVIDIA and Anyscale can help reduce costs and complexity for generative AI development and deployment with a number of application integrations.
NVIDIA TensorRT-LLM, new open-source software announced last week, will support Anyscale offerings to supercharge LLM performance and efficiency to deliver cost savings. Also supported in the NVIDIA AI Enterprise software platform, Tensor-RT LLM automatically scales inference to run models in parallel over multiple GPUs, which can provide up to 8x higher performance when running on NVIDIA H100 Tensor Core GPUs, compared to prior-generation GPUs.
TensorRT-LLM automatically scales inference to run models in parallel over multiple GPUs and includes custom GPU kernels and optimizations for a wide range of popular LLM models. It also implements the new FP8 numerical format available in the NVIDIA H100 Tensor Core GPU Transformer Engine and offers an easy-to-use and customizable Python interface.
NVIDIA Triton Inference Server software supports inference across cloud, data center, edge and embedded devices on GPUs, CPUs and other processors. Its integration can enable Ray developers to boost efficiency when deploying AI models from multiple deep learning and machine learning frameworks, including TensorRT, TensorFlow, PyTorch, ONNX, OpenVINO, Python, RAPIDS XGBoost and more.
With the NVIDIA NeMo framework, Ray users will be able to easily fine-tune and customize LLMs with business data, paving the way for LLMs that understand the unique offerings of individual businesses.
NeMo is an end-to-end, cloud-native framework to build, customize and deploy generative AI models anywhere. It features training and inferencing frameworks, guardrailing toolkits, data curation tools and pretrained models, offering enterprises an easy, cost-effective and fast way to adopt generative AI.
Options for Open-Source or Fully Supported Production AI Ray open source and the Anyscale Platform enable developers to effortlessly move from open source to deploying production AI at scale in the cloud.
The Anyscale Platform provides fully managed, enterprise-ready unified computing that makes it easy to build, deploy and manage scalable AI and Python applications using Ray, helping customers bring AI products to market faster at significantly lower cost.
Whether developers use Ray open source or the supported Anyscale Platform, Anyscale's core functionality helps them easily orchestrate LLM workloads. The NVIDIA AI integration can help developers build, train, tune and scale AI with even greater efficiency.
Ray and the Anyscale Platform run on accelerated computing from leading clouds, with the option to run on hybrid or multi-cloud computing. This helps developers easily scale up as they need more computing to power a successful LLM deployment.
The collaboration will also enable developers to begin building models on their workstations through NVIDIA AI Workbench and scale them easily across hybrid or multi-cloud accelerated computing once it's time to move to production.
NVIDIA AI integrations with Anyscale are in development and expected to be available by the end of the year.
Developers can sign up to get the latest news on this integration as well as a free 90-day evaluation of NVIDIA AI Enterprise.
To learn more, attend the Ray Summit in San Francisco this week or watch the demo video below.
See this notice regarding NVIDIA's software roadmap.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
04/07/2026
April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...
01/06/2026
January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026
Throughout the week, Dolby brings to life the latest innovatio...
02/05/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
01/05/2026
January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...
22/04/2026
Solid State Logic is advancing its System T platform with a stronger focus on IP...
22/04/2026
From immersive audio to live streaming, Dolby Laboratories is focused on the fut...
22/04/2026
Shallow depth-of-field cameras have taken the industry by storm. Its debut a han...
22/04/2026
Riedel Communications (Booth C4908) announced that Eastern Kentucky University (...
22/04/2026
The NAB Show is in full swing, and the SVG and SVG Europe editorial teams are chasing down the hottest stories from all over the Las Vegas Convention Center. He...
22/04/2026
Blackmagic Design has announced the URSA Cine 12K LF 100G, a new model in the URSA Cine family adding 100G Ethernet for SMPTE 2110 live production output up to ...
22/04/2026
Celebrating its 40th anniversary, NEP is leaning into hybrid production with the...
22/04/2026
NEP VP, Platform Dan Murphy sits down at the 2026 NAB Show to unpack what NEP P...
22/04/2026
Spotify and the New York Liberty are teaming up to give music and basketball fan...
22/04/2026
New 20-minute documentary explores iconic design The Focusrite Room in Mesa, Arizona, where John Aquilino hosts the Studio Console 005.
In 2025, Focusrite co...
22/04/2026
Offers compact wireless solution for pedalboards
Taiwanese audio brand Cloudvocal have announced the availability of a new pedalboard-friendly wireless syst...
22/04/2026
Latest hybrid sampling/synthesis instrument arrives
Arturia's Augmented series offerings rely on a mixture of sampling and synthesis, allowing users to ...
22/04/2026
Combines three distinct analogue EQ emulations
The latest addition to Acustica Audio's ever-expanding collection of analogue-emulation plug-ins combines...
22/04/2026
Final instalment in vintage-inspired instrument series
Analog Empire: Bass & Lead marks the final instalment in Melda Production's vintage hardware-insp...
22/04/2026
Fuzz pedal joins all-analogue Series A line
Given that Strymons reputation was built on unapologetically digital pedals, it was a little surprising to see t...
22/04/2026
SBS names shortlisted brands for 2026 SBS Media Sustainability Challenge
22 April, 2026
Media releases
National broadcaster also releases its second annual...
22/04/2026
Why Low Band Electronic Warfare Matters...
22/04/2026
The nation unites around football team's World Cup dream
Warsaw, Poland, 20.04.26: Nielsen, a global leader in audience measurement, data, and media intell...
22/04/2026
Warsaw, Poland, 22.04.26: Nielsen, a global leader in audience measurement, data...
22/04/2026
New market intelligence offering gives businesses a clearer view of local consum...
22/04/2026
Glookast Unveils New UX, YouTube and Social Media Connectors, Premiere Panel, Ci...
22/04/2026
Lightcraft Technology to Preview Spark Story at NAB 2026 with Interactive Previs...
22/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
22/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
22/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
22/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
22/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
22/04/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
22/04/2026
22 Apr 2026
VEON's Banglalink to Bring Starlink Mobile to Customers in Bangladesh Bangladesh becomes the third market where VEON and Starlink Mobile partne...
22/04/2026
U have unveiled exclusive first-look images for their six-part police thriller Hit Point, starring Nick Blood (Day of the Jackal) and BAFTA nominee Saffron Hock...
22/04/2026
What can I watch on UKTV and stream on U this week?
This week on UKTV and the free streaming service U, viewers can watch a range of new and returning programm...
22/04/2026
Wednesday 22 April 2026
Sky announces fifth year of WNT Fund with 30,000 bursa...
22/04/2026
Back to All News
This Earth Day, Discover the Sustainable Productions Behind Our Films and Series
Emma Stewart, Ph.D.
Netflix Sustainability Officer
Enterta...
22/04/2026
The move from Retail Media to Commerce Media is about broadening the scope of th...
22/04/2026
April 22 2026, 07:00 (PDT) Dolby and BMW Bring Dolby Atmos to the BMW 7 Series,...
22/04/2026
RT Documentary On One 7-part series breaks US market for first time
RT Programme Sales has announced its first deal with a US distribution partner for its 7-...
22/04/2026
NVIDIA and Google Cloud have collaborated for more than a decade, co engineering a full stack AI platform that spans every technology layer - from performance o...
21/04/2026
Cloud-based production isnt going anywhere, and BitFire is doubling down by prov...
21/04/2026
The topic of artificial intelligence has a stranglehold on the sports-video-prod...
21/04/2026
5G is still a hot topic in live event production, and this workflow continues to...
21/04/2026
At the 2026 NAB Show, Ed McGivern, GM and President of Appear US, discusses the ...
21/04/2026
Studio Network Solutions (SNS) has announced an on-premise AI suite designed for...
21/04/2026
Suite Studios has integrated its file-streaming technology into the newly announced Frame.io Drive, a desktop application from Adobe company Frame.io. The colla...
21/04/2026
Net Insight has integrated InSync Technology's FrameFormer into the Nimbra E...