Large language model development is about to reach supersonic speed thanks to a collaboration between NVIDIA and Anyscale.
At its annual Ray Summit developers conference, Anyscale - the company behind the fast growing open-source unified compute framework for scalable computing - announced today that it is bringing NVIDIA AI to Ray open source and the Anyscale Platform. It will also be integrated into Anyscale Endpoints, a new service announced today that makes it easy for application developers to cost-effectively embed LLMs in their applications using the most popular open source models.
These integrations can dramatically speed generative AI development and efficiency while boosting security for production AI, from proprietary LLMs to open models such as Code Llama, Falcon, Llama 2, SDXL and more.
Developers will have the flexibility to deploy open-source NVIDIA software with Ray or opt for NVIDIA AI Enterprise software running on the Anyscale Platform for a fully supported and secure production deployment.
Ray and the Anyscale Platform are widely used by developers building advanced LLMs for generative AI applications capable of powering intelligent chatbots, coding copilots and powerful search and summarization tools.
NVIDIA and Anyscale Deliver Speed, Savings and Efficiency Generative AI applications are captivating the attention of businesses around the globe. Fine-tuning, augmenting and running LLMs requires significant investment and expertise. Together, NVIDIA and Anyscale can help reduce costs and complexity for generative AI development and deployment with a number of application integrations.
NVIDIA TensorRT-LLM, new open-source software announced last week, will support Anyscale offerings to supercharge LLM performance and efficiency to deliver cost savings. Also supported in the NVIDIA AI Enterprise software platform, Tensor-RT LLM automatically scales inference to run models in parallel over multiple GPUs, which can provide up to 8x higher performance when running on NVIDIA H100 Tensor Core GPUs, compared to prior-generation GPUs.
TensorRT-LLM automatically scales inference to run models in parallel over multiple GPUs and includes custom GPU kernels and optimizations for a wide range of popular LLM models. It also implements the new FP8 numerical format available in the NVIDIA H100 Tensor Core GPU Transformer Engine and offers an easy-to-use and customizable Python interface.
NVIDIA Triton Inference Server software supports inference across cloud, data center, edge and embedded devices on GPUs, CPUs and other processors. Its integration can enable Ray developers to boost efficiency when deploying AI models from multiple deep learning and machine learning frameworks, including TensorRT, TensorFlow, PyTorch, ONNX, OpenVINO, Python, RAPIDS XGBoost and more.
With the NVIDIA NeMo framework, Ray users will be able to easily fine-tune and customize LLMs with business data, paving the way for LLMs that understand the unique offerings of individual businesses.
NeMo is an end-to-end, cloud-native framework to build, customize and deploy generative AI models anywhere. It features training and inferencing frameworks, guardrailing toolkits, data curation tools and pretrained models, offering enterprises an easy, cost-effective and fast way to adopt generative AI.
Options for Open-Source or Fully Supported Production AI Ray open source and the Anyscale Platform enable developers to effortlessly move from open source to deploying production AI at scale in the cloud.
The Anyscale Platform provides fully managed, enterprise-ready unified computing that makes it easy to build, deploy and manage scalable AI and Python applications using Ray, helping customers bring AI products to market faster at significantly lower cost.
Whether developers use Ray open source or the supported Anyscale Platform, Anyscale's core functionality helps them easily orchestrate LLM workloads. The NVIDIA AI integration can help developers build, train, tune and scale AI with even greater efficiency.
Ray and the Anyscale Platform run on accelerated computing from leading clouds, with the option to run on hybrid or multi-cloud computing. This helps developers easily scale up as they need more computing to power a successful LLM deployment.
The collaboration will also enable developers to begin building models on their workstations through NVIDIA AI Workbench and scale them easily across hybrid or multi-cloud accelerated computing once it's time to move to production.
NVIDIA AI integrations with Anyscale are in development and expected to be available by the end of the year.
Developers can sign up to get the latest news on this integration as well as a free 90-day evaluation of NVIDIA AI Enterprise.
To learn more, attend the Ray Summit in San Francisco this week or watch the demo video below.
See this notice regarding NVIDIA's software roadmap.
Most recent headlines
23/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
23/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
23/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
23/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
23/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
23/12/2025
Taking the Stage at Carnegie Hall-On a Global Scale Boston Conservatory Orchestra students reflect on their epic concert marking the 80th session of the UN Gene...
22/12/2025
SVG New Sponsor Spotlight: Presidio's Neerav Shah on the Role of Its Captiva...
22/12/2025
Hitting the bullseye: Sky Sports readies itself for the biggest PDC World Darts ...
22/12/2025
Unique skillset: Bringing new directors to the world of darts at The Worlds with...
22/12/2025
Gravity Media prepares for a flight of fancy with the PDC World Darts Championsh...
22/12/2025
One hundred and eighty: Gravity Media on hitting the production bullseye at the ...
22/12/2025
The Famous Group's Jon Slusser on Fascinating Fans Through Immersive Content...
22/12/2025
ESPN's Meg Aronowitz on Continuing High-Quality Broadcasts of Collegiate Spo...
22/12/2025
ESPN Takes Data-Driven Storytelling to New Heights with MNF Playbook with Next ...
22/12/2025
For a decade, popular German podcast Fest & Flauschig has hosted an annual Chris...
22/12/2025
Paramount Scores Largest Share Increase Among Distributors as Paramount and CBS...
22/12/2025
New multi-year deal integrates Roku's data to fuel Nielsen's measurement suite
Roku gains access to Nielsen's streaming ratings, showing The Roku C...
22/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
22/12/2025
Berklee Wrapped 2025: Our Top News and Stories A look back at a year highlighted by faculty milestones, major film and television projects, Bob Dylan's ho...
22/12/2025
The works of Plato state that when humans have an experience, some level of change occurs in their brain, which is powered by memory - specifically long-term me...
22/12/2025
Partnership integrates complementary satellite data and AI analytics to enhance security, infrastructure, and environmental monitoring solutions for global cust...
22/12/2025
Workflows allow you to create a sequence of planned events which may be added to your template(s) or inserted directly into your sequential or background playli...
22/12/2025
Monday 22 December 2025
Sky extends PGA TOUR partnership until 2029, as Sky Spo...
22/12/2025
Back to All News
Global Anime Hits and New Releases Take Center Stage at Jump Festa 2026
Entertainment
22 December 2025
GlobalJapan
Link copied to clipboar...
22/12/2025
Siobh n McSweeney, Rory McIlroy, Elon Musk, Catherine Connolly, Jim Gavin, Ivan Yates and Traitor Paudie Moloney lead new characters for Callan Kicks the Year 2...
22/12/2025
Winner announced in the picturesque surroundings of Wicklow's Avondale Tower and Treetop Walk
Andrew Trimble wins the show in his first series as coach
Th...
22/12/2025
The 2025 winners have been announced today, Sunday 21 December, for Ireland's largest choral competition Choirs for Christmas hosted by RT lyric fm.
Ove...
21/12/2025
Back to All News
Legoshi and Haru's Story Reaches Its Finale: BEASTARS Fin...
21/12/2025
John Shortt named Young Sportsperson of the Year Kerry are the Team of the Year
...
20/12/2025
Atomos announced the immediate availability of a new firmware update for its Ninja TX GO and Ninja TX monitor-recorders, unlocking ProRes RAW recording from the...
20/12/2025
CJP Broadcast has completed the digitisation of the European Gymnastics tape archive, converting 328 tapes containing more than forty years of recorded material...
20/12/2025
Bitmovin, the leading provider of video streaming solutions, today announced the launch of the Stream Lab MCP Server, to give AI agents and large language model...
20/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
20/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
20/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
20/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
20/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
20/12/2025
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
20/12/2025
Barack Obama Includes Laufey on His 2025 Favorite Music List The former presidents roundup of books, music, and movies includes a song from the Berklee alums ...
20/12/2025
Study reveals a key hormonal circuit in the kidneys Scripps Research scientists identify the protein that helps kidney cells regulate renin, providing foundatio...
19/12/2025
With Playout Release 2025.4, ToolsOnAir continues to push professional playout w...
19/12/2025
SVG Sit-Down: Diversified's Jared Timmins on AI for Broadcast Sports and Cre...
19/12/2025
2025 SVG Summit Audio Recap: Say What?The Audio Production and Distribution Workshop at the SVG Summit 20 took on issues including speech intelligibility, Next-...
19/12/2025
Gamified fun: Channel 5 on its NFL Big Game Night ambitions with Hungry Bear Med...
19/12/2025
College Football Playoff Preview: For ESPN, Round 1 is a Fantastic Yet Familia...
19/12/2025
AWS's Jason Dvorkin on Developing Partnerships With the NBA and PGA Tour, Em...
19/12/2025
Netflix Kicks Off Packed Sports Week with Paul-Joshua Fight Before Shifting to N...
19/12/2025
SVG New Sponsor Spotlight: Presidio's Nareev Shah on the Role of Its Captiva...
19/12/2025
Mounted to the pylon of an AH-1Z Viper helicopter, a Red Wolf vehicle successful...
Large language model development is about to reach supersonic speed thanks to a collaboration between NVIDIA and Anyscale.
