
Large language model development is about to reach supersonic speed thanks to a collaboration between NVIDIA and Anyscale.
At its annual Ray Summit developers conference, Anyscale - the company behind the fast growing open-source unified compute framework for scalable computing - announced today that it is bringing NVIDIA AI to Ray open source and the Anyscale Platform. It will also be integrated into Anyscale Endpoints, a new service announced today that makes it easy for application developers to cost-effectively embed LLMs in their applications using the most popular open source models.
These integrations can dramatically speed generative AI development and efficiency while boosting security for production AI, from proprietary LLMs to open models such as Code Llama, Falcon, Llama 2, SDXL and more.
Developers will have the flexibility to deploy open-source NVIDIA software with Ray or opt for NVIDIA AI Enterprise software running on the Anyscale Platform for a fully supported and secure production deployment.
Ray and the Anyscale Platform are widely used by developers building advanced LLMs for generative AI applications capable of powering intelligent chatbots, coding copilots and powerful search and summarization tools.
NVIDIA and Anyscale Deliver Speed, Savings and Efficiency Generative AI applications are captivating the attention of businesses around the globe. Fine-tuning, augmenting and running LLMs requires significant investment and expertise. Together, NVIDIA and Anyscale can help reduce costs and complexity for generative AI development and deployment with a number of application integrations.
NVIDIA TensorRT-LLM, new open-source software announced last week, will support Anyscale offerings to supercharge LLM performance and efficiency to deliver cost savings. Also supported in the NVIDIA AI Enterprise software platform, Tensor-RT LLM automatically scales inference to run models in parallel over multiple GPUs, which can provide up to 8x higher performance when running on NVIDIA H100 Tensor Core GPUs, compared to prior-generation GPUs.
TensorRT-LLM automatically scales inference to run models in parallel over multiple GPUs and includes custom GPU kernels and optimizations for a wide range of popular LLM models. It also implements the new FP8 numerical format available in the NVIDIA H100 Tensor Core GPU Transformer Engine and offers an easy-to-use and customizable Python interface.
NVIDIA Triton Inference Server software supports inference across cloud, data center, edge and embedded devices on GPUs, CPUs and other processors. Its integration can enable Ray developers to boost efficiency when deploying AI models from multiple deep learning and machine learning frameworks, including TensorRT, TensorFlow, PyTorch, ONNX, OpenVINO, Python, RAPIDS XGBoost and more.
With the NVIDIA NeMo framework, Ray users will be able to easily fine-tune and customize LLMs with business data, paving the way for LLMs that understand the unique offerings of individual businesses.
NeMo is an end-to-end, cloud-native framework to build, customize and deploy generative AI models anywhere. It features training and inferencing frameworks, guardrailing toolkits, data curation tools and pretrained models, offering enterprises an easy, cost-effective and fast way to adopt generative AI.
Options for Open-Source or Fully Supported Production AI Ray open source and the Anyscale Platform enable developers to effortlessly move from open source to deploying production AI at scale in the cloud.
The Anyscale Platform provides fully managed, enterprise-ready unified computing that makes it easy to build, deploy and manage scalable AI and Python applications using Ray, helping customers bring AI products to market faster at significantly lower cost.
Whether developers use Ray open source or the supported Anyscale Platform, Anyscale's core functionality helps them easily orchestrate LLM workloads. The NVIDIA AI integration can help developers build, train, tune and scale AI with even greater efficiency.
Ray and the Anyscale Platform run on accelerated computing from leading clouds, with the option to run on hybrid or multi-cloud computing. This helps developers easily scale up as they need more computing to power a successful LLM deployment.
The collaboration will also enable developers to begin building models on their workstations through NVIDIA AI Workbench and scale them easily across hybrid or multi-cloud accelerated computing once it's time to move to production.
NVIDIA AI integrations with Anyscale are in development and expected to be available by the end of the year.
Developers can sign up to get the latest news on this integration as well as a free 90-day evaluation of NVIDIA AI Enterprise.
To learn more, attend the Ray Summit in San Francisco this week or watch the demo video below.
See this notice regarding NVIDIA's software roadmap.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
17/11/2026
Fabric Insights and Fonn Group have announced a partnership to connect editorial...
17/11/2026
The National Association of Black Journalists is offering up a Thanksgiving Eve treat on Wednesday, Nov. 25, in the form of a live streaming event featuring for...
17/11/2026
Bridge Technologies has announced that its VB440 production probe now supports both monitoring and signal generation for Apple ProRes video streams. The additio...
17/11/2026
Canon U.S.A., Inc. recently welcomed 21 Jericho High School students to its Melv...
17/11/2026
Spectrum News has deployed the Amagi platform to support its live news operations. Amagi says the platform lets Spectrum News streamline newsroom workflows, red...
17/11/2026
Microsoft has expanded its use of Haivision's live video contribution ecosys...
17/11/2026
Setanta Sports has deployed Ateme's TITAN Live platform to support its OTT video delivery operations. Ateme's transcoding technology is integrated into ...
17/11/2026
Media Links has joined the KAIROS Alliance Partner program for Panasonic's KAIROS IT/IP live video production platform. The company completed interoperabili...
17/11/2026
Behind The Mic provides a roundup of recent news regarding on-air talent, including new deals, departures, and assignments compiled from press releases and repo...
17/11/2026
In live sports and in life, things go by in the blink of an eye. Savoring the mo...
17/11/2026
Satellite operators are adding a new generation of low-Earth-orbit (LEO) network...
17/11/2026
Clear-Com has delivered five customized test systems for electromagnetic securit...
17/11/2026
Stephen Arnold Music (SAM) has created the sonic identity for WTGL, the women's team golf league from TMRW Sports, created in partnership with the LPGA and ...
17/11/2026
The Professional Audio Manufacturers Alliance (PAMA) has introduced Give Sound a Voice, an initiative aimed at raising awareness of the value of sound among the...
17/11/2026
Fanatics Studios, the sports entertainment joint venture between Fanatics and OBB, has named Jamal Henderson its first President of Film & TV, announced Michael...
17/11/2026
The NFL's Thanksgiving Week games will feature halftime performances by U2, ...
17/11/2026
FanSided, a Minute brand with more than 250 fan communities, has launched a redesigned mobile app that combines team news, analysis, live game coverage, stats a...
17/11/2026
Major League Volleyball (MLV), the longest-running indoor professional volleybal...
17/11/2026
Prime Video will present a 15-hour slate of live sports on Black Friday, Nov. 27...
17/11/2026
Catherine Dior was more than a muse. A Resistance fighter, a survivor of deporta...
17/11/2026
Multiband dynamics plug-in boasts FT1 integration
Freqport's latest plug-in combines multiband upward and downward dynamics with the company's renow...
17/11/2026
Hi-fi experts launch two pro-audio models
Berlin-based hi-fi experts Teufel Audio have recently branched out into the pro-audio world with their first pairs...
17/11/2026
A year of ComposerCloud+ for $99
EastWest's annual Black Friday Sale is already live, and new users are being offered one year of the all-encompassing C...
17/11/2026
Pick your favourite gear from 2026
The annual SOS Awards are now underway, and as always, we're asking readers to help us choose the best pro-audio rele...
17/11/2026
As part of an ongoing series production program for a German automaker, production capacity has been increased at SGL Carbon's Austrian site in Ried im Innk...
17/11/2026
eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({...
17/11/2026
Signiant has acquired the media intelligence technology, Ceivo, developed by Burnlab with founder Aaron Sloman joining the company as Vice President, Product Ma...
17/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
17/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
17/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
17/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
17/11/2026
Berklees 2027 Alumni Grammy Nominees: See the Full List From pop and R&B to jazz, musical theater, and video game scores, Berklee talent is represented across...
17/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
17/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
17/11/2026
LA JOLLA, CA-Two Scripps Research faculty, Assistant Professor Christian Diercks...
17/11/2026
Our Biggest Sale of the Year on Ivory 3!
From November 17th through November 30th, enjoy our best savings of the year on Ivory 3 Products!
Whether it's ou...
17/11/2026
Tuesday 17 November 2026
Sky announces major free upgrade to the Sky Glass and ...
17/11/2026
Tuesday 17 November 2026
Passion, Betrayal and Scandal: Sky Arts Reveals the Ma...
17/11/2026
Exact 3D Video, a web service based in Japan, has launched its English version, ...
17/11/2026
Today, Apple Podcasts named Baby, this is Keke Palmer its Show of the Year, hono...
17/11/2026
NVIDIA's Ian Buck kicked off the company's week at SC26 in Chicago on Monday with new tools for AI agents, software that can plan experiments, run code ...
16/11/2026
The ESPN Edge Innovation Conference once again gave key ESPN technology partners as a chance to take to the stage and discuss the future of sports fan experienc...
16/11/2026
As rights holders rethink who tells their stories and where, IMG is preparing th...
16/11/2026
From the sidelines, sonic savants advocate for greater emphasis on sound as sports and entertainment increasingly merge
SVG Live!, a one-day event on Nov. 10 i...
16/11/2026
Deal combines Yamaha's audio technology with Disguise's visual-productio...
16/11/2026
Creative effects designed to encourage experimentation
Berry Effects is the brainchild of Abe and Ralph, a pair of multi-instrumentalists and audio engineer...
16/11/2026
Physics-based synth gets first major update
Anukari have just released a new version of their innovative physics-based software synth, marking the instrumen...
16/11/2026
Both versions kitted out with new tools & improvements
Hit'n'Mix have just launched the latest major version of their innovative DAW software, with ...
16/11/2026
Dynamic bands, unmasking features & new stereo tools
Launched in 2021, Eventide's SplitEQ plug-in introduced an interesting new approach to EQ, with the...