Large language model development is about to reach supersonic speed thanks to a collaboration between NVIDIA and Anyscale.
At its annual Ray Summit developers conference, Anyscale - the company behind the fast growing open-source unified compute framework for scalable computing - announced today that it is bringing NVIDIA AI to Ray open source and the Anyscale Platform. It will also be integrated into Anyscale Endpoints, a new service announced today that makes it easy for application developers to cost-effectively embed LLMs in their applications using the most popular open source models.
These integrations can dramatically speed generative AI development and efficiency while boosting security for production AI, from proprietary LLMs to open models such as Code Llama, Falcon, Llama 2, SDXL and more.
Developers will have the flexibility to deploy open-source NVIDIA software with Ray or opt for NVIDIA AI Enterprise software running on the Anyscale Platform for a fully supported and secure production deployment.
Ray and the Anyscale Platform are widely used by developers building advanced LLMs for generative AI applications capable of powering intelligent chatbots, coding copilots and powerful search and summarization tools.
NVIDIA and Anyscale Deliver Speed, Savings and Efficiency Generative AI applications are captivating the attention of businesses around the globe. Fine-tuning, augmenting and running LLMs requires significant investment and expertise. Together, NVIDIA and Anyscale can help reduce costs and complexity for generative AI development and deployment with a number of application integrations.
NVIDIA TensorRT-LLM, new open-source software announced last week, will support Anyscale offerings to supercharge LLM performance and efficiency to deliver cost savings. Also supported in the NVIDIA AI Enterprise software platform, Tensor-RT LLM automatically scales inference to run models in parallel over multiple GPUs, which can provide up to 8x higher performance when running on NVIDIA H100 Tensor Core GPUs, compared to prior-generation GPUs.
TensorRT-LLM automatically scales inference to run models in parallel over multiple GPUs and includes custom GPU kernels and optimizations for a wide range of popular LLM models. It also implements the new FP8 numerical format available in the NVIDIA H100 Tensor Core GPU Transformer Engine and offers an easy-to-use and customizable Python interface.
NVIDIA Triton Inference Server software supports inference across cloud, data center, edge and embedded devices on GPUs, CPUs and other processors. Its integration can enable Ray developers to boost efficiency when deploying AI models from multiple deep learning and machine learning frameworks, including TensorRT, TensorFlow, PyTorch, ONNX, OpenVINO, Python, RAPIDS XGBoost and more.
With the NVIDIA NeMo framework, Ray users will be able to easily fine-tune and customize LLMs with business data, paving the way for LLMs that understand the unique offerings of individual businesses.
NeMo is an end-to-end, cloud-native framework to build, customize and deploy generative AI models anywhere. It features training and inferencing frameworks, guardrailing toolkits, data curation tools and pretrained models, offering enterprises an easy, cost-effective and fast way to adopt generative AI.
Options for Open-Source or Fully Supported Production AI Ray open source and the Anyscale Platform enable developers to effortlessly move from open source to deploying production AI at scale in the cloud.
The Anyscale Platform provides fully managed, enterprise-ready unified computing that makes it easy to build, deploy and manage scalable AI and Python applications using Ray, helping customers bring AI products to market faster at significantly lower cost.
Whether developers use Ray open source or the supported Anyscale Platform, Anyscale's core functionality helps them easily orchestrate LLM workloads. The NVIDIA AI integration can help developers build, train, tune and scale AI with even greater efficiency.
Ray and the Anyscale Platform run on accelerated computing from leading clouds, with the option to run on hybrid or multi-cloud computing. This helps developers easily scale up as they need more computing to power a successful LLM deployment.
The collaboration will also enable developers to begin building models on their workstations through NVIDIA AI Workbench and scale them easily across hybrid or multi-cloud accelerated computing once it's time to move to production.
NVIDIA AI integrations with Anyscale are in development and expected to be available by the end of the year.
Developers can sign up to get the latest news on this integration as well as a free 90-day evaluation of NVIDIA AI Enterprise.
To learn more, attend the Ray Summit in San Francisco this week or watch the demo video below.
See this notice regarding NVIDIA's software roadmap.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
01/06/2026
January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026
Throughout the week, Dolby brings to life the latest innovatio...
01/05/2026
January 5 2026, 18:30 (PST) NBCUniversal's Peacock to Be First Streamer to ...
01/04/2026
January 4 2026, 18:00 (PST) DOLBY AND DOUYIN EMPOWER THE NEXT GENERATON OF CREATORS WITH DOLBY VISION
Douyin Users Can Now Create And Share Videos With Stun...
21/01/2026
Wisycom, a global leader in advanced wireless RF solutions, launches its new wideband antenna matrix, MATF, which supports RF and fiber for demanding multi-zone...
21/01/2026
Grass Valley will demonstrate how it is powering scalable, future-ready live production at FOMEX 2026, taking place February 2 4 in Riyadh, Saudi Arabia. Exhibi...
21/01/2026
BCNEXXT, the developers of the advanced playout platform Vipe, today announced that OKAST, the monetization-first OTT platform provider, is using BCNEXXT's ...
21/01/2026
Revamped design enables advanced capabilities, leading with powerful IP to HDMI conversion
Magewell, developer of innovative, high-performance video I/O and I...
21/01/2026
Jan 20th 2026, Changsha Kiloview today announced the launch of two major additions to its AV-over-IP ecosystem: the AVX24-4 Media HUB and KiloLink Station, ma...
21/01/2026
Latest version of enterprise-class Buttons brings simple, coherent control to more than 700 professional devices and applications
Bitfocus, the specialist in ...
21/01/2026
Clear-Com is pleased to announce the appointment of Kari Eythorsson as the new Regional Sales Manager (RSM) for Southeast Asia & Australia, based in Singapore,...
21/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
21/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
21/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
21/01/2026
Share Share by:
Copy link
Facebook
X
Linkedin
Pinterest
Bluesky
Email...
21/01/2026
RT News is pleased to announce the appointment of Jackie Fox as its new Washington Correspondent. Jackie is a multimedia journalist with RT News and has repor...
21/01/2026
From skilled trades to startups, AI's rapid expansion is the beginning of th...
20/01/2026
SVG Sit-Down: BitFire's Jim Akimchuk on the Future of Cloud, Remote, Softwar...
20/01/2026
From architecture to reality: What CBC's Milano Cortina 2026 workflows tell ...
20/01/2026
Milano Cortina 2026: Making the Winter Olympics and Paralympics workflow work fo...
20/01/2026
Countdown to Milano Cortina 2026: SVG Launches SportsTechLive Blog in Lead-up to...
20/01/2026
Grand finale: Rally Saudi Arabia pushes WRC to make the most of cameras and dron...
20/01/2026
State of Change: What's Happening in the Remote-Production Business?NEP Group, Game Creek Video, Dome Productions, Mobile TV Group, Program Productions exec...
20/01/2026
CFP National Championship 2026: A Look Back at SVG's Complete CoverageGo behind the scenes with ESPN, Game Creek Video, Van Wagner, and the champion Indiana...
20/01/2026
The Gotham, Film Independent, and Creators Coalition on AI Join Alliance
by Michelle Satter, Founding Senior Director, Sundance Institute's Artist Programs...
20/01/2026
The National Film and Video Foundation (NFVF), an agency of the Department of Sp...
20/01/2026
December, the first month of winter and the holiday season, has traditionally encouraged families to spend time together in front of the television. During this...
20/01/2026
During December, streaming's share of TV viewing in Mexico settled at 24.3%, an increase of 0.1 share points from the previous month.
Disclaimer: YUMI TV,...
20/01/2026
Christmas Day is Most-Streamed Day Ever with 55 Billion Viewing Minutes, led by...
20/01/2026
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
20/01/2026
Disguise and ASB GlassFloor today announced a strategic partnership that turns the arena floor into a fully customizable interactive digital surface. This will ...
20/01/2026
Last summer, ITV celebrated the tenth anniversary of its popular reality dating show, Love Island. To ramp up the excitement of series 12 (launched in the UK on...
20/01/2026
Berklee's Mark Ethier Named One of SPIN's Most Influential People in Mus...
20/01/2026
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
20/01/2026
Share Share by:
Copy link
Facebook
X
Whatsapp
Pinterest
Flipboard...
20/01/2026
Emergent, a provider of AI-powered media production solutions, today announced its presence at ISE 2026 in Barcelona, taking place February 3 6, where it will u...
20/01/2026
Back to All News
Netflix and Warner Bros. Discovery Amend Agreement to All-Cash Transaction
Business
20 January 2026
Global
Link copied to clipboard
All-C...
20/01/2026
Back to All News
Now You Can Play a Role in Our Live Events With Real-Time Voting
Elmar Nubbemeyer
VP, Member Product
Product
20 January 2026
Global
Link...
19/01/2026
CFP National Championship 2026: Indiana Football's Digital Crew Generate Hyp...
19/01/2026
CFP National Championship 2026: Van Wagner Relies on Experience for Indiana-Miam...
19/01/2026
CFP National Championship 2026: Game Creek Video Fields Premiere IP Compound for...
19/01/2026
CFP National Championship 2026: For a Slew of Studio Shows, ESPN Turns Hard Rock...
19/01/2026
CFP National Championship 2026: ESPN's Sweeping Live Game Production Leverag...
19/01/2026
The new L3Harris NOVA binocular system is the every-soldier goggle' - and the beginning of a new era for night-vison capability....
19/01/2026
The Majority of Those Surveyed Prefer to Buy Brands Who Advertise in Content Tha...
19/01/2026
Back to All News
Straight to Hell' Sets April 27 Premiere: Teaser Trailer,...
19/01/2026
New Workflow Enhancements for All Broadcast Pix Systems, Plus 4-Channel ISO Recording for Hybrid and Roadie Tyngsboro, Mass. - January 2025 - Broadcast Pix ann...
19/01/2026
Monday 19 January 2026
Sky News reinvents News at Ten for the modern audience with The Wrap
Sky News today announces The Wrap, a bold evolution of its 10pm ou...
19/01/2026
Back to All News
Netflix Announces the Series Nights & Days, Directed by Kamila...
Large language model development is about to reach supersonic speed thanks to a collaboration between NVIDIA and Anyscale.
