Sony Pixel Power calrec Sony

Ray Shines with NVIDIA AI: Anyscale Collaboration to Help Developers Build, Tune, Train and Scale Production LLMs

18/09/2023

Large language model development is about to reach supersonic speed thanks to a collaboration between NVIDIA and Anyscale.

At its annual Ray Summit developers conference, Anyscale - the company behind the fast growing open-source unified compute framework for scalable computing - announced today that it is bringing NVIDIA AI to Ray open source and the Anyscale Platform. It will also be integrated into Anyscale Endpoints, a new service announced today that makes it easy for application developers to cost-effectively embed LLMs in their applications using the most popular open source models.

These integrations can dramatically speed generative AI development and efficiency while boosting security for production AI, from proprietary LLMs to open models such as Code Llama, Falcon, Llama 2, SDXL and more.

Developers will have the flexibility to deploy open-source NVIDIA software with Ray or opt for NVIDIA AI Enterprise software running on the Anyscale Platform for a fully supported and secure production deployment.

Ray and the Anyscale Platform are widely used by developers building advanced LLMs for generative AI applications capable of powering intelligent chatbots, coding copilots and powerful search and summarization tools.

NVIDIA and Anyscale Deliver Speed, Savings and Efficiency Generative AI applications are captivating the attention of businesses around the globe. Fine-tuning, augmenting and running LLMs requires significant investment and expertise. Together, NVIDIA and Anyscale can help reduce costs and complexity for generative AI development and deployment with a number of application integrations.

NVIDIA TensorRT-LLM, new open-source software announced last week, will support Anyscale offerings to supercharge LLM performance and efficiency to deliver cost savings. Also supported in the NVIDIA AI Enterprise software platform, Tensor-RT LLM automatically scales inference to run models in parallel over multiple GPUs, which can provide up to 8x higher performance when running on NVIDIA H100 Tensor Core GPUs, compared to prior-generation GPUs.

TensorRT-LLM automatically scales inference to run models in parallel over multiple GPUs and includes custom GPU kernels and optimizations for a wide range of popular LLM models. It also implements the new FP8 numerical format available in the NVIDIA H100 Tensor Core GPU Transformer Engine and offers an easy-to-use and customizable Python interface.

NVIDIA Triton Inference Server software supports inference across cloud, data center, edge and embedded devices on GPUs, CPUs and other processors. Its integration can enable Ray developers to boost efficiency when deploying AI models from multiple deep learning and machine learning frameworks, including TensorRT, TensorFlow, PyTorch, ONNX, OpenVINO, Python, RAPIDS XGBoost and more.

With the NVIDIA NeMo framework, Ray users will be able to easily fine-tune and customize LLMs with business data, paving the way for LLMs that understand the unique offerings of individual businesses.

NeMo is an end-to-end, cloud-native framework to build, customize and deploy generative AI models anywhere. It features training and inferencing frameworks, guardrailing toolkits, data curation tools and pretrained models, offering enterprises an easy, cost-effective and fast way to adopt generative AI.

Options for Open-Source or Fully Supported Production AI Ray open source and the Anyscale Platform enable developers to effortlessly move from open source to deploying production AI at scale in the cloud.

The Anyscale Platform provides fully managed, enterprise-ready unified computing that makes it easy to build, deploy and manage scalable AI and Python applications using Ray, helping customers bring AI products to market faster at significantly lower cost.

Whether developers use Ray open source or the supported Anyscale Platform, Anyscale's core functionality helps them easily orchestrate LLM workloads. The NVIDIA AI integration can help developers build, train, tune and scale AI with even greater efficiency.

Ray and the Anyscale Platform run on accelerated computing from leading clouds, with the option to run on hybrid or multi-cloud computing. This helps developers easily scale up as they need more computing to power a successful LLM deployment.

The collaboration will also enable developers to begin building models on their workstations through NVIDIA AI Workbench and scale them easily across hybrid or multi-cloud accelerated computing once it's time to move to production.

NVIDIA AI integrations with Anyscale are in development and expected to be available by the end of the year.

Developers can sign up to get the latest news on this integration as well as a free 90-day evaluation of NVIDIA AI Enterprise.

To learn more, attend the Ray Summit in San Francisco this week or watch the demo video below.

See this notice regarding NVIDIA's software roadmap.
LINK: https://blogs.nvidia.com/blog/2023/09/18/llm-anyscale-nvaie/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

16/11/2026

ESPN Edge Partners Chart a Future Built on Data, AI and a New Generation of Fans

The ESPN Edge Innovation Conference once again gave key ESPN technology partners as a chance to take to the stage and discuss the future of sports fan experienc...

16/11/2026

IMGs Ashley Shaner: How Micro-Dramas Can Build Sports Media Audiences and Why Content Creators Matter

As rights holders rethink who tells their stories and where, IMG is preparing th...

16/11/2026

Heard' at SVG Live!: Audio Needs To Make More Noise

From the sidelines, sonic savants advocate for greater emphasis on sound as sports and entertainment increasingly merge SVG Live!, a one-day event on Nov. 10 i...

16/11/2026

Yamaha To Acquire Disguise, Expanding Into Live Sports, Venue Production, and Immersive Entertainment

Deal combines Yamaha's audio technology with Disguise's visual-productio...

16/11/2026

New plug-in suite from Berry Effects

Creative effects designed to encourage experimentation Berry Effects is the brainchild of Abe and Ralph, a pair of multi-instrumentalists and audio engineer...

16/11/2026

Anukari 1.1 now available

Physics-based synth gets first major update Anukari have just released a new version of their innovative physics-based software synth, marking the instrumen...

16/11/2026

Hit'n'Mix RipX DAW 9 arrives

Both versions kitted out with new tools & improvements Hit'n'Mix have just launched the latest major version of their innovative DAW software, with ...

16/11/2026

Eventide to launch SplitEQ 2

Dynamic bands, unmasking features & new stereo tools Launched in 2021, Eventide's SplitEQ plug-in introduced an interesting new approach to EQ, with the...

16/11/2026

Rohde & Schwarz Japan demonstrates multi-vendor ASA-ML communication with highperformance oscilloscope

Rohde & Schwarz Japan demonstrates multi-vendor ASA-ML communication with high p...

16/11/2026

CNAO and Hitachi launch advanced rotating proton therapy system in Italy

November 16, 2026 Hitachi system enables proton beams to be delivered from 360 degrees around the patient, expanding treatment capacity and supporting more pre...

16/11/2026

NBC Celebrates Centennial

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

16/11/2026

Apple TV Adding Podcasts

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

16/11/2026

Fabric Insights Partners With Fonn Group on Enhancing Media Workflows

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

16/11/2026

SDVI Names Marcy Lefkovitz as VP of Strategic Accounts

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

16/11/2026

Monthly Streaming Costs Up, Internet Fees Down for 2026

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

16/11/2026

Hexagon Introduces Camera-Mounted 3D Capture and Tracking System

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

16/11/2026

Bridge Technologies Adds Apple ProRes Support to VB440 Probe

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

16/11/2026

Rebuilding Broadcast on Air: Inside the 2026 SMPTE Media Technology Summit

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

16/11/2026

Hiltron HMAM Chosen by Major Satcom Service Provider

Hiltron Communications has received a repeat order from one of Europes longest established communications satellite operators. Central to this project is a Hilt...

16/11/2026

Fabric Insights and Fonn Group Partner to Unlock Media Wo...

Fabric Insights and Fonn Group today announce a partnership to connect editorial planning and media operations across their complementary product portfolios. ...

16/11/2026

Density 1.0 brings a complete color grading workflow to Final Cut Pro

Density 1.0 brings a complete color grading workflow to Final Cut Pro Brie Clayton November 16, 2026 0 Comments Filmmaker Pascal Basel's plugin pr...

16/11/2026

Berklee Students to Perform at Boston's CITGO Sign Relighting

Berklee Students to Perform at Boston's CITGO Sign Relighting Kuper Walker, Ollie Mann, and dreamcorp. will perform at the free public celebration in Kenm...

16/11/2026

The Hollywood Reporter Names Berklee Among World's Best Music Schools

The Hollywood Reporter Names Berklee Among World's Best Music Schools The annual list spotlights Berklee's screen scoring programs and new master'...

16/11/2026

Taurus Rooms - A Five-Year Vision Proven In Meeting Rooms

Five years ago, Lightware launched the original Taurus UCX with a clear vision: USB-C would become the central connection in meeting rooms. The goal was simple:...

16/11/2026

Sonniss: bundles over 80% below their libraries' prices, plus up to 60% off over 2,700 sound effects libraries

Sonniss: bundles over 80% below their libraries' prices, plus up to 60% off ...

16/11/2026

Singaporean Comedy Luck My Life Shot with Blackmagic URSA Cine 17K 65

Singaporean Comedy Luck My Life Shot with Blackmagic URSA Cine 17K 65 Brie Clayton November 16, 2026 0 Comments Blackmagic PYXIS 12K and DaVinci Resol...

16/11/2026

U sees new original drama Fleet Street from the creators of Miss Scarlet

Images available HERE UKTV has announced Fleet Street (6x60'), a new period crime drama from Miss Scarlet creators Rachael New and Ben Edwards, coming to f...

16/11/2026

Tribeca Festival Lisboa Unveils 2026 Film Lineup And Talent Roster For Third Annual Edition, December 9-13

November 16th, 2026 TRIBECA FESTIVAL LISBOA UNVEILS 2026 FILM LINEUP AND TALE...

16/11/2026

Sky and NOW bring some of the UKs biggest video podcasts to the big screen at no extra cost with Platform Media as first partner

From Graham Norton to Olivia Attwood and Joe Marler, a new collection of hit vid...

16/11/2026

Can Digital Product Passports help meet climate goals?

As countries gather to discuss climate targets at COP 31 this week, one question that is at the heart of debates is how to measure the environmental impact of t...

16/11/2026

NVIDIA Announces Rubin HPC - A New GPU to Accelerate Agents for Science, Engineering and Analytics

Announced today at SC26, NVIDIA Rubin HPC is a new GPU in the NVIDIA Vera Rubin ...

16/11/2026

CUDA at 20: How the Platform Transformed Scientific Computing

This week at SC26, NVIDIA is marking two decades of CUDA by honoring the researchers, developers and engineers who helped make GPU computing a foundation for mo...

16/11/2026

NVIDIA CUDA-X Libraries Give Agents the Tools to Accelerate Scientific Discovery

With expanded NVIDIA CUDA-X libraries, including cuPhoton, cuPotential and cuSpa, researchers can sift through vast amounts of data to accelerate research, from...

16/11/2026

NVIDIA Sweeps MLPerf Training and Endpoints as Vera Rubin Debuts With Leading Performance

In its MLPerf benchmarks debut, with results released today, the NVIDIA Vera Rub...

16/11/2026

NVIDIA Expands Slurm to Bring Quantum Computing Into Supercomputing

Quantum processor deployments at supercomputing centers have grown nearly sevenfold over the past five years, with nearly 30 supercomputing centers around the w...

16/11/2026

Lambda Boosts AI Factory Resiliency and Efficiency With NVIDIA Photonics

To train an AI model, thousands of GPUs have to work together, requiring a network that's fast, efficient and resilient. Lambda is tackling the network pie...

15/11/2026

Yamaha launch MGX, URX & CC1 updates

Mixers, interfaces & DAW controller receive new firmware Yamaha have just launched a trio of firmware updates that bring some powerful new functionality to ...

15/11/2026

Can VideoProc AI Really Reduce Noise and Enhance Blurry, Old Footage? Real-World Tests

Can VideoProc AI Really Reduce Noise and Enhance Blurry, Old Footage? Real-World...

15/11/2026

AI Day Seoul Brings Together Korea's Industry Leaders, Startups and Developer Communities

Over two days at COEX, more than 2,000 attendees converged for AI Day Seoul, bri...

14/11/2026

MLB Networks David Maybrown, 57, Passes Away

Longtime MLB Network employee David Maybrown passed away Thursday at the age of 57. David joined MLB Network in 2009 shortly after its launch, beginning as an i...

14/11/2026

Rotary Exciter & Resonator Clamps from ADDAC System

Modular innovators introduce two guitar-based products ADDAC System's recently introduced Four String Series makes it possible to integrate physical str...

14/11/2026

Marcelo Dai's Live Session Captured with URSA Cine Immersive

Marcelo Dai's Live Session Captured with URSA Cine Immersive Brie Clayton November 14, 2026 0 Comments Artist takes viewers on an immersive journe...

14/11/2026

Political Ads Spiked a Week Before Midterm Elections, Nielsen Says

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

14/11/2026

FCC Sends 2027 Upper C-Band Auction Plan Into Motion

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

14/11/2026

ZEISS Showcases Latest Cine Lenses at TorunCAMERIMAGE 202...

Newest Lenses, Horizon Anamorphic Seminar & Guided Lens Character Experience Highlight Creative Possibilities At the 2026 Toru CAMERIMAGE Film Festival, ZEISS...

14/11/2026

Sonnet Announces Black Friday Sale on Select Products

Sonnet Announces Black Friday Sale on Select Products Brie Clayton November 14, 2026 0 Comments WHAT: Sonnet Technologies is having a sale on a select...

14/11/2026

Berklee Alumni Shine at the 2026 Latin Grammy Awards

Berklee Alumni Shine at the 2026 Latin Grammy Awards This year's winners included songwriters, engineers, and performers who contributed to acclaimed reco...

13/11/2026

E1 Series Brings Its Remote-First Broadcast Model to Miami for Season Finale

The UIM E1 Series wraps its third season this week with the Bombay Sapphire E1 Miami GP, Nov. 11-14, on Biscayne Bay at the Miami Seaplane Base on Watson Island...

13/11/2026

SVG GameDay, Ep. 39: Carolina Hurricanes Christine Williams & Jason Danby - Celebrating a Stanley Cup in Raleigh

In-venue and creative video staffers at the professional and collegiate level ha...