
Large language model development is about to reach supersonic speed thanks to a collaboration between NVIDIA and Anyscale.
At its annual Ray Summit developers conference, Anyscale - the company behind the fast growing open-source unified compute framework for scalable computing - announced today that it is bringing NVIDIA AI to Ray open source and the Anyscale Platform. It will also be integrated into Anyscale Endpoints, a new service announced today that makes it easy for application developers to cost-effectively embed LLMs in their applications using the most popular open source models.
These integrations can dramatically speed generative AI development and efficiency while boosting security for production AI, from proprietary LLMs to open models such as Code Llama, Falcon, Llama 2, SDXL and more.
Developers will have the flexibility to deploy open-source NVIDIA software with Ray or opt for NVIDIA AI Enterprise software running on the Anyscale Platform for a fully supported and secure production deployment.
Ray and the Anyscale Platform are widely used by developers building advanced LLMs for generative AI applications capable of powering intelligent chatbots, coding copilots and powerful search and summarization tools.
NVIDIA and Anyscale Deliver Speed, Savings and Efficiency Generative AI applications are captivating the attention of businesses around the globe. Fine-tuning, augmenting and running LLMs requires significant investment and expertise. Together, NVIDIA and Anyscale can help reduce costs and complexity for generative AI development and deployment with a number of application integrations.
NVIDIA TensorRT-LLM, new open-source software announced last week, will support Anyscale offerings to supercharge LLM performance and efficiency to deliver cost savings. Also supported in the NVIDIA AI Enterprise software platform, Tensor-RT LLM automatically scales inference to run models in parallel over multiple GPUs, which can provide up to 8x higher performance when running on NVIDIA H100 Tensor Core GPUs, compared to prior-generation GPUs.
TensorRT-LLM automatically scales inference to run models in parallel over multiple GPUs and includes custom GPU kernels and optimizations for a wide range of popular LLM models. It also implements the new FP8 numerical format available in the NVIDIA H100 Tensor Core GPU Transformer Engine and offers an easy-to-use and customizable Python interface.
NVIDIA Triton Inference Server software supports inference across cloud, data center, edge and embedded devices on GPUs, CPUs and other processors. Its integration can enable Ray developers to boost efficiency when deploying AI models from multiple deep learning and machine learning frameworks, including TensorRT, TensorFlow, PyTorch, ONNX, OpenVINO, Python, RAPIDS XGBoost and more.
With the NVIDIA NeMo framework, Ray users will be able to easily fine-tune and customize LLMs with business data, paving the way for LLMs that understand the unique offerings of individual businesses.
NeMo is an end-to-end, cloud-native framework to build, customize and deploy generative AI models anywhere. It features training and inferencing frameworks, guardrailing toolkits, data curation tools and pretrained models, offering enterprises an easy, cost-effective and fast way to adopt generative AI.
Options for Open-Source or Fully Supported Production AI Ray open source and the Anyscale Platform enable developers to effortlessly move from open source to deploying production AI at scale in the cloud.
The Anyscale Platform provides fully managed, enterprise-ready unified computing that makes it easy to build, deploy and manage scalable AI and Python applications using Ray, helping customers bring AI products to market faster at significantly lower cost.
Whether developers use Ray open source or the supported Anyscale Platform, Anyscale's core functionality helps them easily orchestrate LLM workloads. The NVIDIA AI integration can help developers build, train, tune and scale AI with even greater efficiency.
Ray and the Anyscale Platform run on accelerated computing from leading clouds, with the option to run on hybrid or multi-cloud computing. This helps developers easily scale up as they need more computing to power a successful LLM deployment.
The collaboration will also enable developers to begin building models on their workstations through NVIDIA AI Workbench and scale them easily across hybrid or multi-cloud accelerated computing once it's time to move to production.
NVIDIA AI integrations with Anyscale are in development and expected to be available by the end of the year.
Developers can sign up to get the latest news on this integration as well as a free 90-day evaluation of NVIDIA AI Enterprise.
To learn more, attend the Ray Summit in San Francisco this week or watch the demo video below.
See this notice regarding NVIDIA's software roadmap.
Most recent headlines
04/12/2023
Dalet, a leading technology and service provider for media-rich organizations, announced the release of Dalet Cut, the cloud-native, lightning-fast multimedia a...
30/09/2023
With this summers strikes hurting the flow of new content into TV and streaming platforms, live sports have become an even more important programming staple, wi...
30/09/2023
Viewing of the second GOP debate on Sept. 27 fell to 4.1 million homes, a notable 39% drop from the 6.6 million homes that watched that first GOP debate, accord...
30/09/2023
WASHINGTON, D.C. In the long-running regulatory saga of the FCC's station ownership rules, the U.S. Court of Appeals for the D.C. Circuit has issued a rulin...
30/09/2023
Calrec Audio Ltd. will feature its latest solutions to help broadcasters and OB suppliers transition to IP workflows at NAB Show New York, Oct. 25 26....
30/09/2023
Northern Minnesota residents can watch live City Council meetings and more on all viewing platforms thanks to broad network stream support
Between support for...
30/09/2023
The classic Instreamer-to-Exstreamer combination comes through for Spanish language audiences in the United States and Mexico, with the Instreamer handling remo...
30/09/2023
Zeros in Ones Hits the Road with AJA's Dante AV 4K-R and 4K-T
Brie Clayton September 29, 2023
0 Comments
As founder of video production company Zer...
30/09/2023
Arcturus Appoints Former Microsoft and Lucasfilm Veteran Steve Sullivan as Chief...
30/09/2023
Vodafone Studios Boosts Audiovisual Production Capabilities with Blackmagic Desi...
30/09/2023
Moving Above LTO - How Cloud Storage Helps One Texas Church Deliver a Modern Dig...
30/09/2023
New Pass the Mic Podcast to Highlight Inspiring Berklee Alumni Listen to the first episode featuring Captain Marvel composer Pinar Toprak B.M. '00.
By
Dan...
30/09/2023
Live from the 2023 Ryder Cup: Sky Sports Virtual Studio in UK Plays Big Role in ...
29/09/2023
One of the most exciting things about the Sundance Film Festival is having a fro...
29/09/2023
You might know all the songs and albums of your favorite musicians, but do you know the experiences and inspirations behind their work? Luckily, you can find ou...
29/09/2023
There are times when the daily grind might not leave you feeling the most inspired. But it doesn't always have to be like that. Words of wisdom may be just ...
29/09/2023
Sometimes you might notice a thoughtful review from a friend on your feed, or stumble upon a flash summary straight out of your favorite social influencer's...
29/09/2023
Whether it's chilling on the couch or crashing into bed, nothing beats settling into a nice long rest at the end of the day. And few things are more frustra...
29/09/2023
Which was better, the book or the movie?
It's a common argument that has withstood the test of time and has only gotten stronger as more and more hit tele...
29/09/2023
Whether you like to begin your day the moment you get out of bed or ease gently into it with a cup of coffee, you can count on audiobooks to boost your mornings...
29/09/2023
With elaborate storytelling, immersive worlds, and an enthusiastic fandom hyping it up, getting into a new book can be an exciting and fun moment. And with the ...
29/09/2023
Fans flock to music festivals to see dozens of their favorite artists in one go,...
29/09/2023
Now streaming on Disney+, Launchpad Season 2 comprises six original live-action short films from underrepresented filmmakers whose unique voices bring new persp...
29/09/2023
Wasabi hot cloud storage is purpose-built to store the world's data. Wasabi ...
29/09/2023
Sohonet, the global experts in media collaboration for the film and TV industry, announce a series of significant product updates set to redefine and streamline...
29/09/2023
Current and incoming L3Harris innovations provide warfighters immersive battlefield awareness capabilities during the day and at night - with the ability to pus...
29/09/2023
After graduating high school at the age of 16, I left my native Venezuela and moved to Miami, Florida, with the goal of becoming an electrical engineer. The fol...
29/09/2023
L3Harris employees in Mirabel, Quebec, proudly wore their Mother Bear Energy shirts on September 28, 2023....
29/09/2023
The 2023 edition of GSMA MWC Las Vegas saw a rise in attendees from vertical ind...
29/09/2023
Timed to coincide with the Utah Jazz's 50th season celebration, SEG Media has launched the direct-to-consumer streaming service Jazz+, which offers exclusiv...
29/09/2023
WASHINGTON, D.C. The National Association of Broadcasters (NAB) hosted a virtual town hall for members on Sept. 28 where it provided updates on NAB's policy...
29/09/2023
SINGAPORE Ikegami has announced that RE:LIVE Productions, one of Singapore's leading providers of creative video, event management and live streaming servic...
29/09/2023
- The popular showcase for the latest production solutions, technology and insights returns for its third year -
CVP, one of Europe s leading resellers and pro...
29/09/2023
Well known for cross-divisional collaboration and staying connected with our for...
29/09/2023
Berklee Abu Dhabi Convenes Community Leaders for Creative Arts Educators Symposi...
29/09/2023
TransUnion said it is working with Crackle Connex, Chicken Soup For the Soul Entertainment's advertising sales unit, to create custom audiences from first-p...
29/09/2023
Spotify and Roku said they are bringing video ads to the Spotify app on the Roku platform....
29/09/2023
The Rock & Roll Hall of Fame induction ceremony is moving from HBO to Disney Plus and ABC....
29/09/2023
Auto entertainment network Speedvision said it is available as a free ad-supported streaming television (FAST) channel on Xumo Play....
29/09/2023
Scripps News said it hired Chris Nguyen as an anchor....
29/09/2023
The first-ever People's Choice Country Awards happens on NBC and Peacock Thursday, September 28. Little Big Town hosts from the Grand Ole Opry House in Nash...
29/09/2023
Seeing their docuseries Rebuilding Black Wall Street appear on the Oprah Winfrey...
29/09/2023
Scripps News got its first national News Emmy for its docuseries In Real Life. Scripps News won in the outstanding science, technology or environmental coverage...
29/09/2023
John Brohel has been named chief executive officer of Locality, formed in June when CoxReps and Gamut were combined....
29/09/2023
An animal act has won season 18 of America's Got Talent, as Adrian Stoica and his dog Hurricane were the top vote getters. They won $1 million and join the ...
29/09/2023
Zest4.tv's managing director, Simon Roe, looks at the ways in which compact OB Vans are the epitome of efficiency, proving that size does not limit their po...
29/09/2023
TVBEurope talks to the team behind the film about Eddie Jones and the Japanese r...
29/09/2023
Ashish Patel has been appointed president, MediaKind operator platforms, and Tony Goncalves joins as strategic advisor
By Jenny Priestley
Published: Septembe...
29/09/2023
Building on the momentum of major product announcements and award wins at NAB Show earlier this year, Chyron last week concluded a successful showing at IBC2023...
29/09/2023
Visit Matrox Video at Smart Building Expo 2023 in Hall 6P, Stand B37
Technology innovator Matrox Video today announced its lineup for the Smart Building Expo 2...