
An avid cyclist, Thomas Park knows the value of having lots of gears to maintain a smooth, fast ride.
So, when the software architect designed an AI inference platform to serve predictions for Oracle Cloud Infrastructure's (OCI) Vision AI service, he picked NVIDIA Triton Inference Server. That's because it can shift up, down or sideways to handle virtually any AI model, framework and hardware and operating mode - quickly and efficiently.
The NVIDIA AI inference platform gives our worldwide cloud services customers tremendous flexibility in how they build and run their AI applications, said Park, a Zurich-based computer engineer and competitive cycler who's worked for four of the world's largest cloud services providers.
Specifically, Triton reduced OCI's total cost of ownership by 10%, increased prediction throughput up to 76% and reduced inference latency up to 51% for OCI Vision and Document Understanding Service models that were migrated to Triton. The services run globally across more than 45 regional data centers, according to an Oracle blog Park and a colleague posted earlier this year.
Computer Vision Accelerates Insights Customers rely on OCI Vision AI for a wide variety of object detection and image classification jobs. For instance, a U.S.-based transit agency uses it to automatically detect the number of vehicle axles passing by to calculate and bill bridge tolls, sparing busy truckers wait time at toll booths.
OCI AI is also available in Oracle NetSuite, a set of business applications used by more than 37,000 organizations worldwide. It's used, for example, to automate invoice recognition.
Thanks to Park's work, Triton is now being adopted across other OCI services, too.
A Triton-Aware Data Service Our AI platform is Triton-aware for the benefit of our customers , said Tzvi Keisar, a director of product management for OCI's Data Science service, which handles machine learning for Oracle's internal and external users.
If customers want to use Triton, they don't have to worry about the configuration because it will be done automatically by the service, launching a Triton-powered inference endpoint for them, said Keisar.
Triton is included in NVIDIA AI Enterprise, a platform that provides full security and support businesses need - and it's available on OCI Marketplace.
A Massive SaaS Platform OCI's Data Science service is the machine learning platform for both Oracle NetSuite and Oracle Fusion Applications.
These business application suites are massive, with tens of thousands of customers who are also building their frameworks on top of our service, he said.
It's a wide swath of mainly enterprise users in manufacturing, retail, transportation and other industries. They're building and using AI models of nearly every shape and size.
Inference was one of the group's first services, and Triton came on the team's radar not long after its launch.
A Best-in-Class Inference Framework We saw Triton pick up in popularity as a best-in-class serving framework, so we started experimenting with it, Keisar said. We saw really good performance, and it closed a gap in our existing offerings, especially on multi-model inference - it's the most versatile and advanced inferencing framework out there.
Launched on OCI in March, Triton has already attracted the attention of many internal teams at Oracle hoping to use it for inference jobs that require serving predictions from multiple AI models running concurrently.
Triton has a very good track record and performance on multiple models deployed on a single endpoint, he said.
Accelerating the Future Looking ahead, Keisar's team is evaluating NVIDIA TensorRT-LLM software to supercharge inference on the complex large language models (LLMs) that have captured the imagination of many users.
An active blogger, Keisar's latest article detailed quantization techniques for running a Llama 2 LLM with a whopping 70 billion parameters on NVIDIA A10 Tensor Core GPUs.
Even down to four-bit parameters, the quality of model outputs is still quite good, he said. Deploying on NVIDIA GPUs gives us the flexibility to find a good balance in latency, throughput and cost.
After announcements this fall that Oracle is deploying the latest NVIDIA H100 Tensor Core GPUs, H200 GPUs, L40S GPUs and Grace Hopper Superchips, it's just the start of many accelerated efforts to come.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
04/08/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
04/07/2026
April 7 2026, 19:00 (PDT) Detective Conan: Fallen Angel of the Highway Opens in...
01/06/2026
January 6 2026, 05:30 (PST) Dolby Sets the New Standard for Premium Entertainment at CES 2026
Throughout the week, Dolby brings to life the latest innovatio...
04/05/2026
just:play pro 2026 and just:live pro 2026 are available to download!
More Details:At NAB 2026, ToolsOnAir showcased just:play pro 2026 and just:live pro 2026, ...
04/05/2026
just:in mac pro 2026 - The Next Level of Professional Recording on macOS
More Details:The headline innovation in just:in mac pro 2026 is the new Auto format si...
04/05/2026
Hardware is still an emphasis - Supershooter 11 is new, and REMI-based 65 is in ...
04/05/2026
Head of International Business Development Min Joo Kim explores the league's...
04/05/2026
Audio-Technica has announced that its ATND1061 ceiling array microphone and ATUC...
04/05/2026
Triple B Media has launched Bowling TV, a free ad-supported television (FAST) channel dedicated to bowling. The channel is available on Prime Video, LG Channels...
04/05/2026
PlayMetrics, a provider of operations management software for youth sports organizations, has announced the completion of its acquisition of substantially all t...
04/05/2026
IHSE GmbH has announced that Dr. Thomas Niessen has joined as CEO and Managing Director, effective May 1, 2026. He joins Frank Breitenfelder, who has served as ...
04/05/2026
PMY Group deployed its AI-powered crowd intelligence platform, Optic, at the For...
04/05/2026
Behind The Mic provides a roundup of recent news regarding on-air talent, includ...
04/05/2026
Last week, guests gathered in New York City for On Air, In Style: An Evening with Spotify-a night of conversation, culture, and connection celebrating the inter...
04/05/2026
New music & post-production features added
Avid's latest DAW update delivers an array of helpful features aimed at both music and post-production users,...
04/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
04/05/2026
Student Spotlight: Joshua Griffin The New Orleans native, who was named the 2026 student commencement speaker for Boston Conservatory at Berklee, talks about ...
04/05/2026
It's Andrew! stomps onto screens this June 4 May 2026
The ABC and Screen Australia are delighted to announce that brand new preschool series, It's Andr...
03/05/2026
Polysynth now features Mutable Instruments' macro oscillators
Melbourne Instruments have just released a free firmware update that brings the engine beh...
03/05/2026
Introducing the new Mistika Workflows Suite: transformative and cost-effective f...
03/05/2026
Introducing the new Mistake Workflows Suite: transformative and cost-effective f...
03/05/2026
Back to All News
Filming begins on the third and final season of Breathless
Entertainment
03 May 2026
GlobalSpain
Link copied to clipboard
Discover the vi...
02/05/2026
(L-R) Dustin Hoffman and Leo Woodall appear in Tuner by Daniel Roher, an official selection of the 2026 Sundance Film Festival. (Photo courtesy of Sundance In...
02/05/2026
Versatile re-amping tool announced
Warm Audio are best known for their recreations of sought-after vintage studio gear, but their latest release brings a ne...
02/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
02/05/2026
Scripps Research immunologist Dennis Burton elected to American Academy of Arts and Sciences A leader in broadly neutralizing antibodies, Burton has helped driv...
02/05/2026
Dalet, a leading technology and service provider for media-rich organizations, t...
01/05/2026
Ratings Roundup is a rundown of recent rating news and is derived from press rel...
01/05/2026
BKB Bare Knuckle Boxing ( BKB ), today announced the appointment of Will Wright ...
01/05/2026
Lawo has been at the center of the industry's transition to IP and other next-generation technologies. At NAB 2026, its story was the Edge One AV stagebox, ...
01/05/2026
HBA Media, acting on behalf of NBC Sports and Churchill Downs Incorporated, has announced broadcast and streaming distribution for Kentucky Derby 152, taking pl...
01/05/2026
By Bailey Pennick
One of the most exciting things about the Sundance Film Festi...
01/05/2026
Florals for spring? Groundbreaking. But a playlist that tells you which The Devi...
01/05/2026
One of the world's biggest popstars is headed to El Cl sico. Later this mont...
01/05/2026
Limited-edition model celebrates 15th anniversary
Heritage Audio's range of monitor controllers has just gained a new member, the Baby RAM Black Edition...
01/05/2026
Dumble recreation now available as UAD plug-in
Along with their renowned processing plug-ins, Universal Audio have been steadily introducing emulations of c...
01/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
01/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
01/05/2026
Share
Copy link
Facebook
X
Linkedin
Bluesky
Email...
01/05/2026
To celebrate the opening of its new showroom and office, Lightware UK hosted a dedicated launch event at the new London location. The event welcomed partners, c...
01/05/2026
Choice without compromise
The broadcast industrys transformation is accelerating, and traditional broadcasters are having to fundamentally reinvent how they o...
01/05/2026
Beam Dynamics will return to MPTS 2026 with its asset intelligence platform, helping systems integrators, live production teams, media facilities and profession...
01/05/2026
Best-in-class UX design and rapid, scalable delivery for next-generation viewing experiences
Leading video software provider, Synamedia, today announced a coll...
01/05/2026
Compact new cforce MAX lens motor brings unrivaled speed and responsiveness to t...
01/05/2026
Panavision welcomes Fritz Heinzle as Vice President of Sales
Brie Clayton May 1, 2026
0 Comments
Heinzle will support Panavision's global growth s...