
The Hao AI Lab research team at the University of California San Diego - at the forefront of pioneering AI model innovation - recently received an NVIDIA DGX B200 system to elevate their critical work in large language model inference.
Many LLM inference platforms in production today, such as NVIDIA Dynamo, use research concepts that originated in the Hao AI Lab, including DistServe.
How Is Hao AI Lab Using the DGX B200? Members of the Hao AI Lab standing with the NVIDIA DGX B200 system. With the DGX B200 now fully accessible to the Hao AI Lab and broader UC San Diego community at the School of Computing, Information and Data Sciences' San Diego Supercomputer Center, the research opportunities are boundless.
DGX B200 is one of the most powerful AI systems from NVIDIA to date, which means that its performance is among the best in the world, said Hao Zhang, assistant professor in the Hal c o lu Data Science Institute and department of computer science and engineering at UC San Diego. It enables us to prototype and experiment much faster than using previous-generation hardware.
Two Hao AI Lab projects the DGX B200 is accelerating are FastVideo and the Lmgame benchmark.
FastVideo focuses on training a family of video generation models to produce a five-second video based on a given text prompt - in just five seconds.
The research phase of FastVideo taps into NVIDIA H200 GPUs in addition to the DGX B200 system.
Lmgame-bench is a benchmarking suite that puts LLMs to the test using popular online games including Tetris and Super Mario Bros. Users can test one model at a time or put two models up against each other to measure their performance.
The illustrated workflow of Hao AI Lab's Lmgame-Bench project. Other ongoing projects at Hao AI Labs explore new ways to achieve low-latency LLM serving, pushing large language models toward real-time responsiveness.
Our current research uses the DGX B200 to explore the next frontier of low-latency LLM-serving on the awesome hardware specs the system gives us, said Junda Chen, a doctoral candidate in computer science at UC San Diego.
How DistServe Influenced Disaggregated Serving Disaggregated inference is a way to ensure large-scale LLM-serving engines can achieve the optimal aggregate system throughput while maintaining acceptably low latency for user requests.
The benefit of disaggregated inference lies in optimizing what DistServe calls goodput instead of throughput in the LLM-serving engine.
Here's the difference:
Throughput is measured by the number of tokens per second that the entire system can generate. Higher throughput means lower cost to generate each token to serve the user. For a long time, throughput was the only metric used by LLM-serving engines to measure their performance against one another.
While throughput measures the aggregate performance of the system, it doesn't directly correlate to the latency that a user perceives. If a user demands lower latency to generate the tokens, the system has to sacrifice throughput.
This natural trade-off between throughput and latency is what led the DistServe team to propose a new metric, goodput : the measure of throughput while satisfying the user-specified latency objectives, usually called service-level objectives. In other words, goodput represents the overall health of a system while satisfying user experience.
DistServe shows that goodput is a much better metric for LLM-serving systems, as it factors in both cost and service quality. Goodput leads to optimal efficiency and ideal output from a model.
How Can Developers Achieve Optimal Goodput? When a user makes a request in an LLM system, the system takes the user input and generates the first token, known as prefill. Then, the system creates numerous output tokens, one after another, predicting each token's future behavior based on past requests' outcomes. This process is known as decode.
https://blogs.nvidia.com/wp-content/uploads/2025/12/distserve.mp4
Prefill and decode have historically run on the same GPU, but the researchers behind DistServe found that splitting them onto different GPUs maximizes goodput.
Previously, if you put these two jobs on a GPU, they would compete with each other for resources, which could make it slow from a user perspective, Chen said. Now, if I split the jobs onto two different sets of GPUs - one doing prefill, which is compute intensive, and the other doing decode, which is more memory intensive - we can fundamentally eliminate the interference between the two jobs, making both jobs run faster.
This process is called prefill/decode disaggregation, or separating the prefill from decode to get greater goodput.
Increasing goodput and using the disaggregated inference method enables the continuous scaling of workloads without compromising on low-latency or high-quality model responses.
NVIDIA Dynamo - an open-source framework designed to accelerate and scale generative AI models at the highest efficiency levels with the lowest cost - enables scaling disaggregated inference.
In addition to these projects, cross-departmental collaborations, such as in healthcare and biology, are underway at UC San Diego to further optimize an array of research projects using the NVIDIA DGX B200, as researchers continue exploring how AI platforms can accelerate innovation.
Learn more about the NVIDIA DGX B200 system.
Most recent headlines
05/01/2027
Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...
16/11/2026
The ESPN Edge Innovation Conference once again gave key ESPN technology partners as a chance to take to the stage and discuss the future of sports fan experienc...
16/11/2026
As rights holders rethink who tells their stories and where, IMG is preparing th...
16/11/2026
From the sidelines, sonic savants advocate for greater emphasis on sound as sports and entertainment increasingly merge
SVG Live!, a one-day event on Nov. 10 i...
16/11/2026
Deal combines Yamaha's audio technology with Disguise's visual-productio...
16/11/2026
Creative effects designed to encourage experimentation
Berry Effects is the brainchild of Abe and Ralph, a pair of multi-instrumentalists and audio engineer...
16/11/2026
Physics-based synth gets first major update
Anukari have just released a new version of their innovative physics-based software synth, marking the instrumen...
16/11/2026
Both versions kitted out with new tools & improvements
Hit'n'Mix have just launched the latest major version of their innovative DAW software, with ...
16/11/2026
Dynamic bands, unmasking features & new stereo tools
Launched in 2021, Eventide's SplitEQ plug-in introduced an interesting new approach to EQ, with the...
16/11/2026
Rohde & Schwarz Japan demonstrates multi-vendor ASA-ML communication with high p...
16/11/2026
November 16, 2026
Hitachi system enables proton beams to be delivered from 360 degrees around the patient, expanding treatment capacity and supporting more pre...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
16/11/2026
Hiltron Communications has received a repeat order from one of Europes longest established communications satellite operators. Central to this project is a Hilt...
16/11/2026
Fabric Insights and Fonn Group today announce a partnership to connect editorial planning and media operations across their complementary product portfolios.
...
16/11/2026
Density 1.0 brings a complete color grading workflow to Final Cut Pro
Brie Clayton November 16, 2026
0 Comments
Filmmaker Pascal Basel's plugin pr...
16/11/2026
Berklee Students to Perform at Boston's CITGO Sign Relighting Kuper Walker, Ollie Mann, and dreamcorp. will perform at the free public celebration in Kenm...
16/11/2026
The Hollywood Reporter Names Berklee Among World's Best Music Schools The annual list spotlights Berklee's screen scoring programs and new master'...
16/11/2026
Five years ago, Lightware launched the original Taurus UCX with a clear vision: USB-C would become the central connection in meeting rooms. The goal was simple:...
16/11/2026
Sonniss: bundles over 80% below their libraries' prices, plus up to 60% off ...
16/11/2026
Singaporean Comedy Luck My Life Shot with Blackmagic URSA Cine 17K 65
Brie Clayton November 16, 2026
0 Comments
Blackmagic PYXIS 12K and DaVinci Resol...
16/11/2026
Images available HERE
UKTV has announced Fleet Street (6x60'), a new period crime drama from Miss Scarlet creators Rachael New and Ben Edwards, coming to f...
16/11/2026
November 16th, 2026
TRIBECA FESTIVAL LISBOA UNVEILS 2026 FILM LINEUP AND TALE...
16/11/2026
From Graham Norton to Olivia Attwood and Joe Marler, a new collection of hit vid...
16/11/2026
As countries gather to discuss climate targets at COP 31 this week, one question that is at the heart of debates is how to measure the environmental impact of t...
16/11/2026
Announced today at SC26, NVIDIA Rubin HPC is a new GPU in the NVIDIA Vera Rubin ...
16/11/2026
This week at SC26, NVIDIA is marking two decades of CUDA by honoring the researchers, developers and engineers who helped make GPU computing a foundation for mo...
16/11/2026
With expanded NVIDIA CUDA-X libraries, including cuPhoton, cuPotential and cuSpa, researchers can sift through vast amounts of data to accelerate research, from...
16/11/2026
In its MLPerf benchmarks debut, with results released today, the NVIDIA Vera Rub...
16/11/2026
Quantum processor deployments at supercomputing centers have grown nearly sevenfold over the past five years, with nearly 30 supercomputing centers around the w...
16/11/2026
To train an AI model, thousands of GPUs have to work together, requiring a network that's fast, efficient and resilient.
Lambda is tackling the network pie...
15/11/2026
Mixers, interfaces & DAW controller receive new firmware
Yamaha have just launched a trio of firmware updates that bring some powerful new functionality to ...
15/11/2026
Can VideoProc AI Really Reduce Noise and Enhance Blurry, Old Footage? Real-World...
15/11/2026
Over two days at COEX, more than 2,000 attendees converged for AI Day Seoul, bri...
14/11/2026
Longtime MLB Network employee David Maybrown passed away Thursday at the age of 57. David joined MLB Network in 2009 shortly after its launch, beginning as an i...
14/11/2026
Modular innovators introduce two guitar-based products
ADDAC System's recently introduced Four String Series makes it possible to integrate physical str...
14/11/2026
Marcelo Dai's Live Session Captured with URSA Cine Immersive
Brie Clayton November 14, 2026
0 Comments
Artist takes viewers on an immersive journe...
14/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
14/11/2026
Share
{alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...
14/11/2026
Newest Lenses, Horizon Anamorphic Seminar & Guided Lens Character Experience Highlight Creative Possibilities
At the 2026 Toru CAMERIMAGE Film Festival, ZEISS...
14/11/2026
Sonnet Announces Black Friday Sale on Select Products
Brie Clayton November 14, 2026
0 Comments
WHAT: Sonnet Technologies is having a sale on a select...
14/11/2026
Berklee Alumni Shine at the 2026 Latin Grammy Awards This year's winners included songwriters, engineers, and performers who contributed to acclaimed reco...
13/11/2026
The UIM E1 Series wraps its third season this week with the Bombay Sapphire E1 Miami GP, Nov. 11-14, on Biscayne Bay at the Miami Seaplane Base on Watson Island...
13/11/2026
In-venue and creative video staffers at the professional and collegiate level ha...