NVIDIA Advances Robot Learning and Humanoid Development With New AI and Simulation Tools
06/11/2024
Robotics developers can greatly accelerate their work on AI-enabled robots, including humanoids, using new AI and simulation tools and workflows that NVIDIA revealed this week at the Conference for Robot Learning (CoRL) in Munich, Germany.
The lineup includes the general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing.
The open-source Cosmos tokenizer provides robotics developers superior visual tokenization by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenizers, while NeMo Curator provides video processing curation up to 7x faster than unoptimized pipelines.
Also timed with CoRL, NVIDIA presented 23 papers and nine workshops related to robot learning and released training and workflow guides for developers. Further, Hugging Face and NVIDIA announced they're collaborating to accelerate open-source robotics research with LeRobot, NVIDIA Isaac Lab and NVIDIA Jetson for the developer community.
Accelerating Robot Development With Isaac Lab NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation.
Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment - from humanoids to quadrupeds to collaborative robots - to handle increasingly complex movements and interactions.
Leading commercial robot makers, robotics application developers and robotics research entities around the world are adopting Isaac Lab, including 1X, Agility Robotics, The AI Institute, Berkeley Humanoid, Boston Dynamics, Field AI, Fourier, Galbot, Mentee Robotics, Skild AI, Swiss-Mile, Unitree Robotics and XPENG Robotics.
Project GR00T: Foundations for General-Purpose Humanoid Robots Building advanced humanoids is extremely difficult, demanding multilayer technological and interdisciplinary approaches to make the robots perceive, move and learn skills effectively for human-robot and robot-environment interactions.
Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.
Six new Project GR00T workflows provide humanoid developers with blueprints to realize the most challenging humanoid robot capabilities. They include:
GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments
GR00T-Mimic for robot motion and trajectory generation
GR00T-Dexterity for robot dexterous manipulation
GR00T-Control for whole-body control
GR00T-Mobility for robot locomotion and navigation
GR00T-Perception for multimodal sensing
Humanoid robots are the next wave of embodied AI, said Jim Fan, senior research manager of embodied AI at NVIDIA. NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.
New Development Tools for World Model Builders Today, robot developers are building world models - AI representations of the world that can predict how objects and environments respond to a robot's actions. Building these world models is incredibly compute- and data-intensive, with models requiring thousands of hours of real-world, curated image or video data.
NVIDIA Cosmos tokenizers provide efficient, high-quality encoding and decoding to simplify the development of these world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.
Providing high-quality compression and up to 12x faster visual reconstruction, the Cosmos tokenizer paves the path for scalable, robust and efficient development of generative applications across a broad spectrum of visual domains.
1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokenizer.
NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity, said Eric Jang, vice president of AI at 1X Technologies. This allows us to train world models with long horizon video generation in an even more compute-efficient manner.
Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokenizer to manage high-resolution images and videos.
NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.
Curating video data poses challenges due to its massive size, requiring scalable pipelines and efficient orchestration for load balancing across GPUs. Additionally, models for filtering, captioning and embedding need optimization to maximize throughput.
NeMo Curator overcomes these challenges by streamlining data curation with automatic pipeline orchestration, reducing processing time significantly. It supports linear scaling across multi-node, multi-GPU systems, efficiently handling over 100 petabytes of data. This simplifies AI development, reduces costs and accelerates time to market.
Advancing the Robot Learning Community at CoRL The nearly two dozen research papers the NVIDIA robotics team released with CoRL cover breakthroughs in integrating vision language models for improved environmental understanding and task execution, temporal robot navigati
LINK: | https://blogs.nvidia.com/blog/robot-learning-humanoid-development/... |
See more stories from nvidia |
North America Stories
30/11/2024
Amino Fuels Next Frontier for Low Latency Enterprise Vide...
Amino, a leading provider of enterprise video and digital signage solutions, today announces the deployment of its Amino H200 4K media player and Orchestrate re...
30/11/2024
Blackmagic Design Announces Blackmagic Cloud Price Reduction
Blackmagic Design Announces Blackmagic Cloud Price Reduction Brie Clayton November 29, 2024 0 Comments New lower price for Blackmagic Cloud media stor...
30/11/2024
Massive Black Friday savings from Native Instruments, iZotope, and Plugin Alliance starts now
Massive Black Friday savings from Native Instruments, iZotope, and Plugin Allian...
30/11/2024
Recreating Star Trek's Briar Patch with Motion Control
Recreating Star Trek's Briar Patch with Motion Control Graham Quince November 29, 2024 0 Comments This is a collaboration with @nmphoto37 who ha...
29/11/2024
Ross Video Wins Best Innovation Award at 2024 Best Ottawa Business Awards
Recognition for Artimo: The Revolutionary Robotics Solution Redefining Broadcast Technology Ottawa, Canada - November 29, 2024 - Ross Video, a global leader in ...
28/11/2024
Give Me the Backstory: Get to Know 3 Indigenous Filmmakers With Shorts at the 2024 Sundance Film Festival
A still from Vea Mafile'o's Lea Tupu'anga / Mother Tongue, which s...
28/11/2024
Interra Systems Adds Netflixs VMAF Scoring to VEGA Media...
Interra Systems, a leading provider of end-to-end quality assurance solutions for the digital media industry, today announced the integration of Netflix s Video...
28/11/2024
PSI Broadcasting stays ahead of the game by advancing wit...
South East Asia's leading broadcaster and playout service provider, PSI Broadcasting, extends its facilities with the purchase of PlayBox Neo's MultiPla...
28/11/2024
Lifted Entertainment Expands Use of Hitomi MatchBox for M...
Hitomi Broadcast, the market leader in audio/video alignment and latency tools, announces that Lifted Entertainment, one of the world s leading creators and pro...
28/11/2024
Tint Boosts Collaborative Workflows with EditShare Storag...
One of the leading Scandinavian post-production houses, Tint, has supercharged its media workflows with EditShare's high-performance storage across its Goth...
28/11/2024
Last Cut Media Increases EditShare Footprint to Streamlin...
EditShare, the technology leader that enables storytellers to create and manage collaborative workflows at every stage from storyboard to screen, has expanded i...
28/11/2024
CVP Announces Black Friday Promotions
CVP, one of Europe's leading resellers and providers of professional video and broadcast solutions, has unveiled its Black Friday deals for 2024, with some ...
28/11/2024
White Peaks Solutions Taps NPAW To Enhance Its KWIKmotion OTT Platform
BARCELONA and PARIS White Peaks Solutions (WPS), the end-to-end over-the-top solutions provider based in France, has selected NPAW's video-analytics solutio...
28/11/2024
Interra Systems Adds Netflix's VMAF Scoring to VEGA Media Analyzer
CUPERTINO, Calif. Interra Systems has integrated Netflixs Video Multi-Method Assessment Fusion (VMAF) algorithm a widely-trusted standard for video quality asse...
28/11/2024
Survey: Consumers Prioritize Spending on Streaming Over Retirement Savings
MUNCIE, Ind. Despite all the complaints about the rising cost of streaming services, a new survey from First Merchants Bank indicates that setting aside money f...
28/11/2024
LYNX Technik Appoints Vincent Noyer as Director of Product Marketing
LYNX Technik Appoints Vincent Noyer as Director of Product Marketing Brie Clayton November 27, 2024 0 Comments Will Guide Innovation and Drive Market ...
28/11/2024
Unlocking Creative Potential: How Filmmakers and Videographers Can Leverage Adobe Express
Unlocking Creative Potential: How Filmmakers and Videographers Can Leverage Adob...
28/11/2024
Black Friday Deals from LumaTouch
Black Friday Deals from LumaTouch Brie Clayton November 27, 2024 0 Comments LumaTouch Significantly Expands Feature Set Offerings of its Storyblocks ...
28/11/2024
Chinese-Language Period Drama Perfect Match' to Launch Exclusively and Globally on Netflix
Back to All News Chinese-Language Period Drama Perfect Match' to Launch Ex...
28/11/2024
Watch the Spectacular and Intense Trailer for Norwegian Disaster Series 'La Palma'
Back to All News Watch the Spectacular and Intense Trailer for Norwegian Disast...
28/11/2024
Get the Power of GeForce-Powered Gaming in the Cloud Half Off With Black Friday Deal
Turn Black Friday into Green Thursday with a new deal on GeForce NOW Ultimate an...
27/11/2024
Give Me the Backstory: Get to Know Astrid Rondero and Fernanda Valadez, the Co-Directors of Sujo
By Jessica Herndon One of the most exciting things about the Sundance Film Fest...
27/11/2024
Dinner and a Movie: Katie Arthurs on Top End Wedding and Breakfast Burritos
The author watching a film with her mother and brother....
27/11/2024
Inside the Archives: Showcasing Sundance Talent in Oz and the Colosseum
Denzel Washington at the Piper-Heidsieck Tribute press conference at the 1993 Sundance Film Festival. Photo by Sandria Miller...
27/11/2024
L3Harris' Lou Speaight is a Finalist at Women in Defence UK Awards 2024
The Women in Defence U.K. awards ceremony celebrates the extraordinary contributions individuals and teams make to Defence....
27/11/2024
Nielsen launches activation with Advanced Audiences, enhancing digital campaign precision, reach and effectiveness across New Zealand
Auckland, November 26, 2024 Nielsen today announced the launch of Advanced Aud...
27/11/2024
Nielsen releases 2024 Food and Beverage Report, covering a decade of data on the sector's ad spend, media trends and consumer shifts in Australia
Report reveals ad spend of 100 occasional food & beverage brands over last 10 ye...
27/11/2024
Lynx Technik Adds Vincent Noyer as Director of Product Marketing
WEITERSTADT, Germany Broadcast TV equipment provider Lynx Technik has named Vincent Noyer as director of product marketing....
27/11/2024
NBCUniversal Works With Walmart to Bring Live Shopping to Live Sports
NEW YORK NBCUniversal and Walmart said they are bringing new shoppable experiences and measurement capabilities to live sports coverage. Kicking off on Thanksgi...
27/11/2024
Ross Video Hopes To BHAG Private Equity With Ambitious Strategy Shift
Ross Video has outlined a shift in its strategic approach and announced a significant financial target for 2030....
27/11/2024
Bending Spoons To Acquire Brightcove for $233 Million
BOSTON Brightcove said it has entered into a definitive agreement to be acquired by Bending Spoons in an all-cash transaction valued at about $233 million....
27/11/2024
Transmit, Wurl Partner on FAST Channel Ads
NEW YORK Advanced-advertising solutions provider Transmit and connected-TV tech company Wurl have struck a partnership that will bring Transmit's in-stream ...
27/11/2024
Prime Video Adds More FAST Channels for the Holidays
Amazon's Prime Video continues to bulk up its free, ad-supported streaming TV channels with the launch of holiday-themed channels and other services....
27/11/2024
Sinclair's Chris Ripley Lays Out ATSC 3.0 Challenges, Opportunities
HUNT VALLEY, Md. Earlier this month, Sinclair signed a memorandum of understanding with the Institute of Technology Bombay covering their collaboration on broad...
27/11/2024
Krotos Introduces the Creator Toolkit: Tailored Sound Effects for Content Creators
Krotos Introduces the Creator Toolkit: Tailored Sound Effects for Content Creato...
27/11/2024
The Summer Hikaru Died' Set to Bring Eerie Anime Thrills to Netflix
Back to All News The Summer Hikaru Died' Set to Bring Eerie Anime Thrills to NetflixPlay Video Play Video Entertainment 27 November 2024 GlobalJapan ...
27/11/2024
How RTX AI PCs Unlock AI Agents That Solve Complex Problems Autonomously With Generative AI
Editor's note: This post is part of the AI Decoded series, which demystifies...
26/11/2024
L3Harris in the UK Achieves Silver ERS Award Status
Jules Ball and Henry Watts, with the award being presented by Brigadier Adam Fraser-Hitchen ADC DL, Deputy Commander of 3rd UK Division....
26/11/2024
Studio Technologies Introduces Model 352A and Model 354A...
Studio Technologies, manufacturer of high-quality audio, video, and fiber-optic solutions, presents the Model 352A and Model 354A Talk Stations. The units suppo...
26/11/2024
TASCAM Sonicview Recording and Mixing Consoles and SMPTE...
TASCAM, renowned for its versatile and adaptable audio solutions, today announced that its TASCAM Sonicview Digital Mixing Consoles and optional IF-ST2110 Expan...
26/11/2024
Blue Lucy takes on The Americas
London-based media technology business, Blue Lucy, has set up a US division of the company and appointed Dina Behar Hevert as VP Americas. The company's ent...
26/11/2024
Leader Experiences 2024 as Year of Ongoing Innovation in...
The American writer William Sydney Porter, better known as O. Henry, wrote of New York City in the early 1900s that It'll be a great place if they ever fin...
26/11/2024
AGILE CONTENT AND TISCALI TEAM UP TO LAUNCH THE TV SERVIC...
Agile Content joins forces with Tiscali to introduce a new platform in Italy, offering a wide range of premium content for all audiences. Agile Content, a Eur...
26/11/2024
Bulgarian Fund for Women made accessible using SubtitleNE...
For immediate release 26 November 2024, Sofia, Bulgaria Profuz Digital recently sponsored the Bulgarian Fund for Women's 20th Anniversary event by donat...
26/11/2024
Ikegami Sees Accelerating Adoption of IP and Mixed-Format...
Ikegami reports an accelerating migration to high-efficiency IP and mixed-format UHD/HD content creation throughout 2024. Product developments announced during ...
26/11/2024
CVP Announces the Second Belgian Production Technology Sh...
The premier event for cutting-edge production solutions, technologies, and industry insights returns to Brussels. CVP, one of Europe's leading resellers an...
26/11/2024
EMG-Gravity Media to deliver more than 350 hours coverage...
EMG / Gravity Media, a leading force in production and content, media services and facilities, today detailed EMG / Gravity Media's expanding broadcast and ...
26/11/2024
iZotope unveils Cascadia, an intelligent tape delay for clear and present mixes
iZotope unveils Cascadia, an intelligent tape delay for clear and present mixes Brie Clayton November 26, 2024 0 Comments Keep your mixes clear and fo...
26/11/2024
Award Winning Memoir of a Snail Finished in DaVinci Resolve Studio
Award Winning Memoir of a Snail Finished in DaVinci Resolve Studio Brie Clayton November 26, 2024 0 Comments DaVinci Resolve Studio used to create uni...
26/11/2024
Nielsen: Fox, Disney Gain Viewing Share in October
NEW YORK In the battle for viewing among the big TV and media players, Fox had a superb October, earning an 8.4% share of total TV viewing in Nielsen's Octo...