Turning the Tide on Coral Reef Decline: CUREE Robot Dives Deep With Deep Learning
26/10/2023
The Woods Hole Oceanographic Institution (WHOI) Autonomous Robotics and Perception Laboratory (WARPLab) and MIT are developing a robot for studying coral reefs and their ecosystems.
The WARPLab autonomous underwater vehicle (AUV), enabled by an NVIDIA Jetson Orin NX module, is an effort from the world's largest private ocean research institution to turn the tide on reef declines.
Some 25% of coral reefs worldwide have vanished in the past three decades, and most of the remaining reefs are heading for extinction, according to the WHOI Reef Solutions Initiative.
The AUV, dubbed CUREE (Curious Underwater Robot for Ecosystem Exploration), gathers visual, audio, and other environmental data alongside divers to help understand the human impact on reefs and the sea life around them. The robot runs an expanding collection of NVIDIA Jetson-enabled edge AI to build 3D models of reefs and to track creatures and plant life. It also runs models to navigate and collect data autonomously.
WHOI, whose submarine first explored the Titanic in 1986, is developing its CUREE robot for data gathering to scale the effort and aid in mitigation strategies. The oceanic research organization is also exploring the use of simulation and digital twins to better replicate reef conditions and investigate solutions like NVIDIA Omniverse, a development platform for building and connecting 3D tools and applications.
Creating a digital twin of Earth in Omniverse, NVIDIA is developing the world's most powerful AI supercomputer for predicting climate change, called Earth-2.
Underwater AI: DeepSeeColor Model Anyone who's gone snorkeling knows that seeing underwater isn't as clear as seeing on land. Over distance, water attenuates the visible spectrum of light from the sun underwater, muting some colors more than others. At the same time, particles in the water create a hazy view, known as backscatter.
A team from WARPLab recently published a research paper on undersea vision correction that helps mitigate these problems and supports the work of CUREE. The paper describes a model, called DeepSeeColor, that uses a sequence of two convolutional neural networks to reduce backscatter and correct colors in real time on the NVIDIA Jetson Orin NX while undersea.
NVIDIA GPUs are involved in a large portion of our pipeline because, basically, when the images come in, we use DeepSeeColor to color correct them, and then we can do the fish detection and transmit that to a scientist up at the surface on a boat, said Stewart Jamieson, a robotics Ph.D. candidate at MIT and AI developer at WARPLab.
Eyes and Ears: Fish and Reef Detection CUREE packs four forward-facing cameras, four hydrophones for underwater audio capture, depth sensors and inertial measurement unit sensors. GPS doesn't work underwater, so it is only used to initialize the robot's starting position while on the surface.
Using a combination of cameras and hydrophones along with AI models running on the Jetson Orin NX enables CUREE to collect data for producing 3D models of reefs and undersea terrains.
To use the hydrophones for audio data collection, CUREE needs to drift with its motor off so that there's no interference with the audio.
It can build a spatial soundscape map of the reef, using sounds produced by different animals, said Yogesh Girdhar, an associate scientist at WHOI, who leads WARPLab. We currently (in post-processing) detect where all the chatter associated with bioactivity hotspots is, he added, referring to all the noises of sea life.
The team has been training detection models for both audio and video input to track creatures. But a big noise interference with detecting clear audio samples has come from one creature in particular.
The problem is that, underwater, the snapping shrimps are loud, said Girdhar. On land, this classic dilemma of how to separate sounds from background noises is known as the cocktail party problem. If only we could figure out an algorithm to remove the effects of sounds of snapping shrimps from audio, but at the moment we don't have a good solution, said Girdhar.
Despite few underwater datasets in existence, pioneering fish detection and tracking is going well, said Levi Cai, a Ph.D. candidate in the MIT-WHOI joint program. He said they're taking a semi-supervised approach to the marine animal tracking problem. The tracking is initialized using targets detected by a fish detection neural network trained on open-source datasets for fish detection, which is fine-tuned with transfer learning from images gathered by CUREE.
We manually drive the vehicle until we see an animal that we want to track, and then we click on it and have the semi-supervised tracker take over from there, said Cai.
Jetson Orin Energy Efficiency Drives CUREE Energy efficiency is critical for small AUVs like CUREE. The compute requirements for data collection consume roughly 25% of the available energy resources, with driving the robots taking the remainder.
CUREE typically operates for as long as two hours on a charge, depending on the reef mission and the observation requirements, said Girdhar, who goes on the dive missions in St. John in the U.S. Virgin Islands.
To enhance energy efficiency, the team is looking into AI for managing the sensors so that computing resources automatically stay awake while making observations and sleep when not in use.
Our robot is small, so the amount of energy spent on GPU computing actually matters - with Jetson Orin NX our power issues are gone, and it's made our system much more robust, said Girdhar.
Exploring Isaac Sim to Make Improvements The WARPLab team is experimenting with NVIDIA Isaac Sim, a scalable robotics simulation application and synthetic data generation tool powered by Omnive
LINK: | https://blogs.nvidia.com/blog/2023/10/26/coral-reef-decline-curee-robo... |
See more stories from nvidia |
Most recent headlines
04/08/2024
Dalet Appoints Santiago Solanas as CEO to Lead Next Era of Growth and Innovation
Dalet, a leading technology and service provider for media-rich organizations, is excited to announce Santiago Solanas as its new Chief Executive Officer (CEO)....
03/06/2024
Dalet and Veritone Reach Agreement to Distribute, Transact and Monetize Media Archives
Dalet, a leading technology and service provider for media-rich organizations, a...
31/05/2024
Marlow Film Studios considering next steps' as council blocks plans
The proposal had received support from James Cameron, Andy Serkis, Paul Greengrass, Richard Curtis, and Jeremy Irons By Matthew Corrigan Published: May 31, 2...
31/05/2024
Skydance submits new Paramount Global bid, Sony talks still progressing'
According to multiple reports, the new bid from Skydance offers a better deal to Paramounts Class B shareholders, who were said to be unhappy with the companys ...
31/05/2024
2024 Rise Awards open for entries
The awards recognise women and companies who are stand-out performers, innovators and leaders in the broadcast media technology industry By Matthew Corrigan ...
31/05/2024
Samsung, LG Adopt IAB Software Kit for CTV Ad Measurement
NEW YORK The global body that sets technical standards for digital advertising has expanded the reach of its measurement software development kit to include Sam...
31/05/2024
The Old Investment Cycles of Broadcast Technology are Being Run Over'
The rise of digital platforms is fundamentally changing the broadcast newsroom and its investment in technology according to Jon Roberts, director of technology...
31/05/2024
Study: Max, Apple TV Have Fewest 'Committed Customers'
In a new analysis of what services might make up a good bundle and what services might benefit the most from bundling, Antenna has released new data that highli...
31/05/2024
State Broadcasters Urge Passage of Broadcast VOICES Act
WASHINGTON, D.C. Fifty state broadcasters associations, including the District of Columbia and the Commonwealth of Puerto Rico, have sent a letter to Congressio...
31/05/2024
EVS Names Richard Katz SVP Operations for NALA region
EVS has announced the appointment of Richard Katz as senior vice president of operations for the North and Latin American (NALA) region. This appointment is par...
31/05/2024
CommScope to Buy Cable Business Assets of Casa Systems for $45.1M
CommScope has announced that it was selected by Casa Systems, Inc. as the highest to acquire Casa's cable business assets and that the two companies have en...
31/05/2024
NAB Withdraws De Niro's Service Award
Robert De Niro won't be receiving the Service to America Leadership Award from the NAB Leadership Foundation after all....
31/05/2024
Viant Integrates With Google Cloud's BigQuery Clean Rooms
Viant Technology said it formed an integration with Google Cloud's BigQuery data clean rooms...
31/05/2024
The CW To Broadcast Snoop Dogg Arizona Bowl
The CW said it will broadcast the 2024 Snoop Dogg Arizona Bowl presented by Gin & Juice by Dre and Snoop Last year, The CW aired the game when it was called Bar...
31/05/2024
Allen Media Makes Carriage Deal With Amazon Fire Channels
Allen Media Group said it made a deal with Amazon's Fire TV Channel that will make content from several of Allen's outlet available via Fire TV and Echo...
31/05/2024
Robert De Niro Will Not Receive Service to America Award
Robert De Niro, announced as the winner of the NAB's Leadership Foundation's Service to America Leadership Award on May 28, will not get the award. The ...
31/05/2024
Comcast Technology Solutions To Manage Channels for AccuWeather
Comcast Technology Solutions was selected by AccuWeather to create, manage and distribute linear and over-the-top video channels....
31/05/2024
Attention to Connected TV Ads Grew in Q1, TVision Study Finds
Connected TV is becoming a bigger deal for consumers and advertisers, according to a new report from TVision....
31/05/2024
RT 2FM is the home of music this summer
RT 2FM is the home of music this summer, entertaining listeners across the day and night, with the best music, chat and interviews providing the soundtrack to ...
31/05/2024
NVIDIA Grace Hopper Superchip Accelerates Murex MX.3 Analytics Performance, Reduces Power Consumption
After the 2008 financial crisis and increased risk-management regulations that f...
31/05/2024
RT Supporting the Arts: What's On this June
RT Supporting the Arts: What's On this June Watch our promo: RT Supporting the Arts | What's on YouTube This June, RT is delighted to support Lim...
30/05/2024
When to Upgrade Software or Firmware
When to Upgrade Software or Firmware This post is from our blog and news archive. The information may be out of date. Please contact us for further information ...
30/05/2024
The City of San Jose Chooses Utah ScientificAgain!
The City of San Jose Chooses Utah Scientific Again! This post is from our blog and news archive. The information may be out of date. Please contact us for furth...
30/05/2024
Empowering Media Leaders: Transformative Insights from the Executive Learning Series
Navigating Strategy, Ethics, and Innovation in the evolving European media lands...
30/05/2024
Give Me the Backstory: Get to Know Thea Hvistendahl, the Filmmaker Behind Handling the Undead
By Bailey Pennick One of the most exciting things about the Sundance Film Festi...
30/05/2024
CHOOSE CHICAGO, CITY OF CHICAGO ANNOUNCE PROGRAMMING DETAILS OF SUNDANCE INSTITUTE X CHICAGO 2024 (June 28 - 30)
Tickets are now available for the first-of-its-kind event in the United States f...
30/05/2024
UNICEF and Spotify's Award-Winning Mental Health Hub, Our Minds Matter, Comes to Latin America
Mental health and well-being are fundamental components of a child's healthy...
30/05/2024
After 64 days, the winner of Alone Australia has been revealed
After 64 days, the winner of Alone Australia has been revealed 29 May, 2024 Media releases *Contains Spoilers* After 64 days alone in the extreme and wild...
30/05/2024
Alone Australia finale delivers strongly as SBS confirms it will join VOZ streaming
Alone Australia finale delivers strongly as SBS confirms it will join VOZ stream...
30/05/2024
Alone Australia finale delivers strongly for national broadcaster SBS with 4.2m tuning in across the season
Alone Australia finale delivers strongly for national broadcaster SBS with 4.2m ...
30/05/2024
Lockheed Martin Canada Awards L3Harris the Integrated Communications System Contract
Photo credit: BAE...
30/05/2024
The New Standard for Unmanned Ground Vehicle ISR
L3Harris' WESCAM MX -10 RSTA provides advanced ISR and targeting capabilities for land-based platforms including Unmanned Ground Vehicles....
30/05/2024
AccuWeather Inks Deal With Comcast Technology Solutions For Channel Origination
DENVER AccuWeather has selected Comcast Technology Solutions' (CTS') Managed Channel Origination (MCO) to create, manage and distribute linear TV, on-de...
30/05/2024
Samsung, LG Adopt IAB Software Kit
NEW YORK The global body that sets technical standards for digital advertising has expanded the reach of its measurement software development kit to include Sam...
30/05/2024
Amazon Celebrates 10th Anniversary of Fire TV Devices with an AI Upgrade
As Amazon celebrates the 10th anniversary of the launch of the first Fire TV devices in 2014, the company has unveiled new AI-powered search capabilities for th...
30/05/2024
Finding the sustainability balance with personalised video streaming
As the expansion of personalised video streaming continues apace, JUMPs CEO and co-founder, Jer nimo Macan s and Fran ois Polarczyk, sustainability director at ...
30/05/2024
EditShare boosts sales direction with alumnus Grant Carro...
EditShare, the technology leader that enables storytellers to create and manage collaborative workflows at every stage from storyboard to screening, has appoint...
30/05/2024
COW Featured Resume: Paula Zimmerman - Video Editor - Motion Designer
COW Featured Resume: Paula Zimmerman - Video Editor - Motion Designer Brie Clayton May 30, 2024 0 Comments Paula Zimmerman Looking for work? Sign up...
30/05/2024
Palme d'Or Winner Anatomy of a Fall Finished with DaVinci Resolve Studio
Palme d'Or Winner Anatomy of a Fall Finished with DaVinci Resolve Studio Brie Clayton May 30, 2024 0 Comments Winner of the Palme d'Or at the ...
30/05/2024
Real-Time Workflow Masterclass
Real-Time Workflow Masterclass Michael Cioni May 30, 2024 0 Comments Would you rather have your workflow move at the same speed it always has, or woul...
30/05/2024
Ending a Loop Expression in After Effects
Ending a Loop Expression in After Effects Graham Quince May 30, 2024 0 Comments This question comes up quite a bit on forums and it seems to trip peop...
30/05/2024
Parks: Prime Video Has Lowest Churn Rate
DALLAS Consumers who subscribe to streaming services are the least likely to cancel Prime Video among all major providers, according to Parks Associates' St...
30/05/2024
SMPTE/RIS-OSVP tests model for circle of confusion'
According to Camera and Lens Metadata committee, modern digital cinema cameras are not accurately representing the usable depth of field achieved with various l...
30/05/2024
Meet the vice president of product management
Dave MacKinnon, vice president of product management at Clear-Com, explains the value of collaboration and networking in building a media career By Matthew Cor...
30/05/2024
That Station Launches New App, Website, Filled with Features, Music, Interviews and More
That Station's app and website just got a major upgrade. The station crew co...
30/05/2024
SWR Deploys Rohde & Schwarz Pixel Power Software Playout Solution
MUNICH, Germany Regional German broadcaster S dwestrundfunk (SWR) has deployed the Rohde & Schwarz Pixel Power graphics and playout solution....
30/05/2024
Hollyland Technology Unveils Pyro Wireless Video Transmission Series
IRVINE, Calif. Hollyland Technology has launched Pyro, a wireless video transmission system designed for the multi-person, mobile transmission and monitoring re...
30/05/2024
AIMS Opens Call for Presentations for Media-Over-IP Pavilion at AES New York
AIMS has announced it will once again collaborate with the Audio Engineering Society (AES) to bring the popular Media-Over-IP Pavilion to the AES New York show,...
30/05/2024
FCC Announces Opportunity for LPTV Stations to Change Channels
WASHINGTON, D.C. The Federal Communications Commission (FCC) Media Bureau has announced that beginning on August 20, 2024, Class A television, Low Power Televis...
30/05/2024
Comcast's StreamSaver Streaming Bundle Goes Live
Comcast has officially launched its discounted $15-a-month StreamSaver streaming bundle of Netflix, Peacock and Apple TV+ services. The previously announced bun...