Using open data to study the sounds of the ocean and create art

Can you see sounds? Using open data, you can. To celebrate this year’s World Oceans Day, an artist and sustainability application architect at Amazon Web Services (AWS), created an artwork titled Can You See the Sound of the Ocean. To create the art, she drew inspiration from the Pacific Ocean Sound Recordings from the Monterey Bay Aquarium Research Institute (MBARI), available through the Amazon Sustainability Data Initiative (ASDI). Learn more about the dataset and the art work.

The artwork Can You See the Sound of the Ocean by Guyu Ye.

Ocean sounds as open data

The Pacific Ocean Sounds Recordings dataset contains sound recordings from the deep ocean off central California. We acquired the data through MBARI’s cabled observatory, which provides power and communications to the deep sea and allows continuous data acquisition by an underwater microphone, or hydrophone. The hydrophone measures the sounds created by ocean life (marine mammals and fishes), human activities (shipping, resource extraction, and construction), and the Earth (rain, wind, earthquakes, and submarine landslides).

Left: Sound is recorded from the deep ocean off the coast of California, within Monterey Bay National Marine Sanctuary (MBNMS). MBARI’s Monterey Accelerated Research System (MARS) cabled observatory, which resides on the seafloor at 900 meters (nearly 3,000 feet) below the ocean surface, has enabled nearly continuous recording since July 2015. Right: A hydrophone (top image, at the top of the tripod) was connected to MARS using a remotely operated vehicle (bottom image).

Sampled at more than a quarter of a million times each second, these recordings allow detection of sound across a vast range of frequencies, within and far outside the range of human hearing. For example, some species of dolphins echolocate by producing sound at frequencies more than six times greater than the upper limit of human hearing. Whether applied to science or art, the same essential signal is used – sound intensity varying in frequency (pitch) and time.

The high sample rate of the hydrophone creates a large volume of data – approximately two terabytes per month. Through ASDI and the AWS Open Data Sponsorship Program, we are able to share not only the voluminous raw data, but also decimated (lower resolution) data that can serve many purposes. In the month of April, 1.8 petabytes of data were downloaded by users from 11 countries.

What this open data can show us

These recordings represent the soundscape of a biodiversity hotspot. Within these recordings are the sounds produced by many species of animals, opening a window into the behavior and ecology of species that are difficult to study visually. For example, the dataset includes the sounds of the blue whale, the largest animal ever to have evolved on Earth. Blue whales remain endangered following severe population loss to commercial whaling. Supporting their recovery requires deeper understanding of their ecology. These sound recordings have enabled remarkable discoveries about the lives of blue whales. Studying one type of sound produced by blue whales revealed that they may use sound to signal to their regional population when favorable foraging habitat has been located.

Excerpt from a blue whale song recorded in Monterey Bay, California. To make the low-frequency sound more audible, playback speed is 10 times the original recording.

Studying rhythmic sequences of sound produced by blue whales, a behavior described as “song”, revealed that we can hear when they transition from their foraging activity in waters off California to their southward migration for their breeding season. With this acoustic signature of migration, we further discovered that blue whales are highly attuned to the conditions within their foraging habitat. If the ecosystem off California delivers abundant food resources, they will remain for months longer before transitioning to their migration to breeding habitat off Mexico and Central America. This may be an evolutionary strategy to maximize the energy gain and storage that they need to support energy-intensive activities, such as long-distance migration, mating, birthing, and rearing of young.

To tackle the large volume of data, MBARI uses AWS Batch and AWS Fargate to quickly generate spectrograms of all possible blue whale calls. Amazon SageMaker allowed for rapid training and deployment of machine learning (ML) models to accurately classify nearly 200,000 blue whale calls.

Seasonal and year-to-year changes in the number of blue whale A calls classified using machine learning. Accurate classifications were possible using SageMaker services.

Another area of discovery from these sound recordings is noise pollution in the ocean. Because marine animals use sound in essential life activities – communication, socialization, reproduction, foraging, and navigation – noise from human activities can interfere with their lives and cause harm. Studies of anthropogenic noise recorded in MBARI’s acoustic archive have examined shipping noise and its changes due to economic impacts of the COVID-19 pandemic. They have also demonstrated how underwater explosives may drive acoustically sensitive porpoises away from their needed habitat.

Discoveries from this open data archive are not limited to professional researchers. A secondary school student from Canada, Artash Nath, processed years of recordings to examine shipping noise increases following economic rebound from the COVID-19 pandemic. This led Artash to build an award-winning web-based tool for characterizing shipping noise at locations around the world, inspired by the shipping noise tutorial.

Drawing inspiration from open data to create art

Guyu Ye is a sustainability application architect at AWS and an artist. Her artwork, Can You See the Sound of the Ocean, incorporated Pacific Ocean audio data with traditional Chinese painting techniques.

Art is all about discovering and experiencing possibilities. Sometimes creating art starts with exploring interesting datasets. Can You See the Sound of the Ocean was inspired by the spectrograms of the Pacific Ocean sounds. Guyu constructed the canvas using a creative overlay of daily soundscapes from the Pacific Ocean for the entire year of 2021. The swimming jellyfish were painted using ink and water on rice paper, before being transferred to the canvas. This piece of art invites audiences to a unique visual experience of the ocean.

In addition to scientific research and visual arts, MBARI’s ocean sound recordings have inspired and supported audio arts. For example, the internationally traveling immersive experience Sounds of the Ocean combines mindful music, whale and dolphin sounds, visual art, poetry, and dance to support mental health and inspire ocean conservation. Megaptera from composer Boris Lelong, uses the songs of humpback whales (whose genus name is Megaptera) to create a musical composition. Artist Yolande Harris created Melt Me into the Ocean which offers an immersive acoustic dive across the land-sea boundary. Want to experience the ocean sounds yourself? Head over to MBARI’s Soundscape Listening Room.

How to get started with open data on AWS

Get started with this ocean sound data on the AWS Registry of Open Data, where you will find documentation and a link to the Quick Start resource page that includes example tutorials that can be run in a variety of environments.

Learn more about the Amazon Sustainability Data Initiative.

AWS Public Sector Blog

Using open data to study the sounds of the ocean and create art

Ocean sounds as open data

What this open data can show us

Drawing inspiration from open data to create art

How to get started with open data on AWS

Read more stories about open data and sustainability on AWS:

Resources

Follow