Přeskočit navigaci

OOI Seafloor Video

Oceanography Oceans AIforEarth

Video data from the Ocean Observatories Initiative seafloor camera deployed at Axial Volcano on the Juan de Fuca Ridge.

Overview

The OOICloud Project is making data from the Ocean Observatories Initiative (OOI) publicly available on Azure Open Datasets and accessible through a Pangeo interface. A primary goal of the project is to provide these data to the scientific community using a cloud-performant object storage model, and to provide large-scale data-proximate compute capabilities for research investigations. The OOI sensor network consists of 89 scientific platforms with approximately 830 instruments, and provides nearly 5 TB of data each month for the study of the ocean-atmosphere system from the continental margins to the mid-ocean ridges. A core component of OOI is the Regional Cabled Array, which uses a fiber-optic cable to connect and power the largest array of networked oceanographic instruments in the world, delivering data in real-time to shore.

CamHD is a high-definition video camera connected to the OOI’s fiber optic cable at Axial Seamount and provides data that can support a wide range of oceanographic, biological, and geophysical investigations. Every three hours, the camera scans a hydrothermal vent chimney, imaging the entire chimney over the course of about fifteen minutes. The notebook provided under “data access” demonstrates how to load video data from CamHD and demonstrates the basic usage of the pycamhd library, which can be used to extract frames from the ProRes-encoded Quicktime files.

Storage resources

All available video files are listed in a JSON file that has useful information such as the Unix timestamp (seconds) of the first frame in each video, and the total number of frames in each video. Data are stored as bock blobs on Azure Blob storage in the following container:

https://ooiopendata.blob.core.windows.net/camhd

We also provide a read-only SAS (shared access signature) token to allow access to NAIP data via, e.g., BlobFuse, which allows you to mount blob containers as drives:

?sv=2019-12-12&si=camhd-aod-ro&sr=c&sig=zFVfMOqa1YW9mxbEusUsKfPrKjkBFyD2YAUJficSuCo%3D

Mounting instructions for Linux are here. Large-scale processing using this dataset is best performed in the East US Azure data center, where the data are stored. Computational resources are available at ooi.pangeo.io, and if you are using CamHD data for environmental science applications, you may also consider applying for an AI for Earth grant to support your compute requirements.

Pretty picture


The HD camera (orange triangular frame) images the 14 ft-tall actively venting hot spring deposit “Mushroom” located within the caldera for Axial Seamount. The vent rests on an old lava flow. Radiating cracks in the flow are filled with white bacterial mats and small tube worms, marking sites of diffusely flowing fluids that issue from the fractures in the basalt. The 3-D temperature array in the background encloses a tube worm bush, sending 24 temperature measurements live to shore every second.

Contact

For questions about this dataset, contact aiforearthdatasets@microsoft.com.

Notices

MICROSOFT PROVIDES AZURE OPEN DATASETS ON AN “AS IS” BASIS. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, GUARANTEES OR CONDITIONS WITH RESPECT TO YOUR USE OF THE DATASETS. TO THE EXTENT PERMITTED UNDER YOUR LOCAL LAW, MICROSOFT DISCLAIMS ALL LIABILITY FOR ANY DAMAGES OR LOSSES, INCLUDING DIRECT, CONSEQUENTIAL, SPECIAL, INDIRECT, INCIDENTAL OR PUNITIVE, RESULTING FROM YOUR USE OF THE DATASETS.

This dataset is provided under the original terms that Microsoft received source data. The dataset may include data sourced from Microsoft.

Access

Available inWhen to use
Azure Notebooks

Quickly explore the dataset with Jupyter notebooks hosted on Azure or your local machine.

Select your preferred service:

Azure Notebooks

Azure Notebooks

Package: Language: Python

OOI Seafloor Camera Open Datasets Example

This Jupyter notebook shows how to load video data from the OOI Seafloor Camera system deployed at Axial Volcano on the Juan de Fuca Ridge. It demonstrates the basic usage of the pycamhd library, which can be used to extract frames from the ProRes encoded Quicktime files. These data are hosted on Microsoft's Azure Open Datasets.

Imports and constants

In [1]:
# Standard packages
import numpy as np
import pandas as pd
import fsspec
import time
import datetime
import random
import matplotlib.pyplot as plt
from ipywidgets import interact
from ipywidgets import IntSlider

# Non-standard, but still pip- or conda-installable
import pycamhd as camhd

# .json file containing video metadata
dbcamhd_url = 'https://ooiopendata.blob.core.windows.net/camhd/dbcamhd.json'

Load the JSON file that lists all available video files

This file has useful information about each file in Azure Open Datasets, such as the Unix timestamp (seconds) of the first frame in each video, and the total number of frames in each video.

In [2]:
with fsspec.open(dbcamhd_url) as f:
    dbcamhd = pd.read_json(f, orient="records", lines=True)
dbcamhd.tail()
Out[2]:
name url filesize md5 moov timestamp deployment frame_count
18745 CAMHDA301-20201201T160000.mov https://ooiopendata.blob.core.windows.net/camh... 17408035167 9f5d506b42c5fb7f9551622da1ccb733 True 1606837502 7 26947
18746 CAMHDA301-20201201T161500.mov https://ooiopendata.blob.core.windows.net/camh... 16776728895 129ace733db2f1d48005ae4fe6da7aa3 True 1606838401 7 26995
18747 CAMHDA301-20201201T163000.mov https://ooiopendata.blob.core.windows.net/camh... 16367647791 7b63f3c29ab0171b72672e96c2362792 True 1606839302 7 26953
18748 CAMHDA301-20201201T164500.mov https://ooiopendata.blob.core.windows.net/camh... 17174146875 99327a9196ad2196fe4dfb7a3cbc9d8e True 1606840201 7 26998
18749 CAMHDA301-20201201T170000.mov https://ooiopendata.blob.core.windows.net/camh... 17121143943 c1ae06d8105b441cb6a313e1539fa885 True 1606841102 7 26964

Choose a file to work with

In [5]:
# Find files from September 20, 2017...
start_time = datetime.datetime(2017,9,20,0,0,0)
end_time = datetime.datetime(2017,9,21,0,0,0)

start_unixtime = time.mktime(start_time.timetuple())
end_unixtime = time.mktime(end_time.timetuple())

matching_rows = dbcamhd[dbcamhd['timestamp'].between(start_unixtime,end_unixtime)]
matching_rows
Out[5]:
name url filesize md5 moov timestamp deployment frame_count
6997 CAMHDA301-20170920T091500.mov https://ooiopendata.blob.core.windows.net/camh... 15205675185 09fad12381c5bcc15cfa8c8ea67cd81a True 1505898002 4 26959
6998 CAMHDA301-20170920T121500.mov https://ooiopendata.blob.core.windows.net/camh... 15229684709 5acd39433f733a77bcc23e16c514008d True 1505908801 4 26990
6999 CAMHDA301-20170920T151500.mov https://ooiopendata.blob.core.windows.net/camh... 15220788561 9b2811175b1f4ad48bb74e34d37a6174 True 1505919601 4 26989
7000 CAMHDA301-20170920T181500.mov https://ooiopendata.blob.core.windows.net/camh... 15250813073 eaadcc47dcaf8c7a94bb5b9708179db8 True 1505930402 4 26953
7001 CAMHDA301-20170920T211500.mov https://ooiopendata.blob.core.windows.net/camh... 15199072513 7df862c4793e803e086962da67098fb4 True 1505941201 4 26963
7002 CAMHDA301-20170921T001500.mov https://ooiopendata.blob.core.windows.net/camh... 15164009925 6cf2db5e274495b6f758b2a404eefb59 True 1505952001 4 26984
7003 CAMHDA301-20170921T031500.mov https://ooiopendata.blob.core.windows.net/camh... 15208537265 107b63452cb38c5c4a982631f91675fa True 1505962801 4 26959
7004 CAMHDA301-20170921T061500.mov https://ooiopendata.blob.core.windows.net/camh... 15194038145 fda92aa836fe126b93ef7eca21ee5729 True 1505973601 4 26987
In [6]:
# ...and choose the first file from that day.
mov = matching_rows.iloc[0]
mov
Out[6]:
name                               CAMHDA301-20170920T091500.mov
url            https://ooiopendata.blob.core.windows.net/camh...
filesize                                             15205675185
md5                             09fad12381c5bcc15cfa8c8ea67cd81a
moov                                                        True
timestamp                                             1505898002
deployment                                                     4
frame_count                                                26959
Name: 6997, dtype: object

Render one frame from the video

In [7]:
def show_image(frame_number):
    plt.rc('figure', figsize=(12, 6))
    plt.rcParams.update({'font.size': 8})
    frame = camhd.get_frame(mov.url, frame_number)
    fig, ax = plt.subplots();
    im1 = ax.imshow(frame);
    plt.yticks(np.arange(0,1081,270))
    plt.xticks(np.arange(0,1921,480))
    plt.title('Deployment: %s    File: %s    Frame: %s' % (mov.deployment, mov['name'], frame_number));

# Choose a random frame 
initial_frame = random.randrange(0,mov.frame_count)
print('Showing frame {}'.format(initial_frame))
show_image(initial_frame)
Showing frame 24796
Unknown prores profile 0

Interactively explore frames from the video

We include a slider which allows any frame from this video to be shown. If you're viewing an HTML version of this notebook, this cell will look sad, but we swear, the widget you don't see allows you to interactively view frames.

In [8]:
frame_slider = IntSlider(min=0, max=mov.frame_count-1, step=1, value=initial_frame, continuous_update=False)
interact(show_image, frame_number=frame_slider)
Out[8]:
<function __main__.show_image(frame_number)>