{"id":2375,"date":"2018-08-27T00:00:00","date_gmt":"2018-08-27T00:00:00","guid":{"rendered":""},"modified":"2025-04-30T10:34:35","modified_gmt":"2025-04-30T17:34:35","slug":"monitoring-environmental-conditions-near-underwater-datacenters-using-deep-learning","status":"publish","type":"post","link":"https:\/\/azure.microsoft.com\/en-us\/blog\/monitoring-environmental-conditions-near-underwater-datacenters-using-deep-learning\/","title":{"rendered":"Monitoring environmental conditions near underwater datacenters using Deep Learning"},"content":{"rendered":"

This blog post is co-authored by Xiaoyong Zhu, Nile Wilson (Microsoft AI CTO Office), Ben Cutler (Project Natick), and Lucas Joppa (Microsoft Chief Environmental Officer).<\/em><\/p>\n

At Microsoft, we put our cloud and artificial intelligence (AI) tools in the hands of those working to solve global environmental challenges, through programs such as AI for Earth<\/a>. We also use these same tools to understand our own interaction with the environment, such as the work being done in concert with Project Natick<\/a>.<\/p>\n

Project Natick<\/a> seeks to understand the benefits and difficulties in deploying subsea datacenters worldwide; it is the world’s first deployed underwater datacenter and it was designed with an emphasis on sustainability. Phase 2 extends the research accomplished in Phase 1 by deploying a full-scale datacenter module in the North Sea, powered by renewable energy. Project Natick uses AI to monitor the servers and other equipment for signs of failure and to identify any correlations between the environment and server longevity.<\/p>\n

Because Project Natick operates like a standard land datacenter, the computers inside can be used for machine learning to provide AI to other applications, just as in any other Microsoft datacenter. We are also using AI to monitor the surrounding aquatic environment, as a first step to understanding what impact, if any, the datacenter may have.<\/p>\n

\"image\"<\/p>\n

Figure 1. Project Natick datacenter pre-submergence. Photo by Scott Eklund\/Red Box Pictures.<\/em><\/p>\n

Monitoring marine life using object detection<\/h2>\n

The Project Natick datacenter is equipped with various sensors to monitor server conditions and the environment, including two underwater cameras, which are available as live video streams (check out the livestream on the Project Natick homepage<\/a>). These cameras allow us to monitor the surrounding environment from two fixed locations outside the datacenter in real time.<\/p>\n

\"image\"<\/p>\n

\"Natick<\/a><\/p>\n

Figure 2. Live camera feed from Project Natick datacenter (Source: Project Natick homepage<\/a>)<\/em><\/p>\n

We want to count the marine life seen by the cameras. Manually counting the marine life in each frame in the video stream requires a significant amount of effort. To solve this, we can leverage object detection to automate the monitoring and counting of marine life.<\/p>\n

\"image\"<\/p>\n

Figure 3. From the live camera feed, we observe a variety of aquatic life, including left, ray; middle, fish; right, arrow worm<\/em><\/p>\n

In each frame, we count the number of marine creatures. We model this as an object detection problem. Object detection combines the task of classification with localization, and outputs both a category and a set of coordinates representing the bounding box for each object detected in the image. This is illustrated in Figure 4.<\/p>\n

\"image\"<\/p>\n

Figure 4. Object detection tasks in computer vision. left, input image; right, object detection with bounding boxes<\/em><\/p>\n

How to choose an object detection model<\/h2>\n

Over the past few years, many exciting deep learning approaches for object detection have emerged. Models such as Faster R-CNN<\/a> use a two-stage procedure to first propose regions containing some object, followed by classification of the proposed region and adjusting the proposed bounding box. Using a one-stage approach, models such as You Only Look Once (YOLO) and Single Shot MultiBox Detector <\/a>(SSD), or RetinaNet<\/a> with focal loss, consider a fixed set of boxes for detection and skip the region proposal stage, which are usually faster compared with two-stage detectors.<\/p>\n

To achieve real-time object detection for videos, we need to balance between speed and accuracy. From the video stream, we observed that there are only a few types of aquatic life that come within view\u2014fish, arrow worms, and rays. The limited number of animal categories allows us to choose a relatively light-weight object detection model which can be run on CPUs.<\/p>\n

By comparing the speed and accuracy of different deep learning model architectures, we chose to use SSD with MobileNet as our network architecture.<\/p>\n

\"image\"<\/p>\n

Figure 5. Object detection approaches (tradeoffs between accuracy and inference time). Marker shapes indicate meta-architecture and colors indicate feature extractor. Each (meta-architecture, feature extractor) pair corresponds to multiple points on this plot due to changing input sizes, stride, etc. (source)<\/em><\/p>\n

An AI Oriented Architecture for environmental monitoring<\/h2>\n

AI Oriented Architecture (AOA) is a blueprint for enterprises to use AI to accelerate digital transformation. An AOA enables organizations to map a business solution to the set of AI tools, services and infrastructure that will enable\u00a0them to realize the business solution. In this example, we use an AI Oriented Architecture to design a deep learning solution for environmental monitoring.<\/p>\n

Figure 6 shows an AI Oriented Architecture for monitoring marine life near the underwater datacenter. We first save the streamed video file from Azure Media Service using OpenCV, then label the video frames using VoTT and put the labelled data in Azure Blob Storage. Once the dataset is labelled and placed in Azure Blob Storage, we start training an object detection model using Azure. After we have trained the model, we deploy the model to the Natick datacenter, so the model can run inference on the input stream directly. The result is then either posted to PowerBI or displayed directly in the UI for intuitive debugging. Each of these steps is discussed in the following sections.<\/p>\n

\"image\"<\/p>\n

Figure 6. AI Oriented Architecture for detecting underwater wildlife<\/em><\/p>\n

The key ingredients of the environment monitoring solution include the following:<\/p>\n