Open source DC/OS project announced today

Today Microsoft joined Mesosphere and a broad group of industry partners including Accenture, Cisco, and Hewlett Packard Enterprise to announce the open source DC/OS project, a new effort powered by Apache Mesos for running modern distributed applications reliably and at scale. Derived from Mesosphere’s Datacenter Operating System, the 100 percent open source DC/OS offers powerful capabilities for container operations at scale and single-click, app-store-like installation of complex distributed systems, including HDFS, Apache Spark, Apache Kafka, Apache Cassandra and more.

Microsoft’s cloud strategy is founded on customer choice, including support for open systems and open source solutions. To run modern applications, organizations also need proven, flexible systems that support hybrid deployments across public cloud, private datacenters and hosting providers. With today’s announcement, DC/OS meets all these customer criteria: proven, flexible, portable and open source. Microsoft is proud to be a founding member of the new open source DC/OS community, and have made DC/OS a key component of our new Azure Container Service as it enters General Availability today. The Azure Container Service is the most open and flexible way to run your container applications in the cloud. Full support for Docker images provides portability of individual containers while our commitment to using only open source software in the orchestration layer provides portability of your applications across public and private clouds solutions as well as on-premises.

What is DC/OS?

The core of DC/OS is the proven Apache Mesos technology. Mesos has been described as a distributed systems kernel, like the Linux kernel, but running on each machine in the cluster to allow applications to be scheduled and resources managed across an entire datacenter. Started at UC Berkeley in 2009, Mesos was quickly adopted by Twitter and others to build their hyperscale cloud services. It is a well-known maxim that building and running distributed systems and applications is hard, so customers want solutions that have been battle-tested in production. The Powered By Mesos list now includes names such as AirBnb, Apple, Netflix. PayPal and Yelp. Clearly Mesos has a proven track record.

But a kernel alone is not enough to run most applications: other functionality commonly required includes service discovery and load balancing, deployment and package management, logging and metrics, and Web and command line user interfaces. Mesosphere built many of these components as part of their Data Center Operating System, and they are now included in the fully open source DC/OS project, including:

• Marathon: A container orchestrator platform that runs on top of Mesos. Marathon is one of the key technologies powering Azure Container Service.
• Universe: Provides an app store like experience for deploying distributed systems and additional management components, such as Chronos, a distributed job scheduler
• DC/OS UI and DC/OS CLI: For operating DC/OS from the web or command line
• Installers: GUI based installation for on-premises and cloud

DC/OS is the only open source project that bundles all these functions together into a single software distribution. DC/OS provides a complete open source solution, built on a stable, proven kernel for running datacenter scale services, and backed by a community led by Mesosphere and including over 50 technology companies and DC/OS users including Microsoft, Hewlett-Packard Enterprise, Intel, Cisco, Verizon and Yelp.

A growing Universe of solutions

An operating system also needs applications and solutions and an easy way to deploy them. DC/OS Universe is a package management and deployment system, not unlike Apt or Yum, but for distributed systems, including Apache Spark, Kafka, Cassandra and Zeppelin, and others. One of the unique advantages of DC/OS is the ability to run containers and data analytics applications in the same cluster, and Universe is key to making this easy. Furthermore, Universe will be expanding, as it offers a way for developers to package and publish solutions that run on DC/OS. The Universe web UI provides an appstore like experience for finding and deploying these solutions into your DC/OS cluster.

Running DC/OS on Microsoft Azure

Announcing general availability today, the Azure Container Service uses DC/OS to provide an easy-to-use, but powerful and reliable way to orchestrate containers at scale. With Azure Container Service, users get a reference implementation of DC/OS that can be deployed at scale in a matter of minutes by providing a very small set of parameters such as the size and location of the cluster. We believe Azure Container Service is the easiest way to get started with DC/OS for learning and experimentation or for production workloads.

Azure Container Service provides a minimum of configuration to make deployment easy and support the most use cases and known stable configurations. Of course, customers will want more control over the configuration of their DC/OS cluster. For these users, Mesosphere and Microsoft have also partnered to put DC/OS into the Azure Marketplace. The goal of the Marketplace version of DC/OS is to always have the latest edge, fully configurable version of DC/OS available for use on Azure.

Microsoft has been involved in the DC/OS project since its inception. Our customers asked us to build Azure Container Service on open source technology because they wanted a solution that did not lock them in to a single vendor or force them to choose between public cloud, their private datacenters or their favorite local hoster. With our partnership with Mesosphere, a leading provider of open source container orchestration solutions, we are meeting this customer demand. Our collaboration with the new DC/OS community will further expand upon this commitment.

What’s next?

The easiest way to try DC/OS is to launch an instance of Azure Container Service today. Try the tutorial. For more options, try the full DC/OS by launching from the Azure Marketplace. Read the documentation and tutorials for DC/OS for more details.

Microsoft will continue to contribute to DC/OS to make the project better and ensure it works great on Azure. Microsoft is also working with Mesosphere to add Windows support to the Apache Mesos project. When that work is complete it will be possible to extend DC/OS to mixed environments of Windows and Linux.

We also invite you to join the DC/OS community. Join the conversation about DC/OS on the project’s Slack channel. Get the code from GitHub. Or sign up and make a contribution of code or write a tutorial.

Finally, we’d like to thank the whole Mesosphere team for all their work developing DC/OS and organizing this new open source community as well as their partnership with us delivering DC/OS as part of Azure Container Service.