The confluence of cloud, data, and AI is foundational to innovation and is driving unprecedented change. This week at Spark + AI Summit, I talked about how Microsoft enables organizations to take advantage of Azure to build advanced machine learning models and intelligent applications virtually anywhere.
As Satya mentioned during our Build conference last month, applications will increasingly require a ubiquitous computing fabric from the cloud to the edge. These applications also require new machine learning and AI capabilities that enable them to see, hear and predict. The driving force behind these capabilities is data. Data is vital to every app and experience we build today. Organizations are using their data to extract important insights to drive their businesses forward and engage their customers in new ways. Customers like Renault-Nissan are revolutionizing their customer experience with connected cars. Rockwell Automation, a leader in industrial automation has built predictive maintenance capabilities on their equipment to save time and reduce cost associated with device failure. Liebherr, a leader in manufacturing, produces intelligent refrigerators that use object recognition to recommend grocery lists based on refrigerator contents. These are just a few examples of customers leveraging their data, wherever it exists, to turn it into breakthrough insights.
To enable developers and data scientists to build data and AI solutions that impact the world at large, we announced the general availability of Azure Databricks in March. Azure Databricks combines the best of the Apache® Spark™ analytics platform and Microsoft Azure to help customers unleash the power of data like never before. Azure Databricks enables customers like Shell to transform the way they do business.
“At Shell, we are constantly pushing the boundaries of technology, for instance we are using Artificial Intelligence (AI) to help improve our predictive maintenance. We have drones that take pictures of our facilities and use machine vision to help us identify the need for maintenance. With Azure Databricks, we can now run deep learning to build predictive maintenance models that are used to detect and fix issues, increase our operational efficiency and enhance safety.”
– Dan Jeavons, General Manager of Data Science
Organizations also benefit from Azure Databricks' native integration with other services like Azure Blob Storage, Azure Data Factory, Azure SQL Data Warehouse, and Azure Cosmos DB. This enables new analytics solutions that support modern data warehousing, advanced analytics, and real-time analytics scenarios.
To continue driving innovation for our customers, we made a couple of exciting enhancements to Azure Databricks that we announced this week at the Spark + AI Summit.
Support for GPU enabled virtual machines
To quickly build AI models from large volumes of data, specialized hardware like GPUs is required.
Azure Databricks now supports the ability to use GPU enabled VMs as a choice for its clusters. Developers can now easily build, train and deploy AI models at scale using these optimized clusters.
Runtime for machine learning
In addition to GPU support, we have also enhanced Azure Databricks’ AI capabilities with a new machine learning runtime. This runtime enables distributed, multi-GPU training of deep neural networks using Horovod and includes HorovodEstimator for seamless integration with Spark DataFrames. It also comes pre-installed and pre-configured with all the necessary packages such as TensorFlow, Keras, and XGBoost.
This runtime enables developers to build deep learning models with a few lines of code. Previously, developers had to invest considerable time and effort to leverage these toolkits. Now, they no longer have to write their own logic to load data, distribute training code to multiple clusters and validate model accuracy.
Azure Databricks Runtime for Machine Learning is available in preview today, as part of premium SKU in Azure Databricks.
At Microsoft, we are committed to delivering AI into the hands of every developer and data scientist so they can unleash the power of data and reimagine possibilities that will improve our world.
Get started today!
- Already using Azure? Try Azure Databricks now or create a free Azure account to start using Azure Databricks.
- Watch how to get started with Apache Spark on Azure Databricks.
- Download the three practical use cases with Azure Databricks eBook.
- Learn how you can use Azure Databricks and Azure SQL Data Warehouse together.
