Azure Databricks provides a fast, easy, and collaborative Apache Spark™-based analytics platform to accelerate and simplify the process of building big data and AI solutions backed by industry leading SLAs.
With Azure Databricks, customers can set up an optimized Apache Spark environment in minutes. Data scientists and data engineers can collaborate using an interactive workspace with languages and tools of their choice. Native integration with Azure Active Directory (Azure AD) and other Azure services enables customers to build end-to-end modern data warehouse, machine learning and real-time analytics solutions.
We have seen tremendous adoption of Azure Databricks and today we are excited to announce new capabilities that we are bringing to market.
General availability of Data Engineering Light
Customers can now get started with Azure Databricks with a new low-priced workload called Data Engineering Light that enables customers to run batch applications on managed Apache Spark. It is meant for simple, non-critical workloads that don’t need the performance, autoscaling, and other benefits provided by Data Engineering and Data Analytics workloads. Get started with this new workload.
Additionally, we have reduced the price for the Data Engineering workload across both the Standard and Premium SKUs. Both the SKUs are now available at up to 25 percent lower cost. To check out the new pricing for Azure Databricks SKUs, visit the pricing page.
Preview of managed MLflow
MLflow is an open source framework for managing the machine learning lifecycle. With managed MLflow, customers can access it natively from their Azure Databricks environment and leverage Azure Active Directory for authentication. With managed MLflow on Azure Databricks customers can:
- Track experiments by automatically recording parameters, results, code, and data to an out-of-the-box hosted MLflow tracking server. Runs can now be organized in experiments from within the Azure Databricks, and results can be queried from within the Azure Databricks notebooks to identify the best performing models.
- Package machine learning code and dependencies locally in a reproducible project format and execute remotely on a Databricks cluster.
- Quickly deploy models to production.
Learn more about managed MLFlow.
Machine learning on Azure with Azure Machine Learning and Azure Databricks
Since the general availability of Azure Machine Learning service (AML) in December 2018, and its integration with Azure Databricks, we have received overwhelmingly positive feedback from customers who are using this combination to accelerate machine learning on big data. Azure Machine Learning complements the Azure Databricks experience by:
- Unlocking advanced automated machine learning capability which enables data scientists of all skill levels to identify suitable algorithms and hyperparameters faster.
- Enabling DevOps for machine learning for simplified management, monitoring, and updating of machine learning models.
- Deploying models from the cloud and the edge.
- Providing a central registry for experiments, machine learning pipelines, and models that are being created across the organization.
The combination of Azure Databricks and Azure Machine Learning makes Azure the best cloud for machine learning. Customers benefit from an optimized, autoscaling Apache Spark based environment, collaborative workspace, automated machine learning, and end-to-end Machine Learning Lifecycle management.
Get started today!
Try Azure Databricks and let us know your feedback!
- Try Azure Databricks through a 14-day premium trial.
- Try Azure Machine Learning.
- Watch the webinar on Machine Learning on Azure.