As we get ready for SQL PASS Summit 2015, the world’s largest gathering of SQL Server and BI professionals, we’re excited to share that Azure Data Lake Analytics and Azure Data Lake Store are now in Public Preview.
Previously at the Strata conference, we announced our full vision of adding a new analytics service and our existing Azure HDInsight solution to the Azure Data Lake. Our goal is to make big data technology simpler and more accessible to the greatest number of people possible.
To get you productive using Azure Data Lake, we will have a six-part blogging series on different aspects of Azure Data Lake in the next few days. This is the first blog in the series introducing you to both services. Stay tuned on the Azure Big Data channel for more content!
What is Azure Data Lake Analytics?
Azure Data Lake Analytics is a distributed big data service that dynamically scales your code so that you only need to focus on business logic and not on distributed infrastructure. Built on Apache YARN, and designed for the cloud, the analytics service can handle jobs of any scale instantly by simply setting the dial for how much power you need. The analytics service for Azure Data Lake is cost-efficient because you only pay for your job when it is running, and support for Azure Active Directory lets you manage access and roles simply and integrates with your on-premises identity system.
We know that many developers and data scientists struggle to be successful with big data using existing technologies and tools. Azure Data Lake Analytics includes U-SQL, a language that unifies the benefits of SQL with the expressive power of your own code. The U-SQL language is built on the same distributed runtime that powers the big data systems inside Microsoft. Millions of SQL and .NET developers can now process and analyze all of their data with the skills they already have.
The U-SQL support in Azure Data Lake Tools for Visual Studio includes state of the art support for authoring, debugging and advanced performance analysis features for increased productivity when optimizing jobs running across thousands of nodes. Visualizations of your U-SQL code allows you to see how your code runs at scale and identify performance bottlenecks and cost optimizations, making it easier to tune your queries.
What is Azure Data Lake Store?
The Azure Data Lake Store provides a single repository where you can easily capture data of any size, type and speed without forcing changes to your application as data scales. In the store, data can be securely shared for collaboration and is accessible for processing and analytics from HDFS applications and tools.
Where can I get more information?
- Head over to the announcement blog for more details.
- Check out the Visual Studio’s U-SQL blog to learn more about the new big data language.
- Go to the Azure.com marketing Data Lake solution page.
- Watch a video how we make big data easy in Azure.
Documentation and How-To’s
- Azure Data Lake Analytics
- Overview of Azure Data Lake Analytics
- Getting started with Azure Data Lake Analytics in the portal
- Getting started with Azure Data Lake Analytics with PowerShell
- Getting started with Azure Data Lake Analytics and the tooling
- Getting started with Azure Data Lake Analytics with the SDK
- Managing Azure Data Lake Analytics with the portal
- Managing Azure Data Lake Analytics with PowerShell
- Interactive tutorials on Azure Data Lake Analytics
- Analyzing web logs with Azure Data Lake Analytics
- U-SQL
- Azure Data Lake Store
- Overview of Azure Data Lake Store
- Getting started with Azure Data Lake Store from the Portal
- Getting started with Azure Data Lake Store from PowerShell
- Getting started with Azure Data Lake with .NET SDK
- Securing data with Azure Data Lake Store
- Connecting Azure HDInsight with Azure Data Lake Store
- Connecting other OSS applications with Azure Data Lake Store
- WebHDFS APIs with Azure Data Lake Store