New options to modernize your application with Azure SQL Database

Posted on May 10, 2017

General Manager, Database Systems Group

Focus on Your Business, Leave the Rest to Us

In today’s cloud-centric world, enterprises increasingly focus on their core competencies while relying on their cloud provider to offer an affordable, worry-free computing infrastructure that scales with their needs. Using managed services helps the bottom line, improves time to market and ultimately drives business.

Azure SQL Database is a rapidly growing, fully managed, relational Database-As-A-Service (DBaaS) managing millions of production databases across 38 data centers around the world. Customers can rely on Azure SQL Database for supporting a broad range of applications and workloads, from managing straightforward transactional data to driving the most data-intensive, mission-critical applications requiring advanced data processing at global scale.

In this blog post, I want to share a few examples of the industry-leading features that differentiate our platform, announce exciting new capabilities and share updates on our plans to remain the most scalable, secure and customer-friendly database service in the industry.

New Announcement: Easy Lift-and-Shift to PaaS for all Applications

As the cloud industry matures and more enterprises are adopting cloud technologies as a standard practice, we’ve spoken with customers in our database community who feel left behind. They say they absolutely want to enjoy the benefits of a managed database service in the cloud but, until now, have been unable to do so due to their instance-level functionality or isolation requirements. Customers who have made deep investments in instance-level SQL Server capabilities (e.g. SQL CLR integration, SQL Server Agent, cross database querying, etc.) expect similar capabilities in Azure SQL Database. Similarly, customers in highly regulated industries need to maintain isolation for security purposes when moving to the cloud.

Managed Instance

Today we announced a preview of a new deployment option within Azure SQL Database called Managed Instance.  Managed Instance offers near 100% SQL Server compatibility with the benefits of platform as a service! This includes features like SQL CLR, SQL Server Agent and cross-database querying. If you can imagine a managed SQL Server environment where you continue using instance level capabilities without changing your application design – it’s a real game changer

DocuSign, the global leader in eSignature and Digital Transaction Management (DTM), was the first customer to test the Managed Instance private preview in its working environment. As a company that serves more than 300,000 customers and 200 million users in 188 countries, the scalability, performance and manageability of its database is mission-critical. And in testing, DocuSign was able to ‘lift and shift’ the SQL Server code in its core platform over to Managed Instance with no changes, and start running queries in seconds – something its chief architect, Eric Fleishman, says shows incredible promise for the future.

We're also announcing a new Database Migration Service to make the migration to Managed Instance automated, risk free, and with down time measured in minutes.  This service streamlines moving SQL Server and non-Microsoft database systems such as Oracle to Azure SQL Database.

For customers in highly regulated industries, we’re bringing two important innovations: VNETs with support for private IP addresses, and controlled service updates. With VNETs, customers will be able to completely isolate their database tier from the public internet and join it to their other cloud VNETs or on-premise networks where their application and users reside. With controlled service updates, customers can run their test environment deployed in a regular Azure public cloud subscription, where they will receive continuous service updates. After they have validated their application with the latest updates, they will be able to apply the updates in a scheduled manner to the production Managed Instance environment.

Managed Instance is currently in private preview (sign-up page), during which we’ll work with early adopter customers to make sure we satisfy the “lift and shift” requirements for the widest set of existing applications. If you are an ISV interested in making sure your application works with Azure SQL Managed Database Instance, or you would like to leverage it for your own migration effort, enroll today for more information.

With the Managed Instance offering, for those customers in our SQL Server community who have not had a path to enjoy the benefits of Azure SQL Database so far, today we offer them a bridge (or ladder) to the cloud.

Innovation and Evolution

Along with evolutionary change introduced with the new Managed Instance offering, to maintain our lead in offering this premier platform as a service we continuously deliver innovations across performance, intelligence, availability, scalability and security.

Built-in Intelligent Optimization

By choosing Azure SQL Database, you not only get built-in patching, updates, database backups and high availability, you also get built-in intelligence. Built-in intelligence helps you dramatically reduce the costs of running and managing databases and maximizes both performance and security of your application.  Under this feature umbrella, we have SQL Database Advisor, Automatic Tuning, and Adaptive Query Processing.

SQL Database Advisor

Running millions of customer workloads around-the-clock, Azure SQL Database collects and processes a massive amount of telemetry data, while also fully respecting customer privacy behind the scenes. Various algorithms are continuously evaluating the telemetry data so that the service can learn and adapt with your application. Based on this analysis, the service comes up with performance improving recommendations tailored to your specific workload. 

How exactly is this built-in intelligence surfaced? Azure SQL Database automatically identifies the right non-clustered indexes to create or drop and serve them to customers as actionable recommendations, or proactively implements the changes to the database, if the customer opts into automatic tuning mode. The feature will also automatically test and verify each of its actions to ensure the performance keeps improving. This means that the performance of your database is carefully monitored and compared before and after every tuning action, and if the performance doesn’t improve, the tuning action is reverted.

Automatic Tuning

In addition to automatic index tuning which is already available through SQL Database Advisor, Azure SQL Database will soon be receiving a new array of adaptability features. For example, the Automatic Tuning for query plans feature provides a “safety net” for query plan choices, helping your databases always run at top performance by automatically correcting plan regressions. Today, many of our partners running SaaS multi-tenant apps on top of Azure SQL Database are relying on automatic performance tuning to make sure their applications always have stable and predictable performance. For them, this feature tremendously reduces the risk of having a performance incident in the middle of the night. In addition, since part of their customer base also uses SQL Server, they are using the same indexing recommendations provided by Azure SQL Database to help their SQL Server customers.

Adaptive Query Processing

We are also adding the adaptive query processing family of features to Azure SQL Database, including interleaved execution for multi-statement table valued functions, batch mode memory grant feedback and batch mode adaptive joins. Each of these adaptive query processing features apply similar “learn and adapt” techniques, helping further address performance issues related to historically intractable query optimization problems.

Global Scalability and Availability

Azure SQL Database provides out-of-the-box business continuity and global scalability features, including automatic backups, point-in-time restores, and Active Geo-Replication which allows you to configure up to four readable secondary databases in either the same or globally distributed Azure data centers.  For example, if you have a SaaS application with a catalog database that has a high volume of concurrent read-only transactions, with Active Geo-Replication you can enable global read scale and remove bottlenecks on the primary that were due to read workloads. What’s more, with recently released auto-failover groups you can enable high availability and load balancing at global scale, including transparent geo-replication and failover of large sets of databases and elastic pools. This enables creation of globally distributed SaaS applications with minimal administration overhead leaving all the complex monitoring, routing and failover orchestration to SQL Database.

Industry-leading Advanced Security

Securing customer data while maintaining the highest levels of privacy have always been top priorities for Microsoft and the Data Services organization. I have previously blogged about our security investments here and how we deliver industry-leading security features that continue to offer the highest levels of security in the industry. Our service includes Transparent Data Encryption, SQL Threat Detection, Always Encrypted, Multi-Factor Authentication and much more.

Transparent Data Encryption

We recognize that security must be easy to use in order to be effective. For example, starting this month, all newly created Azure SQL databases will automatically be protected with Transparent Data Encryption (TDE). TDE is SQL’s proven encryption-at-rest technology that is required by many compliance standards to protect against theft of storage media. Customers can manage the TDE encryption keys and other secrets in a secure and compliant management using Azure Key Vault.

GEP is a multinational company that offers a cloud-based procurement-software platform which uses Azure SQL Database and TDE.  In their own words:

“Azure managed TDE helped us to be compliant with our customers strict data encryption policy and also gives us peace of mind in securely managing large number of databases. Major benefit is we are able to give TDE encryption to all our customers without impacting application performance.” Sathyan Narasingh, Engineering Manager with GEP

SQL Threat Detection

Additionally, SQL Threat Detection continuously monitors databases for potentially harmful attempts to access sensitive data. For example, in the case of an E-Learning company, SQL Threat Detection identified the activities of a disgruntled ex-employee who was exfiltrating sensitive data after the company forgot to disable the employee’s credentials. SQL Threat Detection compared current to past access locations and query patterns, raised an alert, and was able to help the customer close the leak quickly and minimize damage.

Always Encrypted

Azure SQL Database is the only database system to offer protection of sensitive data in flight, at rest and during query processing with Always Encrypted. Always Encrypted is an industry-first that offers unparalleled data security against breaches involving the theft of critical data. For example, with Always Encrypted, customers’ credit card numbers are stored encrypted in the database always, even during query processing, allowing decryption at the point of use by authorized staff or applications that need to process that data.

Here is how Always Encrypted helped Med+Proctor - a fast-growing healthcare startup:

"There is often a considerable gap between health care and startup SaaS providers and modern and easily auditable data security. Always Encrypted bridged that gap for Med+Proctor by providing us with tools for column encryption, managed access, and key rotation without having to create our own solution or purchase expensive 3rd party tools." - Andy Howden, CIO, Med+Proctor

Multi-Factor Authentication

Users of Azure SQL Database benefit from single sign-on through Azure Active Directory Authentication which now also supports Multi-Factor Authentication (MFA). MFA is an authentication option that works for a growing number of tools and services across SQL Server and Azure SQL Database, such as SSMS or Visual Studio.

Pushing the boundaries

Azure SQL Database provides a powerful platform for all data-intensive applications and it has never been easier to shift your workloads to the cloud. Going forward we will continue to push the boundary in the following four areas:

Scale: A few months ago we added more price-performance choices for elastic pools,: including higher compute pools up to 4000 DTUs, smaller 50 DTU pools with a lower starting price, and higher database limits per pool, effectively reducing the cost per database. In April, we increased the storage limit for P11 and P15 databases from 1 TB to 4 TB.  And in May, the storage limit for the largest Premium elastic pools increased from 750 GB to 4 TB. Over the coming months, Azure SQL Database will increase the storage limits even further, and provide the ability to purchase storage separately from DTUs for both standalone databases and elastic pools.

Monitoring: Recently we released an OMS solution that helps you monitor critical metrics from your cloud databases at scale, as well as set up alerting on metrics of interest from a centralized location. Very soon, we are going to enhance this solution with high precision query insights, continuously fed by the advanced “flight data recorder” feature running on all of your cloud databases, Query Store. This will provide valuable insights for SaaS applications that run the same queries on many databases and help you pin point the most resource consuming queries on all databases in aggregate.

Built-in Intelligence: Over the next several months, we will come much closer to fulfilling our vision of a self-optimizing and self-monitoring database service. This innovative technology has been battle tested and has matured over the last year. As I write this, it is already running around-the-clock on approximately 50,000 databases in the service. We plan to make performance automatic tuning the default setting for all service databases. This means that we will provide non-clustered index tuning and maintenance, as well as automatic plan corrections to all customer databases, at cloud scale. Every developer or IT Professional using Azure SQL Database will benefit from the improved performance and stability brought by the built-in intelligence, without investing any effort.

Security: Our goal is to make it easy for enterprise customers to use the security capabilities available in in Azure SQL Database to implement and maintain effective defense strategies for their databases independent of their level of expertise. For example, we believe that securing a SQL database should be as simple as identifying the desired protection level (e.g., High Business Impact) and applying the appropriate policy to secure the database. You define the level and the system will do the rest, including identifying which data is sensitive and which features are needed to secure the data. While the database is in use, Azure SQL Database will continuously monitor for changes in the configuration and any unusual activities that may be signs of malicious attacks.

Just Getting Started

We will continue to invest in features that provide deep security by-default, enable frictionless migration, and optimize performance using machine learning and adaptive techniques with state-of-the-art relational database technologies to get us closer to our goals. We invite you to take advantage of the cost savings, fast time-to-market, built-in intelligence and the built-in administration that Azure SQL Database provides today! You don’t need a data center or an operations team. Instead, focus on what matters most to you and to your organization and leave the rest to us!

To learn about other announcements across the Microsoft Data Group, be sure to read Joseph Sirosh’s Build 2017 announcement blog post here.