Azure Private Link, Best practices, Networking, Virtual WAN
Enabling and securing ubiquitous compute from intelligent cloud to intelligent edge
By Yousef Khalidi Corporate Vice President, Azure Networking
11 min read
Enterprises are embracing the cloud to run their mission-critical workloads. The number of connected devices on and off-premises, and the data they generate continue to increase requiring new enterprise network edge architectures. We call this the intelligent edge – compute closer to the data sources and users to reduce latency. The intelligent cloud, with its massive compute power, storage and variety of services works in concert with the intelligent edge using similar programming models to enable innovative scenarios and ubiquitous compute. Networking is the crucial enabler integrating the intelligent cloud with the intelligent edge.
The Azure Networking mission is to provide the most secure, reliable, and performant network for your workloads, delivered and managed from the intelligent cloud to the intelligent edge. We continue to innovate to help your services connect and extend to the cloud and the edge, be protected, delivered with optimal performance and provide insightful monitoring.
Microsoft global network
Microsoft runs one of the world’s largest Wide Area Network (WAN) that serves all Microsoft cloud services including Azure, Dynamics 365, Microsoft 365, LinkedIn, Xbox, and Bing. The WAN connects all Microsoft datacenters running our cloud services together and to our customers and partners through edge sites. These edge sites are strategically located around the world. This is where we exchange traffic with internet service providers for internet traffic and ExpressRoute partners for private connectivity traffic. We also use the Azure Front Door and Azure Content Delivery Network services at our edge sites to enhance and accelerate the experience of our own services, such as Microsoft 365. To provide global coverage the WAN has over 130,000 miles of subsea, terrestrial, and metro optical fiber and is fully managed by Microsoft using internal software defined networking (SDN) technologies to provide the best networking experience. Industry leaders such as Thousand Eyes have reported on the performance of our global network and in a 2018 study found it to be the most robust and most consistent. One fundamental principle in providing a great experience is to get the traffic onto the Microsoft network as close to the customer as possible and keep it on Microsoft’s network as long as possible. All traffic between Microsoft services and datacenters remains fully in Microsoft’s network and does not traverse the internet.
Figure 1. Core pillars of Azure Networking
Connect and extend
To get the best internet experience, data should enter and exit the Microsoft network as close as possible to you or your users. With over 160 edge sites today, we have an aggressive plan to increase the number of sites, which you can read more about in our edge site expansion blog. We are also increasing the number of ExpressRoute meet-me sites, providing greater flexibility to privately connect to your Azure workloads.
Staying connected to access and ingest data in today’s highly distributed application environments is paramount for any enterprise. Many businesses need to operate in and across highly unpredictable and challenging conditions. For example, energy, farming, mining, and shipping often operate in remote, rural, or other isolated locations with poor network connectivity. ExpressRoute for Satellites is now generally available, enabling access to Microsoft cloud services using satellite connectivity. With commercial satellite constellations becoming widely available, new solution architectures offer improved and affordable performance to access Microsoft.
MACsec, an industry encryption standard for point to point connections, is now supported on ExpressRoute Direct as a preview ability. ExpressRoute Direct customers can ensure data confidentiality and integrity between physical connections to the ExpressRoute routers to meet security and compliance requirements. Customers fully own and manage the lifecycle of the MACsec keys using Azure Key Vault.
We have invested in optical technologies to greatly reduce the cost of metro networks. We are passing these savings to you with a new ExpressRoute circuit type called ExpressRoute Local, available via ExpressRoute partners. If you select an ExpressRoute site near our datacenters and only access data from that datacenter then egress prices are included in the ExpressRoute Local circuit price. For connectivity to regions in the same geo you can use ExpressRoute Standard, and to get anywhere in the world you can use ExpressRoute Premium.
The new peering service for the Microsoft cloud, now in preview, enables enterprise-grade internet connectivity to access Azure, Dynamics 365, and Microsoft 365, via partnerships with internet providers and internet exchange providers. Peering service also provides internet latency telemetry, route monitoring, and alerting against hijacks, leaks, and other border gateway protocol misconfigurations.
Figure 2. Launch partners supporting the new Peering Service
We have enhanced our VPN service to support up to 10 Gbps of aggregate encrypted bandwidth, IKE v1 on all our VPN gateway SKUs, and packet capture to help debug configuration issues. We have also enhanced our point-to-site VPN service to support Azure Active Directory and multifactor authentication. We also are making available an OpenVPN client that you can download and run to access your Vnet from anywhere.
Azure Virtual WAN brings together our Azure connectivity services into a single operational interface with major SD-WAN partners. Azure Virtual WAN enables a global transit network architecture by providing ubiquitous connectivity between globally distributed sets of spokes such as VNets, sites, applications, and users. Significant enhancements include the preview of hub-to-hub and any-to-any connectivity. Virtual WAN users can connect multiple hubs for full mesh connectivity to further simplify their network architecture. Additionally, ExpressRoute and point-to-site are now generally available with Virtual WAN.
Figure 3. Azure Virtual WAN full topology overview across customers sites and clients connecting to Azure
We have been working closely with industry leaders to expand the ecosystem support for Virtual WAN. Today, we are announcing that Cisco and Microsoft are partnering to modernize the network for the cloud. Cisco, one of our largest global and strategic partners, is working with Microsoft to integrate Cisco SD-WAN technology with both Azure Virtual WAN and Office 365 to enable seamless, distributed and optimal branch office connectivity to Azure and Office 365.
Additionally, other partners including Cloudgenix, Fortinet, Nokia-Nuage, and Silver Peak, have finalized their integrations with Virtual WAN and are immediately available.
Dual stack (IPv4 + IPv6) VNet will be generally available later this month. As a first in the cloud, Azure will enable customers to bring their own IPv6 private space into the VNet thereby avoiding any need for routing changes. IPv6 enables customers to address IPv4 depletion, meet regulatory requirements, and expand into the growing mobile and IoT markets with their Azure-based applications.
Figure 4. Architectural diagram of an Azure VNet routing with IPv6 between VMs, subnet and Load Balancer
Achieving Zero Trust networking
Cloud applications and the mobile workforce have redefined the security perimeter. The new perimeter isn’t defined by the physical location(s) of the organization, it now extends to every access point that hosts, stores, or accesses corporate resources and services.
Instead of believing everything behind the corporate firewall is safe, the Zero Trust model assumes breach and verifies each request as though it originates from an uncontrolled network. Regardless of where the request originates or what resource it accesses, Zero Trust teaches us to “never trust, always verify.”
Azure Networking services provide critical controls to enhance visibility and help prevent bad actors from moving laterally across the network. Networks should be segmented, including deeper software-defined micro-segmentation, and real-time threat protection, end-to-end encryption, monitoring, and analytics should be employed.
Azure Private Link – extended to all Azure regions
Azure Private Link brings Azure services into your private virtual network. Supported Azure services such as Storage, SQL Database, and Azure Synapse Analytics can be consumed over a private IP address thereby not opening the access control lists (ACLs) to public internet. The traffic going through Private Link will always be in the Microsoft backbone network and never entering the public internet. The platform as a service (PaaS) resources can also be accessed privately from on-premises through VPN or ExpressRoute private peering thereby keeping the ACLs simple. Starting today, Private Link will be available in all Azure public regions.
Figure 5. Architectural diagram of Private Link deployed cross-premises
Using Azure Private Link, Azure is the first cloud to provide data governance and compliance by implementing built-in data exfiltration protection. This brings us one step closer to our goal for zero trust networking wherein malicious actors within the trusted network can’t exfiltrate data to non-secure accounts, since individual PaaS instances instead of service frontends are mapped as private endpoints. Private Link also empowers software as a service (SaaS) providers in Azure to extend the same capability to their customers. Snowflake is an early adopter to the program, with more partner services to follow.
Azure Firewall Manager is a new security management service that provides central security policy and route management for cloud-based security perimeters. Azure is currently the only cloud provider to offer traffic governance, routing control, and third party integrated security through Azure Firewall and Firewall Manager. Global admins can centrally create hub and spoke architecture and associate security or routing policies with such a hub, referred to as a secured virtual hub.
Figure 6. Diagram of Azure Firewall Manager deployed inside Secured Virtual WAN Hubs
With trusted security partners, you can use your familiar, industry-leading, third-party security as a service (SECaaS) offerings to protect internet access for your users. We are very pleased to announce our partnership with ZScaler, iboss, and Checkpoint (coming soon) as the trusted security partners.
Azure Firewall threat intelligence-based filtering now general available
Using threat intelligence-based filtering, Azure firewall can now be configured to alert and deny traffic to and from known malicious IP addresses and domains in near real-time. The IP addresses and domains are sourced from the Microsoft threat intelligence feed.
We also extended our web application firewall (WAF) with three new features, WAF bot protection, WAF per-site policies, and geo filtering. Azure managed bot protection rule set in Azure Front Door detects different categories of bots and allows customers to set actions accordingly. Customers can block malicious bots at the network edge, allowing good bots to reach application backends, and log or redirect unknown bots to an alternative site. Azure managed bot protection rule set is also offered as a preview on Azure Application Gateway v2 SKU. WAF per site policy with Application Gateway enables customers to specify WAF policies for different web applications hosted on a single Application Gateway. This allows for finer grained security policy and eliminates the need to create additional deployments per site. Azure Application Gateway is introducing geo filters with existing custom rules in preview on v2 SKU. This capability allows you to extend existing IP/IP range based custom rules to also include countries as a matching criterion and take actions accordingly. This allows you to restrict traffic from a given country or only allow traffic from a set of countries.
We recently announced the general availability of Azure Bastion. The Azure Bastion service is provisioned directly in your Virtual Network, enabling seamless remote desktop (RDP) and secure shell (SSH) access to all virtual machines in the VNet without needing a public IP address. Seamless integration and easy one-time setup of ACLs across your subnets eliminates subsequent and continuous management.
Figure 7. Azure Bastion architecture showing SSL access to VNet resources through the Azure portal
Today we are also announcing a new feature, the Content Delivery Network Rules Engine, which allows the Azure Content Delivery Network to enable customers to customize how http requests are handled. Rules Engine enables very powerful match conditions like device detection, HTTP protocol, and header values and trigger appropriate actions. All the http rules run at our edge sites near end users which gives significant performance benefits compared to running rules at customer origins.
The Application Gateway Ingress Controller allows Azure Application Gateway to be used as the ingress for an Azure Kubernetes Service (AKS.) The ingress controller runs as a pod within the AKS cluster. It consumes Kubernetes Ingress Resources and converts them to an Azure Application Gateway configuration which allows the gateway to load-balance traffic to Kubernetes pods. Using Application Gateway Ingress Controller enables customers to expose a single internet accessible endpoint to communicate with their AKS clusters. Application Gateway directly interacts with pods using private addresses which eliminates the necessity of additional DNAT incurred by Kube-proxy, thus providing more efficient and performant traffic routing to pods. Application Gateway Ingress Controller provides support for all features of Application Gateway including WAF capabilities to secure access to the AKS cluster.
Figure 8. App Gateway Ingress controller explained relative to AKS
Azure Key Vault is a platform managed service to safeguard cryptographic keys and other secrets used by cloud apps and services. Azure Application Gateway v2 now supports direct integration of Key Vault stored TLS certificates for its HTTPS-enabled listeners. This enables better TLS certificate security by having a clear separation of certificate management process from Application Gateway and backend web application management. Application Gateway polls the Key Vault every few hours for newer version of transport layer security (TLS) certificate, thus enabling automatic renewal of certificates.
Azure Internet Analyzer is a new client-side measurement service now available in preview. Internet Analyzer enables A/B testing of networking infrastructures and their impact on your customers’ performance experience. Whether you’re migrating apps and content from on-premises to Azure or evaluating a new Azure service, Internet Analyzer allows you to learn from your users’ data and Microsoft’s rich analytics to better understand and optimize your network architecture with Azure before you migrate. Internet Analyzer is designed to address performance-related questions for cloud migration, deploying to new or additional Azure regions, or testing new application and content delivery platforms in Azure, such as Azure Front Door and Content Delivery Network.
Azure Monitor for Networks service is now available in preview. Azure Monitor for Networks enables customers to monitor key metrics and health of their network resources, discover issues and get troubleshooting help. Azure Monitor for Networks is on by default and doesn’t require any custom setup. Whether it’s about monitoring and troubleshooting the cloud or hybrid networks, Azure Monitor for Networks helps you to setup alerts, get resource-specific diagnostics, and visualize the structure and functional dependencies between resources.
Figure 9. Screenshot of Azure Monitor for Networks illustrating App Gateway metrics and diagnostics
Multi-access Edge Computing (MEC) in preview
Multi-access Edge Computing offers application developers cloud-computing capabilities at the customer premises. This environment is characterized by very low latency and high bandwidth as well as real-time access to radio networks such as Private LTE and 5G. By integrating MEC capabilities with Azure, we will be offering a continuum of compute and network capabilities from the intelligent cloud to the edge. New critical and immersive scenarios such as smart factory and mixed reality require reliable low-latency and high bandwidth connectivity combined with local compute.
Figure 10. Concept draft of Multi-access and network edge compute with Azure
To address these needs, we are introducing a technology preview of Multi-access Edge Compute based on Azure Stack Edge deployed at the customer’s premises for the best possible latency. Key characteristics of the MEC are:
- Enables developers to use GitHub and Azure dev ops CI/CD toolset to write and run container-based applications at the customer’s premises. With a consistent programming-model it is straightforward to develop applications in Azure and then move them to Azure Stack Edge.
- Wireless technology integration, including Private Long-Term Evolution (LTE), LTE-based Citizens Broadband Radio Service (CBRS), and forthcoming 5G technologies. As part of our MEC platform, we have partnered with technology innovators to provide mobile virtual network functions (Evolved Packet Core), device integration, SIM management, and radio access networks.
- MEC is managed from Azure. Curated virtual network function (VNF) images are downloaded from Azure to simplify deploying and running a private mobile network. The platform also provides support for lifecycle management of the VNFs, such as patching, configuration, and monitoring.
- A partner ecosystem including managed service providers to deploy end to end solutions in your network.
For those interested in the early technical preview and options with MEC integration, please reach out to MEC-Networking@microsoft.com.
Figure 11. Overview of Azure Multi-edge Compute (MEC) partner ecosystem
We are fully committed to helping you connect to Azure, by protecting your workloads, delivering a great networking experience, and providing extensive monitoring to simplify your deployment and operational costs while helping you better support your customers. At Microsoft Ignite we will add more details about our announcements, and you can learn more by viewing our technical sessions. We’ll continue providing innovative networking services and guidance to help you take full advantage of the cloud. We’re excited to learn about your new scenarios enabled by our networking services. As always, we welcome your feedback.
Azure. Invent with purpose.