AI + Machine Learning, Azure Machine Learning
Azure.Source – Volume 61
By Rob Caron Sr. Product Marketing Manager, Microsoft Azure
17 min read
Microsoft Connect(); 2018
On Tuesday, December 4th, Microsoft Connect(); 2018 provided a full day of developer-focused content—including updates on Azure and Visual Studio, keynotes, demos, and real-time coding with experts. Scott Guthrie’s keynote provided all the Azure and Visual Studio news on how you can stay productive and focus on what matters to you.
Guthrie’s keynote was followed by a demo-packed session hosted by Scott Hanselman and friends that provided a full tour on how you can build your apps on Azure using best-in-class developer tools.
The day continued with coding sessions hosted on Twitch and Mixer, where developers joined interactive Q&As to share what they’ve been building with Azure. An additional collection of announcement and programming language-focused videos provided a deeper dive into the topics that were covered. You can access all of this content for on-demand viewing on the Microsoft Connect(); 2018 site.
Azure AI – accelerating the pace of AI adoption for organizations
Eric Boyd, Corporate Vice President, Azure AI, shares several new announcements we made at Microsoft Connect(); to enable organizations to easily apply AI to transform their businesses, which are covered in further detail in the next four blog posts below.
Announcing general availability of Azure Machine Learning service: A look under the hood
Azure Machine Learning service, which contains many advanced capabilities designed to simplify and accelerate the process of building, training, and deploying machine learning models, is now generally available. Automated machine learning enables data scientists of all skill levels to identify suitable algorithms and hyperparameters faster. Support for popular open-source frameworks such as PyTorch, TensorFlow, and scikit-learn allow data scientists to use the tools of their choice. DevOps capabilities for machine learning further improve productivity by enabling experiment tracking and management of models deployed in the cloud and on the edge. All these capabilities can be accessed from any Python environment running anywhere, including data scientists’ workstations.
ONNX Runtime is now open source
The Open Neural Network Exchange (ONNX) Runtime is now open source. ONNX is an open format to represent machine learning models that enables data scientists and developers to use the frameworks and tools that work best for them, including PyTorch, TensorFlow, scikit-learn, and more. ONNX Runtime is the first inference engine that fully supports the ONNX specification and delivers an average of 2x in performance gains. You can integrate ONNX Runtime into your code directly from source or from precompiled binaries, but an easy way to operationalize it is to use Azure Machine Learning to deploy a service for your application to call.
Getting started with Cognitive Services Language Understanding container
Language Understanding is now available as part of the preview of Azure Cognitive Services containers. With container support for Language Understanding, you can easily and quickly add cognitive capabilities — such as object detection, vision recognition, and language understanding — into your apps without having deep data science skills. Cognitive Services containers enable customers to build one application architecture that is optimized to take advantage of both robust cloud capabilities and edge locality.
New automated machine learning capabilities in Azure Machine Learning service
As part of Azure Machine Learning service general availability, we are excited to announce the new automated machine learning (automated ML) capabilities. Automated ML enables you to automate model selection and hyperparameter tuning, reducing the time it takes to build machine learning models from weeks or months to days, freeing up more time for them to focus on business problems. This blog post highlights the new automated ML capabilities of that are available today as part of Azure Machine Learning service and provides a walkthrough of the motivation, underlying technology, and principles behind automated ML.
New Azure IoT offerings simplify IoT application development
In this roll-up post from Sam George Director, Azure Internet of Things, he announces additional capabilities to bring rich data streams into IoT applications from cloud to edge, as well as solutions to accelerate the IoT development process, which are covered in further detail in the next five blog posts below.
Announcing key industrial IoT capabilities in Azure Time Series Insights
Azure Time Series Insights is a fully managed end-to-end IoT analytics solution that empowers customers to quickly glean insights from billions of time series measurements generated by IoT devices. To help maximize the value of this data and thereby drive operational intelligence, Microsoft is updating Time Series Insights offering to support a broad range of industrial IoT data analytics scenarios.
Location intelligence for the enterprise: New pricing tier and SDK updates
Azure Maps is a portfolio of mapping, navigation, and traffic services which uses simple and secure location APIs to add geospatial context to data, and geospatial intelligence to IoT devices. To give customers an enhanced service option, Microsoft is introducing a new S1 pricing tier for production-scale deployments of applications using Azure Maps, to complement the Standard S0 offering.
Azure Stream Analytics on IoT Edge now generally available
Azure Stream Analytics (ASA) bring powerful stream processing capabilities to Azure IoT Edge, enabling developers to process data and respond to conditions locally. ASA on Azure IoT Edge makes it easy to move analytics between edge and cloud, which can be critical in scenarios where customers need low-latency command and control, have limited connectivity to the cloud, have limited bandwidth, or have certain compliance needs.
Azure IoT Remote Monitoring extends operator capabilities to the edge
Continued updates to this solution accelerator make it easier than ever to empower operators managing all facets of their IoT solutions, including managing and deploying Azure IoT Edge components from Azure IoT Remote Monitoring and automatic device management.
Accelerating IoT solution development and testing with Azure IoT Device Simulation
Microsoft is enabling developers to create advanced device simulations to test their IoT solutions in development, simplifying the development of robust and scalable solutions. Watch this episode of The Internet of Things Show, Device Simulation with Azure IoT, to learn about the capabilities of Azure IoT Device Simulation, an open source solution that you can deploy directly to your Azure subscription. And discover how Device simulation helps you build simulated devices that look and behave like the real thing.
Migrate mission critical SQL workloads to the most economical cloud destination
Announces the general availability (GA) of the Business Critical service tier in Azure SQL Database Managed Instance, which is a new deployment option in SQL Database that streamlines the migration of SQL Server workloads from on-premises to the cloud and combines the best of SQL Server with all the benefits of a fully managed database service. In addition, instance-level collations and Azure Active Directory (AAD) logins are now available in preview to simplify SQL Server migrations. We have also extended more of our SQL Database platform features to Managed Instance to ensure a smooth production experience once workloads have been migrated. Transparent Data Encryption (TDE) for Managed Instance now supports Bring Your Own Key (BYOK) scenarios through the integration with Azure Key Vault, Azure’s cloud-based external key management system. Networking security is now significantly strengthened with the introduction of a built-in firewall to provide full inbound protection.
Announcing the general availability of Azure Database for MariaDB
The Azure Database for MariaDB service, which is now generally available, offers an enterprise-ready, fully managed community version of MariaDB with easy lift-and-shift capability. It features open-source compatibility, built-in high availability with a 99.99 percent SLA, elastic scaling for storage and compute resources, and flexible pricing. As a fully integrated service, Azure Database for MariaDB also plugs you into the Microsoft global network of datacenters with unparalleled security and around-the-clock monitoring, including Azure IP Advantage.
Azure Cosmos DB Developer experience updates December 2018
Azure Cosmos DB .NET SDK Version 3.0 now in public preview
Azure Cosmos DB now supports Cross-Origin Resource Sharing (CORS)
Bringing serverless to Azure Kubernetes Service
Sean McKenna Principal Program Manager, Azure Cloud Native Services, outlines a series of new capabilities and investments that further demonstrate our commitment to making Kubernetes easier. In addition, Sean covers the donation of the Virtual Kubelet to the Cloud Native Compute Foundation (CNCF) as a sandbox project. We are also adding GPU support to ACI, which will enable a whole new class of compute-intensive applications through AKS virtual nodes.
Announcing Azure API Management for serverless architectures
Azure API Management is a fully managed service that enables customers to publish, secure, transform, maintain, and monitor APIs. API Management handles all the tasks involved in mediating API calls, including request authentication and authorization, rate limit and quota enforcement, request and response transformation, logging and tracing, and API version management. Azure customers can now choose the new Consumption tier when they are creating a new API Management instance, which is designed and implemented around serverless principles. The API Management consumption tier fills helps round out the Azure serverless offering with Azure API Management for serverless applications.
In this episode of Azure Friday, Azure API Management for serverless applications, Mike Budzynski join Scott Hanselman to explain the serverless properties of the new tier and demonstrate how to build a highly scalable serverless application with Function Apps and expose it with API Management.
Linking your GitHub commits with Azure Boards
Aaron Bjork Principal Group Program Manager, Azure DevOps, explains a new integration between Azure Boards and GitHub. Development teams using GitHub can now take advantage of the rich project management capabilities offered by Azure Boards, including Kanban boards, backlogs, sprint planning tools, queries, and multiple work item types.
New Azure Pipelines announcements – VS Code extension, GitHub Releases, and more
Jeremy Epling Principal Group Program Manager, Azure DevOps covers several new features and integrations with Azure Pipelines – two of which were directly inspired by the open source projects using Azure Pipelines to build and release their applications more easily. The Azure Pipelines extension for Visual Studio Code brings syntax highlighting and autocompletion for Azure Pipelines YAML to Visual Studio Code. You can now seamlessly manage GitHub Releases using Azure Pipelines by creating new releases, modifying drafts, or discarding older ones. We also added support in the Azure portal for Azure IoT Edge in the Azure DevOps project workflow to make it easy to get started in those scenarios. Lastly, the ServiceNow Change Management Extension enables integration of ServiceNow Change Management with Azure Pipelines.
Cincinnati Children’s has a great app development experience with Azure services
Cincinnati Children’s Hospital Medical Center is well known for the excellence of its care and its commitment to the well-being of the families it serves. To reduce strain for parents and make their lives easier, the hospital created the Caren mobile app using Microsoft Azure and Azure services. Parents turn to Caren for help navigating the hospital, to get answers to their questions, and to help them entertain their children. Using the app lets parents focus more on their families and less on stressful details. Learn more about this family-friendly app and how the team built it.
Now in preview
Azure Data Lake Storage Gen2 preview – More features, more performance, better availability
Announced in limited public preview in June, Azure Data Lake Storage Gen2 is now fully open to all customers of Azure in all public and sovereign Azure regions. In addition, in response to customer feedback and requests, several new features are now available (for example, Azure Storage Explorer support and the ability to mount an ADLS Gen2 filesystem into the Databricks File System).
Also available in preview
- API Management bring your own cache is in preview
- Azure Container Instances (ACI) GPU support public preview
- Public preview: Virtual network service endpoints for Azure Database for MariaDB
- Serverless Community Library
- Paginated reports in Power BI is in preview
- API Management flexible subscriptions is in preview
Now generally available
General availability of VNet Service Endpoints for Azure SQL Data Warehouse
Virtual Network (VNet) Service Endpoints for Azure SQL Data Warehouse is now generally available in all Azure regions. Azure SQL Data Warehouse is a fast, flexible, and secure cloud data warehouse tuned for running complex queries fast and across petabytes of data. VNet Service Endpoints enable you to isolate connectivity to your logical server from a given subnet or set of subnets within your virtual network.
Disaster recovery of zone pinned Azure Virtual Machines to another region
Azure Virtual Machines now support disaster recovery of virtual machines deployed in Availability Zones to another region using Azure Site Recovery (ASR). You can now replicate and failover zone pinned virtual machines to other regions within a geographic cluster using Azure Site Recovery. This new capability is generally available in all regions supporting Availability Zones. Along with Availability Sets and Availability Zones, Azure Site Recovery completes the resiliency continuum for applications running on Azure Virtual Machines. Azure Availability Zones are unique fault-isolated physical locations, within an Azure region, with independent power, network, and cooling. Each Availability Zone is comprised of one or more datacenters and houses infrastructure to support highly available, mission critical applications with fault tolerance to datacenter failures.
Also generally available
- Azure Cosmos DB new shared database offer now available
- Custom Translation capability of Text Translator
- Now available: Azure Service Fabric Mesh fall refresh
- Now available: Azure Service Fabric runtime version 6.4 & SDK updates
- S1 Tier Now Available
- General availability: Increased compute in Azure Database for MySQL servers
- General availability: Increased compute in Azure Database for PostgreSQL servers
- Azure SQL Database Managed Instance—Business-critical service tier now available
News and updates
Microsoft Azure portal December 2018 update
This month brings updates to help improve the usability of creating and managing virtual machines in Azure, updates to Security Center, and an improved experience to control access to resources.
SONiC: Global support and updates
Microsoft and the community is developing, refining, and making SONiC (Software for Open Networking in the Cloud), our open switch OS, freely available to anyone running global scale or cloud-type networks or just have a healthy interest in advanced networking. This post takes a look at several enhancements and updates that demonstrate SONiC’s powerful ability to run diagnostics, prevent network failures, and provide fast and flexible telemetry.
Azure SQL Data Warehouse Gen2 now supports lower compute tiers
Now you can experience Azure SQL Data Warehouse’s leading performance, flexibility, and security features starting with 100 cDWU (Data Warehouse Units) and scale to 30,000 cDWU in minutes. Starting mid-December 2018, customers can benefit from Gen2 performance and flexibility with lower compute tiers in 15 regions, with remaining regions available during 2019.
An easy way to bring back your Azure VM with In-Place restore
In-Place restore of disks in IaaS VMs along with simplified restore improvements in Azure Backup is now available. This feature helps roll back or fix corrupted virtual machines through in-place restore without the needs of spinning up a new VM. With the introduction of this feature, customers have multiple choices for IaaS VM restore like create new VM, Restore Disks and Replace disks.
Azure IoT Java SDK provides improved Android support
Transforming mobile devices into Internet of Things (IoT) devices have been gaining traction in the IoT space. Mobile devices have a wide range of sensors and a big screen, but most importantly, they are ubiquitous and they are getting cheaper. After releasing native iOS support in April 2018, we have been improving our support for Android. The Azure IoT Java SDK has improved testing on the Android platform, as well as new samples and quickstarts.
- Deploy Service Fabric Ubuntu clusters on-premise using Bosh
- Azure Container Service Will Retire on January 31, 2020
- Azure Backup simplifies data protection and protects against ransomware.
- Application Insights is available in UK South
Modernize your Java Spring Boot application with Azure Database for MySQL
Learn about a Spring Boot application that demonstrates how to build and deploy a purchase order microservice as a containerized application on Azure Kubernetes Service (AKS). Spring Boot framework with MySQL Database backend is one of the established patterns to meet the online transactional processing needs of business applications. More often, such applications are built and deployed on cloud native microservice platforms, such as Kubernetes. Since databases have more stateful requirements with atomicity, consistency, durability, resiliency, and zero data loss across failures, you should run them outside of Kubernetes environment on managed database services that meet these requirements, such as the Azure Database for MySQL service.
Apply the latest Azure Site Recovery capabilities with automatic agent update
Azure Site Recovery has a monthly release cadence where it makes enhancements to existing features or add new capabilities. When you enable disaster recovery of Azure virtual machines, a component known as mobility service agent gets installed on the VM. Once you enable the automatic agent update, the Mobility Service extension gets updated automatically with every new release. You no longer need to plan for deploying the new versions with every release, and automatic update doesn’t require a reboot of your Azure Virtual Machines, nor does it affect on-going replication.
Get up to speed with Azure HDInsight: The comprehensive guide
Azure HDInsight is an easy, cost-effective, enterprise-grade service for open source analytics. With HDInsight, you get managed clusters for various Apache big data technologies, such as Spark, MapReduce, Kafka, Hive, HBase, Storm and ML Services backed by a 99.9% SLA. In addition, you can take advantage of HDInsight’s rich ISV application ecosystem to tailor the solution for your specific scenario. This guide covers the resources you need to learn about big data technologies and get started with HDInsight.
Best practices for naming your Microsoft Azure resources
Naming standards form one small part of Azure Governance, enabling you to structure and manage your Cloud resources in alignment with your organizational requirements. If set up correctly, resource creation, management, monitoring and cost analysis is made much easier. In this blog post, Sonia Cuff takes you through some of the challenges around naming resources and shares tips for improving their naming strategy.
List of Recommended Resources for Getting Started with Machine Learning on Azure
If you’re curious about using machine learning on Azure but don’t know where to start, this post is for you. Ari Bornstein rounds up a list of invaluable resources—from Azure Free Trial, to a list of docs, open source projects, videos, and tutorials—to help you explore machine learning on Azure.
Episode 257 – Azure Firewall | The Azure Podcast
Yair Tor, a Principal PM in the Azure Networking team, gives us the scoop on the newly released Firewall service. He talks to us about the use-cases where it makes sense to use the service, pricing and future plans.
Azure Container Registry updates for security and reliability | Azure Friday
Curious about what’s coming for container lifecycle management in Azure? To help you plan your roadmap, Steve Lasker and Scott Hanselman talk about the Azure Container Registry (ACR) roadmap, including updates for production security, reliability features, and developer security.
Pros and cons of stable and unique tags in Docker image tagging | Azure Friday
Steve Lasker joins Scott Hanselman to talk about best practices for Docker image tagging, and the tradeoffs between stable and unique tagging techniques.
Azure Time Series Insights – end-to-end solution for industrial IoT analytics | Internet of Things Show
To help maximize the value of time series data and drive operational intelligence, Azure Time Series Insights introduces support for a broad range of industrial IoT analytics capabilities. Check out, Chandrika Shankarnarayan, Principal PM Manager for Azure Time Series Insights, showing the new capabilities in this space.
Customizing the Web UI has never been easier with our recently released in code walkthroughs! Follow along as we customize an out of the box solution and make it our own.
An Introduction to the Intelligent Kiosk | AI Show
How to use this sample application to learn cognitive services and test applications that you are infusing with AI.
How to Build a Home Automation Auto-Away Assist with Azure IoT Hub | Azure Makers Series
Get more out of your home automation setup with Azure IoT Hub and Azure Functions. See how you can let your smart thermostat know when you’re in another room (not truly away) using motion sensors, Particle.io, and Azure.
How to work with Azure Logic App using Visual Studio 2017 | Azure Tips and Tricks
Learn how to work with Azure Logic App using Visual Studio 2017. Inside Visual Studio, you can search for the Logic App Visual Studio extension and create a new Logic App.
Jamie Cool on What’s Going On in Azure DevOps – Episode 013 | Azure DevOps Podcast
Jeffrey Palermo is joined by Jamie Cool to discuss what’s going on in Azure DevOps with a dive deep into what the internal roadmap is looking like for Microsoft’s DevOps transformation and a discussion about some of the big shifts that Jamie is currently working on. Jamie also talks about what is happening around the GitHub acquisition, where he sees DevOps headed in the future, and gives his advice on what you should be keeping an eye out for as a Visual Studio Developer.
IoT in Action: 4 innovations that are revolutionizing IoT
Artificial intelligence and cognitive abilities, real-time analytics at the intelligent edge, more secure IoT devices, and provisioning IoT quickly at scale are the four innovations that are revolutionizing IoT. To learn more about how you can take advantage of these innovations, be sure to register for an IoT in Action Virtual Bootcamp. Whether you are an engineer, software architect, or practice owner, this virtual bootcamp (taking place in early 2019 around the world) will give you a clear understanding of IoT from device to cloud and accelerate the development of an IoT solution for your business.
Join the Twitter AMA with Azure Integration Services
Azure Integration Services hosted a joint Twitter Ask Me Anything (AMA), or actually “Ask Us Anything”, session for API Management, Logic Apps, Service Bus, and Event Grid on Thursday, December 6, 2018. You can find the questions and their replies on Twitter by searching for the hashtag, #IntegrationAMA.
Customers, partners and industries
Johnson Controls tackles a $15b building industry problem with Azure Cosmos DB
Modern buildings include multiple systems that handle everything from building management to HVAC to security. To help building operators gather and understand data about their buildings, operations, and occupants, Johnson Controls created Digital Vault to integrate internal and external data sources and present a harmonized view of energy usage, security breaches, fire alarm status, temperature controls, and other building management systems.
How pharma sales operations benefit from centralizing data and process integration
Pharmaceutical companies need to meet demanding sales goals, manage intricate regulatory compliance, and maintain a competitive hold on the market. However, current sales force automation (SFA) solutions for the life sciences industry are focused primarily on sales reps, which leaves a large capability gap for sales operations departments and inhibits their ability to support the sales process. Prescriber360 is a Microsoft Gold Partner with a comprehensive Pharma SalesOps solution designed specifically for the life sciences industry that can reduce, and even close, the capability gap.
Azure obtains automotive industry’s TISAX compliance
Microsoft data centers and operations centers handling Microsoft Azure, Office 365, and Dynamics 365 have been evaluated by independent auditors as meeting the strong security requirements of the Trusted Information Security Assessment Exchange (TISAX). TISAX is used by European automotive companies to provide a common information security assessment for internal assessments, the evaluation of suppliers, and as an information exchange mechanism. See this post to learn which Azure datacenters were assessed at level 2 and 3 of the standard.
A Cloud Guru’s Azure This Week – 7 December 2018
This time on Azure This Week, Lars talks about new security features for Azure functions and Azure Apps, a new Premium Block Blob Storage, and a look into how the Azure teams stay on top of security vulnerabilities on the entire platform.