Azure.Source - Volume 79 | Azure Blog

Move your data from AWS S3 to Azure Storage using AzCopyAzCopy v10 (Preview) now supports Amazon Web Services (AWS) S3 as a data; Announcing general availability of Apache Hadoop 3.0 on Azure HDInsight; Announcing Azure Government Secret private preview and expansion of DoD IL5; Azure Container Registry now supports Singularity Image Format containers; and much more.

Now in preview

Azure Container Registry now supports Singularity Image Format containers

We announced public preview support for storing Singularity Image Files (SIF) in Azure Container Registry based on the OCI Distribution based Container Registries specification. The Singularity project defines a new secure SIF file format which enables untrusted users to run untrusted containers in a trusted way. The work done in collaboration with Sylabs enables customers using Singularity to leverage their investments in Azure Container Registry and other OCI complaint registries, without having to run and maintain another SIF distribution library.

Move your data from AWS S3 to Azure Storage using AzCopy

AzCopy v10 (Preview) now supports Amazon Web Services (AWS) S3 as a data source. Copy an entire AWS S3 bucket, or even multiple buckets, to Azure Blob Storage using AzCopy. Previously, if you wanted to migrate your data from AWS S3 to Azure Blob Storage, you had to bring up a client between the cloud providers to read the data from AWS to then put it in Azure Storage. We addressed this issue in the latest release of AzCopy using a scale out technique thanks to the new Blob API.

Also in preview

Now generally available

Announcing general availability of Apache Hadoop 3.0 on Azure HDInsight

We announced the general availability of Apache Hadoop 3.0 on Azure HDInsight. Microsoft Azure is the first cloud provider to offer customers the benefit of the latest innovations in the most popular open source analytics projects, with unmatched scalability, flexibility, and security. With the general availability of Apache Hadoop 3.0 on Azure HDInsight, we are building upon existing capabilities with a number of key enhancements that further improve performance and security, and deepen support for the rich ecosystem of big data analytics applications.

Manage Azure HDInsight clusters using .NET, Python, or Java

We announced the general availability of the new Azure HDInsight management SDKs for .NET, Python, and Java. Azure HDInsight is an easy, cost-effective, enterprise-grade service for open source analytics that enables customers to easily run popular open source frameworks including Apache Hadoop, Spark, Kafka, and others.

Also generally available

News & updates

Announcing Azure Government Secret private preview and expansion of DoD IL5

We announced a significant milestone in serving our mission customers from cloud to edge with the initial availability of two new Azure Government Secret regions, now in private preview and pending accreditation. In addition, we expanded the scope of all Azure Government regions to enable DoD Impact Level 5 (IL5) data, providing a cost-effective option for L5 workloads with a broad range of available services.

Microsoft open sources Data Accelerator, an easy-to-configure pipeline for streaming at scale

We announced that an internal Microsoft project known as Data Accelerator is now being open sourced. Data Accelerator for Apache Spark simplifies streaming big data using Spark. Data Accelerator has been used for two years within Microsoft for processing streamed data across many internal deployments handling data volumes at Microsoft scale. Offering an easy to use platform to learn and evaluate your streaming needs and requirements, we are excited to share this project with the wider community as open source.

Microsoft driving standards for the token economy with the Token Taxonomy Framework

We announced that the Token Taxonomy Initiative (TTI) is a milestone in the maturity of the blockchain industry, which brings together some of the most important blockchain platforms from the Ethereum ecosystem, Hyperledger and IBM, Intel, R3, and Digital Asset in a joint effort to establish a common taxonomy for tokens.

New Bot Framework v4 Template for QnA Maker

The QnA Maker service lets you easily create and manage a knowledge base from your data, including FAQ pages, support URLs, PDFs, and doc files. You can test and publish your knowledge base and then connect it to a bot using a bot framework sample or template. With this update we have simplified the bot creation process by allowing you to easily create a bot from your knowledge base, without the need for any code or settings changes.

Azure Updates

Learn about important Azure product updates, roadmap, and announcements. Subscribe to notifications to stay informed.

Technical content

Rewrite HTTP headers with Azure Application Gateway

We are pleased to share the capability to rewrite HTTP headers in Azure Application Gateway. With this, you can add, remove, or update HTTP request and response headers while the request and response packets move between the client and backend application. You can also add conditions to ensure that the headers you specify are rewritten only when the conditions are met. The capability also supports several server variables which help store additional information about the requests and responses, thereby enabling you to make powerful rewrite rules.

Machine Learning powered detections with Kusto query language in Azure Sentinel

As cyberattacks become more complex and harder to detect. The traditional correlation rules of a SIEM are not enough, they are lacking the full context of the attack and can only detect attacks that were seen before. This can result in false negatives and gaps in the environment. In addition, correlation rules require significant maintenance and customization since they may provide different results based on the customer environment. Advanced Machine Learning capabilities that are built in into Azure Sentinel can detect indicative behaviors of a threat and helps security analysts to learn the expected behavior in their enterprise. Here you will see three examples.

.NET application migration using Azure App Services and Azure Container Services

Designed for developers and solution architects who need to understand how to move business critical apps to the cloud, this online workshop series gets you hands-on with a proven process for migrating an existing ASP.NET based application to a container-based application. Join us live for 90 minutes on Wednesday and Fridays through May 3 to get expert guidance and to get your questions answered. At the end of this series you will have a good understanding of container concepts, Docker architecture and operations, Azure Container Services, Azure Kubernetes Services and SQL Azure PaaS solutions.

Automated Machine Learning: how do teams work together on an AutoML project?

In this article from Medium, the author shows you an automated machine learning use case (published on GitHub) and, specifically, how a data scientist, a project manager, and a business lead can use automated machine learning to improve team collaboration and learning, and facilitate the successful implementation of data science initiatives.

Uploading your JSON data to Azure Cosmos DB for MongoDB API

If you have built an application and are currently storing the data in a static JSON file, you may want to consider the MongoDB API for Microsoft Azure Cosmos DB. You will have the document data storage you require for your application with the full management of Microsoft Azure with Cosmos DB along with the ability to scale out globally. This will permit you to create replication to regions where your customers are.

Search Like a Boss with Azure Graph Query

Frank Boucher shows how to install the Azure Graph Query extension and explains why you should definitely care about it, and do a few simple queries across multiple Azure subscription.

Securing IoT Data Capture at its Source

What happens when devices only require your organization’s network for connectivity to pass through data or accept commands? Do those attempting to access the IoT devices only access the IoT devices or do they attempt to access other parts of the network now connected to the newly installed IoT device? Enter the new realm of Shadow IT of which “off-the-shelf” IoT devices are being connected to company networks at the request of businesses without understanding the risks or notifying those who govern over the networks themselves, the IT Professional.

How to develop an IoT strategy that yields desired ROI

In an earlier post, we discussed why and how to get started with IoT, recommending that companies shift their mindset, develop a business case, secure ongoing executive sponsorship and budget, and seize the early-mover advantage. This post covers the six elements of crafting an IoT strategy that will yield ongoing ROI.

Azure shows

Episode 275 – Azure Foundations | The Azure Podcast

Derek Martin, a Technology Solutions Principal (TSP) at Microsoft talks about his approach to ensuring that customers get the foundational elements of Azure in place first before deploying anything else. He discusses why Microsoft is getting more opinionated, as a company, when advocating for best practices.

Getting started with Azure App Configuration | Azure Friday

Azure App Configuration is a service that enables you to centralize your application configuration. Built on the simple concept of key-value pairs, this service provides manageability, availability, and ease-of-use. You can use Azure App Configuration to store and retrieve settings for applications, microservices, platforms, and CI/CD pipelines.

Real-time ML Based Anomaly Detection in Azure Stream Analytics | Internet of Things Show

Azure Stream Analytics is a PaaS cloud offering on Microsoft Azure to help customers analyze IoT telemetry data in real-time. Stream Analytics now has embedded ML models for Anomaly Detection, which can be invoked with simple function calls. Learn how you can leverage this powerful feature set for your scenarios.

Using Ethereum Logic Apps to publish ledger data to Azure Search | Block Talk

In this episode we use the Ethereum Logic App connector to push contract data into an Azure Search index. This makes contract data available to a wide range of Enterprise applications via simple search queries.

DevOps for ASP.NET Developers Pt. 1 – What is DevOps? | On .NET

DevOps is the union of people, process, and products to enable continuous delivery of value to our end users. Azure DevOps is everything you need to turn an idea into a working piece of software. In this first episode of the DevOps for ASP.NET Developers series, Abel and Jeremy introduce us the benefits of DevOps.

How to deploy monitored Azure App Services with Azure DevOps | Azure Makers Series

Learn to use Azure DevOps to configure continuous build and release for your web apps. With Application Insights, you'll even be able to monitor everything in real-time—from IDE all the way to production.

How to use Azure Resource Manager | Azure Tips and Tricks

In this edition of Azure Tips and Tricks, learn how to use Azure Resource Manager templates to describe your infrastructure and deploy it.

How to browse your resources in the Azure Portal | Azure Portal Series

The Azure Portal enables you to view and navigate to all your resources more easily. In this video, learn how to go through your Azure resources across locations and subscriptions and customize your views.

How to export your resources to CSV using the Azure Portal | Azure Portal Series

The Azure Portal enables you to customize the information you'd like to export. In this video, learn how easy it is to export your files to CSV.

Udi Dahan on Microservices | Azure DevOps Podcast

This week Udi Dahan, founder of NServiceBus, CEO of Particular Software, and Microsoft Regional Director, joins the Azure DevOps Podcast to discuss microservices and some of the trends, challenges, and problems in the software industry today. Udi gives his advice and recommendations to developers and teams on how to go about making decisions around microservices while giving examples of common mistakes and problems he often sees. He also gives advice on those looking to move forward with an existing legacy system they are trying to modernize as well as those who are looking to build something entirely new.

Episode 7 – “Gaming” March Madness with Azure AI | AzureABILITY Podcast

Doyenne of Data Science Laura Edell visits the pod with new-word-crafter / AI-expert Anthony Franklin to talk about how to use Azure AI to “game” March Madness. During the episode we talk about all sorts of things related to Machine Learning and AI.

Events

Put IoT in action to overcome public building safety challenges

IoT brings transparency to public safety initiatives. Advances in sensors, edge computing, and data analytics give stakeholders a more comprehensive, more immediate view of events as they unfold. Faster, smarter reactions potentially enhance public safety. Given the nature of public safety projects, however, IoT needs collaborative frameworks that provide community members with the network they need to work toward common goals together. This brings challenges that span hardware, software, networks, security, and platform management. Innovative companies are solving these problems, however. Learn how SoloInsight and Microsoft have created secure, manageable and economical IoT solutions for public safety by registering for the IoT in Action webinar, IoT and the New Safety Net. Get insights from industry experts and Microsoft partner SoloInsight around how transparent frameworks create secure buildings.

Customers, partners, and industries

Azure resources to assess risk and compliance

This post walks through some common recommendations for various functions in Financial Services organizations. It is vital for customers in the Financial Services Industry to deliver innovation and value to their customers while adhering to strict security and regulatory requirements. Azure is uniquely positioned to help global FSI customers meet their regulatory requirements and we understand the complexities of trying to innovate fast and effectively, while also ensuring that key regulations and compliance necessities are not overlooked.

Deploying Grafana for production deployments on Azure

Grafana is one of the popular and leading open source tools for visualizing time series metrics. Grafana has quickly become the preferred visualization tool of choice for developers and operations teams for monitoring server and application metrics. Grafana dashboards enable operation teams to quickly monitor and react to performance, availability, and overall health of the service. You can now also use it to monitor Azure services and applications by leveraging the Azure Monitor data source plugin, built by Grafana Labs.

Azure hybrid storage performance & rewrite HTTP headers with Application Gateway | Azure This Week – A Cloud Guru

In this Easter special of Azure This Week, Lars covers hybrid storage performance and a new app service migration assistant. Plus you can now rewrite HTTP headers with Application Gateway.