Azure.Source – Volume 48

Now in preview

Avere vFXT for Microsoft Azure now in public preview – The Avere vFXT for Azure is a high-performance cache for HPC workloads in the cloud that provides low-latency access to Azure compute from Azure Blob and file-based storage locations. The Avere vFXT for Azure has no charge associated with licensing; however, costs associated with consumption do apply at normal rates. Avere Systems joined Microsoft earlier this year.

Powerful Debugging Tools for Spark for Azure HDInsight – Microsoft runs one of the largest big data cluster in the world – internally called “Cosmos”, which runs millions of jobs across hundreds of thousands of servers over multiple exabytes of data. Being able to run and manage jobs of this scale by developers was a huge challenge. We built powerful tools that graphically show the entire job graph, which helped developers greatly and we want to bring that power to all HDInsight Spark developers. The default Spark history server user experience is now enhanced in HDInsight with rich information on your spark jobs with powerful interactive visualization of Job Graphs & Data Flows. The new features greatly assist Spark developers in job data management, data sampling, job monitoring and job diagnosis.

Also in preview

• Public preview: Data Box Disk now available in Australia and Canada
• Azure Data Box: Expanded availability, plus support for NFS and RBAC
• Migration support for PostgreSQL to Azure DB for PostgreSQL
• Azure Database Migration Service: PostgreSQL to Azure DB for PostgreSQL migrations support

Azure Friday

Now generally available

• General availability: Virtual network service endpoints for Azure Key Vault
• General availability: Event Hubs for Python
• Announcing general availability of Azure IoT Hub integration with Azure Event Grid
• BulkExecutor for Azure Cosmos DB Gremlin API now available

The Azure Podcast

News and updates

Exciting new capabilities on Azure HDInsight – Ashish Thapliyal Principal, Program Manager, Azure HDInsight provides a roll-up of the various updates for Azure HDInsight from the past few months, including Apache Phoenix and Zeppelin integration, Oozie support in HDInsight enterprise security package, Azure Data Lake Storage Gen2 integration, Virtual Network Service Endpoints support, support for Spark 2.3, and more.

Additional news and updates

• Azure Security Center update August 29
• Filter VM sizes by current or previous generation
• SQL Data Warehouse now available in Government Cloud
• Azure SQL Data Warehouse enables streaming solutions with Azure Databricks

The IoT Show

Technical content and training

Transparent data encryption or always encrypted? – Transparent Data Encryption (TDE) and Always Encrypted are two different encryption technologies offered by SQL Server and Azure SQL Database. Generally, encryption protects data from unauthorized access in different scenarios. They are complementary features, and this blog post will show a side-by-side comparison to help decide which technology to choose and how to combine them to provide a layered security approach.

Connected arms: A closer look at the winning idea from Imagine Cup 2018 – Joseph Sirosh, Corporate Vice President, Artificial Intelligence & Research, takes a closer look at the champion project of Imagine Cup 2018, smartARM. smartARM is a low-cost, low-function and high-cost, higher-function below-elbow prostheses through its use of low-cost materials and AI. By combining low-cost materials with the cloud and AI, Samin Khan and Hamayal Choudhry present smartARM as an affordable yet personalized alternative to state-of-the-art advanced prosthetic arms.

Additional technical content

• How to use Batch AI to process multiple forecasts
• Accelerate State of DevOps Report 2018 Highlights
• Use Azure managed identities with Azure Kubernetes Services (AKS)
• A Beginner’s Guide to Cloud Gaming
• When is a vulnerability not a vulnerability?

Azure tips & tricks

Events

Join us for two days of live streaming activity that will help you learn all the details of what was announced for Azure DevOps set of services for developers, Azure Pipelines CI/CD platform and new GitHub extensions from Microsoft:

Azure DevOps Keynote – Keynote: Watch our live Azure DevOps keynote on September 11, 2018 from 8:00 – 9:30 AM Pacific Time (GMT -7).

Azure DevOps Workshop – Join our live Mixer workshop with interactive Q&A on September 17, 2018 from 8:30 AM – 2:30 PM Pacific Time (GMT -7).

Tuesdays with Corey

Industries

Delivering innovation in retail with the flexible and productive Microsoft AI platform – Learn how the application of AI is transforming the retail and consumer goods industry resulting in positive business improvements aimed at solving a range of service to production-type problems. Whether you want to use pre-built AI APIs or develop custom models with your data, Microsoft's AI platform offers flexibility using the infrastructure and tools available in the platform. Determine the best path forward by understanding each approach.

Save money on actuarial compute by retiring your on-premises HPC grids – Every insurance company sees the increasing demand from growth in the number of policies processed, and new regulations that require changes to the actuarial and accounting systems. FRS-17 requires changes to workflows, reporting and control throughout the actuarial and accounting process. Read this post to learn why now is the time to move to a cloud-based solution on Azure, and download the Actuarial risk compute and modeling overview to understand your options for moving your compute workload to Azure.

Finish your insurance actuarial modeling in hours, not days – Actuarial departments everywhere work to make sure that their financial and other models produce results which can be used to evaluate their business for regulatory and internal needs. Today, it is common for quarterly reporting to require thousands of hours of compute time. Most actuarial packages on the market support in-house grids with the ability to either extend the grid to Azure, or to use Azure as the primary environment. Learn how actuarial teams review their models and processes to maximize the scale benefits of the cloud.

Anti-money laundering – Microsoft Azure helping banks reduce false positives – Anti-Money Laundering (AML) Transactions Monitoring Systems (TMS) identify suspicious transactions that may involve illicit proceeds or legitimate proceeds used for illegal purposes. AML costs across the banking industry due to this regulatory blunt force approach, has resulted in massive inefficiency and high operating costs for banks. The predominant TMS technologies are antiquated batch rule-based systems that have not fundamentally changed since the late ‘90s, and have not kept pace with increasing regulatory expectations and continually evolving money laundering techniques. Learn how the application of ML and AI can quickly flag changes in patterns of activity, whether caused by new product offerings or money laundering.

Describe, diagnose, and predict with IoT Analytics – The whole point of collecting IoT data is to extract actionable insights. Insights that will trigger some sort of action that will result in some business value such as optimized factory operations, improved product quality, better understanding of customer demand, new sources of revenue, and improved customer experience. This post discusses the extraction of value out of IoT data by focusing on the analytics part of the story, the four types of analytics and how they map to IoT, the role of machine learning in IoT analytics, and an overview of a solution architecture that highlights the components of an IoT solution with analytics.

Current use cases for machine learning in retail and consumer goods – Retail and consumer goods companies are seeing the applicability of machine learning (ML) to drive improvements in customer service and operational efficiency. For example, the Azure cloud is helping retail and consumer brands improve the shopping experience by ensuring shelves are stocked and product is always available when, where and how the consumer wants to shop. Check out 8 ML use cases to improve service and provide benefits of optimization, automation and scale. In addition, this post has links to a lot of other resources you should check out, such as an ML algorithm cheat sheet.

Where is the market opportunity for developers of IoT solutions? – There is a clear opportunity for SIs with practices around IoT and data analytics to develop IoT solutions customized to their customers. The IoT space is a complex, fragmented, and crowded ecosystem. As such, it is hard for software developers to identify where exactly the opportunity is. This post discusses the biggest market opportunities for developers of IoT solutions. There is also an opportunity for ISVs is in the development of IoT platforms and pre-built IoT apps to accelerate the development of IoT solutions.

Reduce costs by optimizing actuarial risk compute on Azure – Actuarial risk modeling is a compute-intensive operation. Employing thousands of server cores, with many uneven workloads such as monthly and quarterly valuation and production runs to meet regulatory requirements, these compute-intensive workloads, are well suited to take advantage of this near bottomless resources of the cloud. Learn how Azure enables you to access more compute power when needed, without having to manage it. And read the actuarial risk compute and modeling overview to understand your options for moving your compute workload to Azure.

A Cloud Guru's Azure This Week