Shubha Vijayasarathy joins Scott Hanselman to discuss Azure Event Hubs, which makes data ingestion simple, secure, and scalable. As a distributed streaming platform, Event Hubs enables you to stream your data from any source—storing and processing millions of events per second— so you can build dynamic data pipelines and respond to business challenges in real time.With Azure Event Hubs for Apache Kafka, we're bringing together two powerful distributed streaming platforms, so you can access the breadth of Kafka ecosystem applications without having to manage servers or networks. Event Hubs for Kafka provides a Kafka endpoint so that any Kafka client running Kafka 1.0 or newer protocols can publish/subscribe events to/from Event Hubs with a simple configuration change.Azure Event Hubs docsAzure Event Hubs samples (GitHub)Azure Event Hubs overviewAzure Event Hubs pricingAnnouncing the general availability of Azure Event Hubs for Apache Kafka® (blog post)Create a free account (Azure)
In this session, hear lessons learned while implementing a scalable rules engine with SQL Server, Azure Event Hub, and Azure Functions. Learn how to secure your application. Understand scaling and idempotency, and monitoring a PAAS using Application Insights. Areas covered include: Event Hubs, Azure SQL Database, Azure Functions, Application Insights, Visual Studio Team Services.
Bringing data to life in a timely manner is every architect's dream. With Azure Databricks Delta, and Spark Streaming, this dream is closer to reality than ever before. Join this session to see how you can create simple pipelines that allow bringing together real-time data and merge with massive batch datasets, with the objective of leveraging the best of both worlds, but with minimal friction. With data driving automated decision-making processes infused into intelligent applications, this session will enable you to develop intelligence integration directly against your in-flight data. We will also introduce Delta Tables on Azure Databricks which help optimize performance and reduce costs for stream analytics pipelines in Azure Databricks.
This session is co-delivered with Ms. Chetna Gupta, Sr. Director of Engineering at Evolve Controls. Continuous streams of data are generated in every industry from sources such as clickstream logs, IoT devices and sensors, business transactions, social feeds, fleet vehicles etc. Within these fast-moving data streams are valuable business insights waiting to be unlocked. Learn how customers are building real-time solutions for building controls, remote monitoring of assets, fleet management, smart grid and network monitoring. See demos and learn how services like Azure Event Hubs, Stream Analytics, Machine Learning and other Azure services work seamlessly together to create your end-to-end real time analytics solutions.
Before you can have Big Data, you must collect the data. There are two popular ways to do this: with batches and with live streams. Apache Kafka has changed the way we look at streaming and logging data and now Azure provides tools and services for streaming data into your Big Data pipeline in Azure. This session will outline the different services in the Big Data Streaming ecosystem in Azure, how they work together, and when to use which including HDInsight Kafka and Event Hubs. We will also talk briefly about when using traditional ETL tools is a better idea.
AI/ML expert Paige Bailey takes you on a tour of the powerful services available on Azure. You'll see how to take your predictive model to production, dynamically train it online with streaming updates, and add realtime data to your models from IoT sources. Paige covers Azure DataBricks, Batch AI, KubeFlow + AKS, Stream Analytics, and Event Hubs in a session you won't want to miss.
Azure Friday visited various Microsoft booths in the Expo Hall at Build 2018 to learn what's new. In this episode, Lara Rubbelke gets an update on Big Data and Analytics from Shubha Vijayasarathy.For more information, see:Big data and analyticsAzure Event HubsText Classification with SQL ServerCreate a free account (Azure)
The Software Engineers in Microsoft's Devices Supply Chain Engineering have replaced their on-premise B2B integration platform with multiple Azure components all working together to create a fully cloud native scalable messaging infrastructure. In this session, you'll learn how the engineers created an advanced event-driven b2b platform with rich telemetry using multiple Azure services to modernize how messages are passed between parties both internal and external to Microsoft. You'll also learn some lessons from the team on building a complex application across multiple Azure services - API Management, Functions, API Apps, Service Bus, Event Hubs, Key Vault, Application Insights, Log Analytics and Azure Active Directory.
This session will outline the different services in the big data streaming ecosystem in Azure, how they work together, and when to use which, including HDInsight Kafka and Event Hubs.