Event-driven architectures are increasingly replacing and outpacing less dynamic polling-based systems, bringing the benefits of serverless computing to IoT scenarios, data processing tasks or infrastructure automation jobs. As the natural evolution of microservices, companies all over the world are taking an event-driven approach to create new experiences in existing applications or bring those applications to the cloud, building more powerful and complex scenarios every day.
Today, we’re incredibly excited to announce a series of updates to Event Grid that will power higher performance and more advanced event-driven applications in the cloud:
- Public preview of IoT Hub device telemetry events
- Public preview of Service Bus as an event handler
- Automatic server-side geo-disaster recovery
- General availability of Event Domains, now with up to 100K topics per Domain
- Public preview of 1MB event support
- List search and pagination APIs
- General availability of advanced filters with increased depth of filtering
Expanded integration with the Azure ecosystem
One of the biggest features we have been asked for since launching the Azure IoT Hub integration with Event Grid is device telemetry events. Today, we’re finally enabling that feature in public preview in all public regions except East US, West US, and West Europe. We are excited for you to try this capability and build more streamlined IoT solutions for your business.
Subscribing to device telemetry events allows you to easily integrate data from your devices into your solution more easily, including serverless applications using Azure Functions or Azure Logic Apps, and any other services by using webhooks, whether they are on Azure or not. This helps simplify IoT architectures by eliminating the need of additional services that poll for device telemetry for further processing.
By publishing device telemetry events to Event Grid, IoT Hub expands the services your data can reach, beyond the endpoints supported through message routing. For example, you can automate downstream workflows by creating different subscriptions to device telemetry events for different device types, identified by the device twin tag, and triggering distinct Azure Functions or third party applications for unique computation per device type. Based on your Event Grid subscriptions to device telemetry events, we create a default route in IoT hub, handling all your Event Grid subscriptions to device telemetry.
Learn more about IoT Hub device telemetry in docs, and continue to submit your suggestions through the Azure IoT User Voice forum.
We are also adding Service Bus as an event handler for Event Grid in public preview, so starting today you can route your events in Event Grid directly to Service Bus queues. Service Bus can now act as either an event source or event handler, making for a more robust experience delivering events and messages in distributed enterprise applications. It is currently in public preview and does not work with Service Bus topics and sessions, but it does work with all tiers of Service Bus queues.
This enables command and control scenarios in which you receive events of activity on other services such as blob created, device created, and job finished passing them along for further processing.
Learn more about Service Bus as a destination in docs.
Server-side geo disaster recovery
Event Grid now has built-in automatic geo disaster recovery (GeoDR) of metadata, applicable to all existing Domains, Topics and Event Subscriptions, not just for new ones. This provides a vastly improved resilience against service interruptions, all fully managed by our platform. In the event of an outage that takes out an entire Azure region, the Event Grid service will already have all of your eventing infrastructure metadata synced to a paired region, and your new events will begin to flow again with no intervention from your side required, avoiding service interruption automatically.
Disaster recovery is generally measured with two metrics:
- Recovery Point Objective (RPO): the minutes or hours of data that may be lost.
- Recovery Time Objective (RTO): the minutes of hours the service may be down.
Event Grid’s automatic failover has different RPO’s and RTO’s for your metadata (event subscriptions, plus more) and data (events). If you need different specification from below, you can still always implement your own client-side failover using the topic health APIs.
- Metadata RPO: Zero minutes. You read that right. Any time a resource is created in Event Grid, its instantly replicated across regions. In the event of a failover, no metadata is lost.
- Metadata RTO: Though generally this happens much more quickly, within 60 minutes Event Grid will begin to accept create/update/delete calls for topics and subscriptions.
- Data RPO: If your system is healthy and caught up on existing traffic at the time of regional failover, the RPO for events is about 5 minutes.
- Data RTO: Like metadata, this generally happens much more quickly, however within 60 minutes Event Grid will begin accepting new traffic after a regional failover.
Here’s the best part, there is no cost for metadata GeoDR on Event Grid. It is included on the current price of the service and won’t incur any additional charges.
Powering advanced event-driven workloads
As we see more advanced event-driven architectures for diverse scenarios such as IoT, CRM, or financial we’ve noticed an increasing need on expanding our capabilities for multitenant applications and workloads handling bigger amount of data in their events.
Event Domains give you the power to organize your entire eventing infrastructure under a single construct, set fine grain auth rules on each topic for who can subscribe, and manage all event publishing with a single endpoint. Classic pub-sub architectures are built exclusively on topics and subscriptions, however as you build more advanced and hi-fidelity event-driven architectures, the burden on maintenance increases exponentially. Event Domains take the headache out of it by handling much of the management for you.
Today we’re happy to announce that Event Domains are now generally available, and with that, you’ll be able to have 100,000 topics per Domain. Here’s the full set of Event Domains limits with general availability:
- 100,000 topics per Event Domain
- 100 Event Domains per Azure Subscription
- 500 event subscriptions per topic in an Event Domain
- 50 ‘firehose’ event subscriptions at the Event Domain scope
- 5,000 events/second into an Event Domain
As always, if these limits don’t suit you, feel free to reach out via support ticket or by emailing askgrid@microsoft.com
so we can get you higher capacity.
We also acknowledge that advanced event-driven architectures don’t always fit in the confines of 64 KB. These workloads require handling larger events for a simpler architecture, and today we’re announcing the public preview of events up to 1MB.
There are no configuration changes required and this will work on existing event subscriptions, and everything under 64 KB will be still be covered by our general availability SLA. To try it out, just push larger events, noticing that events over 64 KB will be charged in 64 KB increments, and the batch size limit for events sent to Event Grid as a JSON array is still 1MB in total.
Simplified management of events
You might have thousands of event subscriptions or, with the general availability of Event Domains, hundreds of thousands of topics floating around your Azure subscription. In order to make searching and managing of these resources easier, we’ve introduced list search and list pagination APIs throughout Event Grid. For more information check out all the details in our, “Azure Event Grid Documentation.”
Advanced filters used to route messages in Event Grid are now generally available, with no restriction on the number of nested objects in your JSON. This allows for more granularity when filtering events before passing it to other services for further processing, reducing compute time, and resources needed by avoiding performing this filtering elsewhere.
If you haven’t played with advanced filters yet, you can use the following operators on any part of the event, making the possibilities nearly endless: StringContains, StringBeginsWith, StringEndsWith, StringIn, StringNotIn, NumberGreaterThan, NumberGreaterThanOrEquals, NumberLessThan, NumberLessThanOrEquals, NumberIn, NumberNotIn, BoolEquals.
Get started today
As always, we love to hear your thoughts, feedback, and wish lists as you get a chance to try out these new features! You can start now with the following resources, and please reach out with your feedback.
- Sign up for an Azure free account if you don’t have one yet
- Subscribe to IoT Hub device telemetry events with Event Grid
- Learn more about using Service Bus as an event handler
- Build more powerful multitenant applications with Event Domains
- Perform searches and pagination over thousands and thousands of events with these new APIs
- Route only the necessary events for processing using advanced filters