Questions? Feedback? powered by Olark live chat software
Přeskočit navigaci

Have some Java with your Event Hub

Publikováno dne 3 března, 2016

Architect, Azure Service Bus

Microsoft Azure Event Hubs are the fastest and most cost efficient way to ingest large data volumes into Azure-based real-time analytics, hyper-scale compute and big data solutions. Using the fast and binary ISO/IEC international standard AMQP 1.0 protocol, our Azure customers’ Event Hubs accept nearly two Trillion events per month in production and deliver more than 10 Trillion event reads to downstream applications during that time span.

Solutions realized with Event Hubs span many industries, from automotive, to industrial automation, to finance, to broadcast media, to online gaming. The blockbuster game Halo 5:Guardians runs all telemetry through Event Hubs with sustained peaks over a million events per second, and sentiment analysis run by several national TV networks during the 2016 State of the Union address was also powered by Event Hubs.

Inside Microsoft, Event Hubs has almost completely displaced prior uses of Apache Kafka. On a performance and reliability equivalent setup, Event Hubs is at least 10 times more cost efficient for basic compute and storage costs. Also, because Event Hubs is a pure PaaS service, customers need zero expertise in setting up and running the complex clusters required for this kind of infrastructure. Creating an Event Hub is a simple portal or API gesture.

Today, we’re happy to announce a brand new, open-source Azure Event Hubs client for Oracle’s Java® programming language, and for all other JVM-based languages leveraging Java libraries. The client library is made available under the MIT license.

The library is available in source code form on GitHub. For Maven builds, the binary distribution is available via Maven Central as described in the publisher and consumer documentation in the repository. The binary release will also available in the releases section of the GitHub repository.

Customers building on this new Java® client can obtain full commercial Microsoft Azure product support just like with the other official Azure SDKs and client libraries.

The client interacts with Event Hubs using the secure AMQP 1.0 protocol, and allows for building event producers that submit events into Event Hubs as well as event consumers that receive events from Event Hubs using a straightforward API. The Event Hub client also fully supports Java 8’s asynchronous programming paradigm using Futures.

Publishing events

Publishing events via the new EventHubClient for Java using a connection string obtained from the Azure Portal is very simple and takes just two lines:

EventHubClient ehClient = EventHubClient.createFromConnectionString(connStr).get();

ehClient.send(sendEvent).get();

Consuming events

Consuming events is a bit more complex than sending events because the receivers need to be aware of Event Hub's partitioning model. Consuming events is also somewhat different compared to other messaging infrastructures as Azure Event Hubs puts the consumer in control of how it wants to process events. Consuming from Event Hubs is less like interaction with a queue and more analogous to a tape drive you can wind back to a particular mark then play back to the freshest data available.

The consumer code creates a PartitionReceiver that will receive the data and which is seeded with the offset from where to start reading; in the simplest case that’s the start of the log. Subsequently, the consumer can keep receiving events in a loop.

 

PartitionReceiver receiver = ehClient.createReceiver(
                                EventHubClient.DefaultConsumerGroupName,
                                partitionId, PartitionReceiver.StartOfStream, false).get();

while ( … )
{
   …
   Iterable<EventData> receivedEvents = receiver.receive().get();

   … process this batch …
}
 

An upcoming update for the client, which curious developers can already observe in development in the public open source repository, will also bring the popular and powerful “event processor host” from C# to Java. The event processor host dramatically simplifies writing high scale, high throughput event consumer applications that distribute the processing load over a dynamic cluster of machines.

We hope that this new client library meets your needs. If you run into a problem, or have feature request or suggestions,  please don’t hesitate to file an issue on GitHub.