As announced at BUILD this week, the Service Bus September 2011 Release is now available. This is the biggest feature update in the production environment since the service launched in January 2010.

The Service Bus provides secure connectivity and messaging capabilities that enable building distributed and loosely coupled applications in the cloud, as well as hybrid applications across both on-premises and the cloud. It enables a variety of communication and messaging protocols and patterns, and eliminates the need for the developer to worry about delivery assurance, reliable messaging and scale. You can learn more about the Service Bus here.

This release introduces enhancements to the Service Bus that improve pub/sub messaging by supporting features such as Queues, Topics and Subscriptions, and Rules. This release also enables new scenarios on the Windows Azure platform, such as:

• Asynchronous Cloud Eventing – Distribute event notifications to occasionally connected clients (for example, phones, remote workers, kiosks, and so on)
• Event-driven Service Oriented Architecture (SOA) – Building loosely coupled systems that can easily evolve over time
• Advanced Intra-App Messaging – Load leveling and load balancing for building highly scalable and resilient applications

The new messaging features that materialize in Queues and Topics were first made available in the May 2011 CTP as a service preview, and are now in the Service Bus production environment.  Several of the detailed capabilities of the new messaging features, like the unique support for sessions along with a facility to track session processing state, were directly informed by the needs of customer projects that we’ve followed in early adopter programs and other Microsoft development efforts that are taking a long-term architectural bet on Service Bus.

What Changed

What developers will notice right away after looking at the documentation and exploring the samples is that the API for the new messaging capabilities has changed quite a bit since the May 2011 CTP release –in response to direct customer feedback. One key goal of the API changes was to streamline the API and reduce the number of lines of code that are needed to use the new Service Bus features; I’ll give a few examples below.

Another goal was to make the runtime pieces of the API much more robust – example: developers had to handle exceptions due to torn network connections explicitly in the CTP and take explicit steps to replace ‘faulted’ receiver or sender objects; in this production release the messaging client API will automatically try to reconnect just like the Relay listeners of Service Bus and client objects won‘t go into a ‘faulted’ state requiring explicit recovery by the application.

What Didn’t Change

What developers will not see is a discontinuity in service or a disruptive change to Service Bus behavior in production. Even though we’ve proverbially pulled the tablecloth out from under everyone and put a new one on – with a very much expanded feature set – none of the crystal, china, or silverware on the table moved. Existing apps using Service Bus still run and the Microsoft.ServiceBus.dll assembly 1.0.x.x from the previous production SDK ‘just works’ – there is no work required to adapt existing apps to the new release.

How to Use the New Features

Taking advantage of all of the new Service Bus capabilities from .NET does, however, require using the new SDK with the new Microsoft.ServiceBus.dll version 1.5.x.x that contains the new Messaging API. Our recommendation is that even applications that are only using Relay features be recompiled and tested against this newest assembly and that customer deployments of the 1.0.x.x assemblies are starting phased out as part of regular upgrade and deployment cycles.

Keep in mind that the new 1.5.x.x assembly is only available for the full .NET 4.0 framework – for access to the new messaging capabilities from Silverlight, the new Windows 8 client profile, the .NET 4.0 client profile, or from applications requiring .NET 3.5, developers can leverage the client samples provided for Silverlight to gain access to the vast majority of features through the Service Bus REST API. These Silverlight samples as well as code for accessing Service Bus from Java, PHP, and other platforms we will be made available over the course of the upcoming weeks –  each of them will use the appropriate native runtime APIs, but echo the .NET API in terms of terminology and patterns.

Using the REST API is also a good choice, for this release, if applications depending on Service Bus Queues and Topics need access from tightly managed networking environments where outbound HTTPS access is possible, but TCP 9354 cannot be made available for outbound traffic.

The grand upside of using the new .NET API directly is that the TCP protocol used by the new Microsoft.ServiceBus.dll client bits is vastly more efficient than HTTP. The TCP protocol is also, for now, a prerequisite for several advanced features like sessions and transaction support.  

New Opportunities for Existing Applications

And even though we’re committed to backwards compatibility, there are a few things developers with existing Service Bus apps may want to consider looking at over the next few months. We will provide more concrete guidance in coming posts, but for now, here are a few high-level examples:

• For the vast majority of use cases, the new Queue capability is a better choice than the Message Buffer. There are some special use-cases where the Message Buffer may be preferred, such as where the application relies on the Message Buffer being ephemeral (it automatically expires and vanishes) or where the application relies on the Message Buffer’s overflow policy to exert backpressure on the sender – in most other cases, moving to the Queue is a splendid idea. 
• The NetEventRelayBinding provides relayed multicast one-way messaging – albeit within rather narrow limits of up to 25 concurrent listeners. For many cases, it’s well worth looking into replacing these paths with a Topic where every destination has its own Subscription and that receives messages using the new NetMessagingBinding. That provides more scale (up to 2000 concurrent subscriptions) and the subscriptions can also be filtered. The remaining differentation for NetEventRelayBinding is that it does, as all direct TCP-based connectivity mechanisms do, exert backpressure into the client if the listeners have a slower receive rate than the sender’s theoretically possible send rate.   

API Changes Compared to the May CTP

Based on customer feedback and our own experiences in building apps with the API we’ve presented back in May, we’ve made a significant number of API improvements, some of which also benefit the existing Service Bus Relay capabilities – while staying compatible with existing code.

One of the most apparent changes is around security where we’ve introduced the notion of ‘token providers’ instead of feeding credentials directly into the API. Using the credential classes and setting credentials on the TransportClientEndpointBehavior is still possible for the Relay, but is now labeled as deprecated.  Service Bus uses federated security with access tokens issued by the Windows Azure AppFabric Access Control service (ACS). The API factoring now reflects that federated nature, with token providers being independent entities that acquire and dispense tokens into the Service Bus client infrastructure as needed. The upside is that this new factoring allows customer code to plug new token providers into the API that acquire credentials and interact with the Access Control service in special ways, possibly popping a dialog window that hosts a web-browser control asking for a Facebook login, passing the resulting Facebook token to ACS, and then returning the Service Bus access token to the application.

The management API surface also got a bit of a makeover. ServiceBusNamespaceClient is now called NamespaceManager and there are now – of course – methods that allow checking whether a particular Queue, Topic, or Subscription already exists. The Namespace Manager also now allows direct management of Subscription rules. Creating rules on Subscriptions is done using filters and the respective classes have also been slightly reorganized, e.g. SqlFilterExpression is now just SqlFilter.

The most significant changes – and, as we hope, improvements – were done in the core Messaging API where we’ve changed a number of things ‘under the hood’, allowing us to shed quite a bit of complexity in the API surface.

What you will find browsing through the new samples is that the MessagingFactory and all objects dispensed by it now have a much simpler state management model. You no longer need to ‘Open’ the objects for use and they also no longer go into a faulted state. As much as those changes may seem like minor details, they are the direct result of improvements in the underlying binary protocol that now allows connection sharing across multiple senders and receivers, message prefetching, and – probably most importantly – automatically reconnects sessions when connections get lost as mentioned earlier. In other words, an application can now have a pending ‘Receive’ request, get disconnected or even be hibernated along with the OS, and will reconnect and automatically retry getting the message once the network is available again.

NetMessagingBinding is the new name for the binding for Queues and Topics that is providing full integration with WCF, and is functionally similar to its cousin NetMsmqBinding. On the service-side, the NetMessagingBinding provides an automatic message-pump that pulls messages off a Queue or Subscription and it’s integrated with WCF’s ReceiveContext mechanism.

Where? How?

As mentioned above – the new release is available right now and you can instantly use all of these features. The required client assemblies and samples are available for download here. You can also easily install the runtime assemblies through NuGet or the Web Platform Installer as part of the Windows Azure SDK and start building applications.

Please note that the September 2011 release is only available in the regular production environment for Service Bus and not in the CTP environment (“appfabriclabs.com”) that you may have been using before. Applications that run against the CTP environment should be migrated to the regular Service Bus service no later than October 31, 2011 as the environment may change, unannounced, after that date.

Here is a list of resources for learning more about this release:

• Service Bus Overview
• Video: An Introduction to Service Bus
• Video: An Introduction to Service Bus Relay
• Video: An Introduction to Service Bus Queue
• Video: An Introduction to Service Bus Topics
• Video & Code Sample: How to use Service Bus Relay
• Video & Code Sample: How to use Service Bus Queues
• Video & Code Sample: How to use Service Bus Topics
• Blog post: An Introduction to Service Bus Topics
• Blog post: An Introduction to Service Bus Queues
• Blog post: Using the Service Bus via REST / HTTP
• Service Bus Detailed FAQs
• Updated SDK
• Documentation on MSDN

To read more about all of the Windows Azure-related announcements made at BUILD, please read the blog post, “JUST ANNOUNCED @ BUILD: New Windows Azure Toolkit for Windows 8, Windows Azure SDK 1.5, Geo-Replication for Windows Azure Storage, and More”.  For more information about BUILD or to watch the keynotes, please visit the BUILD Virtual Press Room.  And follow @WindowsAzure and @STBNewsBytes for the latest news and real-time talk about BUILD. 

Clemens Vasters is the Principal Technical Lead on Service Bus. Follow Clemens @clemensv.