Announcing the 0.3.0-beta preview of Microsoft Azure WebJobs SDK

We are releasing another preview of the Microsoft Azure WebJobs SDK introduced by Scott Hanselman here. To read more about the previous preview, read this post.

This release has same general feature set as alpha2 and also new features.

Download this release

You can download the WebJobs SDK from the NuGet gallery. You can install or update these packages through NuGet gallery using the NuGet Package Manager Console, like this:

Install-Package Microsoft.Azure.Jobs –Pre

If you want to use Microsoft Azure Service Bus triggers, install the following package:

Install-Package Microsoft.Azure.Jobs.ServiceBus -Pre

Since the package names have changed from alpha2, we have uploaded redirection packages which will help you update from alpha2 – beta.

Update-Package Microsoft.WindowsAzure.Jobs –Pre
Update-Package Microsoft.WindowsAzure.Jobs.Host –Pre

What is the WebJobs SDK?

The WebJobs feature of Microsoft Azure Web Sites provides an easy way for you to run programs such as services or background tasks in a Web Site. You can upload and run an executable file such as an .exe, .cmd, or .bat file to your web site. You can run these as triggered or continuous WebJobs. Without the WebJobs SDK, connecting and running background task requires a lot of complex programming. The SDK provides a framework that lets you write a minimum amount of code to get common tasks done.

The WebJobs SDK has a binding and trigger system which works with Microsoft Azure Storage Blobs, Queues and Tables as well as Service Bus. The binding system makes it easy to write code that reads or writes Microsoft Azure Storage objects. The trigger system calls a function in your code whenever any new data is received in a queue or blob.

Scenarios for the WebJobs SDK

Here are some typical scenarios you can handle more easily with the Azure WebJobs SDK:

• Image processing or other CPU-intensive work.
• Other long-running tasks that you want to run in a background thread, such as sending emails. Until now you couldn’t do this in ASP.NET because IIS would recycle your app if your app was idle for some time. Now with AlwaysOn in Azure Web Sites you can keep the web site from being recycled when the app is idle. AlwaysOn ensures that the site does not go to sleep, which means you can run long-running tasks or services using WebJobs and the WebJobs SDK.
• Queue processing. A common way for a web frontend to communicate with a backend service is to use queues. This is a common producer – consumer pattern.
• RSS aggregation. If you have a site that maintains a list of RSS feeds, you could pull in all of the articles from the feeds in a background process.
• File maintenance, such as aggregating or cleaning up log files.

Goals of the SDK

• Provide a way to make it easier to use Azure Storage when doing any background processing work.
• The SDK makes it easier to consume Azure Storage within your application. You do not have to deal with writing code to read/ write from storage.
• Provider a rich diagnostics and monitoring experience without having the developer write any diagnostics and logging code.

Updates in this preview

Add support for Microsoft Azure Service Bus

The WebJobs SDK now supports Microsoft Azure Service Bus. In the earlier previews you were able to trigger a function on Azure Queues. Now you can use Azure Service Bus Queues and Topics.

For example, the following code shows how you can use WebJobs SDK to trigger a function on a new message on a Service Bus Queue and write the message to a different Service Bus QueueYou can monitor these functions in the dashboard and run them, abort them and get all the dashboard functionality for Service Bus as you had for Azure Queues.

You can specify a connectionString called “AzureJobsServiceBus” which points to your ServiceBus connection. Look at our samples site for more samples on using Service Bus Queues and Topics

class Program
{
    static void Main(string[] args)
    {
        JobHost host = new JobHost();
        host.RunAndBlock();
    }

    public static void TriggerOnSBQueue(
        [ServiceBusTrigger("inputqueue")] string inputText,
        [ServiceBus("outputqueue")] out string outputText)
    {
        outputText = inputText;
    }
}

Made triggers and bindings usage more explicit

The SDK allows you to trigger a function and bind the message to your specified type or to a CLR type such as String, TextReader, TextWriter, and more. In the earlier previews the triggers and bindings usage was specified in the attributes being used on the function parameters. The rules around the attribute usage were not very clear and to avoid confusion we changed the name and usage of the attributes to make triggers and bindings more explicit.

Following are the high-level changes.

• A Triggered QueueInput is now QueueTrigger, and QueueOutput is now Queue

Before

public static void OnQueue(
    [QueueInput("input")] string message, 
    [QueueOutput("output")] out string newMessage)
{     
    newMessage = message;
}

After

public static void OnQueue(
    [QueueTrigger("input")] string message, 
    [Queue("output")] out string newMessage)
{     
    newMessage = message;
}

• A Triggered BlobInput is now BlobTrigger but a BlobInput or BlobOutput is Blob

Before

public static void BlobToBlob(
    [BlobInput("input/{name}")] TextReader input, 
    [BlobOutput("output/{name}")] out string output)
{
    output = input.ReadToEnd();
}

After

public static void BlobToBlob(
    [BlobTrigger("input/{name}")] TextReader input, 
    [Blob("output/{name}")] out string output)
{
    output = input.ReadToEnd();
}

Branding changes

In line with the branding change to Microsoft Azure we are changing the package name, assembly name, and namespaces to match this branding. Following are the changes needed in your app.

Improved function discovery

We added an ITypeLocator and INameResolver to enable customizing how the WebJobs SDK looks for functions. This enables scenarios such as the following:

• You can define functions where the QueueName is not explicit. You can read Queue names from a config source and specify this value at runtime.
• Restrict function discovery to a particular class or assembly.
• Dynamic functions at indexing time: you can define the function signature at runtime.

Access to the dequeue count on Queue message

While the SDK makes it easier to use Azure Queues and covers most of the common usage patterns, there are cases when you want to have access to the low level queues. The SDK now gives you access to some of the commonly used properties of message which may be needed for advanced scenarios. For example, in this sample you can access the dequeueCount of your queue message. Beyond this support the SDK allows you to bind to Azure Storage SDK Queue types.

public static void PropertyBinding(
    [QueueTrigger("inputqueue")] string inputText,
    int dequeueCount)
{
   // Do something with the dequeueCount            
}

Added support for Cancellation Tokens

The functions can take a CancellationToken parameter in your function and the function can receive a cancellation request from the Host.

Added support for WebJobs shutdown notifications

WebJobs added support for graceful shutdown. This allows your WebJob to finish executing the function before the WebJob gets stopped. The SDK supports this graceful shutdown in WebJobs by notifying you when the WebJob is shutting down. This information is flowed to the function using the CancellationToken. The following function takes in a CancellationToken which will receive the Cancellation request when the WebJob is stopping.

public static void UseCancellationToken(
    [QueueTrigger("inputqueue")] string inputText,
    TextWriter log,
    CancellationToken token)
{
      // This is a long running function which can be cancelled  
      while (!token.IsCancellationRequested)
      {
          Thread.Sleep(2000);
          log.WriteLine("Not cancelled");
      }
      log.WriteLine("cancelled");
}

Triggers are now supported on host.Call()

You can now have Triggers on functions which are invoked through host.Call()

class Program
{
    static void Main(string[] args)
    {
        JobHost host = new JobHost();
        host.Call(typeof(Program).GetMethod("TriggerOnQueue"),
            new { inputText="input" });
    }

    public static void TriggerOnQueue(
    [QueueTrigger("inputqueue")] string inputText)
    {
        //Process Queue message
    }
}

JobHost Configuration

If you want to override the default connectionString names for your Azure Storage and Service Bus accounts, you can do it through the JobHostConfiguration.

static void Main(string[] args)
{
    var _storageConn = ConfigurationManager
        .ConnectionStrings["MyStorageConnection"].ConnectionString;

    var _servicesBusConn = ConfigurationManager
        .ConnectionStrings["MyServiceBusConnection"].ConnectionString;

    JobHostConfiguration config = new JobHostConfiguration(_storageConn)
        {
            ServiceBusConnectionString = _servicesBusConn
        };

    JobHost host = new JobHost(config);
    host.RunAndBlock();
}

Updated Azure Storage dependency

The WebJobs SDK will depend on the Azure Storage 4.0.1 package.

Existing features of the SDK

Following is the feature set that was supported in alpha2 and continues to be supported in this release.

Azure Storage

The SDK works with Azure Blobs, Queues and Tables.

Triggers

Functions get executed when a new input is detected on a Queue or a Blob. For example. In the following code ProcessQueue function will be triggered when a new message comes on a queue called “inputqueue”. For more details on triggers please see this post.

public static void ProcessQueue(
    [QueueTrigger("inputqueue")] string input)
{
    // Process the Queue message 
}

Bindings

The SDK supports binding to provides model binding between C# primitive types and Azure storage like Blobs, Tables, and Queues. This makes it easy for a developer to read/ write from Blobs, Tables and Queues as they do not have to learn about the code around reading/ writing from Azure Storage.

The following Bindings are currently supported: Stream, TextReader/Writer, and String. You can add support for binding to your custom types and other types from the Storage SDK as well.

For more details on how Bindings work against Azure Storage, please read Blobs, Queues and Tables

Hosting

A JobHost is an execution container which knows what functions you have in your program. A JobHost object (which lives in Microsoft.Azure.Jobs ) reads the bindings, listens on the triggers, and invokes the functions. In the following example, you create an instance of JobHost and call RunAndBlock(), which will cause the JobHost to listen for any triggers on any functions that you define in this Host.

static void Main(string[] args)
{
    JobHost host = new JobHost();
    host.RunAndBlock();
}

Dashboard for monitoring WebJobs.

As WebJobs (written in any language and of any type) execute, you can monitor them in real time. You can see their state (Running, Stopped, Successfully completed), last run time and the logs of a particular execution. The following screenshot shows you a view of all WebJobs running in your Website.

Function execution details

When you are monitoring a particular execution of this “ImageProcessing” WebJob, you can view invocation details about the functions in the program such as:

• The parameters of this function.
• How much time it took for the function to execute.
• How much time it took to read from a Blob and how many bytes were read/ written.

Following is the code for this ImageProcessing WebJob

public class ImageProcessing
{
    static void Main(string[] args)
    {
        JobHost host = new JobHost();
        host.RunAndBlock();
    }

    public static void Resize(
        [BlobTrigger(@"images-input/{name}")] WebImage input,
        [Blob(@"images2-output/{name}")] out WebImage output)
    {
        var width = 80;
        var height = 80;
        output = input.Resize(width, height);
    }

    public static void WaterMark(
        [BlobTrigger(@"images2-output/{name}")] WebImage input,
        [Blob(@"image2-output/{name}")] out WebImage output)
    {
        output = input.AddTextWatermark("WebJobs", fontSize: 6);
    }
}

public class WebImageBinder : ICloudBlobStreamBinder
{
    public WebImage ReadFromStream(System.IO.Stream input)
    {
        return new WebImage(input);
    }

    public void WriteToStream(WebImage result, System.IO.Stream output)
    {
        var bytes = result.GetBytes();
        output.Write(bytes, 0, bytes.Length);
    }
}

Invoke & Replay

In the above example if the Resize function fails for some reason, you can upload a new image and Replay Resize function, which will trigger the execution chain and call Watermark function as well. This is useful to diagnose and debug an issue when you have a complicated graph for chaining functions together. You can also run a function from the dashboard.

Causality of functions

In the above example, we know that when the Resize function writes to a Blob, it will trigger the WaterMark function. The dashboard will show this causality between functions. If you have chained lots of functions which will get triggered as new inputs are detected then it can be useful to see this causality graph.

Search Blobs

You can click on Search for a Blob and get information on what happened to that Blob. For example, in the case of the ImageProcessing, the Blob was written because the Resize function got executed. For more details on Search Blobs see this post.

Samples

Samples for WebJobs SDK can be found at https://aspnet.codeplex.com/SourceControl/latest#Samples/AzureWebJobs/ReadMe.txt

• You can find samples on how to use triggers and bindings for Blobs, Tables, Queues and Service Bus.
• There is a sample called PhluffyShuffy which is an Image processing Website where a customer can upload pictures which will trigger a function to process those pictures from Blob storage.

Documentation

• Tutorial: Getting Started with the Windows Azure WebJobs SDK
• Channel 9 video on Making Your Jobs Easier With Windows Azure WebJobs SDK
• Introduction to WebJobs and SDK by Scott Hanselman
• A web site monitor built using WebJobs and ASP.NET by Brady Gaster
• Azure WebJobs – Recommended Resources

Deploying WebJobs with SDK

If you don’t want to use the WebJobs portal page to upload your programs, you can use FTP, Git, or Web Deploy. For more information, see How to deploy Azure WebJobs and Git deploying a .NET console app to Azure using WebJobs

If you want to deploy your WebJobs along with your Websites, check out the following Visual Studio extension.

Known Issues when migrating from 0.2.0-alpha2 to 0.3.0-beta

Update namespaces to match new APIs

Update connectionString names

When you are setting the connectionStrings, in the app.config of your WebJob or in “Configure Tab” in your Microsoft Azure Web sites, you will have to change the connectionString names to match the names in 0.3.0-beta

Bindings for Tables

In this release we have dropped support for IDictionary bindings for Azure Table Storage. You can use CloudTableEntity to bind to tables. The following example shows the difference and how you can bind to Azure Tables. For a more complete example please see the table usage on samples site.

Before

public static void CountAndSplitInWords(
    [QueueInput] string textInput, 
    [Table] IDictionary, WordCount> words) 
{ 
}

After

public static void CountAndSplitInWords(
    [QueueTrigger("textInput")] string textInput, 
    [Table("words")] CloudTable wordsTable) 
{ 
}

Logging

If you want to write any logging information then you can use Console.Write(). These logs will show up in the dashboard when you are viewing the WebJob details

If you want to do any function level logging you should use TextWriter which will log any information into Blob storage. These logs will show up in the dashboard when you see the function execution details.

public static void Logging(
    [QueueTrigger("inputqueue")] string inputText,
    TextWriter log)
{
   log.WriteLine(inputText);
}

Dashboard will only work for WebJobs deployed with 0.3.0-beta

If you had a WebJob deployed with 0.2.0-alpha2 of SDK and, if you access the dashboard to see the logs for the WebJob, then you will see a warning about “Host not running”. This happens because as part of this release a newer version of the dashboard gets deployed to all Azure Websites. The new dashboard has some protocol changes which are not compatible with 0.2.0-alpha2. To work around this error, please update your WebJob to use 0.3.0-beta NuGet package and redeploy your WebJob.

Give feedback and get help

The WebJobs feature of Microsoft Azure Web Sites and the Microsoft Azure WebJobs SDK are in preview. Feedback will be considered in changes we make to future versions.

If you have questions that are not directly related to the tutorial, you can post them to the Azure forum, the ASP.NET forum, or StackOverflow.com. Use #AzureWebJobs for Twitter and the tag Azure-WebJobsSDK for StackOverflow.