Announcing preview of WebJobs SDK

Today, we are releasing an update to Windows Azure WebJobs SDK which Scott Hanselman introduced.

Download this release

You can download WebJobs SDK from the NuGet gallery. In a Console Application, you can install or update to these packages through NuGet gallery using the NuGet Package Manager Console, like this:

Install-Package Microsoft.WindowsAzure.Jobs -Pre 
Install-Package Microsoft.WindowsAzure.Jobs.Host -Pre

Scenarios for WebJobs SDK

Here are some typical scenarios you can handle more easily with the Windows Azure WebJobs SDK:

    • Image processing or other CPU-intensive work. A common feature of web sites is the ability to upload images or videos. Often you want to manipulate the content after it’s uploaded
    • Queue processing.[VH1]  A common way for a web frontend to communicate with a backend service is to use queues. When the web site needs to get work done, it pushes a message onto a queue. A backend service pulls messages from the queue and does the work.
    • RSS aggregation. If you have a site that maintains a list of RSS feeds, you could pull in all of the articles from the feeds in a background process.
    • File maintenance, such as aggregating or cleaning up log files.  You might have log files being created by several sites or for separate time spans which you want to combine in order to run analysis jobs on them. Or you might want to schedule a task to run weekly to clean up old log files.
    • Other long-running tasks that you want to run in a background thread, such as sending emails. Until now you couldn’t do this in ASP.NET because IIS would recycle your app if your app was idle for some time. Now with AlwaysOn in Windows Azure Web Sites you can keep the web site from being recycled when the app is idle.AlwaysOn ensures that the site does not go to sleep, which means you can run long-running tasks or services using WebJobs and the WebJobs SDK.

What is WebJobs SDK?

The WebJobs feature of Windows Azure Web Sites provides an easy way for you to run programs such as services or background tasks in a Web Site. You can upload and run an executable file such as an .exe, .cmd, or .bat file to your web site. However, most useful work that you might want to do requires a lot of complex programming. That’s where the Windows Azure WebJobs SDK comes in:  it provides a framework that lets you write a minimum amount of code to get common tasks done.

The WebJobs SDK has a binding and trigger system which works with Windows Azure Storage  Blobs, Queues and Tables. The binding system makes it easy to write code that reads or writes Windows Azure Storage objects. The trigger system calls a function in your code whenever any new data is received in a queue or blob.

The WebJobs SDK includes the following components:

    • NuGet packages. The NuGet packages enable your code to use the WebJobs SDK binding and trigger system with Windows Azure Storage tables, blobs and queues.
    • Dashboard. Part of the WebJobs SDK is already installed in Windows Azure Web Sites and provides rich monitoring and diagnostics for the programs that you write by using the NuGet packages. You don’t have to write code to use these monitoring and diagnostics features.

Goals of the SDK?

    • Provide a simple way to make it easier to use Windows Azure Storage when doing any background processing work.
      • The SDK makes it easier to consume Azure Storage within your application. You do not have to deal with writing code to read/ write from storage.
    • Provider a rich diagnostics and monitoring experience without having the user write any diagnostics and logging code.

Features of the SDK?

Azure Storage

The SDK works with Azure Blobs, Queues and Tables.

Triggers

Functions get executed when a new input is detected on a Queue or a Blob. For eg. in the following snippet ProcessQueue function will be triggered when a new message comes on a queue called “longqueue”. For more details on triggers please seehttp://blogs.msdn.com/b/jmstall/archive/2014/01/28/trigger-bindings-and-route-parameters-in-azurejobs.aspx

Bindings

The SDK supports binding to provides model binding between C# BCL types and Azure storage like Blobs, Tables, and Queues. This makes it easy for a developer to read/ write from Blobs, Tables and Queues as they do not have to learn about the mundane code around reading/ writing from Azure Storage.

  1. Convenience. You can pick the type that’s most useful for you to consume and the WebJobs SDK will take care of the glue code. If you’re doing string operations on a blob, you can bind directly to TextReader/TextWriter, rather than worr about how to convert to a TextWriter.
  2. Flushing and Closing: The WebJobs SDK will automatically flush and close outstanding outputs.
  3. Unit testability. The SDK makes it possible to unit test your code since you can mock BCL types like TextWriter rather than ICloudBlob.
  4. Diagnostics.  model binding works with the dashboard to give you real time diagnostics on your parameter usage.

Following are the Bindings currently supported: Stream, TextReader/Writer, String

For more details on how Bindings work against Azure Storage, please read Blobs, Queues andTables

Hosting

A Host is an execution container which knows what all functions do you have in your program. A JobHost object (which lives in Microsoft.WindowsAzure.Jobs.Host ) reads the bindings, listens on the triggers, and invokes the functions.

Dashboard for monitoring WebJobs.

As WebJobs (written in any language and of any type) execute, you can monitor them in real time. You can see whether they are running or not and also see the logs of a particular execution of a WebJob.

If you have written a WebJob using the SDK, then you get diagnostics and monitoring experience for the functions in your program. For eg. Let’s say that you have an Image processing WebJob called “ImageResizeAndWaterMark” that has the following flow. When a user uploads an image to a Blob to “images1-input” container, it will trigger WaterMark function. Watermark will process the image and write to images2-input conainer which will trigger the Resize function which will resize the image and write it to “images2-output” Blob container. Following image shows the code for this WebJob.

When you run the WebJob in Azure, you can view the WebJobs Dashboard by clicking the logs link of the “ImageResizeAndWaterMark” in the WEBJOBS tab of Windows Azure Websites portal. Since the Dashboard is a SiteExtension you can access it by going to the url: https://mysite.scm.azurewebsites.net/azurejobs.  You will need your deployment credentials to access the SiteExtension. For more information on accessing Site Extension, see the documentation on the Kudu project https://github.com/projectkudu/kudu/wiki/Accessing-the-kudu-service

Function execution details

When you are monitoring a particular execution of this “ImageResizeAndWaterMark” WebJob, you can view details about the function such as:

    • What are the parameters of this function?
    • How much time did it take for the function to execute.
    • How much time did it take to read from Blob and how many bytes were read/ written.

Invoke & Replay

In the above example if WaterMark function fails for some reason, then you can upload a new image can Replay WaterMark which will trigger the execution chain and call Resize as well. This is useful to diagnose and debug an issue when you have a complicated graph for chaining functions together. You can also Invoke a function from the dashboard.

Causality of functions

In the above example, we know that when WaterMark function writes to a Blob, then it will trigger Resize function. The dashboard will show this causality between functions. You can imagine if you have chained lots of functions which will get triggered as new inputs are detected then it is useful to see this causality graph.

Search Blobs

You can click on Search for a Blob and get information on what happened to that Blob. eg.[VH1]  In the case of the ImageResizeAndWaterMark, the Blob was written because WaterMark function got executed. For more details on Search Blobs please seehttp://blogs.msdn.com/b/jmstall/archive/2014/02/19/who-wrote-that-blob.aspx

Samples

We have all our samples for WebJobs SDKhttps://aspnet.codeplex.com/SourceControl/latest#Samples/AzureWebJobs/ReadMe.txt

    • You can find samples on how to use triggers and bindings for Blobs, Tables and Queues.
    • There is a sample called PhluffySuffy which is an Image processing Website where a customer can upload pictures which will trigger a function to process those pictures from Blob storage.

Documentation

Deploying WebJobs with SDK

You can use deploy WebJobs using Kudu or Web deploy. If you want to deploy your WebJob along with your Website please check out the following prototype.

Known Issues from 0.1.0-alpha1 to 0.2.0-alpha2

Dashboard will only work for WebJobs deployed with 0.2.0-alpha2

If you had a WebJob deployed with 0.1.0-alpha1 of SDK and, if you access the dashboard to see the logs for the WebJob, then you will see a warning about “Host not running”.This happens because as part of this release a newer version of the dashboard gets deployed to all Azure Websites. The new dashboard has some protocol changes which are not compatible with 0.1.0-alpha1. To work around this error, please update your WebJob to use 0.2.0-alpha2 NuGet package and redeploy your WebJob.

Give feedback and get help

The WebJobs feature of Windows Azure Web Sites and the Windows Azure WebJobs SDK are in preview and not formally supported. Feedback will be considered in changes we make to future versions.

If you have questions that are not directly related to the tutorial, you can post them to the Windows Azure forum, the ASP.NET forum, or StackOverflow.com. Use #AzureWebJobs for Twitter.