Adoption Program Insights: Sending Emails from Windows Azure (Part 1 of 2)

The Adoption Program Insights series describes experiences of Microsoft Services consultants involved in the Windows Azure Technical Adoption Program assisting customers deploy solutions on the Windows Azure platform. This post is by Patrick Butler Monterde and Tom Hollander.

Many custom-developed applications need to send email, whether it is a part of the registration process, a way of notifying users when important events occur or something else.  If you’re a .NET developer you’ve probably used classes of the System.Web.Mail namespace to accomplish this.  However these classes require access to an SMTP e-mail server to send messages and Windows Azure does not currently provide such a capability.  However all is not lost.  This two-part series describes some patterns for enabling emailing capabilities for applications deployed to the Windows Azure platform.

1. Using a custom on-premise Email Forwarder Service: This pattern, described in this post, utilizes an on-premise email server to send emails on behalf of the application running on Windows Azure. This is accomplished by creating a custom service using a distributed asynchronous model that uses Windows Azure storage queues and blobs to deliver emails generated in Windows Azure to the an on-premise email server.
2. Using Email Server’s Web Services APIs:  This pattern, that will be described in Part 2, uses the web services API provided by Microsoft Exchange to send email directly from Windows Azure. This pattern can be applied to other messaging products that provide a similar web services interface.
3. Using a third party SMTP Service: This pattern, described in Steve Marx’s blog post EmailTheInternet.com: Sending and Receiving Email in Windows Azure  utilizes a 3^rd party Email service like SendGrid or AuthSMTP to relay emails. The solution described in this post goes one step further and also shows how to receive email from a Windows Azure application by listening for SMTP traffic on port 25.

Pattern 1:  Using a Custom On-premise Email Forwarder Service

This pattern utilizes your existing on premise email server to send email on behalf of a Windows Azure application. This is accomplished by creating a custom on-premise Email Forwarder Service that uses Windows Azure Storage queues and blobs to deliver emails generated in Windows Azure to an on-premise email server.  The pattern is divided into two main sections: 

1. Preparing and sending email work items: This is the implementation of a Windows Azure Web/Worker Role that generates the email. It serializes the email object and creates an email work item in Windows Azure Storage.
2. Receiving and sending email work items: This is the implementation of an Email Forwarder Service which retrieves the email work items from Windows Azure storage, deserializes the email object and sends it to the email server.

For the distribution of emails from Windows Azure to the on premise email servers, we will define a concept of a “work item”. A work item is a logical container composed of:

1. One Queue Item: The queue item stores the reference (URI) of Blob where the email message is stored. It can also hold up to 8k of metadata you may need.
2. One Blob Item: The Blob item contains the serialized email object. Because Blobs can have up to 1TB of size, the email object could hold multiple large attachments.

The following diagram shows the pattern’s workflow:

This is what happens when an application hosted in Windows Azure needs to send an email message:

1. A worker/web role generates an email message. This email message is in the form of a System.Net.Mail.MailMessage instance. This mail object could include any number of attachments.
2. The email object is serialized, and stored into a blob. The blob’s URL is then added to a Queue item. The combination of the queue item and the blob become the email work item. You can make use of both the Queue and Blob items metadata to store additional information.
3. On premise, an Email Forwarder Service constantly monitors the Queues for emails. Queue items can be retrieved a rate of 32 items at the time. The Email Forwarder Service first retrieves the queue item, and then it extracts the Blob URI and retrieves the serialized email.
4. Once deserialized, the Email Forwarder Service uses the on-premise email server information to send the email.  After delivering the email, it removes the work item from the queue and blob storage.
5. The on-premise email server receives the emails. Because it is an on-premise application, the authentication and authorization should be straightforward.
6. The email server sends the email to the appropriate user.

To better illustrate the pattern, a sample implementation of the Email Forwarder Service and a Windows Azure application that uses it can be downloaded below as a .zip file.  The code sample contains the following projects:

• Email Forwarder Service: Implementation of the on-premise Email Forwarder Service. For simple demonstration purposes it is implemented as a Windows Form application; however for real-world deployment you would implement this in a Windows service. To test the sample service, edit the app.config file to include the details of your on-premise SMTP Server.
• Entity: Class Library that contains the email message serialization capabilities and the operations to add and remove email work items from Windows Azure storage. Both the Email Forwarder Service and the Web/Worker roles use this project.
• Email Generator Web Role: Implementation of a simple Web role that can send email. The role provides a web user interface that lets you enter details of the email to be sent.
• Email Generator Worker Role: Implementation of a simple worker role that can send Email. The role generates and sends email messages every 10 seconds using details found in the role’s app.config file.

Architectural Considerations

It is important to understand the architectural implications for any solution. Some of the considerations for the custom Email Forwarder Service include:

• Cost: Data storage in Blobs and Queues and the data flow to the on-premise service will incur the additional cost to the overall solution. The overall cost impact will vary based on the email volume in individual solution, which must be taken in consideration before implementing this pattern. Use of compression may be desirable to reduce the size of serialized email objects. To minimize bandwidth costs, the Windows Azure Storage account used for the blobs and queues should be located in the same affinity group as the web/worker roles sending the emails.
• Performance: There are two main observations regarding performance:

1. Serialized email objects that contain large email attachments may pose some performance impact since these needs to be serialized, moved to storage and then retrieved and de-serialized by the Email Forwarder Service.
2. Due to asynchronous nature of this pattern, the Email Forwarder Service checks the Windows Azure Storage Queues periodically for work items. This generated a marginal delay sending the emails to the email server. This must be studied carefully and set as per the individual needs.

• Management: This service should be monitored. We recommend adding logging and monitoring capabilities to the implementation.
• Reliability: Proper retry mechanisms (including exponential back-off) should be implemented in the Email Forwarder Service to take care of any connection failures. 

Part 2 will be posted here later this week.

Tweet