As part of the “2016-05-31” REST API version, we have introduced the pop receipt on add message functionality, which has been a commonly requested feature by our users.
Pop receipt functionality for the Queue service is a great tool for developers to easily identify an enqueued message for further processing. Prior to the “2016-05-31” version, pop receipt value could only be retrieved when a user gets a message from the queue. To simplify this, we now make pop receipt value available in the Put Message (aka Add Message) response which allows users to update/delete a message without the need to retrieve the message first.
Below is a short code snippet that make use of this new feature using Azure Storage Client Library 8.0 for .NET.
// create initial message
CloudQueueMessage message = new CloudQueueMessage("");
// add the message to the queue, but keep it hidden for 3 min
queue.AddMessage(message, null, TimeSpan.FromSeconds(180));
//message.PopReceipt is now populated, and only this client can operate on the message until visibility timeout expires
.
.
.
// update the message (now no need to receive the message first, since we already have a PopReceipt for the message)
message.SetMessageContent("");
queue.UpdateMessage(message, TimeSpan.FromSeconds(180), MessageUpdateFields.Content | MessageUpdateFields.Visibility);
// remove the message using the PopReceipt before any other process sees it
await queue.DeleteMessageAsync(message.Id, message.PopReceipt);
A common problem in cloud applications is to help coordinate updates across non-transactional resources. As an example, an application that processes images or videos may:
1. Process an image
2. Upload it to a blob
3. Save metadata in a table entity
These steps can be tracked using the Queue service as the processes complete successfully using the following flow:
1. Add a state as a message to the Queue service
2. Process an image
3. Upload it to a blob
4. Save metadata in a table entity
5. Delete the message if all were successful
Remaining messages in the queue are simply images that failed to be processed, and can be consumed by a worker for cleanup. The scenario above is now made simpler with the popreceipt on add message feature, since in the 5th step the message can be deleted with the popreceipt value retrieved in the 1st step.
Quick Sample using the Face API from Azure Cognitive Services
In the following sample, we are going to be uploading photos from a local folder to the Blob service and we will also make use of the Face API to estimate each person’s age in the photos, storing as an entity in a table. This process will be tracked in a queue and once completed, the message will be deleted with the pop receipt value. The workflow for the sample is:
1. Find JPG files in ‘testfolder’
2. For each photo, repeat steps 2-7:
3. Upload a queue message representing the processing of this photo.
4. Call the Face API to estimate the age of each person in the photo.
5. Store the age information as an entity in the table.
6. Upload the image to a blob if at least one face is detected.
7. If both the blob and the table entity operation succeeded, delete the message from queue using the pop receipt.
// Iterate over photos in 'testfolder'
var images = Directory.EnumerateFiles("testfolder", "*.jpg");
foreach (string currentFile in images)
{
string fileName = currentFile.Replace("testfolder", "");
Console.WriteLine("Processing image {0}", fileName);
// Add a message to the queue for each photo. Note the visibility timeout
// as blob and table operations in the following process may take up to 180 seconds.
// After the 180 seconds, the message will be visible and a worker role can pick up
// the message from queue for cleanup. Default time to live for the message is 7 days.
CloudQueueMessage message = new CloudQueueMessage(fileName);
queue.AddMessage(message, null, TimeSpan.FromSeconds(180));
// read the file
using (var fileStream = File.OpenRead(currentFile))
{
// detect face and estimate the age
var faces = await faceClient.DetectAsync(fileStream, false, true, new FaceAttributeType[] { FaceAttributeType.Age });
Console.WriteLine(" > " + faces.Length + " face(s) detected.");
CloudBlockBlob blob = container.GetBlockBlobReference(fileName);
var tableEntity = new DynamicTableEntity(DateTime.Now.ToString("yyMMdd"), fileName);
// iterate over detected faces
int i = 1;
foreach (var face in faces)
{
// append the age info as property in the table entity
tableEntity.Properties.Add("person" + i.ToString(), new EntityProperty(face.FaceAttributes.Age.ToString()));
i++;
}
// upload the blob if a face was detected
if (faces.Length > 0)
await blob.UploadFromFileAsync(currentFile);
// store the age info in the table
table.Execute(TableOperation.InsertOrReplace(tableEntity));
// delete the queue message with the pop receipt since previous operations completed successfully
await queue.DeleteMessageAsync(message.Id, message.PopReceipt);
}
}
Check out the full sample in our Github sample repository.
As always, if you have any feature requests please let us know by submitting your ideas to Azure Storage Feedback.
