Simple Web Site Backup retention policy with WebJobs
Azure provides the capability of running code on your website context with WebJobs. They provide great versatility in terms of application scenario solutions for websites, while accessing Azure resources, and different ways in which you can solve them. The website backup feature runs on a schedule and makes use of Azure storage, which makes this a particularly interesting example. This post covers the details on how to build a simple WebJob that will clean-up your old website backup files on a regular basis.
Backup files layout
To set the context, let’s cover some details of the backup feature that are relevant for understanding this solution. One of the best aspects of its design is how accessible and manageable the backup content is made for customers. After each successful backup run, two files are stored in the specified storage account: an XML file describing the backup contents, and a ZIP file with the actual files that have been backed up. The files’ naming convention is <website_name>_<timestamp>.<extension>. For example:
Most customers will set up the backup operation to run on a schedule, such as once a day. This will create 2 files on a daily basis that will accumulate over time in the backup storage account. While this feature is in preview mode, retention policy hasn’t been offered out of the box yet, but you can configure your site to do this cleanup.
One simple way to cleanup old backups is to run a scheduled WebJob that will apply your backup files retention policy. To familiarize yourself with WebJobs, check out this article on the Azure website. To use WebJobs for this, you need to create a simple application and configure a WebJob to run it every day. The application will go through all the files in the backup folder, check when they were created, and delete them if they are older than your defined policy.
The application can be written and deployed in a variety of ways. The example below is written in C# and deployed as a console app (.exe), but it could easily be done in other languages. Note: if you would like to use this code directly, follow the instructions in the next section on how to download the full version that can be used to build and deploy this as a WebJob
This first part of the code gathers the configuration for the backup storage account, the backup folder location, the max age to keep (in days) and the Website name. The configuration values (uppercase strings below) are obtained from the application settings for your website. Later in this post you’ll see how to set up values for those settings.
With the settings in place, the second part of the code connects to the blob storage, where the backup files are located. It then calculates the Date and Time offset from the current time to the max-age policy. Next, it uses a foreach loop to go through each of the files in the folder and for files that have the zip or xml extension and are named according to the backup process convention. The program then calculates each file age, if that age is older than the max defined, the code deletes the files.
Building and packaging the binaries (.exe)
- Create new Console App –> Templates Visual C# -> Console Application –> Name it, e.g. “WebJobProject”
- Replace the content from Program.cs with the full version of the code above, available here.
- It has the references to the libraries (using statements), logging progress + errors to standard output, namespace/class definitions, and additional code comments.
- Add References (in the Solution Explorer dialog –> right click references –> Add Reference)
- Framework –> System.Configuration
- Extensions –> Microsoft.WindowsAzure.Storage
- Build the solution (Ctrl+Shift+B)
- Zip the contents of the project “Debug” and name it, e.g. “WebJobProject.zip”
- To get to the Debug folder you can go to Solution Explorer –> click Show All Files icon –> expand “bin” folder to show the “Debug” folder –> right click to Open Folder in File Explorer
- To zip up the contents in a Windows 8 machine: Select All content –> right click to Send To –> “Compressed (zipped) folder”