Automate Linux VM Customization Tasks Using CustomScript Extension

You might already be familiar with the CustomScript extension for Windows VM in the Windows Extension Blog, the good news is that, the extension is also available for Linux VM now. Cloud-init was introduced in a previous blog as well, which enables you to inject a script or metadata into an Azure Linux VM during provision time. CustomScript extension provides even more power for customizing your VM, it can run on any Azure supported Linux VM.

What CustomScript can do for you?

After creating the Linux VM on Azure, you may need to deploy your workload (web server, database, etc.) on the VM, ideally this can be done via a serial of scripts.

• CustomScript extension can automatically download and execute these scripts for you, from the location you’ve specified, with parameters you’ve entered.
• CustomScript extension supports scripts written in any scripting language for Linux (Python, Linux shell scripts etc.), as long as the correspondent script interpreter is installed on your VM.
• You can install and configure CustomScript Extension any time after the provision.
• You can deploy the CustomScript Extension using Azure PowerShell Cmdlets,  xPlat scripts or Azure Management Portal. 

Updates in Feburary, 2015, following are new features added since original blog post:
1. Deployment through xPlat scripts
2. Deployment through Azure management portal
3. Support Inline command
4. Display the last 4KB outputs (stdout, stderr) to portal or as the result for  “Get Deployment” cmdlet

Updates in December 2015, following are new features added since last update:
1. Supporting Azure Resource Manager model, for sample Powershell and CLI scripts on running CustomScript Extension for Linux under Azure Resource Manager model, please see github release document: (https://github.com/Azure/azure-linux-extensions/blob/master/CustomScript/README.md).
2. CustomScript Extention for Linux 1.4 is released, supporting passing data in “commandToExecute” parameter under protected configuration section, this gives extra protection for sensitive data. For details, see (https://github.com/Azure/azure-linux-extensions/blob/master/CustomScript/README.md).

Pre-requisites

• Microsoft Azure Linux Agent version 2.0.6 or later. Note most Azure VM Linux gallery images included version 2.0.6. You can run waagent -version to confirm the version installed in the VM. If the VM is running a version earlier than 2.0.6, you can follow these instructions to update.
• Azure PowerShell and Cross-Platform CLI for the script scenario.
• Prepare the script you want to run, make sure it is uploaded to Azure Storage blob or Github location if you want to deploy the extension through the script. If you want to deploy it through Azure portal, you can save your script to your local machine.

Deploy the CustomScript Extension through Scripts

Deploy CustomScript Extension via scripting gives you the flexibility to automate it on multiple machines.   There are two options for the location of your script: Azure storage or GitHub. You can also run it through Azure PowerShell or Cross-Platform CLI.  In addition, an “inline” mode is also supported.

Running the script stored in Azure storage blob

1. User writes and uploads the script to an Azure blob location.
2. He adds the extension to his VM either during the provision or post the provision.
3. He enters the Azure blob location of the script and the optional parameters.
4. Inside the VM the extension downloads the script.
5. The extension launches the script with user specified parameters.

Power Shell Sample:

#Sample PowerShell Script to run a Linux Shell script stored in Azure blob
#Enter the VM name, Service name, Azure storage account name and key
$vm = Get-AzureVM -ServiceName "MyService" -Name "MyVM"
$PrivateConfiguration = '{"storageAccountName": "MyAccount","storageAccountKey":"Mykey"}' 
#Specify the Location of the script from Azure blob, and command to execute
$PublicConfiguration = '{"fileUris":["https://MyAccount.blob.core.windows.net/vhds/MyShellScript.sh"], "commandToExecute": "sh MyShellScript.sh" }' 
 
#Deploy the extension to the VM, always use the latest version by specify version “1.*”
$ExtensionName = 'CustomScriptForLinux'  
$Publisher = 'Microsoft.OSTCExtensions'  
$Version = '1.*' 
Set-AzureVMExtension -ExtensionName $ExtensionName -VM  $vm -Publisher $Publisher -Version $Version -PrivateConfiguration $PrivateConfiguration -PublicConfiguration $PublicConfiguration  | Update-AzureVM

Xplat Sample:

#Sample xPlat Script 
node bin/azure vm extension set MyVM CustomScriptForLinux Microsoft.OSTCExtensions  1.2 -i '{"fileUris":["https://MyAccount.blob.core.windows.net/vhds/MyShellScript.sh"], "commandToExecute": " sh MyShellScript.sh " }' -t '{"storageAccountName":"MyAccount ","storageAccountKey":" Mykey"}'  

Running scripts stored in GitHub

1. The user writes the scripts and uploads them to GitHub.
2. He adds the extension to his VM either during provision or post provision.
3. He enters the GitHub location of the script and optional parameters.
4. Inside the VM the extension downloads the scripts.
5. The extension launches the script with user specified parameters.

PowerShell Sample:

#Identify the VM
$vm = Get-AzureVM -ServiceName ‘MyServiceName’ -Name ‘MyVMName’
#Specify the Location of the script and the command to execute
$PublicConfiguration = '{"fileUris":["https://github.com/MyProject/Archive/MyPythonScript.py"], "commandToExecute": "python MyPythonScript.py" }' 

#Deploy the extension to the VM, pick up the latest version of the extension
$ExtensionName = 'CustomScriptForLinux'  
$Publisher = 'Microsoft.OSTCExtensions'  
$Version = '1.*' 
Set-AzureVMExtension -ExtensionName $ExtensionName -VM  $vm -Publisher $Publisher -Version $Version -PublicConfiguration $PublicConfiguration  | Update-AzureVM

Xplat Sample:

#Sample xPlat Script 
node bin/azure vm extension set MyVM CustomScriptForLinux Microsoft.OSTCExtensions  1.2 -i '{"fileUris":[" https://github.com/MyProject/Archive/MyPythonScript.py "], "commandToExecute": " python MyPythonScript.py " }' 

Running inline scripts

IF you just want to run one command instead of a script, you don't need to write a separate script, you only need to copy the command inline into the deployment script, utilizing the  parameter. Here is a PowerShell script example.

#Sample PowerShell Script
#Sample script to run a Python script inline
#Enter the VM name and Service name
$vm = Get-AzureVM -ServiceName "MyService" -Name "MyVM"
#Specify the command to execute
$PublicConfiguration = '{"commandToExecute": " python -c "print "hello,azure!"""}' 

#Deploy the extension to the VM
$ExtensionName = 'CustomScriptForLinux'  
$Publisher = 'Microsoft.OSTCExtensions'  
$Version = '1.*' 
Set-AzureVMExtension -ExtensionName $ExtensionName -VM  $vm -Publisher $Publisher -Version $Version -PublicConfiguration $PublicConfiguration  | Update-AzureVM

Deploy the CustomScript Extension through Azure Portal

For this scenario, user does not need to write a script to deploy the CustomScript Extension; user can store the script to local desktop or laptop, and upload the script to Azure via the portal. The portal will initiate the extension handler.

1. Log in the portal, locate the virtual machine you want to run the extension, click “Setting”, and choose the “Extension” section.

2. This will brings up the Extension page. Click  “Add“ button.

3. From the Extension list, choose the CustomScript Extension, and click “Create” button on the bottom of the page.

4. Now, the CustomScript Extension page pops up, allowing user to enter two parameters: 1. Path to the script 2. Parameters for the script

5. Browse to the script you want to run from your local machine, enter parameter for the script if needed,  click “Create”, the extension will be deployed to your VM, the scrip will be kicked off.

Please be aware that deployment via portal has following requirements: 1.  You can only upload one script. If you have multiple scripts you want to upload and run, you will want to user the script approach listed above. 2. There is no “Command to Execute” parameter in the UI, so you need to specify the interpreter at the beginning for your script;  for example ” #!/usr/bin/env python”.

Query the Result

• The CustomScript extension will kick off user’s scripts right after the deployment of the extension.
• The execution result can be retrieved via Azure PowerShell Cmdlet Get-AzureVM or Get-Deployment. You can also check the status through xPlat scripts and Azure portal “Status” tab. This includes the last 4K standard output, standard error from user’s script.
• You can find the CustomScript extension’s log file at /var/log/azure/Microsoft.OSTCExtensions. CustomScriptForLinux/1.0/extension.log.  Note the standard output of the user’s script will also be captured in this log file.

Additional notes

• If you have multiple scripts, you can author an entry point script that calls the dependent scripts, then upload the entry point script, dependent scripts and any other dependent binaries to the script location(Azure storage blob or GitHub). You can use following format to ensure multiple files are uploaded:  “fileUris” : [“ScriptFileUri1”, “ScriptFileUri2”, “ScriptFileUri3”]

• The extension is designed for one off tasks after VM creation. If the extension is called the second time with the same configuration, it will not execute the script you specified. However if you do need to run the CustomScript extension multiple times, you can run it with different configurations. For example:

1. Change the name of your script.
2. Use following code to add a timestamp as the [PublicConfiguration] parameter:

   #Generate the current timestamp, add to the configuration, this ensures the configuration is different every time the CustomScript extension runs
   $TimeStamp = (Get-Date).Ticks
   $PublicConfiguration ='{"fileUris":["MyAccount.blob.core.windows.net/vhds/MyShellScript.sh"], "commandToExecute": "sh MyShellScript.sh", "timestamp": "' + $TimeStamp + '"}'

Known Issues

When you run the PowerShell command “Set-AzureVMExtension” on Linux VM, you may hit following error: “Provision Guest Agent must be enabled on the VM object before setting IaaS VM Access Extension”. This does not happen if you are using the new portal. Root Cause: when you create the image via portal, the value of guest agent on the VM is not always  set to “True”. If your VM is created using PowerShell, you will not see this issue. Resolution: Add the following PowerShell command to set the ProvisionGuestAgent to “True”; $vm = Get-AzureVM -ServiceName 'MyServiceName' -Name 'MyVMName' $vm.GetInstance().ProvisionGuestAgent = $true