Skip navigation

Deploy HDInsight cluster with existing linked storage

Last updated: 14/06/2021

This template allows you to create an Hadoop cluster in HDInsight and the dependent default storage account. The template also links an existing storage account. The linked storage account usually contains the business data.

This Azure Resource Manager template was created by a member of the community and not by Microsoft. Each Resource Manager template is licensed to you under a licence agreement by its owner, not Microsoft. Microsoft is not responsible for Resource Manager templates provided and licensed by community members and does not screen for security, compatibility, or performance. Community Resource Manager templates are not supported under any Microsoft support programme or service, and are made available AS IS without warranty of any kind.

Parameters

Parameter Name Description
ClusterName The name of the HDInsight cluster to create. This must be unique, less than 59 characters, can contain letters, numbers, and hyphens (the first and last characters must be a letter or number). The name cannot contain a reserved keyword.
clusterType The type of the HDInsight cluster to create.
clusterVersion The type of the HDInsight cluster to create.
clusterLoginUserName These credentials can be used to submit jobs to the cluster and to log into cluster dashboards. The cluster login username must be 2-20 characters and consist of digits, lowercase letters, and/or the following special characters: (!#$%&'()-^_`{}~). The name cannot contain a reserved keyword.
clusterLoginPassword The password must be at least 10 characters in length and must contain at least one digit, one upper case letter, one lower case letter, and one non-alphanumeric character. The following characters are not allowed (single-quote, double-quote, backslash, right-bracket, full-stop). Also, the password must not contain 3 consecutive characters from the cluster username or SSH username.
sshUserName These credentials can be used to remotely access the cluster. The SSH username can only consist of digits, upper or lowercase letters, and/or the following special characters (%&'^_`{}~). Also, it cannot be the same as the cluster login username or a reserved word.
sshPassword SSH password must be 6-72 characters long and must contain at least one digit, one upper case letter, and one lower case letter. It must not contain any 3 consecutive characters from the SSH username.
storageAccountResourceGroup The existing linked storage resource groupname
storageAccountName The short name of the linked Azure storage account name. This account needs to be secure transfer enabled.
headNodeVirtualMachineSize This is the headnode Azure Virtual Machine size, and will affect the cost. If you don't know, just leave the default value.
workerNodeVirtualMachineSize This is the workdernode Azure Virtual Machine size, and will affect the cost. If you don't know, just leave the default value.
defaultStorageAccountName Name for the storage account
location Location for all resources.

Use the template

PowerShell

New-AzResourceGroup -Name <resource-group-name> -Location <resource-group-location> #use this command when you need to create a new resource group for your deployment
New-AzResourceGroupDeployment -ResourceGroupName <resource-group-name> -TemplateUri https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/quickstarts/microsoft.hdinsight/hdinsight-linux-with-existing-linked-storage-account/azuredeploy.json
Install and configure Azure PowerShell

Command line

az group create --name <resource-group-name> --location <resource-group-location> #use this command when you need to create a new resource group for your deployment
az group deployment create --resource-group <my-resource-group> --template-uri https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/quickstarts/microsoft.hdinsight/hdinsight-linux-with-existing-linked-storage-account/azuredeploy.json
Install and Configure the Azure Cross-Platform Command-Line Interface