This template creates a data factory pipeline with Copy and HDInsight Hive activities.
This Azure Resource Manager template was created by a member of the community and not by Microsoft. Each Resource Manager template is licensed to you under a licence agreement by its owner, not Microsoft. Microsoft is not responsible for Resource Manager templates provided and licensed by community members and does not screen for security, compatibility, or performance. Community Resource Manager templates are not supported under any Microsoft support programme or service, and are made available AS IS without warranty of any kind.
Parameters
| Parameter Name | Description |
|---|---|
| storageAccountResourceGroupName | The resource group that contains your Azure storage account that contains the input/output data. |
| storageAccountName | Name of the Azure storage account that contains the input/output data. |
| storageAccountKey | Key for the Azure storage account. |
| blobContainer | Name of the blob container in the Azure Storage account. |
| inputBlobFolder | The folder in the blob container that has the input file. |
| inputBlobName | Name of the input file/blob. |
| ftpHost | Name or IP address of FTP server. |
| ftpUser | User account that has access to the FTP server. |
| ftpPassword | Password for the user account that has access to the FTP server. |
| ftpFolderName | The folder in FTP that has the input file. |
| ftpFileName | Name of the input file in FTP. |
| outputBlobFolder | The folder in the blob container that will hold the transformed data. |
| hiveScriptFolder | The folder in the blob container that contains the Hive query file. |
| hiveScriptFile | Name of the hive query (HQL) file. |
| sqlServerName | Name of the Azure SQL Server that will hold the output/copied data. |
| sqlDatabaseName | Name of the Azure SQL Database in the Azure SQL server. |
| sqlServerUserName | Name of the user that has access to the Azure SQL server. |
| sqlServerPassword | Password for the user. |
| targetSQLTable | Table in the Azure SQL Database that will hold the copied data. |
Use the template
PowerShell
New-AzResourceGroup -Name <resource-group-name> -Location <resource-group-location> #use this command when you need to create a new resource group for your deploymentInstall and configure Azure PowerShell
New-AzResourceGroupDeployment -ResourceGroupName <resource-group-name> -TemplateUri https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/quickstarts/microsoft.datafactory/data-factory-ftp-hive-blob/azuredeploy.json
Command line
az group create --name <resource-group-name> --location <resource-group-location> #use this command when you need to create a new resource group for your deploymentInstall and Configure the Azure Cross-Platform Command-Line Interface
az group deployment create --resource-group <my-resource-group> --template-uri https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/quickstarts/microsoft.datafactory/data-factory-ftp-hive-blob/azuredeploy.json