This template deploys a new Data Factory and requisite objects (linked services, datasets, pipelines, gateways, etc.) to fascilitate a two-activity chained Data Factory pipeline. The first leg of the pipeline leverages data management gateway to pull data from an on-premises SQL server source into Azure Data Lake Store in Apache orc columnar storage format. The second leg of the pipeline pulls data from orc files in Azure Data Lake Store and inserts into Azure SQL as a final destination. This pipeline can be easily customized to accomodated a wide variety of additional sources and targets.
This Azure Resource Manager (ARM) template was created by a member of the community and not by Microsoft. Each ARM template is licensed to you under a licence agreement by its owner, not Microsoft. Microsoft is not responsible for ARM templates provided and licensed by community members and does not screen for security, compatibility or performance. Community ARM templates are not supported under any Microsoft support programme or service, and are made available AS IS without warranty of any kind.
Parameters
| Parameter Name | Description |
|---|---|
| dataLakeStoreUri | URI of Azure Data Lake store |
| dataLakeStoreServicePrincipalID | ID of Azure Service Principal used for accessing Data Lake |
| dataLakeStoreServicePrincipalKey | Key for Azure Service Principal used for accessing Data Lake |
| azureSQLConnectionString | Connection string for Azure SQL Database |
| location | Location for all resources. |
Use the template
PowerShell
New-AzResourceGroup -Name <resource-group-name> -Location <resource-group-location> #use this command when you need to create a new resource group for your deploymentInstalling and configuring Azure PowerShell
New-AzResourceGroupDeployment -ResourceGroupName <resource-group-name> -TemplateUri https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/101-data-factory-chained-copy-activities/azuredeploy.json
Command line
az group create --name <resource-group-name> --location <resource-group-location> #use this command when you need to create a new resource group for your deploymentInstalling and configuring the Azure cross-platform command-line interface
az group deployment create --resource-group <my-resource-group> --template-uri https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/101-data-factory-chained-copy-activities/azuredeploy.json