This template deploys a new Data Factory and requisite objects (linked services, datasets, pipelines, gateways, etc.) to fascilitate a two-activity chained Data Factory pipeline. The first leg of the pipeline leverages data management gateway to pull data from an on-premises SQL server source into Azure Data Lake Store in Apache orc columnar storage format. The second leg of the pipeline pulls data from orc files in Azure Data Lake Store and inserts into Azure SQL as a final destination. This pipeline can be easily customized to accomodated a wide variety of additional sources and targets.
This Azure Resource Manager template was created by a member of the community and not by Microsoft. Each Resource Manager template is licensed to you under a license agreement by its owner, not Microsoft. Microsoft is not responsible for Resource Manager templates provided and licensed by community members and does not screen for security, compatibility, or performance. Community Resource Manager templates are not supported under any Microsoft support program or service, and are made available AS IS without warranty of any kind.
|dataLakeStoreUri||URI of Azure Data Lake store|
|dataLakeStoreServicePrincipalID||ID of Azure Service Principal used for accessing Data Lake|
|dataLakeStoreServicePrincipalKey||Key for Azure Service Principal used for accessing Data Lake|
|azureSQLConnectionString||Connection string for Azure SQL Database|
|location||Location for all resources.|
Use the template
New-AzResourceGroup -Name <resource-group-name> -Location <resource-group-location> #use this command when you need to create a new resource group for your deploymentInstall and configure Azure PowerShell
New-AzResourceGroupDeployment -ResourceGroupName <resource-group-name> -TemplateUri https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/quickstarts/microsoft.datafactory/data-factory-chained-copy-activities/azuredeploy.json
az group create --name <resource-group-name> --location <resource-group-location> #use this command when you need to create a new resource group for your deploymentInstall and Configure the Azure Cross-Platform Command-Line Interface
az group deployment create --resource-group <my-resource-group> --template-uri https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/quickstarts/microsoft.datafactory/data-factory-chained-copy-activities/azuredeploy.json