Azure Data Factory (ADF) is the fully-managed data integration service for analytics workloads in Azure. Using ADF users can load the lake from 70+ data sources, on premises and in the cloud, use rich set of transform activities to prep, cleanse, process the data using Azure analytics engines, and finally land the curated data into a data warehouse for reporting and app consumption. With ADF you can iteratively develop, debug, and continuously integrate and deploy into dev, QA, and production environments, enabling you to achieve productivity during development phrase as well as operationalize and manage your Extract Transform Load /Extract Load Transform workflows holistically.
All analytics solutions start with loading data from diverse data source into data lake. As part of January 2018 release of ADF Visual Tool, we released Copy Data Tool which allows you to easily set up a pipeline to accomplish the data loading task in minutes, without having to understand or explicitly set up Linked Services and datasets for source and destination. We continuously listened to your feedback and today we are happy to announce the latest set of enhancements to the Copy Data Tool making it easier to ingest data at scale:
Support ingesting data at scale for all 70+ on-prem and cloud data sources
Copy Data Tool now supports all 70+ on-prem and cloud data sources, and we will continue to add more connectors in the coming months. Tell us if you do not find the connector you are looking for in the list.
Ingest tens of thousands of tables into Azure at scale
Using Copy Data Tool you can now browse and select tens or thousands of tables from data sources to be loaded. Use default settings or customize per-table column mapping logic that best suites your specific needs. Upon deployment, Copy Data Tool automatically generates a parameterized pipeline containing each plus copy activity, which means you do not end up with large number of pipelines and datasets for loading multiple tables.
Support both schedule trigger and tumbling window trigger
Please send us your feedback as you get started on your modern data warehouse journey. Starting with building your data ingest pipelines! We are eagerly awaiting your feedback so we can continuously improve our feature set and experience offering. We look forward to hearing from you!