Azure Data Factory’s (ADF) public preview launched on 10/27 and one of the consistent pieces of customer feedback we received, is to enable a rich interactive authoring experience allowing users to create, configure and deploy data integration pipelines without any friction. We listened to your feedback and are happy to announce the release of “ADF Editor” which is a light weight web editor for creating, editing and deploying the JSON files of all ADF entities.
The main goal of the ADF Editor is to allow users to be productive with ADF by getting pipelines up & running quickly without requiring any PowerShell installation or ramp up on the cmdlets. You can now do all of your JSON editing, deployment and management operations from the ADF within the Azure preview portal. The ADF editor is our first step towards a full authoring experience.
Below is a video showcasing an end to end use case with ADF Editor:
Launching the ADF Editor
On the Data Factory summary blade, click Author and deploy to launch the Editor:
Figure 1 : Shows the launch the ADF Editor.
ADF Editor Layout
The Editor has 2 main sections:
1. Navigation tree on the left
Displays the list of the deployed entities to the Data Factory in a tree view. You can browse and view them on the editing canvas. On top of the navigation tree, there are buttons for create and delete actions on ADF entities.
2. JSON editing canvas on the right
This is the JSON editor for new or deployed ADF entities. On top of the editing canvas, there are buttons to for corresponding actions that you may want to perform on the entities in the canvas.
Figure 2 : Shows the ADF Editor layout.
Create new Linked Services, Datasets and Pipelines
You can browse all deployed ADF entities ,edit and/or create new entities by using the JSON templates. There are 3 categories for “New” actions
1. New Linked Service
a. New data store
b. New compute
2. New dataset
3. New pipeline
Each of these categories have corresponding sub-categories for the types that they support. On selecting the category and the subcategory, the JSON templates appear on the editor canvas. You can quickly configure by replacing the template properties with the actual values.
Figure 3 : Browse deployed entities from the tree on the left navigation
Figure 4 : JSON templates for easy creation of ADF entities
Clone and Drafts
Clone is useful in cases you are trying to create the exact copy of a deployed entity. You can now select any entity from the tree and click the “Clone” button to create the copy.
Drafts allows you to temporarily save your work when you are context switching or navigating to a different entity in the Data Factory. The lifetime of the Drafts are associated with the browser session. If you close the browser or use another machine, the drafts are not going to be available.
Starting a Pipeline
Once you have created all the required entities, you want to deploy the pipelines and set the processing active period. You can do it using the editor in the Pipeline definition as shown below:
Figure 5 : Set properties to start the pipeline
Editing a deployed entity
You can edit a deployed entity in one of 2 ways:
1. Launch the Editor, browse the entity on the left tree and edit the definition on the canvas
2. Browse through the Datasets and Pipelines and click “Table source” and/or “Pipeline source” to navigate to the Editor
Figure 6 : Edit a deployed pipeline
We believe all these capabilities of the ADF Editor will alleviate the configuration and deployment pains encountered by our customers today. We are continuously thriving to improve the user experience and will be iterating on the ADF editor to enrich the capabilities in upcoming releases.
You can now easily copy/paste JSON contents from the Azure Data Factory Getting Started Tutorial and/or the GitHub samples and see the end to end pipelines up and running quickly.
We are looking forward to hear your feedback on the ADF Editor. If you are missing a specific functionality or encounter any issues, please visit the Azure Data Factory Forums and provide your feedback.