News
Azure Data Factory Takes Visual, Low-Code Route
In response to developer feedback, Microsoft is taking a page from the growing visual, low-code development approach to simplify Big Data analytics in the cloud with Azure Data Factory.
The new visual, drag-and-drop functionality comes in Azure Data Factory (ADF) v2, a new public preview of Microsoft's cloud-based data integration service that debuted as v1 in October 2015. ADF is used to create data-driven workflows -- such as extract, transform, load (ETL) and extract, load, transform (ELT) -- in the cloud in order to orchestrate and automate data movement and data transformation.
Processing and transforming the data is done in conjunction with compute services such as Azure HDInsight Hadoop, Spark, Azure Data Lake Analytics and Azure Machine Learning.
One of ADF's main uses is the creation and scheduling of those data-driven workflows (Microsoft calls them "pipelines": logical groupings of activities that together perform a task) that can ingest data from disparate data stores.
That process is now point-and-click simple in the v2 preview.
"One of the consistent pieces of customer feedback we received, is to enable a rich interactive visual authoring and monitoring experience allowing users to create, configure, test, deploy and monitor data integration pipelines without any friction," said Microsoft's Gaurav Malhotra, senior program manager, in a blog post last week.
"We listened to your feedback and are happy to announce the release of visual tools for ADF v2. The main goal of the ADF visual tools is to allow you to be productive with ADF by getting pipelines up & running quickly without requiring [you] to write a single line of code."
The new visual functionality lets developers create control flow pipelines, drag-and-drop activities onto a canvas and "connect them on-success, on-failure, on-completion." Visual monitoring of pipeline, activity and trigger runs is also available in a simple list view interface.
ADF v2 also includes guided tours that help developers understand how to use the new visual authoring and monitoring features while also providing a mechanism to give the ADF team more valuable feedback.
See Malhotra's post for more details on the above functionality and much more. Additional guidance is provided in the Jan. 16 article "Create a data factory using the Azure Data Factory UI," which covers the v2 preview.
About the Author
David Ramel is an editor and writer at Converge 360.