Data Orchestration with Azure Data Factory V2
2 h 20 m
In this lab, you will learn to build, monitor, manage and troubleshoot data pipelines with Azure Data Factory V2. You will learn to use the Copy Data wizard to build pipeline with no coding. You will build a custom pipeline to copy data from Blob storage to a table in Azure SQL Database. You will build a tumbling window pipeline to pick up data on a daily basis. Finally, you will learn to use the Management & Monitoring tools to troubleshoot pipeline failures.
- Provision Azure Data Factory using the Azure portal
- Build data pipelines using the Copy Data wizard in Azure Data Factory V2
- Build a custom data pipeline using Azure Data Factory V2
- Schedule and trigger data pipelines in Azure Data Factory V2
- Manage and monitor data pipelines in Azure Data Factory V2
- Troubleshoot failed pipelines in Azure Data Factory V2
In this exercise, you will configure the lab environment and provision an Azure Data Factory.
In this exercise, we will leverage the Azure Data Factory Copy Wizard to copy data from a file in our storage account to our Azure SQL Database.
You can create custom data flows by defining the linked services, datasets, and pipelines required to transfer and transform your data. In this exercise, you will create a simple pipeline that performs the same copy data task you accomplished with the Copy Data wizard.
In this exercise, you will create an Azure Data Factory pipeline to implement a daily copy operation for online game data. You will create a pipeline that copies data every day using a tumbling window trigger. You will then monitor the pipeline status and use Data Factory Monitoring & Management to troubleshoot pipeline issues.