Total Page Preview: 000000000723
What is Datasets in Azure Data Factory
In this article we will discussed about Datasets In Azure Data Factory.
Datasets:
Datasets represent data structures within the data stores, which simply points the data you want to use in your activities as inputs or outputs. An activity takes a zero or more datasets as inputs and one or more datasets as outputs. Example, an Azure Blob dataset specifies the blob container and folder in the Azure Blob Storage from which the pipeline should read the data. Or, an Azure SQL Table dataset specifies the table to which the output data is written by the activity.
Create Datasets in Azure Data Factory using portal :
{
"name": "SourceAzureBlobDataset",
"properties": {
"structure": [
{
"name": "ProductId",
"type": "Int64"
},
{
"name": "ProductName",
"type": "String"
},
{
"name": "Quantity",
"type": "Int64"
},
{
"name": "City",
"type": "String"
}
],
"published": false,
"type": "AzureBlob",
"linkedServiceName": "SourceAzureStorageLinkedService",
"typeProperties": {
"fileName": "Products.txt",
"folderPath": "products",
"format": {
"type": "TextFormat",
"columnDelimiter": ",",
"firstRowAsHeader": true
}
},
"availability": {
"frequency": "Minute",
"interval": 15
},
"external": true,
"policy": {}
}
}
Thank You
About Author
Brijesh Kumar
Database Developer
I have more then 6 years Experience in Microsoft Technologies - SQL Server Database, ETL
Azure Cloud - Azure SQL Database, CosmosDB, Azure Data Factory, PowerBI, Web Job, Azure Function, Azure Storage, Web Apps, Powershall
and Database Migration On-Premise to Azure Cloud.
LinkedIn : https://www.linkedin.com