Total Page Preview:   000000000723

What is Datasets in Azure Data Factory

In this article we will discussed about Datasets In Azure Data Factory.

Datasets:

Datasets represent data structures within the data stores, which simply points the data you want to use in your activities as inputs or outputs. An activity takes a zero or more datasets as inputs and one or more datasets as outputs. Example, an Azure Blob dataset specifies the blob container and folder in the Azure Blob Storage from which the pipeline should read the data. Or, an Azure SQL Table dataset specifies the table to which the output data is written by the activity.

Create Datasets in Azure Data Factory using portal :

 
 
{
    "name": "SourceAzureBlobDataset",
    "properties": {
        "structure": [
            {
                "name": "ProductId",
                "type": "Int64"
            },
            {
                "name": "ProductName",
                "type": "String"
            },
            {
                "name": "Quantity",
                "type": "Int64"
            },
            {
                "name": "City",
                "type": "String"
            }
        ],
        "published": false,
        "type": "AzureBlob",
        "linkedServiceName": "SourceAzureStorageLinkedService",
        "typeProperties": {
            "fileName": "Products.txt",
            "folderPath": "products",
            "format": {
                "type": "TextFormat",
                "columnDelimiter": ",",
                "firstRowAsHeader": true
            }
        },
        "availability": {
            "frequency": "Minute",
            "interval": 15
        },
        "external": true,
        "policy": {}
    }
}

 

Thank You

About Author

Brijesh Kumar

Database Developer

I have more then 6 years Experience in Microsoft Technologies - SQL Server Database, ETL Azure Cloud - Azure SQL Database, CosmosDB, Azure Data Factory, PowerBI, Web Job, Azure Function, Azure Storage, Web Apps, Powershall and Database Migration On-Premise to Azure Cloud.
LinkedIn : https://www.linkedin.com



Comments

Feaster
07-Feb-2019
Howdy! I'm at work surfing around your blog from my new iphone! Just wanted to say I love reading through your blog and look forward to all your posts! Carry on the superb work!

                           
                           

                           

                           

Facebook User: