How to Connect to Databricks SQL Endpoint from Azure Data Factory?

A Databricks SQL Endpoint is a compute cluster, quite similar to the cluster we have known in the Databricks that allows you execute SQL commands on data objects within the Databricks environment. The Databricks allows you to connect using various tools such as DBT, connect to Notebook using Azure Data Factory, etc. But there is no direct method to connect Databricks SQL endpoint warehouse. In this article, we will check how to connect to Databricks SQL endpoint from Azure Data Factory (ADF). Connect to Databricks SQL Endpoint from Azure Data…

Continue ReadingHow to Connect to Databricks SQL Endpoint from Azure Data Factory?
Comments Off on How to Connect to Databricks SQL Endpoint from Azure Data Factory?

How to Access Azure Blob Storage Files from Databricks?

Azure blob storage is a Microsoft Azure cloud service to store large amount of structured and unstructured data such as text files, database export files, json files, etc. Azure blob storage allows you to store data publicly or you can store application data privately. You can access public Azure blob data without using any additional credentials. But, to access private data, you need to generate access key. In this article, we will check how to access Azure Blob storage files from Databricks? Access Azure Blob Storage Files from Databricks Similar…

Continue ReadingHow to Access Azure Blob Storage Files from Databricks?
Comments Off on How to Access Azure Blob Storage Files from Databricks?

How to Export Azure Synapse Table to Local CSV using BCP?

It is common practice to offload large table to a flat file such as CSV, TSV or fixed width files. Many application stores data in the form of flat files such as CSV. Managing flat files such as CSV is easy and it is easy to transport using any electronic medium. In this article, we will check how to export Azure Synapse table to local CSV using BCP command? In my other article, we have discussed how to export an Azure table to local CSV using SQLCMD command. Azure Synapse…

Continue ReadingHow to Export Azure Synapse Table to Local CSV using BCP?
Comments Off on How to Export Azure Synapse Table to Local CSV using BCP?

How to Load Local File to Azure Synapse using BCP?

It is common practice to offload large table to a flat file such as CSV, TSV or fixed width files. Managing flat files such as CSV is easy and it is easy to transport using any electronic medium. In this article, we will check how to load or import local flat file to Azure Synapse using the BCP command with some examples. Load Local Flat File to Azure Synapse using BCP Utility The bulk copy program utility (bcp) bulk copies data from a file in a user-specified format to an instance such as…

Continue ReadingHow to Load Local File to Azure Synapse using BCP?
Comments Off on How to Load Local File to Azure Synapse using BCP?

Azure Synapse INSERT with VALUES Limitations and Alternative

Azure Synapse Analytics data warehouse is based on TSQL. But, not all TSQL features are available in Azure Synapse dedicated SQL pool. The feature such as using an expression or UDF in VALUES clause of INSERT INTO table statement is not supported. In this article, we will check Azure Synapse INSERT with VALUES limitations and alternative methods. Azure Synapse INSERT with VALUES Limitations The VALUES clause Introduces the list or lists of data values to be inserted. There must be one data value for each column in column_list, if specified, or…

Continue ReadingAzure Synapse INSERT with VALUES Limitations and Alternative
Comments Off on Azure Synapse INSERT with VALUES Limitations and Alternative

Azure Synapse Analytics Cursor Alternative

SQL Cursor is a database object to retrieve data from a result set one row at a time. Database such as Oracle, Teradata, Microsoft SQL Server support cursors. The cursor are useful in many scenarios such as retrieving records in a loop. SQL Cursor always returns one row at a time, you can perform your calculation on returned values. In this article, we will check what is Azure Synapse Analytics Cursor Alternative with an example. Does Azure Synapse Analytics support SQL Cursor Variable? The Microsoft SQL Server, which is based…

Continue ReadingAzure Synapse Analytics Cursor Alternative
Comments Off on Azure Synapse Analytics Cursor Alternative

Azure Synapse DROP TABLE IF EXISTS Alternatives

Many relational databases such as Netezza supports DROP TABLE IF EXISTS syntax to safely drop table if it is already present in the database. The Microsoft SQL Server 2016 or higher version supports drop table if exists. However, same command won't work on Azure Synapse dedicated SQL pool server. In this article, we will check what are the Azure Synapse analytics DROP TABLE IF EXISTS alternatives with some examples. Azure Synapse DROP TABLE IF EXISTS Alternatives The DROP TABLE IF EXISTS statement checks the existence of the table in the schema, and…

Continue ReadingAzure Synapse DROP TABLE IF EXISTS Alternatives
Comments Off on Azure Synapse DROP TABLE IF EXISTS Alternatives

QUALIFY Clause in Synapse and TSQL- Alternative and Examples

The Azure Synapse data warehouse is based on the Microsoft TSQL. If you are migrating from Teradata to Azure Synapse, you will notice that many features are missing in the Synapse date warehouse. The feature such as QUALIFY clause is widely used in Teradata. But, the same feature is not available in Synapse. In this article, we will check what is QUALIFY Clause alternative in Azure Synapse and TSQL. QUALIFY Clause in Azure Synapse and TSQL The relational databases such as Teradata uses QUALIFY clause to filter the result of…

Continue ReadingQUALIFY Clause in Synapse and TSQL- Alternative and Examples
Comments Off on QUALIFY Clause in Synapse and TSQL- Alternative and Examples