Databricks Archives - DWgeek.com

How to Connect to Databricks SQL Endpoint from Azure Data Factory?

A Databricks SQL Endpoint is a compute cluster, quite similar to the cluster we have known in the Databricks that allows you execute SQL commands on data objects within the Databricks environment. The Databricks allows you to connect using various tools such as DBT, connect to Notebook using Azure Data Factory, etc. But there is no direct method to connect Databricks SQL endpoint warehouse. In this article, we will check how to connect to Databricks SQL endpoint from Azure Data Factory (ADF). Connect to Databricks SQL Endpoint from Azure Data…

Comments Off

January 27, 2023

Apache Spark

How to Access Azure Blob Storage Files from Databricks?

Azure blob storage is a Microsoft Azure cloud service to store large amount of structured and unstructured data such as text files, database export files, json files, etc. Azure blob storage allows you to store data publicly or you can store application data privately. You can access public Azure blob data without using any additional credentials. But, to access private data, you need to generate access key. In this article, we will check how to access Azure Blob storage files from Databricks? Access Azure Blob Storage Files from Databricks Similar…

Comments Off

May 4, 2022

Snowflake

How to Connect to Snowflake from Databricks?

Many organizations use the hybrid model to process the data. They use databricks to perform operations such as Machine Learning tasks and copy end results to Snowflake for reporting or further analysis. In this article, we will check how to connect to Snowflake from databricks to build hybrid architecture. Connect to Snowflake from Databricks The Snowflake is one of the relational databases that provide connector for Spark. You can use the Snowflake Spark connector to connect to Snowflake server and copy data from databricks to Snowflake. Test Data We will…

Comments Off

June 16, 2021