Databricks

Lateral Column Alias in Databricks – Example

The lateral column alias in Databricks allows users to reuse an expression specified earlier in the same SELECT list, eliminating the need to use nested subqueries and Common Table Expressions (CTEs) in many cases. This blog post discusses the use cases of the feature and the benefits it brings to Spark and Databricks users. Page Contents Introduction What is Lateral Column Alias Support? Benefits of using Lateral Column Alias in Databricks SQL Alternative Methods to Lateral Column Alias in Databricks SQL Conclusion Introduction Databricks has introduced a much needed lateral…

Continue ReadingLateral Column Alias in Databricks – Example
Comments Off on Lateral Column Alias in Databricks – Example
September 24, 2024
General

How to Connect to Databricks SQL Endpoint from Azure Data Factory?

A Databricks SQL Endpoint is a compute cluster, quite similar to the cluster we have known in the Databricks that allows you execute SQL commands on data objects within the Databricks environment. The Databricks allows you to connect using various tools such as DBT, connect to Notebook using Azure Data Factory, etc. But there is no direct method to connect Databricks SQL endpoint warehouse. In this article, we will check how to connect to Databricks SQL endpoint from Azure Data Factory (ADF). Connect to Databricks SQL Endpoint from Azure Data…

Continue ReadingHow to Connect to Databricks SQL Endpoint from Azure Data Factory?
Comments Off on How to Connect to Databricks SQL Endpoint from Azure Data Factory?
January 27, 2023
Apache Spark

How to Access Azure Blob Storage Files from Databricks?

Azure blob storage is a Microsoft Azure cloud service to store large amount of structured and unstructured data such as text files, database export files, json files, etc. Azure blob storage allows you to store data publicly or you can store application data privately. You can access public Azure blob data without using any additional credentials. But, to access private data, you need to generate access key. In this article, we will check how to access Azure Blob storage files from Databricks? Access Azure Blob Storage Files from Databricks Similar…

Continue ReadingHow to Access Azure Blob Storage Files from Databricks?
Comments Off on How to Access Azure Blob Storage Files from Databricks?
May 4, 2022
Snowflake

How to Connect to Snowflake from Databricks?

Many organizations use the hybrid model to process the data. They use databricks to perform operations such as Machine Learning tasks and copy end results to Snowflake for reporting or further analysis. In this article, we will check how to connect to Snowflake from databricks to build hybrid architecture. Connect to Snowflake from Databricks The Snowflake is one of the relational databases that provide connector for Spark. You can use the Snowflake Spark connector to connect to Snowflake server and copy data from databricks to Snowflake. Test Data We will…

Continue ReadingHow to Connect to Snowflake from Databricks?
Comments Off on How to Connect to Snowflake from Databricks?
June 16, 2021