Databricks

How to Use the QUALIFY Clause in Databricks SQL (Syntax & Examples)?

If you frequently write data transformation queries in Databricks, you probably use window functions to rank, row-number, or aggregate data. The QUALIFY clause in Databricks SQL is a massive time-saver that allows you to filter the results of those window functions directly without needing to wrap your query in a clunky subquery or Common Table Expression (CTE). Page Contents Introduction What is the QUALIFY Clause in Databricks SQL? Databricks QUALIFY Clause Syntax Using QUALIFY with a Window Function in the SELECT List Using QUALIFY with a Window Function Directly QUALIFY…

0 Comments
July 22, 2026
GCP BigQuery

How to Call BigQuery Stored Procedure from Cloud Composer?

Automating your data warehouse workflows in Google Cloud is critical for scaling your analytics. If you want to orchestrate complex SQL logic seamlessly, knowing how to call a BigQuery stored procedure from Cloud Composer is an essential skill for any data engineer. Here is a complete, step-by-step guide and Airflow DAG example to get your pipelines running smoothly. Page Content Introduction Step 1: Create the BigQuery Stored Procedure Step 2: Write the Cloud Composer (Airflow) DAG Best Practices for Orchestrating BigQuery Conclusion Introduction Google Cloud offers several ways to orchestrate…

0 Comments
July 7, 2026
Databricks

Databricks Recursive CTE Example: Querying Hierarchical Data

If you have ever tried to query hierarchical data in Databricks like an employee organizational chart, a bill of materials, or category trees you know it used to be a headache. Historically, data engineers had to write complex Python loops in PySpark to walk through parent-child relationships step-by-step. Thankfully, that is no longer necessary. Databricks now natively supports recursive Common Table Expressions (CTEs). Using the WITH RECURSIVE clause, you can easily query hierarchies and graph data directly in Databricks SQL. Let’s break down exactly how recursive CTEs work and look…

0 Comments
July 6, 2026
Databricks

Databricks vs Snowflake: Which is Right for You in 2026?

Choosing between Databricks and Snowflake used to be a simple decision, but massive platform updates in 2026 have completely blurred the lines between the two. With both platforms aggressively expanding into each other's territory, from AI workloads to business intelligence, picking the right one requires a deep understanding of their modern architectures. Here is a complete, up-to-date breakdown of how they compare on performance, pricing, and ideal use cases. Page Contents Introduction What is Snowflake? (The Cloud Data Warehouse) What is Databricks? (The Data Lakehouse) Core Differences: Architecture and Pricing…

0 Comments
July 6, 2026
Databricks

Databricks Common Table Expressions (CTEs): A Simple Guide

If you read our recent guide on Databricks SQL subqueries, you know that nesting queries inside one another can get messy very quickly. If you nest them three or four levels deep, your code becomes incredibly hard to read and debug. This is where Databricks Common Table Expressions (CTEs) save the day. CTEs are the secret to writing clean, professional, and easily maintainable SQL. In this guide, we will look at what CTEs are, why you should use them over subqueries, and how to write them in Databricks. Page Content…

0 Comments
July 6, 2026
Databricks

Databricks SQL Subqueries: Examples and Best Practices

If you write SQL in Databricks, you are eventually going to need a subquery. Whether you are trying to filter data based on another table, calculate a moving average, or just organize a massive query into readable chunks, subqueries are an essential tool for any data engineer. In this guide, we will break down exactly how subqueries work in Databricks SQL, look at the specific types supported, and walk through practical examples you can use right away. Page Content Introduction What is a Subquery in Databricks? Types of subqueries in…

0 Comments
July 6, 2026
Databricks

Databricks Temporary Tables Support

If you have been waiting for Databricks to add temporary tables, good news—they are finally here in Databricks Runtime 16.4. If you are migrating from databases like Snowflake, Oracle, or Netezza, you already know how incredibly useful these are. Let's break down how to create Databricks temporary tables, why you should use them, and how they actually differ from temporary views. Page Contents Introduction What Are Temporary Tables? Why Use Temporary Tables in Databricks? Temporary Tables vs Temporary Views Databricks Temporary Tables Examples Conclusion Introduction With the release of Databricks Runtime…

Comments Off on Databricks Temporary Tables Support
June 23, 2025
Databricks

Lateral Column Alias in Databricks – Example

If you have ever written a complex SQL query, you have probably run into this frustrating problem: you calculate a new column using an alias, but when you try to use that alias again in the exact same SELECT statement, the database throws an error. Historically, the only way around this was to write a subquery or a Common Table Expression (CTE). That is no longer the case. The lateral column alias in Databricks allows you to reuse an expression specified earlier in the same SELECT list. This is a…

Comments Off on Lateral Column Alias in Databricks – Example
September 24, 2024
Snowflake

Database Migration to Snowflake: Best Practices and Tips

Moving your legacy data warehouse to the Snowflake Data Cloud is one of the best decisions you can make for performance, scalability, and cost-efficiency. However, migrating from traditional systems like Oracle, Netezza, Teradata, or SQL Server is rarely a simple "lift and shift" operation. Without a solid strategy, you risk data loss, security gaps, and unexpected cloud computing costs. In this guide, we will break down the essential steps for a successful Snowflake migration and the best practices you need to follow to optimize your new environment. Page Content Introduction…

Comments Off on Database Migration to Snowflake: Best Practices and Tips
April 20, 2023
GCP BigQuery

Reuse Column Aliases in BigQuery – Lateral Column alias

BigQuery lateral Column alias are columns that are derived from the previously computed columns in same SELECT statement. Derived columns or lateral Column alias are virtual columns that are not physically stored in the table. Their values are re-calculated every time they are referenced in a query. Many PostgreSQL relational databases such as Netezza supports reuse of the column aliases within the same SELECT statement but GCP BigQuery does not support reuse of calculated derived columns. In this article, we will identify the alternate methods reuse column aliases in Google…

Comments Off on Reuse Column Aliases in BigQuery – Lateral Column alias
April 18, 2023