Databricks Archives - DWgeek.com

Databricks Recursive CTE Example: Querying Hierarchical Data

If you have ever tried to query hierarchical data in Databricks like an employee organizational chart, a bill of materials, or category trees you know it used to be a headache. Historically, data engineers had to write complex Python loops in PySpark to walk through parent-child relationships step-by-step. Thankfully, that is no longer necessary. Databricks now natively supports recursive Common Table Expressions (CTEs). Using the WITH RECURSIVE clause, you can easily query hierarchies and graph data directly in Databricks SQL. Let’s break down exactly how recursive CTEs work and look…

0 Comments

July 6, 2026

Databricks

Databricks vs Snowflake: Which is Right for You in 2026?

Choosing between Databricks and Snowflake used to be a simple decision, but massive platform updates in 2026 have completely blurred the lines between the two. With both platforms aggressively expanding into each other's territory, from AI workloads to business intelligence, picking the right one requires a deep understanding of their modern architectures. Here is a complete, up-to-date breakdown of how they compare on performance, pricing, and ideal use cases. Page Contents Introduction What is Snowflake? (The Cloud Data Warehouse) What is Databricks? (The Data Lakehouse) Core Differences: Architecture and Pricing…

0 Comments

July 6, 2026

Databricks

Databricks Common Table Expressions (CTEs): A Simple Guide

If you read our recent guide on Databricks SQL subqueries, you know that nesting queries inside one another can get messy very quickly. If you nest them three or four levels deep, your code becomes incredibly hard to read and debug. This is where Databricks Common Table Expressions (CTEs) save the day. CTEs are the secret to writing clean, professional, and easily maintainable SQL. In this guide, we will look at what CTEs are, why you should use them over subqueries, and how to write them in Databricks. Page Content…

0 Comments

July 6, 2026

Databricks

Databricks SQL Subqueries: Examples and Best Practices

If you write SQL in Databricks, you are eventually going to need a subquery. Whether you are trying to filter data based on another table, calculate a moving average, or just organize a massive query into readable chunks, subqueries are an essential tool for any data engineer. In this guide, we will break down exactly how subqueries work in Databricks SQL, look at the specific types supported, and walk through practical examples you can use right away. Page Content Introduction What is a Subquery in Databricks? Types of subqueries in…

0 Comments

July 6, 2026

Databricks

Databricks Temporary Tables Support

If you have been waiting for Databricks to add temporary tables, good news—they are finally here in Databricks Runtime 16.4. If you are migrating from databases like Snowflake, Oracle, or Netezza, you already know how incredibly useful these are. Let's break down how to create Databricks temporary tables, why you should use them, and how they actually differ from temporary views. Page Contents Introduction What Are Temporary Tables? Why Use Temporary Tables in Databricks? Temporary Tables vs Temporary Views Databricks Temporary Tables Examples Conclusion Introduction With the release of Databricks Runtime…

Comments Off

June 23, 2025

Databricks

Lateral Column Alias in Databricks – Example

If you have ever written a complex SQL query, you have probably run into this frustrating problem: you calculate a new column using an alias, but when you try to use that alias again in the exact same SELECT statement, the database throws an error. Historically, the only way around this was to write a subquery or a Common Table Expression (CTE). That is no longer the case. The lateral column alias in Databricks allows you to reuse an expression specified earlier in the same SELECT list. This is a…

Comments Off

September 24, 2024