Snowflake

How to Add an Identity Column to Existing Snowflake Table?

Because of its unique architecture, Snowflake does not support column constraints such as primary, foreign or unique key column. The only constraint supported is NOT NULL. But, many traditional database architects would like to keep primary keys and populate with unique key values. You can define the identity columns or create sequences and add sequence value as a default. But, you can't add an identity column to the existing table with data. In This article, we will check how to add an identity column to the existing Snowflake table. Add…

Continue ReadingHow to Add an Identity Column to Existing Snowflake Table?
Comments Off on How to Add an Identity Column to Existing Snowflake Table?
June 21, 2021
Snowflake

How to Create Parameterized Views in Snowflake?

A view allows the result of a query to be accessed as if it were a table. It will encapsulate the business logic by combining two or more tables. It can also refer other views. Views serve a many different purposes, including combining tables or views, segregating, and protecting the underlying data. For example, you can create separate views that meet the needs of different types of employees, such as doctors and nurses at a hospital. In this article, we will check how to create parameterized views in Snowflake. Though,…

Continue ReadingHow to Create Parameterized Views in Snowflake?
Comments Off on How to Create Parameterized Views in Snowflake?
June 17, 2021
Snowflake

How to Connect to Snowflake from Databricks?

Many organizations use the hybrid model to process the data. They use databricks to perform operations such as Machine Learning tasks and copy end results to Snowflake for reporting or further analysis. In this article, we will check how to connect to Snowflake from databricks to build hybrid architecture. Connect to Snowflake from Databricks The Snowflake is one of the relational databases that provide connector for Spark. You can use the Snowflake Spark connector to connect to Snowflake server and copy data from databricks to Snowflake. Test Data We will…

Continue ReadingHow to Connect to Snowflake from Databricks?
Comments Off on How to Connect to Snowflake from Databricks?
June 16, 2021
Snowflake

How to Update JSON field in Snowflake Variant Column?

The update is a common operation in any relational databases. A Snowflake, a leading cloud data warehouse supports some unique features such as built-in support for semi structured data. Snowflake support many built-in functions that allow you yo manipulate semi-structured data, such as JSON and XML data. Its universal data type VARIANT allows you to store semi-structured data including parquet. In this article, we will check how to update JSON field in Snowflake. Update or Replace JSON field in Snowflake Snowflake support functionalities that are present in almost all relational…

Continue ReadingHow to Update JSON field in Snowflake Variant Column?
Comments Off on How to Update JSON field in Snowflake Variant Column?
June 15, 2021
Apache Spark

Spark SQL Array Functions – Syntax and Examples

Similar to relational databases such as Snowflake, Teradata, Spark SQL support many useful array functions. You can use these array manipulation functions to manipulate the array types. In this article, we will check how to work with Spark SQL Array Functions its Syntax and Examples. Spark SQL Array Functions Following is the list of Spark SQL array functions with brief descriptions: Spark SQL Array FunctionDescriptionarray(expr, ...) Returns an array with the given elements.array_contains(array, value)Returns true if the array contains the value.array_distinct(array)This function removes duplicate values from the arrayarray_except(array1, array2)Returns an array…

Continue ReadingSpark SQL Array Functions – Syntax and Examples
Comments Off on Spark SQL Array Functions – Syntax and Examples
June 14, 2021
Apache Spark

How to Add Column with Default Value to Pyspark DataFrame?

Since the inception, Spark has made a lot of improvement and added many useful DataFrame API's. If you are from SQL background, you might have noticed that adding default value to a column when you add new column is a common practice. This is just to make sure the new column does not hold junk or NULL values. In this article, we will check how to add a column with a default or constant value to a Pyspark DataFrame. Add a Column with Default Value to Pyspark DataFrame Adding a…

Continue ReadingHow to Add Column with Default Value to Pyspark DataFrame?
Comments Off on How to Add Column with Default Value to Pyspark DataFrame?
June 11, 2021
Snowflake

How to combine two arrays in Snowflake?

Snowflake is the one of the databases that combines many useful functions from other relational databases such as Oracle, PostgreSQL, Teradata, etc. Like many other relational databases, Snowflake support many array functions. In this article, we will check how to combine/merge/concatenate two or more arrays in Snowflake. Combine Two Arrays in Snowflake Snowflake allows you to deal with many different kinds of data sets. For example, you can work with JSON, XML or array variables with an ease. As Snowflake integrates many heterogeneous data sets, you may get requirement such…

Continue ReadingHow to combine two arrays in Snowflake?
Comments Off on How to combine two arrays in Snowflake?
June 10, 2021
Snowflake

How to Get First Row of each Group in Snowflake?

In the data warehouse reporting, you will encounter many different scenarios. One of such scenario is to get first row of each group. For example, identify the department wise highest salary. In this article, we will check how to select or get first row of each group in Snowflake. Select First Record of each Group in Snowflake Selecting first row of each group in SQL is one of the common query in reporting. You need to use proper function that does not take much time to return results. Following are…

Continue ReadingHow to Get First Row of each Group in Snowflake?
Comments Off on How to Get First Row of each Group in Snowflake?
June 9, 2021
Apache Spark

PySpark – Search Table in Spark Database

In a real world scenario, you will be dealing with petabytes of data and thousands of tables in a hundred of databases within Spark or Hive catalog. It is practically, time consuming to identify the particular table in a database, hence it is always good idea to develop reusable code that you can use to search table(s) in a given database. The usability includes safely dropping table(s) and identify table structure. In this article, we will learn how to search table in a database using PySpark. Search Table in Database…

Continue ReadingPySpark – Search Table in Spark Database
Comments Off on PySpark – Search Table in Spark Database
June 8, 2021
Snowflake

Snowflake Concat Function and Operator – Examples

The Snowflake cloud architecture supports data ingestion from multiple sources, hence it is a common requirement to combine data from multiple columns to come up with required results. You may also get a requirement to concatenate multiple strings before loading them to target table. For example, you may get requirement to combine state and city columns before loading data to the customer address table. In this article, we will check Snowflake CONCAT function, its Syntax and examples. We will also check how to combine two or more columns using Snowflake…

Continue ReadingSnowflake Concat Function and Operator – Examples
Comments Off on Snowflake Concat Function and Operator – Examples
June 8, 2021