DWgeek.com

How to Find Snowflake Table Size? – Query and Examples

Snowflake is a fully managed cloud data warehouse solution. It is fast, reliable and comes with near zero management. Similar to many relational databases, Snowflake support many system tables and views. System tables can store the important information such as records count, constraint details, etc. In this article, we will check how to find the Snowflake table size using system tables present in the information schema. How to Find Snowflake Table Size? As mentioned earlier, Snowflake maintains the important information in system tables. One of such view is TABLE_STORAGE_METRICS view.…

Comments Off

February 17, 2020

General

Connect Teradata using Python pyodbc – Steps and example

Teradata is one of the widely used MPP databases. Teradata provides many connectors such as its own Python connector Teradata, Teradatasql, etc. You can use those to interact with the Teradata database. There are many different options to connect to Teradata. Teradata provides support for JDBC and ODBC drivers. In this article, we will check method on connect Teradata using the Python pyodbc module and odbc driver with a working example. Teradata ODBC Driver Being a popular MPP database, Teradata database server comes with ODBC support. Before attempting to connect…

Comments Off

February 12, 2020

Snowflake

Optimize Snowflake Table Structure to Improve Performance

The performance of the Snowflake cloud data warehouse is directly dependent on the optimal table structure and design. Optimizing Snowflake table structure is one of the important aspect to improve the performance of query, data loading and unloading process. In this article, we will check how to optimize the Snowflake table structure to improve query performance. Optimize Snowflake Table Structure There are no specific best practices that you can apply to optimize the table structure. Creating optimal table structure that uses right data type and length is one of the…

Comments Off

February 6, 2020

Snowflake

Snowflake Transient Tables, Usage and Examples

Snowflake Transient tables are similar to permanent tables with the key difference that they do not have a Fail-safe period. The transient tables are similar to temporary tables, but, you have to explicitly drop transient tables at the end of the session. Snowflake Transient Tables Snowflake transient tables persist until explicitly dropped and are available to all users with the appropriate privileges. The transient tables are designed for transitory data that needs to be maintained beyond the current session. Because transient tables do not have a Fail-safe period, they provide…

Comments Off

February 5, 2020

Snowflake

Snowflake Temporary Tables, Usage and Examples

Similar to other relational databases, Snowflake support creating temp or temporary tables to hold non-permanent data. i.e. Data which is used in the current session. In this article, we will check how to create Snowflake temp tables, syntax, usage and restrictions with some examples. Snowflake Temporary Tables The temporary table in Snowflake is visible only within the current session. Temporary tables only exist within the session in which they were created and persist only for the remainder of the session. Once the session ends, the system will purge the data…

Comments Off

February 5, 2020

Snowflake

Snowflake Load Local CSV File using COPY and Example

Many organizations use flat files such as CSV or TSV files to offload large tables. Managing flat files such as CSV is easy and it can be transported by any electronic medium. In this article, we will check how to load or import local CSV file into Snowflake using COPY command with some examples. Load Local CSV File using Snowflake COPY Command There are a couple of methods that you can use to load a csv file present in your local system. Following are the methods. Use SnowSQL command Line…

Comments Off

February 4, 2020

Snowflake

Snowflake Type of Subqueries and Examples

In general, subquery in a database is a nested query block in a query statement. It is simply a SELECT expression enclosed in a parenthesis. The Subquery may return zero to one or more values to its upper or parent SELECT statements. In this article, we will check Snowflake type of subqueries with an examples. Snowflake Subqueries A subquery in Snowflake is a nested select statement, that return zero or more records to is upper select statement. The outer SELECT statement that contains subquery is sometimes referred to as a…

Comments Off

February 4, 2020

Snowflake

Snowflake Fixed-Width File Loading Options and Examples

Fixed width text files are special cases of text files where the format is specified by column widths, pad character and left or right alignment. Many telecom companies use fixed-width file to store call detail records (CDR) data. In this format, column width are in terms of units of characters. In this article, we will learn about Snowflake Fixed-Width file loading options and examples. Snowflake Fixed-Width File Loading The fixed-width data files have uniform lengths for each column of data. Each field in a fixed-width data file has exactly the…

Comments Off

February 4, 2020

Apache Spark

Spark SQL CASE WHEN on DataFrame – Examples

In general, the CASE expression or command is a conditional expression, similar to if-then-else statements found in other languages. Spark SQL supports almost all features that are available in Apace Hive. One of such a features is CASE statement. In this article, how to use CASE WHEN and OTHERWISE statement on a Spark SQL DataFrame. Spark SQL CASE WHEN on DataFrame The CASE WHEN and OTHERWISE function or statement tests whether any of a sequence of expressions is true, and returns a corresponding result for the first true expression. Spark…

Comments Off

February 4, 2020

Apache Spark

Import CSV file to Pyspark DataFrame – Example

Many organization uses a flat file format such as CSV or TSV to offload their tables. Managing flat file is easy and can be transported by any electronic medium. In this article we will check how to import CSV file to Pyspark DataFrame with some examples. Import CSV file to Pyspark DataFrame There are many methods that you can use to import CSV file into pyspark or Spark DataFrame. But, the following methods are easy to use. Read Local CSV using com.databricks.spark.csv FormatRun Spark SQL Query to Create Spark DataFrame…

Comments Off

February 3, 2020