Database Migration to Snowflake: Best Practices and Tips

The Snowflake cloud data warehouse has become widely recognized as a flexible, high-performing, and scalable solution for both data warehousing and analytics needs. This article will explore how to migrate a database to Snowflake cloud data warehouse and also provide insights into some best practices for the migration. Page Content Introduction Preparing for Migration Migrating to Snowflake Best Practices for Database Migration to Snowflake Best Practices for File Sizing and Format Best Practices for Data Transfer Best Practices for Running Source and Snowflake Databases Best Practices for Temporary and Transient…

Continue ReadingDatabase Migration to Snowflake: Best Practices and Tips
Comments Off on Database Migration to Snowflake: Best Practices and Tips

Data Warehouse and Data Lake – Definition and differences

You will hear a lot about data warehouse and data lake when you work on Big Data. Both are widely used for storing Big Data but, they are not interchangeable. In this article, we will check data warehouse and data lake, its definition and differences. Data Warehouse and Data Lake As mentioned earlier, both are used for storing big data. But, they server different purpose when it comes to data usage. Data Warehouse A Data warehouse is an electronic storage of business data for analysis. It is a technique for…

Continue ReadingData Warehouse and Data Lake – Definition and differences
Comments Off on Data Warehouse and Data Lake – Definition and differences

Database Table Denormalization Example

Bigdata technologies such as Hive, HBase, NoSQL taking over industry, thanks to its fast and distributed processing. Hadoop works on commodity hardware, so it is cheap too. Every organization wants to move its data to Bigdata world. If you are reading this article, your organization may be planning to migrate your relational database to Hadoop. Hadoop works best with denormalized tables. In this article, we will check how database Table denormalization works with an example. What is Table Denormalization? Before jumping into denormalization process, let us first understand what is…

Continue ReadingDatabase Table Denormalization Example
Comments Off on Database Table Denormalization Example

Data Vault Modeling Methodology Architecture

Based on what you are working and expected results, you have to use different methodologies and best practices. A data warehouse is no different, you have to use different modeling methodologies based on the type of source data and integration. Big data is a hot cake now, everybody wants to move their data to bigdata world. Traditional methods such as Kibmal’s Star schema and Inmon’s relational 3NF may not work. You have to choose a different approach based on your ecosystem and data. In this article, we will check new…

Continue ReadingData Vault Modeling Methodology Architecture
Comments Off on Data Vault Modeling Methodology Architecture

Data Warehouse fact-less fact Tables and Examples

A Data Warehouse fact-less fact table is a fact that does not have any measures stored in it. This table will only contain keys from different dimension tables. The fact-less fact is often used to resolve a many-to-many cardinality issue. Types of Fact-less fact tables in Data Warehouse? There are two types of fact-less fact tables Event capturing fact-less fact This type of fact table establishes the relationship among the various dimension members from various dimension tables without any measured value. For examples, Student attendance (student-teacher relation table) capturing table…

Continue ReadingData Warehouse fact-less fact Tables and Examples
2 Comments

Data Warehouse Two-tier Architecture in Details

Usually, data warehouse adapts either two-tier or three-tier architecture. We have discussed three-tire architecture in my other post 'Data Warehouse Three-tier Architecture'. In this article, we will discuss on the data warehouse two-tier architecture. Data Warehouse Two-tier Architecture  The data warehouse two-tier architecture is a client - server application. There is a direct communication between client and data source server, we call it as data layer or database layer. Usually, there is no intermediate application between client and database layer. Below diagram depicts data warehouse two-tier architecture: As shown in…

Continue ReadingData Warehouse Two-tier Architecture in Details
Comments Off on Data Warehouse Two-tier Architecture in Details

Data Warehouse Surrogate Key Design – Advantages and Disadvantages

If you are working on Data warehouse project, than you might have heard lot about surrogate keys. Surrogate keys are widely accepted data warehouse design standard. In this article, we will check data warehouse surrogate key design, advantages and disadvantages. What are surrogate keys in Data warehouse? If you are a data warehouse developer, that you might be thinking what is surrogate key? How and where it is being used? You will get answers to all your questions here. Data warehouse surrogate keys are sequentially generated meaningless numbers associated with…

Continue ReadingData Warehouse Surrogate Key Design – Advantages and Disadvantages
2 Comments

Data Warehouse Project Life Cycle and Design

Building data warehouse is not different than executing other development project such as front-end application. You need to be technical and business person who understand technical details along with organizations business to successfully design and implement data warehouse project. In this article, we will check what the data warehouse project life cycle is and different steps in designing data warehouse project! Steps of Data Warehouse Project Life Cycle Design Following are steps generally followed in any data warehouse projects you can consider these steps as data warehouse lifecycle: Requirements gathering…

Continue ReadingData Warehouse Project Life Cycle and Design
Comments Off on Data Warehouse Project Life Cycle and Design

Data Warehouse Three-tier Architecture in Details

Usually, data warehouse adapts the three-tier architecture. In this article, we will discuss on the data warehouse three-tier architecture. You can read about read about two-tier architecture in my other post 'Data Warehouse Two-tier architecture in details' Data Warehouse Three-tier Architecture Following are the three-tiers of data warehouse architecture: Bottom Tier The bottom tier of the architecture is the data warehouse database server. It is usually the relational database (RDBMS) system. Data from operational databases and external sources are extracted using application program interfaces and ETL/ELT utilities. You generally use…

Continue ReadingData Warehouse Three-tier Architecture in Details
Comments Off on Data Warehouse Three-tier Architecture in Details

Data Warehouse Physical Data Model

The Data warehouse physical data model describes how the model will be built in the database. A physical database model shows all table structures, including column name, column data type, column constraints, primary key, foreign key, and relationships between tables. Read: Data Warehouse Project Life Cycle and Design Star Schema model in Data Warehouse Snowflake Schem Model in Data Warehouse Step by Step Guide to Dimensional Data Modeling Slowly Changing Dimensions (SCD) in Data Warehouse Rapidly Changing Dimension in Data Warehouse Features of Data Warehouse Physical Data Model Following are…

Continue ReadingData Warehouse Physical Data Model
Comments Off on Data Warehouse Physical Data Model