General posts

General

How to Connect to Databricks SQL Endpoint from Azure Data Factory?

A Databricks SQL Endpoint is a compute cluster, quite similar to the cluster we have known in the Databricks that allows you execute SQL commands on data objects within the Databricks environment. The Databricks allows you to connect using various tools such as DBT, connect to Notebook using Azure Data Factory, etc. But there is no direct method to connect Databricks SQL endpoint warehouse. In this article, we will check how to connect to Databricks SQL endpoint from Azure Data Factory (ADF). Connect to Databricks SQL Endpoint from Azure Data…

Continue ReadingHow to Connect to Databricks SQL Endpoint from Azure Data Factory?
Comments Off on How to Connect to Databricks SQL Endpoint from Azure Data Factory?
January 27, 2023
General

How to Export SQL Server Table to S3 using Spark?

Apache Spark is one of the emerging Bigdata technology. Due to its in memory distributed and fast computation, you can use it to perform heavy jobs such as analyzing petabytes of data or export millions or billions of records from any relational database to cloud storage such as Amazon S3, Azure Blob or Google cloud storage. In this article, we will check how to export SQL Server table to the Amazon cloud S3 bucket using Spark. We will use PySpark to demonstrate the method. In my other article, we have…

Continue ReadingHow to Export SQL Server Table to S3 using Spark?
Comments Off on How to Export SQL Server Table to S3 using Spark?
January 19, 2023
General

Connect to SQL Server From Spark – PySpark

Due to its in memory distributed and fast computation, Apache Spark is one of the emerging Bigdata technology. Apache Spark in memory distributed computation allows you to analyze petabytes of data without any performance issue. In this article, we will check one of methods to connect SQL Server database from Spark program. Preferably, we will use PySpark to read SQL Server table. Connection method is similar to that have already discussed for Oracle, Netezza, Snowflake, Teradata, etc. Steps to Connect SQL Server From Spark To access SQL Server from Apache…

Continue ReadingConnect to SQL Server From Spark – PySpark
Comments Off on Connect to SQL Server From Spark – PySpark
January 19, 2023
General

DBT – Export Snowflake Table to S3 Bucket

Are you looking to move your data out of Snowflake and into an AWS S3 bucket using dbt? While dbt is built primarily for transforming data inside your warehouse, you can still use it to unload data with a simple workaround. This guide will show you exactly how to export a DBT Snowflake table to an S3 bucket using custom dbt macros. DBT - Export Snowflake Table to S3 Bucket Page Content Introduction Set Up a Snowflake External Stage Step1: Create Snowflake External Stage Step 2: Write a dbt Macro…

Continue ReadingDBT – Export Snowflake Table to S3 Bucket
Comments Off on DBT – Export Snowflake Table to S3 Bucket
September 30, 2022
General

QUALIFY Clause in Oracle: Native Example and Alternative

If you are moving your database from Teradata to Oracle, you might be searching for a QUALIFY Clause in Oracle. While older versions needed a workaround to filter ordered analytical functions, the highly anticipated QUALIFY clause has officially been introduced in Oracle 26. Below, we will check how to use this new feature and the best fallback method if you are still on an older Oracle version. Page Content Introduction QUALIFY Clause in Oracle 26 QUALIFY Clause Alternative in Oracle Older Versions Conclusion Introduction When writing SQL queries, filtering the…

Continue ReadingQUALIFY Clause in Oracle: Native Example and Alternative
Comments Off on QUALIFY Clause in Oracle: Native Example and Alternative
May 19, 2022
General

How to Generate Teradata Object DDL (Tables, Views, UDFs, SPs, & Macros)?

Extracting Data Definition Language (DDL) scripts from your Teradata environment doesn't have to be a frustrating manual process. Whether you are migrating databases, backing up schemas, or setting up version control, knowing how to quickly generate DDL for your objects is an essential database administration skill. Page Content Introduction Generating Table DDL in Teradata Extracting View DDL in Teradata Exporting Stored Procedure DDLs via BTEQ Generating User-Defined Function (UDF) DDLs Extracting Teradata Macro DDLs Conclusion Introduction The Teradata database remains one of the most powerful and widely used relational databases…

Continue ReadingHow to Generate Teradata Object DDL (Tables, Views, UDFs, SPs, & Macros)?
Comments Off on How to Generate Teradata Object DDL (Tables, Views, UDFs, SPs, & Macros)?
October 8, 2021
General

How to Duplicate or Clone SQL Tables – Methods

In an application development, there may be situations where you need to create a similar table of the table which is already present in the database. In other words, you need to create a duplicate or clone of the existing table. The methods to duplicate or clone SQL table vary from database to database. For example, data warehouse such as Snowflake, Redshift provide methods which are not present in any other databases. In this article, we will check basic methods that you can use to clone or duplicate SQL tables…

Continue ReadingHow to Duplicate or Clone SQL Tables – Methods
Comments Off on How to Duplicate or Clone SQL Tables – Methods
April 15, 2021
General

Commonly used Teradata BTEQ commands and Examples

The BTEQ stands for Basic Teradata Query is one of the commonly used tools to interact with Teradata server. BTEQ tool was the original way that SQL was submitted to Teradata as a means of getting an answer in a desired format. In this article, we will check commonly used Teradata BTEQ commands with some examples. Commonly used Teradata BTEQ commands Following BTEQ commands are commonly used when writing scripts. BTEQ CommandDescription=Repeats the previous Teradata SQL request a specified number of times.AUTOKEYRETRIEVEThis control enables users to specify whether the database…

Continue ReadingCommonly used Teradata BTEQ commands and Examples
Comments Off on Commonly used Teradata BTEQ commands and Examples
December 24, 2020
General

Teradata Extract Function Syntax and Example

The extract function in Teradata extracts date or time part of the date, time, timestamp, or interval fields. The extract function retrieves the sub part such as day, month, year, etc. This extract function is equivalent to date_part() function in other relational databases. In this article, we will check how to use an extract function to get specified subpart. The extract function also converts the extracted value exact numeric format. The output is always a numeric value. Teradata Extract Function In a data warehouse, you will be working with many…

Continue ReadingTeradata Extract Function Syntax and Example
Comments Off on Teradata Extract Function Syntax and Example
October 22, 2020
General

What is isnull Alternative in Teradata SQL?

The relational databases such as SQL server supports isnull function. It returns the alternate value if input argument or expression is null. But, Teradata does not support isnull function. Instead, you have to use an alternate method to replace null values. In this article, we will check what is isnull alternative in Teradata SQL with some examples. isnull Function Alternative in Teradata SQL As mentioned earlier, Teradata does not support isnull function. There are many alternative methods that you can use as an alternative to isnull function. The SQL Server ISNULL() function…

Continue ReadingWhat is isnull Alternative in Teradata SQL?
Comments Off on What is isnull Alternative in Teradata SQL?
September 9, 2020