General posts

General

How to Connect to Databricks SQL Endpoint from Azure Data Factory?

A Databricks SQL Endpoint is a compute cluster, quite similar to the cluster we have known in the Databricks that allows you execute SQL commands on data objects within the Databricks environment. The Databricks allows you to connect using various tools such as DBT, connect to Notebook using Azure Data Factory, etc. But there is no direct method to connect Databricks SQL endpoint warehouse. In this article, we will check how to connect to Databricks SQL endpoint from Azure Data Factory (ADF). Connect to Databricks SQL Endpoint from Azure Data…

Continue ReadingHow to Connect to Databricks SQL Endpoint from Azure Data Factory?
Comments Off on How to Connect to Databricks SQL Endpoint from Azure Data Factory?
January 27, 2023
General

How to Export SQL Server Table to S3 using Spark?

Apache Spark is one of the emerging Bigdata technology. Due to its in memory distributed and fast computation, you can use it to perform heavy jobs such as analyzing petabytes of data or export millions or billions of records from any relational database to cloud storage such as Amazon S3, Azure Blob or Google cloud storage. In this article, we will check how to export SQL Server table to the Amazon cloud S3 bucket using Spark. We will use PySpark to demonstrate the method. In my other article, we have…

Continue ReadingHow to Export SQL Server Table to S3 using Spark?
Comments Off on How to Export SQL Server Table to S3 using Spark?
January 19, 2023
General

Connect to SQL Server From Spark – PySpark

Due to its in memory distributed and fast computation, Apache Spark is one of the emerging Bigdata technology. Apache Spark in memory distributed computation allows you to analyze petabytes of data without any performance issue. In this article, we will check one of methods to connect SQL Server database from Spark program. Preferably, we will use PySpark to read SQL Server table. Connection method is similar to that have already discussed for Oracle, Netezza, Snowflake, Teradata, etc. Steps to Connect SQL Server From Spark To access SQL Server from Apache…

Continue ReadingConnect to SQL Server From Spark – PySpark
Comments Off on Connect to SQL Server From Spark – PySpark
January 19, 2023
General

DBT – Export Snowflake Table to S3 Bucket

dbt stands for data build tool is a data transformation tool that enables data analysts and engineers to transform data in a cloud analytics warehouse. dbt basically focuses on the Transformation part in ELT (Extract, Load, Transform) processes. It supports cloud data warehouses such as Snowflake, Redshift, etc. In this article, we will check how to export your Snowflake table to S3 bucket from dbt. DBT - Export Snowflake Table to S3 Bucket What is dbt? Before going into details on exporting Snowflake table to S3 bucket using DBT, let us…

Continue ReadingDBT – Export Snowflake Table to S3 Bucket
Comments Off on DBT – Export Snowflake Table to S3 Bucket
September 30, 2022
General

QUALIFY Clause in Oracle – Alternative

If you are migrating from Teradata to Oracle or Redshift, you will notice that QUALIFY clause is not available. The feature such as QUALIFY clause is widely used in Teradata. But, the same feature is not available in Oracle. In this article, we will check what is QUALIFY Clause alternative in Oracle. QUALIFY Clause in Oracle The relational databases such as Teradata uses QUALIFY clause to filter the result of ordered analytical function. The Oracle database does not support this feature yet. But, when you are migrating Teradata scripts to…

Continue ReadingQUALIFY Clause in Oracle – Alternative
Comments Off on QUALIFY Clause in Oracle – Alternative
May 19, 2022
General

How to Generate Teradata Object DDL? – Tables,Views,UDF,SP,Macros

Teradata database is one of the widely used relational databases. The Teradata system is available on premises and cloud version. Teradata supports many useful tools such as Fast Export, Fast Load, BTEQ, etc. But, there are no tools to pull object DDL's from the Teradata database. In this article, we will check one of the administrative tasks, How to Generate Teradata Object DDL using System Tables and show command. We will generate DDL's for Teradata objects such as tables, views, stored procedures, macros and user defined functions. How to Generate…

Continue ReadingHow to Generate Teradata Object DDL? – Tables,Views,UDF,SP,Macros
Comments Off on How to Generate Teradata Object DDL? – Tables,Views,UDF,SP,Macros
October 8, 2021
General

How to Duplicate or Clone SQL Tables – Methods

In an application development, there may be situations where you need to create a similar table of the table which is already present in the database. In other words, you need to create a duplicate or clone of the existing table. The methods to duplicate or clone SQL table vary from database to database. For example, data warehouse such as Snowflake, Redshift provide methods which are not present in any other databases. In this article, we will check basic methods that you can use to clone or duplicate SQL tables…

Continue ReadingHow to Duplicate or Clone SQL Tables – Methods
Comments Off on How to Duplicate or Clone SQL Tables – Methods
April 15, 2021
General

Commonly used Teradata BTEQ commands and Examples

The BTEQ stands for Basic Teradata Query is one of the commonly used tools to interact with Teradata server. BTEQ tool was the original way that SQL was submitted to Teradata as a means of getting an answer in a desired format. In this article, we will check commonly used Teradata BTEQ commands with some examples. Commonly used Teradata BTEQ commands Following BTEQ commands are commonly used when writing scripts. BTEQ CommandDescription=Repeats the previous Teradata SQL request a specified number of times.AUTOKEYRETRIEVEThis control enables users to specify whether the database…

Continue ReadingCommonly used Teradata BTEQ commands and Examples
Comments Off on Commonly used Teradata BTEQ commands and Examples
December 24, 2020
General

Teradata Extract Function Syntax and Example

The extract function in Teradata extracts date or time part of the date, time, timestamp, or interval fields. The extract function retrieves the sub part such as day, month, year, etc. This extract function is equivalent to date_part() function in other relational databases. In this article, we will check how to use an extract function to get specified subpart. The extract function also converts the extracted value exact numeric format. The output is always a numeric value. Teradata Extract Function In a data warehouse, you will be working with many…

Continue ReadingTeradata Extract Function Syntax and Example
Comments Off on Teradata Extract Function Syntax and Example
October 22, 2020
General

What is isnull Alternative in Teradata SQL?

The relational databases such as SQL server supports isnull function. It returns the alternate value if input argument or expression is null. But, Teradata does not support isnull function. Instead, you have to use an alternate method to replace null values. In this article, we will check what is isnull alternative in Teradata SQL with some examples. isnull Function Alternative in Teradata SQL As mentioned earlier, Teradata does not support isnull function. There are many alternative methods that you can use as an alternative to isnull function. The SQL Server ISNULL() function…

Continue ReadingWhat is isnull Alternative in Teradata SQL?
Comments Off on What is isnull Alternative in Teradata SQL?
September 9, 2020