Different Methods to Export Netezza Data into Amazon S3 Bucket

  • Post author:
  • Post last modified:January 19, 2023
  • Post category:Netezza
  • Reading time:6 mins read

Now a days, Amazon AWS is gaining popularity as cloud based web services. It will just take few clicks to make your system or storage up and running. In this article, we will check how to integrate Netezza and Amazon S3. We will also check how to export Netezza data into Amazon S3 bucket using Amazon web services command line interface (aws cli) with an example.

Export Netezza Data into Amazon S3 Bucket

You may also interested in load data from Amazon S3 to Netezza table:

We will be using Amazon AWS CLI to export Netezza data to S3 bucket. 

Install Amazon Web Services Command Line Interface – AWS CLI

The AWS Command Line Interface (AWS CLI) is a unified tool that provides a consistent interface for interacting with AWS. You can use AWS CLI commands to interact with S3 bucket. Follow below link to install AWS CLI on Linux system.

Once you have AWS CLI installed and configured with your access and secret key, next step would be to export Netezza data to Amazon S3.

Export Netezza Data into Amazon S3 Bucket

There are three ways you can upload Netezza data to Amazon S3:

  • Unload Netezza Data to Flat file and upload to S3 using AWS CLI
  • Uploading a Netezza standard output stream to S3
  • Unload Netezza Data to Flat file and upload to S3 using web console – Drag and drop

Unload Netezza Data to Flat file and upload to S3 using AWS CLI

To use this method, you have to export Netezza table data to a flat file and upload that flat file to Amazon S3 using AWS CLI commands.

You can follow below article to export data to CSV file:

You can either copy folder containing Netezza data to S3 bucket or simply sync the folder to copy only changed and new files.

Upload Data Folder or File from Local to S3

You can perform recursive uploads and downloads of multiple files in a single folder-level command. To upload the Netezza unloaded files from Local folder to the Amazon S3 use following command

vithal@vithal-Inspiron-3558:~$ aws s3 cp nz_data.txt s3://testnetezzabucket/upload/
upload: ./nz_data.txt to s3://testnetezzabucket/upload/nz_data.txt

vithal@vithal-Inspiron-3558:~$ aws s3 ls s3://testnetezzabucket/upload/
2017-11-14 09:08:31 0 
2017-11-14 18:05:04 506 nz_data.txt
vithal@vithal-Inspiron-3558:~$

Sync Local Folder with Amazon S3 using AWS CLI

A sync command makes it easy to synchronize the contents of a local folder with a copy in an S3 bucket.

aws s3 sync upload s3://testnetezzabucket/upload/ --exclude *.tmp

Uploading a Netezza standard output Stream Data to S3

Following combination of nzsql command and AWS CLI will upload Netezza stream data into Amazon S3.

vithal@vithal-Inspiron-3558:~$ nzsql -host 192.168.0.106 -u admin -pw password -db training -F ',' -A -t -c "select * from patient_1" | aws s3 cp - s3://testnetezzabucket/upload/test.txt

vithal@vithal-Inspiron-3558:~$ aws s3 ls s3://testnetezzabucket/upload/
2017-11-14 09:08:31 0 
2017-11-14 18:05:04 506 nz_data.txt
2017-11-14 17:57:32 42 test.txt
vithal@vithal-Inspiron-3558:~$

Unload Netezza Data to Flat file and upload to S3 using AWS Management Console – Drag and Drop

The final and easy step is to drag and drop the Netezza unloaded data file into Amazon S3 bucket using web console.

You have to login using AWS credentials and just upload or drag and drop the file that you want to upload to S3 bucket.

Export Netezza Data to Amazon S3 Bucket Methods and Examples

Related  Article,

Hope it helps 🙂

Tags: ,