Netezza Hadoop Connector and its Usage

  • Post author:
  • Post last modified:February 28, 2018
  • Post category:Netezza
  • Reading time:3 mins read

Netezza Hadoop connector for Sqoop is an implementation of the Sqoop connector interfaces for accessing a Netezza data warehouse appliance from Hadoop cluster. Yom can export and import he data to a Hadoop cluster from various Netezza data warehouse environment.

netezza hadoop connector

Netezza Hadoop connector is designed to use Netezza high-throughput data-transfer mechanisms to import and export data to Hadoop HDFS. This Connector for Netezza is a standard Sqoop extension that allows Sqoop to inter operate with Netezza Data warehouse appliance through Netezza JDBC drivers.

This connector is already Cloudera Hadoop distribution (CDH). For other distributions, Copy the Netezza JDBC version 3.0 driver (nzjdbc3.jar) into the /var/lib/sqoop directory of the Sqoop installation. You can download this driver from the Netezza Client software distribution central. Without this driver, the sqoop extension for this connector will not function correctly.

Install Netezza Hadoop Connector

The Hadoop 2 Sqoop distribution includes Netezza connector software. You can also download the Netezza sqoop connector from Cloudera Website if you have CDH.

To deploy it, the only requirement is that you acquire the JDBC jar file (named nzjdbc.jar) from IBM and copy it to the /usr/local/nz/lib directory.

Netezza Hadoop Connector Connection String

The connection string must be of the form:

jdbc:netezza://<nzhost >/<nz-db>

Where:

  • <nzhost>: hostname or IP address of the Netezza Server.
  • <nz-db> : Netezza database name.

To use the Netezza connector, you must specify the –direct option along with a number of mappers greater than one.

Import Data using Sqoop Import

Import data to HDFS from Netezza server.

For example, the following command invokes the Sqoop import tool with four mappers:

$ sqoop import --connect jdbc:netezza://Netezza123/TEST --username Vithal \
--password xxxxx --direct --table TEST_TABLE --num-mappers 4 --escaped-by '\\' \
--fields-terminated-by '~'

Read:

Export Data using Sqoop Export

Export data to Netezza server from HDFS.

For example, the following command invokes the Sqoop export tool with four mappers:

$ sqoop export --connect jdbc:netezza://Netezza123/TEST --username Vithal \
--password xxxxx --direct --export-dir /user/arvind/MY_TABLE --table MY_TABLE_TARGET \
--num-mappers 8 --input-escaped-by '\\'

Read:

Install Netezza Hadoop Connector Limitations

If you are using the Cloudera Hadoop distribution (CDH) Netezza connector then there are several limitations

  • Direct mode does not support import into HBase.
  • This connector does not support data type TIME.
  • The Direct mode does not support export using updates.
  • Direct mode does not support import-all-tables.
  • The Direct mode does not support imports from views or materialized views.
  • Direct mode does not support import into or export from Avro.
Tags: , , , ,