sqoop

Sqoop is a tool designed to transfer data between Hadoop and relational databases. You can use Sqoop to import data from a relational database management system (RDBMS) such as MySQL or Oracle into the Hadoop Distributed File System (HDFS), transform the data in Hadoop MapReduce, and then export the data back into an RDBMS

$ sqoop list-databases –connect jdbc:postgresql://phdmst:5432 –username gpadmin –password  xxxxxx
15/06/05 14:19:29 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
15/06/05 14:19:29 INFO manager.SqlManager: Using default fetchSize of 1000
template0
template1
postgres
gpadmin
ovi
RCDB

-bash-4.1$ sqoop list-databases –connect jdbc:postgresql://uphdmst02.uat.mydev.com:5432 –username gpadmin –password xxxxxx
15/08/04 14:20:05 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
15/08/04 14:20:05 INFO manager.SqlManager: Using default fetchSize of 1000
template0
template1
postgres
gpadmin
CCDM
testcmri
benchmark
testovi
ITS

Netezza Connector

Netezza connector for Sqoop is an implementation of the Sqoop connector interfaces for accessing a Netezza data warehouse appliance, so that data can be exported and imported to a Hadoop environment from Netezza data warehousing environments.

The HDP 2 Sqoop distribution includes Netezza connector software. To deploy it, the only requirement is that you acquire the JDBC jar file (named nzjdbc.jar) from IBM and copy it to the /usr/local/nz/lib directory.

Here is an example of a complete command line for import using the Netezza external table feature:

$ sqoop import \
 --direct \
 --connect jdbc:netezza://nzhost:5480/sqoop \
 --table nztable \
 --username nzuser \
 --password nzpass \
 --target-dir hdfsdir \
 -- --log-dir /tmp

Here is an example of a complete command line for export with tab (\t) as the field terminator character:

$ sqoop export \
 --direct \
 --connect jdbc:netezza://nzhost:5480/sqoop \
 --table nztable \
 --username nzuser \
 --password nzpass \
 --export-dir hdfsdir \
 --input-fields-terminated-by "\t"