You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by "Sambit Tripathy (RBEI/PJ-NBS)" <Sa...@in.bosch.com> on 2014/09/26 00:06:28 UTC
Sqoop connector configuration
Hi,
I have recently upgraded my cluster to CDH 5.1 and Sqoop2.
My Sqoop 2 server is "sqoop-server-main"
1. On "sqoop-server-main" I have copied "ojdbc6.jar" and "mysql-connector-java-version-bin.jar" to /var/lib/sqoop2/
2. Restarted the Sqoop2 service
3. To verify I logged in to "sqoop-server-main" and typed "sqoop2" to get into the sqoop shell which worked.
4. Now to see the available connectors I typed "show connector --all"
I get
1 connector(s) to show:
Connector with id 1:
Name: generic-jdbc-connector
Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector
Version: 1.99.3-cdh5.1.0
I am not able to figure out what has gone wrong here! I should see both oracle and mysql connectors along with the generic-jdbc-driver.
Are there any config changes required? As I understand in Sqoop 2 the connectors/drivers should be present on the server in contrast with Sqoop 1
I always appreciate your suggestions and pointers.
Regards,
Sambit.
RE: Sqoop connector configuration
Posted by "Sambit Tripathy (RBEI/PJ-NBS)" <Sa...@in.bosch.com>.
Many thanks Gwen and Abey.
-----Original Message-----
From: Gwen Shapira [mailto:gshapira@cloudera.com]
Sent: Thursday, September 25, 2014 3:18 PM
To: user@sqoop.apache.org
Subject: Re: Sqoop connector configuration
Sqoop2 does not have specific connectors for Oracle and Netezza yet.
Its not you, its us :)
Suggestions:
1. Use Sqoop2 with the generic JDBC connector
2. Use Sqoop 1.4.X with the Oracle and Netezza connectors.
Gwen
On Thu, Sep 25, 2014 at 3:06 PM, Sambit Tripathy (RBEI/PJ-NBS)
<Sa...@in.bosch.com> wrote:
> Hi,
>
> I have recently upgraded my cluster to CDH 5.1 and Sqoop2.
>
> My Sqoop 2 server is “sqoop-server-main”
>
> 1. On “sqoop-server-main” I have copied “ojdbc6.jar” and
> “mysql-connector-java-version-bin.jar” to /var/lib/sqoop2/
>
> 2. Restarted the Sqoop2 service
>
> 3. To verify I logged in to “sqoop-server-main” and typed “sqoop2” to get
> into the sqoop shell which worked.
>
> 4. Now to see the available connectors I typed “show connector --all”
>
> I get
>
> 1 connector(s) to show:
> Connector with id 1:
> Name: generic-jdbc-connector
> Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector
> Version: 1.99.3-cdh5.1.0
>
>
> I am not able to figure out what has gone wrong here! I should see both
> oracle and mysql connectors along with the generic-jdbc-driver.
>
> Are there any config changes required? As I understand in Sqoop 2 the
> connectors/drivers should be present on the server in contrast with Sqoop 1
>
> I always appreciate your suggestions and pointers.
>
>
> Regards,
> Sambit.
>
>
Re: Sqoop connector configuration
Posted by Gwen Shapira <gs...@cloudera.com>.
Sqoop2 does not have specific connectors for Oracle and Netezza yet.
Its not you, its us :)
Suggestions:
1. Use Sqoop2 with the generic JDBC connector
2. Use Sqoop 1.4.X with the Oracle and Netezza connectors.
Gwen
On Thu, Sep 25, 2014 at 3:06 PM, Sambit Tripathy (RBEI/PJ-NBS)
<Sa...@in.bosch.com> wrote:
> Hi,
>
> I have recently upgraded my cluster to CDH 5.1 and Sqoop2.
>
> My Sqoop 2 server is “sqoop-server-main”
>
> 1. On “sqoop-server-main” I have copied “ojdbc6.jar” and
> “mysql-connector-java-version-bin.jar” to /var/lib/sqoop2/
>
> 2. Restarted the Sqoop2 service
>
> 3. To verify I logged in to “sqoop-server-main” and typed “sqoop2” to get
> into the sqoop shell which worked.
>
> 4. Now to see the available connectors I typed “show connector --all”
>
> I get
>
> 1 connector(s) to show:
> Connector with id 1:
> Name: generic-jdbc-connector
> Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector
> Version: 1.99.3-cdh5.1.0
>
>
> I am not able to figure out what has gone wrong here! I should see both
> oracle and mysql connectors along with the generic-jdbc-driver.
>
> Are there any config changes required? As I understand in Sqoop 2 the
> connectors/drivers should be present on the server in contrast with Sqoop 1
>
> I always appreciate your suggestions and pointers.
>
>
> Regards,
> Sambit.
>
>
Re: Sqoop connector configuration
Posted by Abraham Elmahrek <ab...@cloudera.com>.
I don't think Sqoop needs exec. priviliges on jars... just read permission.
On Thu, Sep 25, 2014 at 4:06 PM, Sambit Tripathy (RBEI/PJ-NBS) <
Sambit.Tripathy@in.bosch.com> wrote:
> One thing I could not find in the documentation and that is after doing
> step 1, should you be doing a chmod 755 on those jars? Usually root or
> sudoer copies the jars into /var/lib/sqoop2.
>
>
>
> -bash-4.1$ ls -la /var/lib/sqoop2/
>
> total 4756
>
> drwxr-xr-x 4 sqoop2 sqoop 160 Sep 23 00:27 .
>
> drwxr-xr-x 28 root root 600 Sep 16 05:48 ..
>
> -rw-r--r-- 1 root root 954041 Sep 10 07:51
> mysql-connector-java-5.1.30-bin.jar
>
> -rw-r--r-- 1 root root 2739616 Sep 10 07:51 ojdbc6.jar
>
> -rw-r--r-- 1 root root 588974 Sep 10 07:51
> postgresql-9.3-1101.jdbc4.jar
>
> drwxr-xr-x 3 sqoop2 sqoop 60 Sep 2 00:56 repository
>
> -rw-r--r-- 1 root root 584207 Sep 10 07:51 sqljdbc4.jar
>
> drwxr-xr-x 5 sqoop2 sqoop 100 Sep 23 00:27 tomcat-deployment
>
>
>
>
>
>
>
>
>
> *From:* Abraham Elmahrek [mailto:abe@cloudera.com]
> *Sent:* Thursday, September 25, 2014 3:18 PM
> *To:* user@sqoop.apache.org
> *Subject:* Re: Sqoop connector configuration
>
>
>
> Hey there,
>
>
>
> Sqoop2 is still under active and heavy development. The Generic JDBC
> connector will be able to connect to Oracle and MySQL, but the optimized
> connectors are not yet available.
>
>
>
> If you feel like using Sqoop2, try creating a job with the Generic JDBC
> Connector.
>
>
>
> If you feel like an optimized connector, then please use Sqoop1.
>
>
>
> -Abe
>
>
>
> On Thu, Sep 25, 2014 at 3:06 PM, Sambit Tripathy (RBEI/PJ-NBS) <
> Sambit.Tripathy@in.bosch.com> wrote:
>
> Hi,
>
>
>
> I have recently upgraded my cluster to CDH 5.1 and Sqoop2.
>
>
>
> My Sqoop 2 server is “sqoop-server-main”
>
>
>
> 1. On “sqoop-server-main” I have copied “ojdbc6.jar” and
> “mysql-connector-java-*version*-bin.jar” to */var/lib/sqoop2/*
>
>
>
> 2. Restarted the Sqoop2 service
>
>
>
> 3. To verify I logged in to “sqoop-server-main” and typed “sqoop2” to get
> into the sqoop shell which worked.
>
>
>
> 4. Now to see the available connectors I typed “show connector --all”
>
>
>
> I get
>
>
>
> *1 connector(s) to show:*
>
> Connector with id 1:
>
> Name: generic-jdbc-connector
>
> Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector
>
> Version: 1.99.3-cdh5.1.0
>
>
>
>
>
> I am not able to figure out what has gone wrong here! I should see both
> oracle and mysql connectors along with the generic-jdbc-driver.
>
>
>
> Are there any config changes required? As I understand in Sqoop 2 the
> connectors/drivers should be present on the server in contrast with Sqoop 1
>
>
>
> I always appreciate your suggestions and pointers.
>
>
>
>
>
> Regards,
>
> Sambit.
>
>
>
>
>
>
>
RE: Sqoop connector configuration
Posted by "Sambit Tripathy (RBEI/PJ-NBS)" <Sa...@in.bosch.com>.
One thing I could not find in the documentation and that is after doing step 1, should you be doing a chmod 755 on those jars? Usually root or sudoer copies the jars into /var/lib/sqoop2.
-bash-4.1$ ls -la /var/lib/sqoop2/
total 4756
drwxr-xr-x 4 sqoop2 sqoop 160 Sep 23 00:27 .
drwxr-xr-x 28 root root 600 Sep 16 05:48 ..
-rw-r--r-- 1 root root 954041 Sep 10 07:51 mysql-connector-java-5.1.30-bin.jar
-rw-r--r-- 1 root root 2739616 Sep 10 07:51 ojdbc6.jar
-rw-r--r-- 1 root root 588974 Sep 10 07:51 postgresql-9.3-1101.jdbc4.jar
drwxr-xr-x 3 sqoop2 sqoop 60 Sep 2 00:56 repository
-rw-r--r-- 1 root root 584207 Sep 10 07:51 sqljdbc4.jar
drwxr-xr-x 5 sqoop2 sqoop 100 Sep 23 00:27 tomcat-deployment
From: Abraham Elmahrek [mailto:abe@cloudera.com]
Sent: Thursday, September 25, 2014 3:18 PM
To: user@sqoop.apache.org
Subject: Re: Sqoop connector configuration
Hey there,
Sqoop2 is still under active and heavy development. The Generic JDBC connector will be able to connect to Oracle and MySQL, but the optimized connectors are not yet available.
If you feel like using Sqoop2, try creating a job with the Generic JDBC Connector.
If you feel like an optimized connector, then please use Sqoop1.
-Abe
On Thu, Sep 25, 2014 at 3:06 PM, Sambit Tripathy (RBEI/PJ-NBS) <Sa...@in.bosch.com>> wrote:
Hi,
I have recently upgraded my cluster to CDH 5.1 and Sqoop2.
My Sqoop 2 server is “sqoop-server-main”
1. On “sqoop-server-main” I have copied “ojdbc6.jar” and “mysql-connector-java-version-bin.jar” to /var/lib/sqoop2/
2. Restarted the Sqoop2 service
3. To verify I logged in to “sqoop-server-main” and typed “sqoop2” to get into the sqoop shell which worked.
4. Now to see the available connectors I typed “show connector --all”
I get
1 connector(s) to show:
Connector with id 1:
Name: generic-jdbc-connector
Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector
Version: 1.99.3-cdh5.1.0
I am not able to figure out what has gone wrong here! I should see both oracle and mysql connectors along with the generic-jdbc-driver.
Are there any config changes required? As I understand in Sqoop 2 the connectors/drivers should be present on the server in contrast with Sqoop 1
I always appreciate your suggestions and pointers.
Regards,
Sambit.
Re: Sqoop connector configuration
Posted by Abraham Elmahrek <ab...@cloudera.com>.
Hey there,
Sqoop2 is still under active and heavy development. The Generic JDBC
connector will be able to connect to Oracle and MySQL, but the optimized
connectors are not yet available.
If you feel like using Sqoop2, try creating a job with the Generic JDBC
Connector.
If you feel like an optimized connector, then please use Sqoop1.
-Abe
On Thu, Sep 25, 2014 at 3:06 PM, Sambit Tripathy (RBEI/PJ-NBS) <
Sambit.Tripathy@in.bosch.com> wrote:
> Hi,
>
> I have recently upgraded my cluster to CDH 5.1 and Sqoop2.
>
> My Sqoop 2 server is “sqoop-server-main”
>
> 1. On “sqoop-server-main” I have copied “ojdbc6.jar” and
> “mysql-connector-java-*version*-bin.jar” to */var/lib/sqoop2/*
>
> 2. Restarted the Sqoop2 service
>
> 3. To verify I logged in to “sqoop-server-main” and typed “sqoop2” to get
> into the sqoop shell which worked.
>
> 4. Now to see the available connectors I typed “show connector --all”
>
> I get
>
> *1 connector(s) to show:*
> Connector with id 1:
> Name: generic-jdbc-connector
> Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector
> Version: 1.99.3-cdh5.1.0
>
>
> I am not able to figure out what has gone wrong here! I should see both
> oracle and mysql connectors along with the generic-jdbc-driver.
>
> Are there any config changes required? As I understand in Sqoop 2 the
> connectors/drivers should be present on the server in contrast with Sqoop 1
>
> I always appreciate your suggestions and pointers.
>
>
> Regards,
> Sambit.
>
>
>