You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Suhas Satish <su...@gmail.com> on 2014/04/01 02:30:07 UTC
starting sqoop2 server
Hi
I am trying to start cdh5b2 sqoop2 server from the linux command line.
I see that the catalina tomcat server starts listening on port 8080 instead
of the sqoop server starting up on port 12000. There are no logs that tell
me whats going wrong.
Any ideas?
Here's some diagnosis -
$SQOOP_HOME/bin/sqoop.sh server start
Sqoop home directory:
/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2
Setting SQOOP_HTTP_PORT: 12000
Setting SQOOP_ADMIN_PORT: 12001
Using CATALINA_OPTS:
Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
Using CATALINA_BASE:
/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server
Using CATALINA_HOME:
/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server
Using CATALINA_TMPDIR:
/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server/temp
Using JRE_HOME: /usr
Using CLASSPATH:
/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server/bin/bootstrap.jar
------------------------------------
ps axu | grep sqoop2
root 14130 6.1 0.7 3168652 69424 pts/0 Sl 17:00 0:01
/usr/bin/java
-Djava.util.logging.config.file=/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
-Djava.endorsed.dirs=/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server/endorsed
-classpath
/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server/bin/bootstrap.jar
-Dcatalina.base=/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server
-Dcatalina.home=/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server
-Djava.io.tmpdir=/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server/temp
org.apache.catalina.startup.Bootstrap start
root 14545 0.0 0.0 9384 920 pts/0 S+ 17:01 0:00 grep
--color=auto sqoop2
-------------------------------------------
ls $SQOOP_HOME
bin client-lib cloudera server webapps
sqoop.war is under
webapps/sqoop.war
Thanks,
Suhas.
Re: starting sqoop2 server
Posted by Suhas Satish <su...@gmail.com>.
Thanks a lot for pointing me in the right direction Vasanth, you saved me
a great deal of time!
I see the change in server.xml which makes the port configurable now
<Connector port="${sqoop.http.port}" protocol="HTTP/1.1"
Cheers,
Suhas.
On Wed, Apr 2, 2014 at 4:25 AM, vasanth kumar <rj...@gmail.com>wrote:
> Hi Suhas,
>
> Check tomcat's connector port in server.xml
>
> You are using old svn code build and setup.
>
> Check this link to build:
> http://sqoop.apache.org/docs/1.99.3/BuildingSqoop2.html
>
> Installation part given in :
> http://sqoop.apache.org/docs/1.99.3/Installation.html
>
>
>
>
> On Wed, Apr 2, 2014 at 1:50 AM, Suhas Satish <su...@gmail.com>wrote:
>
>> I am doing a manual deploy of sqoop2 server from instructions here.
>> https://cwiki.apache.org/confluence/display/SQOOP/Sqoop2+Manual+Setup
>>
>> As detailed in the instructions, I am able to start sqoop server
>> listening on
>> http://localhost:8080/sqoop
>>
>> But I also see a couple of properties being passed to CATALINA_OPTS for
>> starting sqoop server on port 12000, which is completely ignored. Any idea *how
>> I can send the sqoop server web application to listen on a configurable
>> port*?
>>
>> Also, *whats the best location for putting the mysql jdbc driver jar
>> file*?
>> Right now I have it under
>> s2test/tomcat-deployment/lib/
>> along with tomcat jars and I include this directory on the tomcat
>> classpath thru setenv.sh
>>
>> Note: s2test is the equivalent of /var/lib/sqoop2 in cdh.
>>
>> ps axu | grep 17349
>>
>> root 17349 11.3 1.9 4229428 158440 pts/7 Sl 13:07 0:09
>> /usr/lib/jvm/java-6-oracle/bin/java
>> -Djava.util.logging.config.file=/opt/mapr/sqoop/s2test/server/conf/logging.properties
>> -Dsqoop.config.dir=/opt/mapr/sqoop/s2test/conf
>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager *-Dsqoop.http.port=12000
>> -Dsqoop.admin.port=12001*-Djava.endorsed.dirs=/opt/mapr/sqoop/s2test/server/endorsed -classpath
>> /opt/mapr/sqoop/s2test/server/bin/tomcat-juli.jar:/opt/mapr/sqoop/s2test/server/bin/bootstrap.jar::server/bin/../lib/annotations-api.jar:server/bin/../lib/catalina-ant.jar:server/bin/../lib/catalina-ha.jar:server/bin/../lib/catalina-tribes.jar:server/bin/../lib/catalina.jar:server/bin/../lib/ecj-3.7.2.jar:server/bin/../lib/el-api.jar:server/bin/../lib/jasper-el.jar:server/bin/../lib/jasper.jar:server/bin/../lib/jsp-api.jar:server/bin/../lib/mysql-connector-java-5.1.30-bin.jar:server/bin/../lib/servlet-api.jar:server/bin/../lib/tomcat-api.jar:server/bin/../lib/tomcat-coyote.jar:server/bin/../lib/tomcat-dbcp.jar:server/bin/../lib/tomcat-i18n-es.jar:server/bin/../lib/tomcat-i18n-fr.jar:server/bin/../lib/tomcat-i18n-ja.jar:server/bin/../lib/tomcat-jdbc.jar:server/bin/../lib/tomcat-util.jar:
>>
>>
>> Thanks,
>> Suhas.
>>
>
>
>
> --
> Regards
> Vasanth kumar RJ
>
Re: starting sqoop2 server
Posted by vasanth kumar <rj...@gmail.com>.
Hi Suhas,
Check tomcat's connector port in server.xml
You are using old svn code build and setup.
Check this link to build:
http://sqoop.apache.org/docs/1.99.3/BuildingSqoop2.html
Installation part given in :
http://sqoop.apache.org/docs/1.99.3/Installation.html
On Wed, Apr 2, 2014 at 1:50 AM, Suhas Satish <su...@gmail.com> wrote:
> I am doing a manual deploy of sqoop2 server from instructions here.
> https://cwiki.apache.org/confluence/display/SQOOP/Sqoop2+Manual+Setup
>
> As detailed in the instructions, I am able to start sqoop server listening
> on
> http://localhost:8080/sqoop
>
> But I also see a couple of properties being passed to CATALINA_OPTS for
> starting sqoop server on port 12000, which is completely ignored. Any idea *how
> I can send the sqoop server web application to listen on a configurable
> port*?
>
> Also, *whats the best location for putting the mysql jdbc driver jar file*
> ?
> Right now I have it under
> s2test/tomcat-deployment/lib/
> along with tomcat jars and I include this directory on the tomcat
> classpath thru setenv.sh
>
> Note: s2test is the equivalent of /var/lib/sqoop2 in cdh.
>
> ps axu | grep 17349
>
> root 17349 11.3 1.9 4229428 158440 pts/7 Sl 13:07 0:09
> /usr/lib/jvm/java-6-oracle/bin/java
> -Djava.util.logging.config.file=/opt/mapr/sqoop/s2test/server/conf/logging.properties
> -Dsqoop.config.dir=/opt/mapr/sqoop/s2test/conf
> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager *-Dsqoop.http.port=12000
> -Dsqoop.admin.port=12001*-Djava.endorsed.dirs=/opt/mapr/sqoop/s2test/server/endorsed -classpath
> /opt/mapr/sqoop/s2test/server/bin/tomcat-juli.jar:/opt/mapr/sqoop/s2test/server/bin/bootstrap.jar::server/bin/../lib/annotations-api.jar:server/bin/../lib/catalina-ant.jar:server/bin/../lib/catalina-ha.jar:server/bin/../lib/catalina-tribes.jar:server/bin/../lib/catalina.jar:server/bin/../lib/ecj-3.7.2.jar:server/bin/../lib/el-api.jar:server/bin/../lib/jasper-el.jar:server/bin/../lib/jasper.jar:server/bin/../lib/jsp-api.jar:server/bin/../lib/mysql-connector-java-5.1.30-bin.jar:server/bin/../lib/servlet-api.jar:server/bin/../lib/tomcat-api.jar:server/bin/../lib/tomcat-coyote.jar:server/bin/../lib/tomcat-dbcp.jar:server/bin/../lib/tomcat-i18n-es.jar:server/bin/../lib/tomcat-i18n-fr.jar:server/bin/../lib/tomcat-i18n-ja.jar:server/bin/../lib/tomcat-jdbc.jar:server/bin/../lib/tomcat-util.jar:
>
>
> Thanks,
> Suhas.
>
--
Regards
Vasanth kumar RJ
Re: starting sqoop2 server
Posted by Suhas Satish <su...@gmail.com>.
I am doing a manual deploy of sqoop2 server from instructions here.
https://cwiki.apache.org/confluence/display/SQOOP/Sqoop2+Manual+Setup
As detailed in the instructions, I am able to start sqoop server listening
on
http://localhost:8080/sqoop
But I also see a couple of properties being passed to CATALINA_OPTS for
starting sqoop server on port 12000, which is completely ignored. Any idea *how
I can send the sqoop server web application to listen on a configurable
port*?
Also, *whats the best location for putting the mysql jdbc driver jar file*?
Right now I have it under
s2test/tomcat-deployment/lib/
along with tomcat jars and I include this directory on the tomcat classpath
thru setenv.sh
Note: s2test is the equivalent of /var/lib/sqoop2 in cdh.
ps axu | grep 17349
root 17349 11.3 1.9 4229428 158440 pts/7 Sl 13:07 0:09
/usr/lib/jvm/java-6-oracle/bin/java
-Djava.util.logging.config.file=/opt/mapr/sqoop/s2test/server/conf/logging.properties
-Dsqoop.config.dir=/opt/mapr/sqoop/s2test/conf
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
*-Dsqoop.http.port=12000
-Dsqoop.admin.port=12001*-Djava.endorsed.dirs=/opt/mapr/sqoop/s2test/server/endorsed
-classpath
/opt/mapr/sqoop/s2test/server/bin/tomcat-juli.jar:/opt/mapr/sqoop/s2test/server/bin/bootstrap.jar::server/bin/../lib/annotations-api.jar:server/bin/../lib/catalina-ant.jar:server/bin/../lib/catalina-ha.jar:server/bin/../lib/catalina-tribes.jar:server/bin/../lib/catalina.jar:server/bin/../lib/ecj-3.7.2.jar:server/bin/../lib/el-api.jar:server/bin/../lib/jasper-el.jar:server/bin/../lib/jasper.jar:server/bin/../lib/jsp-api.jar:server/bin/../lib/mysql-connector-java-5.1.30-bin.jar:server/bin/../lib/servlet-api.jar:server/bin/../lib/tomcat-api.jar:server/bin/../lib/tomcat-coyote.jar:server/bin/../lib/tomcat-dbcp.jar:server/bin/../lib/tomcat-i18n-es.jar:server/bin/../lib/tomcat-i18n-fr.jar:server/bin/../lib/tomcat-i18n-ja.jar:server/bin/../lib/tomcat-jdbc.jar:server/bin/../lib/tomcat-util.jar:
Thanks,
Suhas.
Re: starting sqoop2 server
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
This seems as a CDH specific issue, so I would suggest to ask that question on CDH specific mailing list - cdh-user@cloudera.org [1].
You seem to be using parcels that are meant to be used from within Cloudera Manager, any reason why you are trying to start the service from console?
Jarcec
Links:
1: https://groups.google.com/a/cloudera.org/forum/#!forum/cdh-user
On Mon, Mar 31, 2014 at 05:30:07PM -0700, Suhas Satish wrote:
> Hi
> I am trying to start cdh5b2 sqoop2 server from the linux command line.
> I see that the catalina tomcat server starts listening on port 8080 instead
> of the sqoop server starting up on port 12000. There are no logs that tell
> me whats going wrong.
>
> Any ideas?
>
> Here's some diagnosis -
>
> $SQOOP_HOME/bin/sqoop.sh server start
>
> Sqoop home directory:
> /opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2
> Setting SQOOP_HTTP_PORT: 12000
> Setting SQOOP_ADMIN_PORT: 12001
> Using CATALINA_OPTS:
> Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
> Using CATALINA_BASE:
> /opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server
> Using CATALINA_HOME:
> /opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server
> Using CATALINA_TMPDIR:
> /opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server/temp
> Using JRE_HOME: /usr
> Using CLASSPATH:
> /opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server/bin/bootstrap.jar
> ------------------------------------
> ps axu | grep sqoop2
>
> root 14130 6.1 0.7 3168652 69424 pts/0 Sl 17:00 0:01
> /usr/bin/java
> -Djava.util.logging.config.file=/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server/conf/logging.properties
> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
> -Djava.endorsed.dirs=/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server/endorsed
> -classpath
> /opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server/bin/bootstrap.jar
> -Dcatalina.base=/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server
> -Dcatalina.home=/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server
> -Djava.io.tmpdir=/opt/cloudera/parcels/CDH-5.0.0-0.cdh5b2.p0.27/lib/sqoop2/server/temp
> org.apache.catalina.startup.Bootstrap start
> root 14545 0.0 0.0 9384 920 pts/0 S+ 17:01 0:00 grep
> --color=auto sqoop2
> -------------------------------------------
> ls $SQOOP_HOME
> bin client-lib cloudera server webapps
>
> sqoop.war is under
> webapps/sqoop.war
>
>
>
> Thanks,
> Suhas.