You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Cyril Bogus <cy...@gmail.com> on 2013/03/02 21:11:40 UTC
Java to Sqoop commands
So, I am running a java program to import data from a sqlserver using sqoop.
I have the Microsoft SQL - Hadoop connector in the lib and whenever I run
commands from the command line the import runs fine.
Problem is when my Java is the one trying to run the command then sqoop
can't find the lib somehow so it does not connect to the DB.
I try running the exact same query directly in command line and everything
works fine.
Is there a way where I can define both the driver and connection manager
through Java as not to worry about it not finding the proper manager for
import?
Regards
Cyril
Re: Java to Sqoop commands
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Cyril,
would you mind rerunning the command with --verbose argument? I would like to see what factories is Sqoop trying to use.
Jarcec
On Tue, Mar 05, 2013 at 12:21:44PM -0500, Cyril Bogus wrote:
> Hi Jarek, I have managed to get it to internet. Here is the output with the
> content of the $CLASSPATH passed.
>
> owner@owner-7:~/Desktop$ $CLASSPATH
> bash:
> .:/home/owner/hadoop/hadoop-core-1.0.4.jar:/home/owner/hive/conf:/home/owner/hive/lib/antlr-runtime-3.0.1.jar:/home/owner/hive/lib/commons-cli-1.2.jar:/home/owner/hive/lib/commons-codec-1.3.jar:/home/owner/hive/lib/commons-collections-3.2.1.jar:/home/owner/hive/lib/commons-dbcp-1.4.jar:/home/owner/hive/lib/commons-lang-2.4.jar:/home/owner/hive/lib/commons-logging-1.0.4.jar:/home/owner/hive/lib/commons-logging-api-1.0.4.jar:/home/owner/hive/lib/commons-pool-1.5.4.jar:/home/owner/hive/lib/datanucleus-connectionpool-2.0.3.jar:/home/owner/hive/lib/datanucleus-core-2.0.3.jar:/home/owner/hive/lib/datanucleus-enhancer-2.0.3.jar:/home/owner/hive/lib/datanucleus-rdbms-2.0.3.jar:/home/owner/hive/lib/derby-10.4.2.0.jar:/home/owner/hive/lib/guava-r09.jar:/home/owner/hive/lib/hbase-0.92.0.jar:/home/owner/hive/lib/hbase-0.92.0-tests.jar:/home/owner/hive/lib/hive-builtins-0.9.0.jar:/home/owner/hive/lib/hive-cli-0.9.0.jar:/home/owner/hive/lib/hive-common-0.9.0.jar:/home/owner/hive/lib/hive-contrib-0.9.0.jar:/home/owner/hive/lib/hive_contrib.jar:/home/owner/hive/lib/hive-exec-0.9.0.jar:/home/owner/hive/lib/hive-hbase-handler-0.9.0.jar:/home/owner/hive/lib/hive-hwi-0.9.0.jar:/home/owner/hive/lib/hive-jdbc-0.9.0.jar:/home/owner/hive/lib/hive-metastore-0.9.0.jar:/home/owner/hive/lib/hive-pdk-0.9.0.jar:/home/owner/hive/lib/hive-serde-0.9.0.jar:/home/owner/hive/lib/hive-service-0.9.0.jar:/home/owner/hive/lib/hive-shims-0.9.0.jar:/home/owner/hive/lib/jackson-core-asl-1.8.8.jar:/home/owner/hive/lib/jackson-jaxrs-1.8.8.jar:/home/owner/hive/lib/jackson-mapper-asl-1.8.8.jar:/home/owner/hive/lib/jackson-xc-1.8.8.jar:/home/owner/hive/lib/JavaEWAH-0.3.2.jar:/home/owner/hive/lib/jdo2-api-2.3-ec.jar:/home/owner/hive/lib/jline-0.9.94.jar:/home/owner/hive/lib/json-20090211.jar:/home/owner/hive/lib/libfb303-0.7.0.jar:/home/owner/hive/lib/libfb303.jar:/home/owner/hive/lib/libthrift-0.7.0.jar:/home/owner/hive/lib/libthrift.jar:/home/owner/hive/lib/log4j-1.2.16.jar:/home/owner/hive/lib/mysql-connector-java-5.1.23-bin.jar:/home/owner/hive/lib/slf4j-api-1.6.1.jar:/home/owner/hive/lib/slf4j-log4j12-1.6.1.jar:/home/owner/hive/lib/sqljdbc4.jar:/home/owner/hive/lib/stringtemplate-3.1-b1.jar:/home/owner/hive/lib/zookeeper-3.4.3.jar:/home/owner/hadoop/lib/asm-3.2.jar:/home/owner/hadoop/lib/aspectjrt-1.6.5.jar:/home/owner/hadoop/lib/aspectjtools-1.6.5.jar:/home/owner/hadoop/lib/commons-beanutils-1.7.0.jar:/home/owner/hadoop/lib/commons-beanutils-core-1.8.0.jar:/home/owner/hadoop/lib/commons-cli-1.2.jar:/home/owner/hadoop/lib/commons-codec-1.4.jar:/home/owner/hadoop/lib/commons-collections-3.2.1.jar:/home/owner/hadoop/lib/commons-configuration-1.6.jar:/home/owner/hadoop/lib/commons-daemon-1.0.1.jar:/home/owner/hadoop/lib/commons-digester-1.8.jar:/home/owner/hadoop/lib/commons-el-1.0.jar:/home/owner/hadoop/lib/commons-httpclient-3.0.1.jar:/home/owner/hadoop/lib/commons-io-2.1.jar:/home/owner/hadoop/lib/commons-lang-2.4.jar:/home/owner/hadoop/lib/commons-logging-1.1.1.jar:/home/owner/hadoop/lib/commons-logging-api-1.0.4.jar:/home/owner/hadoop/lib/commons-math-2.1.jar:/home/owner/hadoop/lib/commons-net-1.4.1.jar:/home/owner/hadoop/lib/core-3.1.1.jar:/home/owner/hadoop/lib/hadoop-capacity-scheduler-1.0.4.jar:/home/owner/hadoop/lib/hadoop-fairscheduler-1.0.4.jar:/home/owner/hadoop/lib/hadoop-thriftfs-1.0.4.jar:/home/owner/hadoop/lib/hsqldb-1.8.0.10.jar:/home/owner/hadoop/lib/jackson-core-asl-1.8.8.jar:/home/owner/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/home/owner/hadoop/lib/jasper-compiler-5.5.12.jar:/home/owner/hadoop/lib/jasper-runtime-5.5.12.jar:/home/owner/hadoop/lib/jdeb-0.8.jar:/home/owner/hadoop/lib/jersey-core-1.8.jar:/home/owner/hadoop/lib/jersey-json-1.8.jar:/home/owner/hadoop/lib/jersey-server-1.8.jar:/home/owner/hadoop/lib/jets3t-0.6.1.jar:/home/owner/hadoop/lib/jetty-6.1.26.jar:/home/owner/hadoop/lib/jetty-util-6.1.26.jar:/home/owner/hadoop/lib/jsch-0.1.42.jar:/home/owner/hadoop/lib/junit-4.5.jar:/home/owner/hadoop/lib/kfs-0.2.2.jar:/home/owner/hadoop/lib/log4j-1.2.15.jar:/home/owner/hadoop/lib/mockito-all-1.8.5.jar:/home/owner/hadoop/lib/oro-2.0.8.jar:/home/owner/hadoop/lib/servlet-api-2.5-20081211.jar:/home/owner/hadoop/lib/slf4j-api-1.4.3.jar:/home/owner/hadoop/lib/slf4j-log4j12-1.4.3.jar:/home/owner/hadoop/lib/xmlenc-0.52.jar:/home/owner/sqoop/lib/ant-contrib-1.0b3.jar:/home/owner/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar:/home/owner/sqoop/lib/avro-1.5.3.jar:/home/owner/sqoop/lib/avro-ipc-1.5.3.jar:/home/owner/sqoop/lib/avro-mapred-1.5.3.jar:/home/owner/sqoop/lib/commons-io-1.4.jar:/home/owner/sqoop/lib/hsqldb-1.8.0.10.jar:/home/owner/sqoop/lib/jackson-core-asl-1.7.3.jar:/home/owner/sqoop/lib/jackson-mapper-asl-1.7.3.jar:/home/owner/sqoop/lib/jopt-simple-3.2.jar:/home/owner/sqoop/lib/paranamer-2.3.jar:/home/owner/sqoop/lib/snappy-java-1.0.3.2.jar:/home/owner/sqoop/lib/sqljdbc4.jar:/home/owner/sqoop/lib/sqoop-sqlserver-1.0.jar:/home/owner/trunk/lib/*.jar:
> File name too long
> owner@owner-7:~/Desktop$ javac Scheduler.java
> owner@owner-7:~/Desktop$ java -cp $CLASSPATH Scheduler
> /home/owner/sqoop/bin/sqoop import --connect
> 'jdbc:sqlserver://111.111.111.111:1111;username=sa;password=owner%2007;database=database'
> --table minedrive_driver_scoring_table --hive-import --hive-overwrite
> Warning: $HADOOP_HOME is deprecated.
>
> 13/03/05 12:16:51 INFO tool.BaseSqoopTool: Using Hive-specific delimiters
> for output. You can override
> 13/03/05 12:16:51 INFO tool.BaseSqoopTool: delimiters with
> --fields-terminated-by, etc.
> 13/03/05 12:16:53 ERROR tool.BaseSqoopTool: Got error creating database
> manager: java.io.IOException: No manager for connect string:
> 'jdbc:sqlserver://111.111.111.111:1111
> ;username=sa;password=owner%2007;database=database'
> at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:185)
> at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:200)
> at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:464)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
>
> Process exitValue: 1
> PROGRAM DONE
> owner@owner-7:~/Desktop$
>
>
> On Mon, Mar 4, 2013 at 11:07 AM, Jarek Jarcec Cecho <ja...@apache.org>wrote:
>
> > Hi Cyril,
> > I do understand that your second computer might not be connector to the
> > internet, however it's very hard to help you without logs. Would you mind
> > saving entire generated log with the exception and your classpath into file
> > and move it to a computer where you have internet connectivity and share
> > that with us? (floppy, usb stick, ...)
> >
> > Jarcec
> >
> > On Sat, Mar 02, 2013 at 01:55:00PM -0800, Venkat wrote:
> > > It looks like your JDBC spec is wrong. Can you please make sure for
> > > SQLServer your connect string is of the form
> > >
> > jdbc:sqlserver://[serverName[\instanceName][:portNumber]][;property=value[;property=value]]
> > >
> > > See http://msdn.microsoft.com/en-us/library/ms378428(v=sql.90).aspx
> > >
> > >
> > > On Sat, Mar 2, 2013 at 1:33 PM, Cyril Bogus <cy...@gmail.com>
> > wrote:
> > >
> > > > Hi Jarek, thank you for your reply.
> > > >
> > > > The computer On which I run the job is not connected to the internet so
> > > > all I can do is describe the output very briefly.
> > > >
> > > > And by the way I am running Sqoop 1.4.2
> > > >
> > > > Following your advice I have done the following
> > > >
> > > > javac Importer.java
> > > >
> > > > java -cp $CLASSPATH Importer
> > > >
> > > > CLASSPATH contains all libraries for sqoop, hive and Hadoop
> > > > this is a brief description of the output:
> > > >
> > > >
> > > > Using Hive Specific Import ....
> > > > delimiters with --field-terminated by, etc.
> > > > Got error creating database manager: java.io.IOException. No manager
> > for
> > > > connect string 'myserver:port/database;user;pass'
> > > >
> > > >
> > > > Again the classpath contains all the jars AND when I just copy and
> > paste
> > > > the same command line that gave me an error in the program as a linux
> > > > command everything runs smoothly to the end.
> > > >
> > > >
> > > >
> > > > On Sat, Mar 2, 2013 at 3:28 PM, Jarek Jarcec Cecho <jarcec@apache.org
> > >wrote:
> > > >
> > > >> Hi Cyril,
> > > >> when executing Sqoop from within your Java application, do you have
> > all
> > > >> the jars (connector, JDBC Driver) in the classpath? Also it would be
> > > >> helpful if you could share exactly the error message that you're
> > getting.
> > > >>
> > > >> Jarcec
> > > >>
> > > >> On Sat, Mar 02, 2013 at 03:11:40PM -0500, Cyril Bogus wrote:
> > > >> > So, I am running a java program to import data from a sqlserver
> > using
> > > >> sqoop.
> > > >> >
> > > >> > I have the Microsoft SQL - Hadoop connector in the lib and whenever
> > I
> > > >> run
> > > >> > commands from the command line the import runs fine.
> > > >> >
> > > >> > Problem is when my Java is the one trying to run the command then
> > sqoop
> > > >> > can't find the lib somehow so it does not connect to the DB.
> > > >> > I try running the exact same query directly in command line and
> > > >> everything
> > > >> > works fine.
> > > >> >
> > > >> > Is there a way where I can define both the driver and connection
> > manager
> > > >> > through Java as not to worry about it not finding the proper
> > manager for
> > > >> > import?
> > > >> >
> > > >> > Regards
> > > >> >
> > > >> > Cyril
> > > >>
> > > >
> > > >
> > >
> > >
> > > --
> > > Regards
> > >
> > > Venkat
> >
Re: Java to Sqoop commands
Posted by Cyril Bogus <cy...@gmail.com>.
Hi Jarek, I have managed to get it to internet. Here is the output with the
content of the $CLASSPATH passed.
owner@owner-7:~/Desktop$ $CLASSPATH
bash:
.:/home/owner/hadoop/hadoop-core-1.0.4.jar:/home/owner/hive/conf:/home/owner/hive/lib/antlr-runtime-3.0.1.jar:/home/owner/hive/lib/commons-cli-1.2.jar:/home/owner/hive/lib/commons-codec-1.3.jar:/home/owner/hive/lib/commons-collections-3.2.1.jar:/home/owner/hive/lib/commons-dbcp-1.4.jar:/home/owner/hive/lib/commons-lang-2.4.jar:/home/owner/hive/lib/commons-logging-1.0.4.jar:/home/owner/hive/lib/commons-logging-api-1.0.4.jar:/home/owner/hive/lib/commons-pool-1.5.4.jar:/home/owner/hive/lib/datanucleus-connectionpool-2.0.3.jar:/home/owner/hive/lib/datanucleus-core-2.0.3.jar:/home/owner/hive/lib/datanucleus-enhancer-2.0.3.jar:/home/owner/hive/lib/datanucleus-rdbms-2.0.3.jar:/home/owner/hive/lib/derby-10.4.2.0.jar:/home/owner/hive/lib/guava-r09.jar:/home/owner/hive/lib/hbase-0.92.0.jar:/home/owner/hive/lib/hbase-0.92.0-tests.jar:/home/owner/hive/lib/hive-builtins-0.9.0.jar:/home/owner/hive/lib/hive-cli-0.9.0.jar:/home/owner/hive/lib/hive-common-0.9.0.jar:/home/owner/hive/lib/hive-contrib-0.9.0.jar:/home/owner/hive/lib/hive_contrib.jar:/home/owner/hive/lib/hive-exec-0.9.0.jar:/home/owner/hive/lib/hive-hbase-handler-0.9.0.jar:/home/owner/hive/lib/hive-hwi-0.9.0.jar:/home/owner/hive/lib/hive-jdbc-0.9.0.jar:/home/owner/hive/lib/hive-metastore-0.9.0.jar:/home/owner/hive/lib/hive-pdk-0.9.0.jar:/home/owner/hive/lib/hive-serde-0.9.0.jar:/home/owner/hive/lib/hive-service-0.9.0.jar:/home/owner/hive/lib/hive-shims-0.9.0.jar:/home/owner/hive/lib/jackson-core-asl-1.8.8.jar:/home/owner/hive/lib/jackson-jaxrs-1.8.8.jar:/home/owner/hive/lib/jackson-mapper-asl-1.8.8.jar:/home/owner/hive/lib/jackson-xc-1.8.8.jar:/home/owner/hive/lib/JavaEWAH-0.3.2.jar:/home/owner/hive/lib/jdo2-api-2.3-ec.jar:/home/owner/hive/lib/jline-0.9.94.jar:/home/owner/hive/lib/json-20090211.jar:/home/owner/hive/lib/libfb303-0.7.0.jar:/home/owner/hive/lib/libfb303.jar:/home/owner/hive/lib/libthrift-0.7.0.jar:/home/owner/hive/lib/libthrift.jar:/home/owner/hive/lib/log4j-1.2.16.jar:/home/owner/hive/lib/mysql-connector-java-5.1.23-bin.jar:/home/owner/hive/lib/slf4j-api-1.6.1.jar:/home/owner/hive/lib/slf4j-log4j12-1.6.1.jar:/home/owner/hive/lib/sqljdbc4.jar:/home/owner/hive/lib/stringtemplate-3.1-b1.jar:/home/owner/hive/lib/zookeeper-3.4.3.jar:/home/owner/hadoop/lib/asm-3.2.jar:/home/owner/hadoop/lib/aspectjrt-1.6.5.jar:/home/owner/hadoop/lib/aspectjtools-1.6.5.jar:/home/owner/hadoop/lib/commons-beanutils-1.7.0.jar:/home/owner/hadoop/lib/commons-beanutils-core-1.8.0.jar:/home/owner/hadoop/lib/commons-cli-1.2.jar:/home/owner/hadoop/lib/commons-codec-1.4.jar:/home/owner/hadoop/lib/commons-collections-3.2.1.jar:/home/owner/hadoop/lib/commons-configuration-1.6.jar:/home/owner/hadoop/lib/commons-daemon-1.0.1.jar:/home/owner/hadoop/lib/commons-digester-1.8.jar:/home/owner/hadoop/lib/commons-el-1.0.jar:/home/owner/hadoop/lib/commons-httpclient-3.0.1.jar:/home/owner/hadoop/lib/commons-io-2.1.jar:/home/owner/hadoop/lib/commons-lang-2.4.jar:/home/owner/hadoop/lib/commons-logging-1.1.1.jar:/home/owner/hadoop/lib/commons-logging-api-1.0.4.jar:/home/owner/hadoop/lib/commons-math-2.1.jar:/home/owner/hadoop/lib/commons-net-1.4.1.jar:/home/owner/hadoop/lib/core-3.1.1.jar:/home/owner/hadoop/lib/hadoop-capacity-scheduler-1.0.4.jar:/home/owner/hadoop/lib/hadoop-fairscheduler-1.0.4.jar:/home/owner/hadoop/lib/hadoop-thriftfs-1.0.4.jar:/home/owner/hadoop/lib/hsqldb-1.8.0.10.jar:/home/owner/hadoop/lib/jackson-core-asl-1.8.8.jar:/home/owner/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/home/owner/hadoop/lib/jasper-compiler-5.5.12.jar:/home/owner/hadoop/lib/jasper-runtime-5.5.12.jar:/home/owner/hadoop/lib/jdeb-0.8.jar:/home/owner/hadoop/lib/jersey-core-1.8.jar:/home/owner/hadoop/lib/jersey-json-1.8.jar:/home/owner/hadoop/lib/jersey-server-1.8.jar:/home/owner/hadoop/lib/jets3t-0.6.1.jar:/home/owner/hadoop/lib/jetty-6.1.26.jar:/home/owner/hadoop/lib/jetty-util-6.1.26.jar:/home/owner/hadoop/lib/jsch-0.1.42.jar:/home/owner/hadoop/lib/junit-4.5.jar:/home/owner/hadoop/lib/kfs-0.2.2.jar:/home/owner/hadoop/lib/log4j-1.2.15.jar:/home/owner/hadoop/lib/mockito-all-1.8.5.jar:/home/owner/hadoop/lib/oro-2.0.8.jar:/home/owner/hadoop/lib/servlet-api-2.5-20081211.jar:/home/owner/hadoop/lib/slf4j-api-1.4.3.jar:/home/owner/hadoop/lib/slf4j-log4j12-1.4.3.jar:/home/owner/hadoop/lib/xmlenc-0.52.jar:/home/owner/sqoop/lib/ant-contrib-1.0b3.jar:/home/owner/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar:/home/owner/sqoop/lib/avro-1.5.3.jar:/home/owner/sqoop/lib/avro-ipc-1.5.3.jar:/home/owner/sqoop/lib/avro-mapred-1.5.3.jar:/home/owner/sqoop/lib/commons-io-1.4.jar:/home/owner/sqoop/lib/hsqldb-1.8.0.10.jar:/home/owner/sqoop/lib/jackson-core-asl-1.7.3.jar:/home/owner/sqoop/lib/jackson-mapper-asl-1.7.3.jar:/home/owner/sqoop/lib/jopt-simple-3.2.jar:/home/owner/sqoop/lib/paranamer-2.3.jar:/home/owner/sqoop/lib/snappy-java-1.0.3.2.jar:/home/owner/sqoop/lib/sqljdbc4.jar:/home/owner/sqoop/lib/sqoop-sqlserver-1.0.jar:/home/owner/trunk/lib/*.jar:
File name too long
owner@owner-7:~/Desktop$ javac Scheduler.java
owner@owner-7:~/Desktop$ java -cp $CLASSPATH Scheduler
/home/owner/sqoop/bin/sqoop import --connect
'jdbc:sqlserver://111.111.111.111:1111;username=sa;password=owner%2007;database=database'
--table minedrive_driver_scoring_table --hive-import --hive-overwrite
Warning: $HADOOP_HOME is deprecated.
13/03/05 12:16:51 INFO tool.BaseSqoopTool: Using Hive-specific delimiters
for output. You can override
13/03/05 12:16:51 INFO tool.BaseSqoopTool: delimiters with
--fields-terminated-by, etc.
13/03/05 12:16:53 ERROR tool.BaseSqoopTool: Got error creating database
manager: java.io.IOException: No manager for connect string:
'jdbc:sqlserver://111.111.111.111:1111
;username=sa;password=owner%2007;database=database'
at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:185)
at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:200)
at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:464)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
Process exitValue: 1
PROGRAM DONE
owner@owner-7:~/Desktop$
On Mon, Mar 4, 2013 at 11:07 AM, Jarek Jarcec Cecho <ja...@apache.org>wrote:
> Hi Cyril,
> I do understand that your second computer might not be connector to the
> internet, however it's very hard to help you without logs. Would you mind
> saving entire generated log with the exception and your classpath into file
> and move it to a computer where you have internet connectivity and share
> that with us? (floppy, usb stick, ...)
>
> Jarcec
>
> On Sat, Mar 02, 2013 at 01:55:00PM -0800, Venkat wrote:
> > It looks like your JDBC spec is wrong. Can you please make sure for
> > SQLServer your connect string is of the form
> >
> jdbc:sqlserver://[serverName[\instanceName][:portNumber]][;property=value[;property=value]]
> >
> > See http://msdn.microsoft.com/en-us/library/ms378428(v=sql.90).aspx
> >
> >
> > On Sat, Mar 2, 2013 at 1:33 PM, Cyril Bogus <cy...@gmail.com>
> wrote:
> >
> > > Hi Jarek, thank you for your reply.
> > >
> > > The computer On which I run the job is not connected to the internet so
> > > all I can do is describe the output very briefly.
> > >
> > > And by the way I am running Sqoop 1.4.2
> > >
> > > Following your advice I have done the following
> > >
> > > javac Importer.java
> > >
> > > java -cp $CLASSPATH Importer
> > >
> > > CLASSPATH contains all libraries for sqoop, hive and Hadoop
> > > this is a brief description of the output:
> > >
> > >
> > > Using Hive Specific Import ....
> > > delimiters with --field-terminated by, etc.
> > > Got error creating database manager: java.io.IOException. No manager
> for
> > > connect string 'myserver:port/database;user;pass'
> > >
> > >
> > > Again the classpath contains all the jars AND when I just copy and
> paste
> > > the same command line that gave me an error in the program as a linux
> > > command everything runs smoothly to the end.
> > >
> > >
> > >
> > > On Sat, Mar 2, 2013 at 3:28 PM, Jarek Jarcec Cecho <jarcec@apache.org
> >wrote:
> > >
> > >> Hi Cyril,
> > >> when executing Sqoop from within your Java application, do you have
> all
> > >> the jars (connector, JDBC Driver) in the classpath? Also it would be
> > >> helpful if you could share exactly the error message that you're
> getting.
> > >>
> > >> Jarcec
> > >>
> > >> On Sat, Mar 02, 2013 at 03:11:40PM -0500, Cyril Bogus wrote:
> > >> > So, I am running a java program to import data from a sqlserver
> using
> > >> sqoop.
> > >> >
> > >> > I have the Microsoft SQL - Hadoop connector in the lib and whenever
> I
> > >> run
> > >> > commands from the command line the import runs fine.
> > >> >
> > >> > Problem is when my Java is the one trying to run the command then
> sqoop
> > >> > can't find the lib somehow so it does not connect to the DB.
> > >> > I try running the exact same query directly in command line and
> > >> everything
> > >> > works fine.
> > >> >
> > >> > Is there a way where I can define both the driver and connection
> manager
> > >> > through Java as not to worry about it not finding the proper
> manager for
> > >> > import?
> > >> >
> > >> > Regards
> > >> >
> > >> > Cyril
> > >>
> > >
> > >
> >
> >
> > --
> > Regards
> >
> > Venkat
>
Re: Java to Sqoop commands
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Cyril,
I do understand that your second computer might not be connector to the internet, however it's very hard to help you without logs. Would you mind saving entire generated log with the exception and your classpath into file and move it to a computer where you have internet connectivity and share that with us? (floppy, usb stick, ...)
Jarcec
On Sat, Mar 02, 2013 at 01:55:00PM -0800, Venkat wrote:
> It looks like your JDBC spec is wrong. Can you please make sure for
> SQLServer your connect string is of the form
> jdbc:sqlserver://[serverName[\instanceName][:portNumber]][;property=value[;property=value]]
>
> See http://msdn.microsoft.com/en-us/library/ms378428(v=sql.90).aspx
>
>
> On Sat, Mar 2, 2013 at 1:33 PM, Cyril Bogus <cy...@gmail.com> wrote:
>
> > Hi Jarek, thank you for your reply.
> >
> > The computer On which I run the job is not connected to the internet so
> > all I can do is describe the output very briefly.
> >
> > And by the way I am running Sqoop 1.4.2
> >
> > Following your advice I have done the following
> >
> > javac Importer.java
> >
> > java -cp $CLASSPATH Importer
> >
> > CLASSPATH contains all libraries for sqoop, hive and Hadoop
> > this is a brief description of the output:
> >
> >
> > Using Hive Specific Import ....
> > delimiters with --field-terminated by, etc.
> > Got error creating database manager: java.io.IOException. No manager for
> > connect string 'myserver:port/database;user;pass'
> >
> >
> > Again the classpath contains all the jars AND when I just copy and paste
> > the same command line that gave me an error in the program as a linux
> > command everything runs smoothly to the end.
> >
> >
> >
> > On Sat, Mar 2, 2013 at 3:28 PM, Jarek Jarcec Cecho <ja...@apache.org>wrote:
> >
> >> Hi Cyril,
> >> when executing Sqoop from within your Java application, do you have all
> >> the jars (connector, JDBC Driver) in the classpath? Also it would be
> >> helpful if you could share exactly the error message that you're getting.
> >>
> >> Jarcec
> >>
> >> On Sat, Mar 02, 2013 at 03:11:40PM -0500, Cyril Bogus wrote:
> >> > So, I am running a java program to import data from a sqlserver using
> >> sqoop.
> >> >
> >> > I have the Microsoft SQL - Hadoop connector in the lib and whenever I
> >> run
> >> > commands from the command line the import runs fine.
> >> >
> >> > Problem is when my Java is the one trying to run the command then sqoop
> >> > can't find the lib somehow so it does not connect to the DB.
> >> > I try running the exact same query directly in command line and
> >> everything
> >> > works fine.
> >> >
> >> > Is there a way where I can define both the driver and connection manager
> >> > through Java as not to worry about it not finding the proper manager for
> >> > import?
> >> >
> >> > Regards
> >> >
> >> > Cyril
> >>
> >
> >
>
>
> --
> Regards
>
> Venkat
Re: Java to Sqoop commands
Posted by Venkat <ve...@gmail.com>.
It looks like your JDBC spec is wrong. Can you please make sure for
SQLServer your connect string is of the form
jdbc:sqlserver://[serverName[\instanceName][:portNumber]][;property=value[;property=value]]
See http://msdn.microsoft.com/en-us/library/ms378428(v=sql.90).aspx
On Sat, Mar 2, 2013 at 1:33 PM, Cyril Bogus <cy...@gmail.com> wrote:
> Hi Jarek, thank you for your reply.
>
> The computer On which I run the job is not connected to the internet so
> all I can do is describe the output very briefly.
>
> And by the way I am running Sqoop 1.4.2
>
> Following your advice I have done the following
>
> javac Importer.java
>
> java -cp $CLASSPATH Importer
>
> CLASSPATH contains all libraries for sqoop, hive and Hadoop
> this is a brief description of the output:
>
>
> Using Hive Specific Import ....
> delimiters with --field-terminated by, etc.
> Got error creating database manager: java.io.IOException. No manager for
> connect string 'myserver:port/database;user;pass'
>
>
> Again the classpath contains all the jars AND when I just copy and paste
> the same command line that gave me an error in the program as a linux
> command everything runs smoothly to the end.
>
>
>
> On Sat, Mar 2, 2013 at 3:28 PM, Jarek Jarcec Cecho <ja...@apache.org>wrote:
>
>> Hi Cyril,
>> when executing Sqoop from within your Java application, do you have all
>> the jars (connector, JDBC Driver) in the classpath? Also it would be
>> helpful if you could share exactly the error message that you're getting.
>>
>> Jarcec
>>
>> On Sat, Mar 02, 2013 at 03:11:40PM -0500, Cyril Bogus wrote:
>> > So, I am running a java program to import data from a sqlserver using
>> sqoop.
>> >
>> > I have the Microsoft SQL - Hadoop connector in the lib and whenever I
>> run
>> > commands from the command line the import runs fine.
>> >
>> > Problem is when my Java is the one trying to run the command then sqoop
>> > can't find the lib somehow so it does not connect to the DB.
>> > I try running the exact same query directly in command line and
>> everything
>> > works fine.
>> >
>> > Is there a way where I can define both the driver and connection manager
>> > through Java as not to worry about it not finding the proper manager for
>> > import?
>> >
>> > Regards
>> >
>> > Cyril
>>
>
>
--
Regards
Venkat
Re: Java to Sqoop commands
Posted by Cyril Bogus <cy...@gmail.com>.
Hi Jarek, thank you for your reply.
The computer On which I run the job is not connected to the internet so all
I can do is describe the output very briefly.
And by the way I am running Sqoop 1.4.2
Following your advice I have done the following
javac Importer.java
java -cp $CLASSPATH Importer
CLASSPATH contains all libraries for sqoop, hive and Hadoop
this is a brief description of the output:
Using Hive Specific Import ....
delimiters with --field-terminated by, etc.
Got error creating database manager: java.io.IOException. No manager for
connect string 'myserver:port/database;user;pass'
Again the classpath contains all the jars AND when I just copy and paste
the same command line that gave me an error in the program as a linux
command everything runs smoothly to the end.
On Sat, Mar 2, 2013 at 3:28 PM, Jarek Jarcec Cecho <ja...@apache.org>wrote:
> Hi Cyril,
> when executing Sqoop from within your Java application, do you have all
> the jars (connector, JDBC Driver) in the classpath? Also it would be
> helpful if you could share exactly the error message that you're getting.
>
> Jarcec
>
> On Sat, Mar 02, 2013 at 03:11:40PM -0500, Cyril Bogus wrote:
> > So, I am running a java program to import data from a sqlserver using
> sqoop.
> >
> > I have the Microsoft SQL - Hadoop connector in the lib and whenever I run
> > commands from the command line the import runs fine.
> >
> > Problem is when my Java is the one trying to run the command then sqoop
> > can't find the lib somehow so it does not connect to the DB.
> > I try running the exact same query directly in command line and
> everything
> > works fine.
> >
> > Is there a way where I can define both the driver and connection manager
> > through Java as not to worry about it not finding the proper manager for
> > import?
> >
> > Regards
> >
> > Cyril
>
Re: Java to Sqoop commands
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Cyril,
when executing Sqoop from within your Java application, do you have all the jars (connector, JDBC Driver) in the classpath? Also it would be helpful if you could share exactly the error message that you're getting.
Jarcec
On Sat, Mar 02, 2013 at 03:11:40PM -0500, Cyril Bogus wrote:
> So, I am running a java program to import data from a sqlserver using sqoop.
>
> I have the Microsoft SQL - Hadoop connector in the lib and whenever I run
> commands from the command line the import runs fine.
>
> Problem is when my Java is the one trying to run the command then sqoop
> can't find the lib somehow so it does not connect to the DB.
> I try running the exact same query directly in command line and everything
> works fine.
>
> Is there a way where I can define both the driver and connection manager
> through Java as not to worry about it not finding the proper manager for
> import?
>
> Regards
>
> Cyril