You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by sam liu <li...@gmail.com> on 2013/08/06 03:52:08 UTC
Why sqoop picks up hsqldb driver jar when connect to mysql db?
Hi Experts,
Under SQOOP/lib, I have both hsqldb driver jar file and mysql driver jar
file:
- lib/hsqldb-1.8.0.10.jar
- lib/mysql-connector-java-5.1.20-bin.jar
But, when I try to connect to mysql db using sqoop import tool, sqoop will
pick up hsqldb driver jar and returned following exception. Even if I add
'--connection-manager org.apache.sqoop.manager.MySQLManager' into the
command, this issue still exists. But, after I removed the hsqldb jar file
from lib folder, this issue does not exist any more.
Why? What's the correct method to invoke import tool to avoid such issue?
Thanks!
> bin/sqoop import --connect jdbc:mysql://hostname:3306/test
--connection-manager org.apache.sqoop.manager.MySQLManager --username user
--password pwd --table testtbl -m 1 --target-dir /tmp/test5
13/08/05 18:44:03 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
13/08/05 18:44:03 INFO manager.MySQLManager: Preparing to use a MySQL
streaming resultset.
13/08/05 18:44:03 INFO tool.CodeGenTool: Beginning code generation
Exception in thread "main" java.lang.NoSuchMethodError:
org/hsqldb/DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:402)
at java.sql.DriverManager.getConnection(DriverManager.java:450)
at
org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:745)
at
org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
at
org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
at
org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
at
org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:283)
at
org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1255)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1072)
at
org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
--
Sam Liu
Re: Why sqoop picks up hsqldb driver jar when connect to mysql db?
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Sam,
my best guess is that there is somewhere rogue HSQLDB 2.0 jar that is not expected and is taking precedence over the 1.8 version. Are you building your own Sqoop or are you using provided artifacts?
Jarcec
On Mon, Aug 19, 2013 at 05:27:24PM +0800, sam liu wrote:
> Hi Jarek, I am using Hadoop-2.0.4-alpha.
>
>
> 2013/8/12 Jarek Jarcec Cecho <ja...@apache.org>
>
> > Hi Sam,
> > Sqoop uses HSQLDB version 1.8, not 2.0. There was period of time when
> > Hadoop was forcing HSQLDB 2.0 into downstream component classpath, but that
> > was resolved. Please check out SQOOP-571 for more background.
> >
> > I've tried both 1.4.3 and 1.4.3 releases on BigTop 0.6 (that is based on
> > Hadoop 2.0.5-alpha) and both were working for me without any issues, so I'm
> > wondering what Hadoop version are you using?
> >
> > Jarcec
> >
> > On Wed, Aug 07, 2013 at 02:56:47PM +0800, sam liu wrote:
> > > Hi Abe,
> > >
> > > I tried with --driver param, but it does not work for me, and returned
> > the
> > > same exception.
> > >
> > > My command is 'bin/sqoop import --driver com.mysql.jdbc.Driver --connect
> > > jdbc:mysql://hostname:3306/test --username user --password pwd --table
> > > testtbl -m 1 --target-dir /tmp/test6'.
> > >
> > > Thanks!
> > >
> > >
> > > 2013/8/7 Abraham Elmahrek <ab...@cloudera.com>
> > >
> > > > Sam,
> > > >
> > > > Did you try the --driver parameter? It allows you to choose the JDBC
> > > > driver class.
> > > >
> > > > -Abe
> > > >
> > > >
> > > > On Tue, Aug 6, 2013 at 7:54 PM, sam liu <li...@gmail.com>
> > wrote:
> > > >
> > > >> I think this is a defect of Sqoop, and opened a jira for it:
> > > >> https://issues.apache.org/jira/browse/SQOOP-1164. I set the priority
> > as
> > > >> Minor, because it has the workaround: only leave the target driver
> > jar file
> > > >> in the sqoop/lib folder, but remove other driver jar files from the
> > folder.
> > > >> However, it should be mentioned that when we have many dirver jar
> > files
> > > >> under sqoop/lib folder for multiple kinds of databases, or a script
> > will
> > > >> connect to different databases, this workaround can not work well.
> > > >>
> > > >> Thanks!
> > > >>
> > > >>
> > > >> 2013/8/6 sam liu <li...@gmail.com>
> > > >>
> > > >>> Hi Experts,
> > > >>>
> > > >>> Under SQOOP/lib, I have both hsqldb driver jar file and mysql driver
> > jar
> > > >>> file:
> > > >>> - lib/hsqldb-1.8.0.10.jar
> > > >>> - lib/mysql-connector-java-5.1.20-bin.jar
> > > >>>
> > > >>> But, when I try to connect to mysql db using sqoop import tool, sqoop
> > > >>> will pick up hsqldb driver jar and returned following exception.
> > Even if I
> > > >>> add '--connection-manager org.apache.sqoop.manager.MySQLManager'
> > into the
> > > >>> command, this issue still exists. But, after I removed the hsqldb
> > jar file
> > > >>> from lib folder, this issue does not exist any more.
> > > >>>
> > > >>> Why? What's the correct method to invoke import tool to avoid such
> > > >>> issue? Thanks!
> > > >>>
> > > >>> > bin/sqoop import --connect jdbc:mysql://hostname:3306/test
> > > >>> --connection-manager org.apache.sqoop.manager.MySQLManager
> > --username user
> > > >>> --password pwd --table testtbl -m 1 --target-dir /tmp/test5
> > > >>> 13/08/05 18:44:03 WARN tool.BaseSqoopTool: Setting your password on
> > the
> > > >>> command-line is insecure. Consider using -P instead.
> > > >>> 13/08/05 18:44:03 INFO manager.MySQLManager: Preparing to use a MySQL
> > > >>> streaming resultset.
> > > >>> 13/08/05 18:44:03 INFO tool.CodeGenTool: Beginning code generation
> > > >>> Exception in thread "main" java.lang.NoSuchMethodError:
> > > >>>
> > org/hsqldb/DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
> > > >>> at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
> > > >>> at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
> > > >>> at
> > java.sql.DriverManager.getConnection(DriverManager.java:402)
> > > >>> at
> > java.sql.DriverManager.getConnection(DriverManager.java:450)
> > > >>> at
> > > >>>
> > org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:745)
> > > >>> at
> > > >>>
> > org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
> > > >>> at
> > > >>> org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
> > > >>> at
> > > >>> org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
> > > >>> at
> > > >>>
> > org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
> > > >>> at
> > > >>>
> > org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
> > > >>> at
> > > >>>
> > org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:283)
> > > >>> at
> > > >>>
> > org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1255)
> > > >>> at
> > > >>> org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1072)
> > > >>> at
> > > >>> org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
> > > >>> at
> > > >>> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
> > > >>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
> > > >>> at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> > > >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > > >>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> > > >>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> > > >>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> > > >>> at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> > > >>>
> > > >>> --
> > > >>>
> > > >>> Sam Liu
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >>
> > > >> Sam Liu
> > > >>
> > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Sam Liu
> >
>
>
>
> --
>
> Sam Liu
Re: Why sqoop picks up hsqldb driver jar when connect to mysql db?
Posted by sam liu <li...@gmail.com>.
Hi Jarek, I am using Hadoop-2.0.4-alpha.
2013/8/12 Jarek Jarcec Cecho <ja...@apache.org>
> Hi Sam,
> Sqoop uses HSQLDB version 1.8, not 2.0. There was period of time when
> Hadoop was forcing HSQLDB 2.0 into downstream component classpath, but that
> was resolved. Please check out SQOOP-571 for more background.
>
> I've tried both 1.4.3 and 1.4.3 releases on BigTop 0.6 (that is based on
> Hadoop 2.0.5-alpha) and both were working for me without any issues, so I'm
> wondering what Hadoop version are you using?
>
> Jarcec
>
> On Wed, Aug 07, 2013 at 02:56:47PM +0800, sam liu wrote:
> > Hi Abe,
> >
> > I tried with --driver param, but it does not work for me, and returned
> the
> > same exception.
> >
> > My command is 'bin/sqoop import --driver com.mysql.jdbc.Driver --connect
> > jdbc:mysql://hostname:3306/test --username user --password pwd --table
> > testtbl -m 1 --target-dir /tmp/test6'.
> >
> > Thanks!
> >
> >
> > 2013/8/7 Abraham Elmahrek <ab...@cloudera.com>
> >
> > > Sam,
> > >
> > > Did you try the --driver parameter? It allows you to choose the JDBC
> > > driver class.
> > >
> > > -Abe
> > >
> > >
> > > On Tue, Aug 6, 2013 at 7:54 PM, sam liu <li...@gmail.com>
> wrote:
> > >
> > >> I think this is a defect of Sqoop, and opened a jira for it:
> > >> https://issues.apache.org/jira/browse/SQOOP-1164. I set the priority
> as
> > >> Minor, because it has the workaround: only leave the target driver
> jar file
> > >> in the sqoop/lib folder, but remove other driver jar files from the
> folder.
> > >> However, it should be mentioned that when we have many dirver jar
> files
> > >> under sqoop/lib folder for multiple kinds of databases, or a script
> will
> > >> connect to different databases, this workaround can not work well.
> > >>
> > >> Thanks!
> > >>
> > >>
> > >> 2013/8/6 sam liu <li...@gmail.com>
> > >>
> > >>> Hi Experts,
> > >>>
> > >>> Under SQOOP/lib, I have both hsqldb driver jar file and mysql driver
> jar
> > >>> file:
> > >>> - lib/hsqldb-1.8.0.10.jar
> > >>> - lib/mysql-connector-java-5.1.20-bin.jar
> > >>>
> > >>> But, when I try to connect to mysql db using sqoop import tool, sqoop
> > >>> will pick up hsqldb driver jar and returned following exception.
> Even if I
> > >>> add '--connection-manager org.apache.sqoop.manager.MySQLManager'
> into the
> > >>> command, this issue still exists. But, after I removed the hsqldb
> jar file
> > >>> from lib folder, this issue does not exist any more.
> > >>>
> > >>> Why? What's the correct method to invoke import tool to avoid such
> > >>> issue? Thanks!
> > >>>
> > >>> > bin/sqoop import --connect jdbc:mysql://hostname:3306/test
> > >>> --connection-manager org.apache.sqoop.manager.MySQLManager
> --username user
> > >>> --password pwd --table testtbl -m 1 --target-dir /tmp/test5
> > >>> 13/08/05 18:44:03 WARN tool.BaseSqoopTool: Setting your password on
> the
> > >>> command-line is insecure. Consider using -P instead.
> > >>> 13/08/05 18:44:03 INFO manager.MySQLManager: Preparing to use a MySQL
> > >>> streaming resultset.
> > >>> 13/08/05 18:44:03 INFO tool.CodeGenTool: Beginning code generation
> > >>> Exception in thread "main" java.lang.NoSuchMethodError:
> > >>>
> org/hsqldb/DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
> > >>> at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
> > >>> at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
> > >>> at
> java.sql.DriverManager.getConnection(DriverManager.java:402)
> > >>> at
> java.sql.DriverManager.getConnection(DriverManager.java:450)
> > >>> at
> > >>>
> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:745)
> > >>> at
> > >>>
> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
> > >>> at
> > >>> org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
> > >>> at
> > >>> org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
> > >>> at
> > >>>
> org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
> > >>> at
> > >>>
> org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
> > >>> at
> > >>>
> org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:283)
> > >>> at
> > >>>
> org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1255)
> > >>> at
> > >>> org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1072)
> > >>> at
> > >>> org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
> > >>> at
> > >>> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
> > >>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
> > >>> at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> > >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > >>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> > >>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> > >>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> > >>> at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> > >>>
> > >>> --
> > >>>
> > >>> Sam Liu
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >>
> > >> Sam Liu
> > >>
> > >
> > >
> >
> >
> > --
> >
> > Sam Liu
>
--
Sam Liu
Re: Why sqoop picks up hsqldb driver jar when connect to mysql db?
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Sam,
Sqoop uses HSQLDB version 1.8, not 2.0. There was period of time when Hadoop was forcing HSQLDB 2.0 into downstream component classpath, but that was resolved. Please check out SQOOP-571 for more background.
I've tried both 1.4.3 and 1.4.3 releases on BigTop 0.6 (that is based on Hadoop 2.0.5-alpha) and both were working for me without any issues, so I'm wondering what Hadoop version are you using?
Jarcec
On Wed, Aug 07, 2013 at 02:56:47PM +0800, sam liu wrote:
> Hi Abe,
>
> I tried with --driver param, but it does not work for me, and returned the
> same exception.
>
> My command is 'bin/sqoop import --driver com.mysql.jdbc.Driver --connect
> jdbc:mysql://hostname:3306/test --username user --password pwd --table
> testtbl -m 1 --target-dir /tmp/test6'.
>
> Thanks!
>
>
> 2013/8/7 Abraham Elmahrek <ab...@cloudera.com>
>
> > Sam,
> >
> > Did you try the --driver parameter? It allows you to choose the JDBC
> > driver class.
> >
> > -Abe
> >
> >
> > On Tue, Aug 6, 2013 at 7:54 PM, sam liu <li...@gmail.com> wrote:
> >
> >> I think this is a defect of Sqoop, and opened a jira for it:
> >> https://issues.apache.org/jira/browse/SQOOP-1164. I set the priority as
> >> Minor, because it has the workaround: only leave the target driver jar file
> >> in the sqoop/lib folder, but remove other driver jar files from the folder.
> >> However, it should be mentioned that when we have many dirver jar files
> >> under sqoop/lib folder for multiple kinds of databases, or a script will
> >> connect to different databases, this workaround can not work well.
> >>
> >> Thanks!
> >>
> >>
> >> 2013/8/6 sam liu <li...@gmail.com>
> >>
> >>> Hi Experts,
> >>>
> >>> Under SQOOP/lib, I have both hsqldb driver jar file and mysql driver jar
> >>> file:
> >>> - lib/hsqldb-1.8.0.10.jar
> >>> - lib/mysql-connector-java-5.1.20-bin.jar
> >>>
> >>> But, when I try to connect to mysql db using sqoop import tool, sqoop
> >>> will pick up hsqldb driver jar and returned following exception. Even if I
> >>> add '--connection-manager org.apache.sqoop.manager.MySQLManager' into the
> >>> command, this issue still exists. But, after I removed the hsqldb jar file
> >>> from lib folder, this issue does not exist any more.
> >>>
> >>> Why? What's the correct method to invoke import tool to avoid such
> >>> issue? Thanks!
> >>>
> >>> > bin/sqoop import --connect jdbc:mysql://hostname:3306/test
> >>> --connection-manager org.apache.sqoop.manager.MySQLManager --username user
> >>> --password pwd --table testtbl -m 1 --target-dir /tmp/test5
> >>> 13/08/05 18:44:03 WARN tool.BaseSqoopTool: Setting your password on the
> >>> command-line is insecure. Consider using -P instead.
> >>> 13/08/05 18:44:03 INFO manager.MySQLManager: Preparing to use a MySQL
> >>> streaming resultset.
> >>> 13/08/05 18:44:03 INFO tool.CodeGenTool: Beginning code generation
> >>> Exception in thread "main" java.lang.NoSuchMethodError:
> >>> org/hsqldb/DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
> >>> at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
> >>> at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
> >>> at java.sql.DriverManager.getConnection(DriverManager.java:402)
> >>> at java.sql.DriverManager.getConnection(DriverManager.java:450)
> >>> at
> >>> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:745)
> >>> at
> >>> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
> >>> at
> >>> org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
> >>> at
> >>> org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
> >>> at
> >>> org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
> >>> at
> >>> org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
> >>> at
> >>> org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:283)
> >>> at
> >>> org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1255)
> >>> at
> >>> org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1072)
> >>> at
> >>> org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
> >>> at
> >>> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
> >>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
> >>> at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> >>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> >>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> >>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> >>> at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> >>>
> >>> --
> >>>
> >>> Sam Liu
> >>>
> >>
> >>
> >>
> >> --
> >>
> >> Sam Liu
> >>
> >
> >
>
>
> --
>
> Sam Liu
Re: Why sqoop picks up hsqldb driver jar when connect to mysql db?
Posted by sam liu <li...@gmail.com>.
Hi Abe,
I tried with --driver param, but it does not work for me, and returned the
same exception.
My command is 'bin/sqoop import --driver com.mysql.jdbc.Driver --connect
jdbc:mysql://hostname:3306/test --username user --password pwd --table
testtbl -m 1 --target-dir /tmp/test6'.
Thanks!
2013/8/7 Abraham Elmahrek <ab...@cloudera.com>
> Sam,
>
> Did you try the --driver parameter? It allows you to choose the JDBC
> driver class.
>
> -Abe
>
>
> On Tue, Aug 6, 2013 at 7:54 PM, sam liu <li...@gmail.com> wrote:
>
>> I think this is a defect of Sqoop, and opened a jira for it:
>> https://issues.apache.org/jira/browse/SQOOP-1164. I set the priority as
>> Minor, because it has the workaround: only leave the target driver jar file
>> in the sqoop/lib folder, but remove other driver jar files from the folder.
>> However, it should be mentioned that when we have many dirver jar files
>> under sqoop/lib folder for multiple kinds of databases, or a script will
>> connect to different databases, this workaround can not work well.
>>
>> Thanks!
>>
>>
>> 2013/8/6 sam liu <li...@gmail.com>
>>
>>> Hi Experts,
>>>
>>> Under SQOOP/lib, I have both hsqldb driver jar file and mysql driver jar
>>> file:
>>> - lib/hsqldb-1.8.0.10.jar
>>> - lib/mysql-connector-java-5.1.20-bin.jar
>>>
>>> But, when I try to connect to mysql db using sqoop import tool, sqoop
>>> will pick up hsqldb driver jar and returned following exception. Even if I
>>> add '--connection-manager org.apache.sqoop.manager.MySQLManager' into the
>>> command, this issue still exists. But, after I removed the hsqldb jar file
>>> from lib folder, this issue does not exist any more.
>>>
>>> Why? What's the correct method to invoke import tool to avoid such
>>> issue? Thanks!
>>>
>>> > bin/sqoop import --connect jdbc:mysql://hostname:3306/test
>>> --connection-manager org.apache.sqoop.manager.MySQLManager --username user
>>> --password pwd --table testtbl -m 1 --target-dir /tmp/test5
>>> 13/08/05 18:44:03 WARN tool.BaseSqoopTool: Setting your password on the
>>> command-line is insecure. Consider using -P instead.
>>> 13/08/05 18:44:03 INFO manager.MySQLManager: Preparing to use a MySQL
>>> streaming resultset.
>>> 13/08/05 18:44:03 INFO tool.CodeGenTool: Beginning code generation
>>> Exception in thread "main" java.lang.NoSuchMethodError:
>>> org/hsqldb/DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
>>> at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
>>> at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
>>> at java.sql.DriverManager.getConnection(DriverManager.java:402)
>>> at java.sql.DriverManager.getConnection(DriverManager.java:450)
>>> at
>>> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:745)
>>> at
>>> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
>>> at
>>> org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
>>> at
>>> org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
>>> at
>>> org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
>>> at
>>> org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
>>> at
>>> org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:283)
>>> at
>>> org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1255)
>>> at
>>> org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1072)
>>> at
>>> org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
>>> at
>>> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
>>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
>>> at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>>> at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>>>
>>> --
>>>
>>> Sam Liu
>>>
>>
>>
>>
>> --
>>
>> Sam Liu
>>
>
>
--
Sam Liu
Re: Why sqoop picks up hsqldb driver jar when connect to mysql db?
Posted by Abraham Elmahrek <ab...@cloudera.com>.
Sam,
Did you try the --driver parameter? It allows you to choose the JDBC driver
class.
-Abe
On Tue, Aug 6, 2013 at 7:54 PM, sam liu <li...@gmail.com> wrote:
> I think this is a defect of Sqoop, and opened a jira for it:
> https://issues.apache.org/jira/browse/SQOOP-1164. I set the priority as
> Minor, because it has the workaround: only leave the target driver jar file
> in the sqoop/lib folder, but remove other driver jar files from the folder.
> However, it should be mentioned that when we have many dirver jar files
> under sqoop/lib folder for multiple kinds of databases, or a script will
> connect to different databases, this workaround can not work well.
>
> Thanks!
>
>
> 2013/8/6 sam liu <li...@gmail.com>
>
>> Hi Experts,
>>
>> Under SQOOP/lib, I have both hsqldb driver jar file and mysql driver jar
>> file:
>> - lib/hsqldb-1.8.0.10.jar
>> - lib/mysql-connector-java-5.1.20-bin.jar
>>
>> But, when I try to connect to mysql db using sqoop import tool, sqoop
>> will pick up hsqldb driver jar and returned following exception. Even if I
>> add '--connection-manager org.apache.sqoop.manager.MySQLManager' into the
>> command, this issue still exists. But, after I removed the hsqldb jar file
>> from lib folder, this issue does not exist any more.
>>
>> Why? What's the correct method to invoke import tool to avoid such issue?
>> Thanks!
>>
>> > bin/sqoop import --connect jdbc:mysql://hostname:3306/test
>> --connection-manager org.apache.sqoop.manager.MySQLManager --username user
>> --password pwd --table testtbl -m 1 --target-dir /tmp/test5
>> 13/08/05 18:44:03 WARN tool.BaseSqoopTool: Setting your password on the
>> command-line is insecure. Consider using -P instead.
>> 13/08/05 18:44:03 INFO manager.MySQLManager: Preparing to use a MySQL
>> streaming resultset.
>> 13/08/05 18:44:03 INFO tool.CodeGenTool: Beginning code generation
>> Exception in thread "main" java.lang.NoSuchMethodError:
>> org/hsqldb/DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
>> at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
>> at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
>> at java.sql.DriverManager.getConnection(DriverManager.java:402)
>> at java.sql.DriverManager.getConnection(DriverManager.java:450)
>> at
>> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:745)
>> at
>> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
>> at
>> org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
>> at
>> org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
>> at
>> org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
>> at
>> org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
>> at
>> org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:283)
>> at
>> org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1255)
>> at
>> org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1072)
>> at
>> org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
>> at
>> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
>> at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>> at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>>
>> --
>>
>> Sam Liu
>>
>
>
>
> --
>
> Sam Liu
>
Re: Why sqoop picks up hsqldb driver jar when connect to mysql db?
Posted by sam liu <li...@gmail.com>.
I think this is a defect of Sqoop, and opened a jira for it:
https://issues.apache.org/jira/browse/SQOOP-1164. I set the priority as
Minor, because it has the workaround: only leave the target driver jar file
in the sqoop/lib folder, but remove other driver jar files from the folder.
However, it should be mentioned that when we have many dirver jar files
under sqoop/lib folder for multiple kinds of databases, or a script will
connect to different databases, this workaround can not work well.
Thanks!
2013/8/6 sam liu <li...@gmail.com>
> Hi Experts,
>
> Under SQOOP/lib, I have both hsqldb driver jar file and mysql driver jar
> file:
> - lib/hsqldb-1.8.0.10.jar
> - lib/mysql-connector-java-5.1.20-bin.jar
>
> But, when I try to connect to mysql db using sqoop import tool, sqoop will
> pick up hsqldb driver jar and returned following exception. Even if I add
> '--connection-manager org.apache.sqoop.manager.MySQLManager' into the
> command, this issue still exists. But, after I removed the hsqldb jar file
> from lib folder, this issue does not exist any more.
>
> Why? What's the correct method to invoke import tool to avoid such issue?
> Thanks!
>
> > bin/sqoop import --connect jdbc:mysql://hostname:3306/test
> --connection-manager org.apache.sqoop.manager.MySQLManager --username user
> --password pwd --table testtbl -m 1 --target-dir /tmp/test5
> 13/08/05 18:44:03 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> 13/08/05 18:44:03 INFO manager.MySQLManager: Preparing to use a MySQL
> streaming resultset.
> 13/08/05 18:44:03 INFO tool.CodeGenTool: Beginning code generation
> Exception in thread "main" java.lang.NoSuchMethodError:
> org/hsqldb/DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
> at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
> at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
> at java.sql.DriverManager.getConnection(DriverManager.java:402)
> at java.sql.DriverManager.getConnection(DriverManager.java:450)
> at
> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:745)
> at
> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
> at
> org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
> at
> org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
> at
> org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:283)
> at
> org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1255)
> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1072)
> at
> org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
> at
> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>
> --
>
> Sam Liu
>
--
Sam Liu