You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@sqoop.apache.org by prabhu k <pr...@gmail.com> on 2012/07/12 14:55:22 UTC

Import from mysql db to HDFS

Hi user,

While i am import from mysql db to HDFS, I am getting following error.

DB name: books
Table name: authors


sqoop-1.4.1-incubating__hadoop-0.20# bin/sqoop import --connect jdbc:mysql:
10.10.10.10:54310/books --username root --table authors --target-dir
/user/root
Warning: /usr/lib/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: $HADOOP_HOME is deprecated.
12/07/12 18:13:34 ERROR tool.BaseSqoopTool: Got error creating database
manager: java.io.IOException: No manager for connect string: jdbc:mysql:
10.10.10.10:54310/books
        at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:119)
        at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
        at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:441)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
        at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)

Please suggest me, what i am doing wrong?

Thanks,
Prabhu.

Re: Import from mysql db to HDFS

Posted by Jarek Jarcec Cecho <ja...@apache.org>.

Hi Prabhu,
I do not see anything wrong on your log fragment. Could you please execute it again with --verbose parameter and let it running for couple of minutes to see the results?

Jarcec

On Thu, Jul 12, 2012 at 07:02:49PM +0530, prabhu k wrote:
> Thanks for the quick response.
> 
> I have already added the mysql-connector-java-5.1.20-bin.jar into the sqoop
> lib directory.
> 
> I have executed following command with your suggestion, it's executing now,
> but it seems got struck, or executing fine?
> 
> the authors table having only 3 rows. i did't understand what it is going
> on..
> 
> 
> 
> sqoop-1.4.1-incubating__hadoop-0.20# bin/sqoop import --connect
> jdbc:mysql://10.10.10.10:54310/books --username root --table authors
> --target-dir /user/root/authors --append
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: $HADOOP_HOME is deprecated.
> 
> 12/07/12 18:55:04 INFO manager.MySQLManager: Preparing to use a MySQL
> streaming resultset.
> 12/07/12 18:55:04 INFO tool.CodeGenTool: Beginning code generation
> Please suggest me on this issue.
> 
> Thanks,
> Prabhu.
> 
> On Thu, Jul 12, 2012 at 6:42 PM, Jarek Jarcec Cecho <ja...@apache.org>wrote:
> 
> > Hi sir,
> > Your jdbc path seems incorrect to me as I believe that it should be
> > "jdbc:mysql://10.10.10.10:54310/books" (notice two slashes after "mysql:")
> >
> > Also did you downloaded and added mysql jdbc connector to sqoop lib
> > directory?
> >
> > You can download it on following link:
> >
> > http://dev.mysql.com/downloads/connector/j/
> >
> > Jarcec
> >
> > On Thu, Jul 12, 2012 at 06:25:22PM +0530, prabhu k wrote:
> > > Hi user,
> > >
> > > While i am import from mysql db to HDFS, I am getting following error.
> > >
> > > DB name: books
> > > Table name: authors
> > >
> > >
> > > sqoop-1.4.1-incubating__hadoop-0.20# bin/sqoop import --connect
> > jdbc:mysql:
> > > 10.10.10.10:54310/books --username root --table authors --target-dir
> > > /user/root
> > > Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> > > Please set $HBASE_HOME to the root of your HBase installation.
> > > Warning: $HADOOP_HOME is deprecated.
> > > 12/07/12 18:13:34 ERROR tool.BaseSqoopTool: Got error creating database
> > > manager: java.io.IOException: No manager for connect string: jdbc:mysql:
> > > 10.10.10.10:54310/books
> > >         at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:119)
> > >         at
> > org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
> > >         at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
> > >         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:441)
> > >         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> > >         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> > >         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> > >         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> > >         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> > >         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> > >         at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
> > >
> > > Please suggest me, what i am doing wrong?
> > >
> > > Thanks,
> > > Prabhu.
> >

Re: Import from mysql db to HDFS

Posted by prabhu k <pr...@gmail.com>.

Thanks for the quick response.

I have already added the mysql-connector-java-5.1.20-bin.jar into the sqoop
lib directory.

I have executed following command with your suggestion, it's executing now,
but it seems got struck, or executing fine?

the authors table having only 3 rows. i did't understand what it is going
on..



sqoop-1.4.1-incubating__hadoop-0.20# bin/sqoop import --connect
jdbc:mysql://10.10.10.10:54310/books --username root --table authors
--target-dir /user/root/authors --append
Warning: /usr/lib/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: $HADOOP_HOME is deprecated.

12/07/12 18:55:04 INFO manager.MySQLManager: Preparing to use a MySQL
streaming resultset.
12/07/12 18:55:04 INFO tool.CodeGenTool: Beginning code generation
Please suggest me on this issue.

Thanks,
Prabhu.

On Thu, Jul 12, 2012 at 6:42 PM, Jarek Jarcec Cecho <ja...@apache.org>wrote:

> Hi sir,
> Your jdbc path seems incorrect to me as I believe that it should be
> "jdbc:mysql://10.10.10.10:54310/books" (notice two slashes after "mysql:")
>
> Also did you downloaded and added mysql jdbc connector to sqoop lib
> directory?
>
> You can download it on following link:
>
> http://dev.mysql.com/downloads/connector/j/
>
> Jarcec
>
> On Thu, Jul 12, 2012 at 06:25:22PM +0530, prabhu k wrote:
> > Hi user,
> >
> > While i am import from mysql db to HDFS, I am getting following error.
> >
> > DB name: books
> > Table name: authors
> >
> >
> > sqoop-1.4.1-incubating__hadoop-0.20# bin/sqoop import --connect
> jdbc:mysql:
> > 10.10.10.10:54310/books --username root --table authors --target-dir
> > /user/root
> > Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> > Please set $HBASE_HOME to the root of your HBase installation.
> > Warning: $HADOOP_HOME is deprecated.
> > 12/07/12 18:13:34 ERROR tool.BaseSqoopTool: Got error creating database
> > manager: java.io.IOException: No manager for connect string: jdbc:mysql:
> > 10.10.10.10:54310/books
> >         at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:119)
> >         at
> org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
> >         at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
> >         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:441)
> >         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> >         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> >         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> >         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> >         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> >         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> >         at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
> >
> > Please suggest me, what i am doing wrong?
> >
> > Thanks,
> > Prabhu.
>

Re: Import from mysql db to HDFS

Posted by Jarek Jarcec Cecho <ja...@apache.org>.

Hi sir,
Your jdbc path seems incorrect to me as I believe that it should be "jdbc:mysql://10.10.10.10:54310/books" (notice two slashes after "mysql:")

Also did you downloaded and added mysql jdbc connector to sqoop lib directory?

You can download it on following link:

http://dev.mysql.com/downloads/connector/j/

Jarcec

On Thu, Jul 12, 2012 at 06:25:22PM +0530, prabhu k wrote:
> Hi user,
> 
> While i am import from mysql db to HDFS, I am getting following error.
> 
> DB name: books
> Table name: authors
> 
> 
> sqoop-1.4.1-incubating__hadoop-0.20# bin/sqoop import --connect jdbc:mysql:
> 10.10.10.10:54310/books --username root --table authors --target-dir
> /user/root
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: $HADOOP_HOME is deprecated.
> 12/07/12 18:13:34 ERROR tool.BaseSqoopTool: Got error creating database
> manager: java.io.IOException: No manager for connect string: jdbc:mysql:
> 10.10.10.10:54310/books
>         at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:119)
>         at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
>         at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
>         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:441)
>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>         at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
> 
> Please suggest me, what i am doing wrong?
> 
> Thanks,
> Prabhu.