You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Prasad C <pr...@gmail.com> on 2015/10/11 20:51:15 UTC
sqoop error while running import
Hello,
I have a problem and I doubt the problem is connecting to HDFS, but unable
to figure out how to fix this.
My MySQl and HDFS are fine, all services are up and running.
To confirm sqoop connects, I ran this.
pman@pman-lappy:~/hadoop-1.0.3$ sqoop eval --connect
jdbc:mysql://localhost/sqoop --username root --password root --query
"select * from sqoop_test"
Warning: /usr/lib/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: $HADOOP_HOME is deprecated.
15/10/12 00:00:18 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
15/10/12 00:00:18 INFO manager.MySQLManager: Preparing to use a MySQL
streaming resultset.
------------------------
| name |
------------------------
| pMan |
| Sudhagar |
| Sarun |
| Praveen |
| Ragu |
------------------------
pman@pman-lappy:~/hadoop-1.0.3$
And I rant he import command which fails.
pman@pman-lappy:~/hadoop-1.0.3$ sqoop import --connect
jdbc:mysql://localhost/sqoop --username root --password root --table
sqoop_testWarning: /usr/lib/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: $HADOOP_HOME is deprecated.
15/10/12 00:02:12 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
15/10/12 00:02:12 INFO manager.MySQLManager: Preparing to use a MySQL
streaming resultset.
15/10/12 00:02:12 INFO tool.CodeGenTool: Beginning code generation
15/10/12 00:02:12 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM `sqoop_test` AS t LIMIT 1
15/10/12 00:02:12 INFO orm.CompilationManager: HADOOP_HOME is
/home/pman/hadoop-1.0.3/libexec/..
Note:
/tmp/sqoop-pman/compile/0af08da3934f9f78a9f819acdf7ad52c/sqoop_test.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
15/10/12 00:02:14 ERROR orm.CompilationManager: Could not rename
/tmp/sqoop-pman/compile/0af08da3934f9f78a9f819acdf7ad52c/sqoop_test.java to
/home/pman/hadoop-1.0.3/./sqoop_test.java
org.apache.commons.io.FileExistsException: Destination
'/home/pman/hadoop-1.0.3/./sqoop_test.java' already exists
at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:2378)
at
org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:227)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:367)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:453)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
15/10/12 00:02:14 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-pman/compile/0af08da3934f9f78a9f819acdf7ad52c/sqoop_test.jar
15/10/12 00:02:14 WARN manager.MySQLManager: It looks like you are
importing from mysql.
15/10/12 00:02:14 WARN manager.MySQLManager: This transfer can be faster!
Use the --direct
15/10/12 00:02:14 WARN manager.MySQLManager: option to exercise a
MySQL-specific fast path.
15/10/12 00:02:14 INFO manager.MySQLManager: Setting zero DATETIME behavior
to convertToNull (mysql)
15/10/12 00:02:14 ERROR tool.ImportTool: Error during import: No primary
key could be found for table sqoop_test. Please specify one with --split-by
or perform a sequential import with '-m 1'.
pman@pman-lappy:~/hadoop-1.0.3$
Wondering anybody had this error. How did you fix it?
Any thought is appreciated.
Prasad Cholakkottil
Re: sqoop error while running import
Posted by Prasad C <pr...@gmail.com>.
Hello Jarcec,
Thank you so much!
Adding -m1 resolved my import problem, and thanks for the --split-by option.
Prasad Cholakkottil
On Mon, Oct 12, 2015 at 11:43 PM, Jarek Jarcec Cecho <ja...@apache.org>
wrote:
> Hi Prasad,
> it seems that the following is the root problem:
>
> > 15/10/12 00:02:14 ERROR tool.ImportTool: Error during import: No primary
> key could be found for table sqoop_test. Please specify one with --split-by
> or perform a sequential import with '-m 1’.
>
> Have you tried to specify the —split-by or the -m 1?
>
> Also it’s worth mentioning that Hadoop 1.0.3 is very, very old. You might
> want consider upgrading a newer version.
>
> Jarcec
>
> > On Oct 11, 2015, at 11:51 AM, Prasad C <pr...@gmail.com> wrote:
> >
> > Hello,
> >
> > I have a problem and I doubt the problem is connecting to HDFS, but
> unable to figure out how to fix this.
> >
> > My MySQl and HDFS are fine, all services are up and running.
> >
> > To confirm sqoop connects, I ran this.
> >
> > pman@pman-lappy:~/hadoop-1.0.3$ sqoop eval --connect
> jdbc:mysql://localhost/sqoop --username root --password root --query
> "select * from sqoop_test"
> > Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> > Please set $HBASE_HOME to the root of your HBase installation.
> > Warning: $HADOOP_HOME is deprecated.
> >
> > 15/10/12 00:00:18 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> > 15/10/12 00:00:18 INFO manager.MySQLManager: Preparing to use a MySQL
> streaming resultset.
> > ------------------------
> > | name |
> > ------------------------
> > | pMan |
> > | Sudhagar |
> > | Sarun |
> > | Praveen |
> > | Ragu |
> > ------------------------
> > pman@pman-lappy:~/hadoop-1.0.3$
> >
> > And I rant he import command which fails.
> >
> >
> > pman@pman-lappy:~/hadoop-1.0.3$ sqoop import --connect
> jdbc:mysql://localhost/sqoop --username root --password root --table
> sqoop_testWarning: /usr/lib/hbase does not exist! HBase imports will fail.
> > Please set $HBASE_HOME to the root of your HBase installation.
> > Warning: $HADOOP_HOME is deprecated.
> >
> > 15/10/12 00:02:12 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> > 15/10/12 00:02:12 INFO manager.MySQLManager: Preparing to use a MySQL
> streaming resultset.
> > 15/10/12 00:02:12 INFO tool.CodeGenTool: Beginning code generation
> > 15/10/12 00:02:12 INFO manager.SqlManager: Executing SQL statement:
> SELECT t.* FROM `sqoop_test` AS t LIMIT 1
> > 15/10/12 00:02:12 INFO orm.CompilationManager: HADOOP_HOME is
> /home/pman/hadoop-1.0.3/libexec/..
> > Note:
> /tmp/sqoop-pman/compile/0af08da3934f9f78a9f819acdf7ad52c/sqoop_test.java
> uses or overrides a deprecated API.
> > Note: Recompile with -Xlint:deprecation for details.
> > 15/10/12 00:02:14 ERROR orm.CompilationManager: Could not rename
> /tmp/sqoop-pman/compile/0af08da3934f9f78a9f819acdf7ad52c/sqoop_test.java to
> /home/pman/hadoop-1.0.3/./sqoop_test.java
> > org.apache.commons.io.FileExistsException: Destination
> '/home/pman/hadoop-1.0.3/./sqoop_test.java' already exists
> > at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:2378)
> > at
> org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:227)
> > at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
> > at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:367)
> > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:453)
> > at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> > at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> > at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
> > 15/10/12 00:02:14 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-pman/compile/0af08da3934f9f78a9f819acdf7ad52c/sqoop_test.jar
> > 15/10/12 00:02:14 WARN manager.MySQLManager: It looks like you are
> importing from mysql.
> > 15/10/12 00:02:14 WARN manager.MySQLManager: This transfer can be
> faster! Use the --direct
> > 15/10/12 00:02:14 WARN manager.MySQLManager: option to exercise a
> MySQL-specific fast path.
> > 15/10/12 00:02:14 INFO manager.MySQLManager: Setting zero DATETIME
> behavior to convertToNull (mysql)
> > 15/10/12 00:02:14 ERROR tool.ImportTool: Error during import: No primary
> key could be found for table sqoop_test. Please specify one with --split-by
> or perform a sequential import with '-m 1'.
> > pman@pman-lappy:~/hadoop-1.0.3$
> >
> > Wondering anybody had this error. How did you fix it?
> > Any thought is appreciated.
> >
> > Prasad Cholakkottil
>
>
Re: sqoop error while running import
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Prasad,
it seems that the following is the root problem:
> 15/10/12 00:02:14 ERROR tool.ImportTool: Error during import: No primary key could be found for table sqoop_test. Please specify one with --split-by or perform a sequential import with '-m 1’.
Have you tried to specify the —split-by or the -m 1?
Also it’s worth mentioning that Hadoop 1.0.3 is very, very old. You might want consider upgrading a newer version.
Jarcec
> On Oct 11, 2015, at 11:51 AM, Prasad C <pr...@gmail.com> wrote:
>
> Hello,
>
> I have a problem and I doubt the problem is connecting to HDFS, but unable to figure out how to fix this.
>
> My MySQl and HDFS are fine, all services are up and running.
>
> To confirm sqoop connects, I ran this.
>
> pman@pman-lappy:~/hadoop-1.0.3$ sqoop eval --connect jdbc:mysql://localhost/sqoop --username root --password root --query "select * from sqoop_test"
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: $HADOOP_HOME is deprecated.
>
> 15/10/12 00:00:18 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
> 15/10/12 00:00:18 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
> ------------------------
> | name |
> ------------------------
> | pMan |
> | Sudhagar |
> | Sarun |
> | Praveen |
> | Ragu |
> ------------------------
> pman@pman-lappy:~/hadoop-1.0.3$
>
> And I rant he import command which fails.
>
>
> pman@pman-lappy:~/hadoop-1.0.3$ sqoop import --connect jdbc:mysql://localhost/sqoop --username root --password root --table sqoop_testWarning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: $HADOOP_HOME is deprecated.
>
> 15/10/12 00:02:12 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
> 15/10/12 00:02:12 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
> 15/10/12 00:02:12 INFO tool.CodeGenTool: Beginning code generation
> 15/10/12 00:02:12 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `sqoop_test` AS t LIMIT 1
> 15/10/12 00:02:12 INFO orm.CompilationManager: HADOOP_HOME is /home/pman/hadoop-1.0.3/libexec/..
> Note: /tmp/sqoop-pman/compile/0af08da3934f9f78a9f819acdf7ad52c/sqoop_test.java uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 15/10/12 00:02:14 ERROR orm.CompilationManager: Could not rename /tmp/sqoop-pman/compile/0af08da3934f9f78a9f819acdf7ad52c/sqoop_test.java to /home/pman/hadoop-1.0.3/./sqoop_test.java
> org.apache.commons.io.FileExistsException: Destination '/home/pman/hadoop-1.0.3/./sqoop_test.java' already exists
> at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:2378)
> at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:227)
> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:367)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:453)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
> 15/10/12 00:02:14 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-pman/compile/0af08da3934f9f78a9f819acdf7ad52c/sqoop_test.jar
> 15/10/12 00:02:14 WARN manager.MySQLManager: It looks like you are importing from mysql.
> 15/10/12 00:02:14 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
> 15/10/12 00:02:14 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
> 15/10/12 00:02:14 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
> 15/10/12 00:02:14 ERROR tool.ImportTool: Error during import: No primary key could be found for table sqoop_test. Please specify one with --split-by or perform a sequential import with '-m 1'.
> pman@pman-lappy:~/hadoop-1.0.3$
>
> Wondering anybody had this error. How did you fix it?
> Any thought is appreciated.
>
> Prasad Cholakkottil