You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by "Peleg, Eyal" <ey...@intel.com> on 2013/06/09 14:04:56 UTC
failed export command
My export command is as follows:
--table testing --export-dir /user/eyapeleg/test --input-fields-terminated-by '\t' --input-escaped-by '\t' --lines-terminated-by '\n'
* note! I used tab delimited format.
* I omitted the source name
I'm able to export the following table:
a a
b b
but fail to export the next one:
aa a
bb a
I get a parse exception:
java.io.IOException: Could not parse record: aa a
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:80)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183)
at org.apache.hadoop.mapred.MapTask.runNewMapper_aroundBody4(MapTask.java:813)
at org.apache.hadoop.mapred.MapTask$AjcClosure5.run(MapTask.java:1)
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
Best Regards,
---------------------------------------------------------------------
Intel Electronics Ltd.
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
RE: failed export command
Posted by "Peleg, Eyal" <ey...@intel.com>.
13/06/09 05:49:44 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
13/06/09 05:49:44 ERROR sqoop.ConnFactory: Error loading ManagerFactory information from file /usr/lib/sqoop/conf/managers.d/connectors: java.io.IOException: Could not load jar /usr/lib/ into JVM. (Could not find class com.cloudera.sqoop.manager.NetezzaManagerFactory.)
at org.apache.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:92)
at com.cloudera.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:36)
at org.apache.sqoop.ConnFactory.addManagersFromFile(ConnFactory.java:159)
at org.apache.sqoop.ConnFactory.loadManagersFromConfDir(ConnFactory.java:218)
at org.apache.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:83)
at org.apache.sqoop.ConnFactory.<init>(ConnFactory.java:60)
at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:36)
at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:86)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
Caused by: java.lang.ClassNotFoundException: com.cloudera.sqoop.manager.NetezzaManagerFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:627)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:88)
... 15 more
13/06/09 05:49:44 ERROR sqoop.ConnFactory: Could not load ManagerFactory com.cloudera.sqoop.manager.NetezzaManagerFactory (not found)
13/06/09 05:49:44 INFO manager.SqlManager: Using default fetchSize of 1000
13/06/09 05:49:44 INFO tool.CodeGenTool: Beginning code generation
13/06/09 05:49:44 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM testing AS t WHERE 1=0
13/06/09 05:49:44 INFO orm.CompilationManager: HADOOP_HOME is /usr/lib/hadoop/libexec/..
13/06/09 05:49:44 INFO orm.CompilationManager: Found hadoop core jar at: /usr/lib/hadoop/libexec/../hadoop-core.jar
Note: /tmp/sqoop-eyapeleg/compile/a480b3a617b2f64c5964790087de8c94/testing.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
13/06/09 05:49:45 ERROR orm.CompilationManager: Could not rename /tmp/sqoop-eyapeleg/compile/a480b3a617b2f64c5964790087de8c94/testing.java to /home/eyapeleg/./testing.java
org.apache.commons.io.FileExistsException: Destination '/home/eyapeleg/./testing.java' already exists
at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:2378)
at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:227)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:64)
at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:97)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
13/06/09 05:49:45 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-eyapeleg/compile/a480b3a617b2f64c5964790087de8c94/testing.jar
13/06/09 05:49:45 INFO mapreduce.ExportJobBase: Beginning export of testing
13/06/09 05:49:51 INFO input.FileInputFormat: Total input paths to process : 1
13/06/09 05:49:51 INFO input.FileInputFormat: Total input paths to process : 1
13/06/09 05:49:51 INFO mapred.JobClient: Running job: job_201305130941_2944
13/06/09 05:49:52 INFO mapred.JobClient: map 0% reduce 0%
13/06/09 05:50:08 INFO mapred.JobClient: Task Id : attempt_201305130941_2944_m_000000_0, Status : FAILED
java.io.IOException: Could not parse record: aa a
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:80)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183)
at org.apache.hadoop.mapred.MapTask.runNewMapper_aroundBody4(MapTask.java:813)
at org.apache.hadoop.mapred.MapTask$AjcClosure5.run(MapTask.java:1)
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
...
...
at org.apache.hadoop.mapred.MapTask
13/06/09 05:50:16 INFO mapred.JobClient: Task Id : attempt_201305130941_2944_m_000000_1, Status : FAILED
java.io.IOException: Could not parse record: aa a
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:80)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183)
at org.apache.hadoop.mapred.MapTask.runNewMapper_aroundBody4(MapTask.java:813)
at org.apache.hadoop.mapred.MapTask$AjcClosure5.run(MapTask.java:1)
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
...
...
at org.apache.hadoop.mapred.MapTask
13/06/09 05:50:23 INFO mapred.JobClient: Task Id : attempt_201305130941_2944_m_000000_2, Status : FAILED
java.io.IOException: Could not parse record: aa a
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:80)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183)
at org.apache.hadoop.mapred.MapTask.runNewMapper_aroundBody4(MapTask.java:813)
at org.apache.hadoop.mapred.MapTask$AjcClosure5.run(MapTask.java:1)
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
...
...
at org.apache.hadoop.mapred.MapTask
13/06/09 05:50:34 INFO mapred.JobClient: Job complete: job_201305130941_2944
13/06/09 05:50:34 INFO mapred.JobClient: Counters: 8
13/06/09 05:50:34 INFO mapred.JobClient: Job Counters
13/06/09 05:50:34 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=33660
13/06/09 05:50:34 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
13/06/09 05:50:34 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
13/06/09 05:50:34 INFO mapred.JobClient: Rack-local map tasks=3
13/06/09 05:50:34 INFO mapred.JobClient: Launched map tasks=4
13/06/09 05:50:34 INFO mapred.JobClient: Data-local map tasks=1
13/06/09 05:50:34 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
13/06/09 05:50:34 INFO mapred.JobClient: Failed map tasks=1
13/06/09 05:50:35 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 48.7006 seconds (0 bytes/sec)
13/06/09 05:50:35 INFO mapreduce.ExportJobBase: Exported 0 records.
13/06/09 05:50:35 ERROR tool.ExportTool: Error during export: Export job failed!
From: abhijeet gaikwad [mailto:abygaikwad17@gmail.com]
Sent: Sunday, June 09, 2013 15:44
To: user@sqoop.apache.org
Subject: Re: failed export command
Can you run this command with --verbose option in the command and post whole console log here?
Thanks,
Abhijeet
My export command is as follows:
--table testing --export-dir /user/eyapeleg/test --input-fields-terminated-by '\t' --input-escaped-by '\t' --lines-terminated-by '\n'
* note! I used tab delimited format.
* I omitted the source name
I'm able to export the following table:
a a
b b
but fail to export the next one:
aa a
bb a
I get a parse exception:
java.io.IOException: Could not parse record: aa a
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:80)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183)
at org.apache.hadoop.mapred.MapTask.runNewMapper_aroundBody4(MapTask.java:813)
at org.apache.hadoop.mapred.MapTask$AjcClosure5.run(MapTask.java:1)
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
---------------------------------------------------------------------
Intel Electronics Ltd.
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
Re: failed export command
Posted by abhijeet gaikwad <ab...@gmail.com>.
Can you run this command with --verbose option in the command and post
whole console log here?
Thanks,
Abhijeet
On Sun, Jun 9, 2013 at 5:34 PM, Peleg, Eyal <ey...@intel.com> wrote:
> *My export command is as follows:*
>
> ** **
>
> --table testing --export-dir /user/eyapeleg/test
> --input-fields-terminated-by '\t' --input-escaped-by '\t' --lines-terminated-by
> '\n'****
>
> ** **
>
> *** ***note! I used tab delimited format.*
>
> *** ***I omitted the source name*
>
> * *
>
> I’m *able* to export the following table:****
>
> * *
>
> *a a*
>
> b b****
>
> ** **
>
> but *fail* to export the next one:****
>
> ** **
>
> aa a****
>
> bb a****
>
> ** **
>
> I get a parse exception:****
>
> ** **
>
> *java.io.IOException: Could not parse record: aa a*
>
> at
> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:80)*
> ***
>
> at
> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)*
> ***
>
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)****
>
> at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183)
> ****
>
> at
> org.apache.hadoop.mapred.MapTask.runNewMapper_aroundBody4(MapTask.java:813)
> ****
>
> at org.apache.hadoop.mapred.MapTask$AjcClosure5.run(MapTask.java:1)
> ****
>
> at
> org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)*
> ***
>
> ** **
>
> ** **
>
> Best Regards,****
>
> ** **
>
> ---------------------------------------------------------------------
> Intel Electronics Ltd.
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>