You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Vincent Barat <vb...@capptain.com> on 2012/08/08 12:31:08 UTC
First tests does not work
Hi,
I'm currently starting evaluating Sqoop, and I've trouble doing a
simple import export. I use a local Hadoop cluster and LySQL server.
This works fine:
$ sqoop import --connect jdbc:mysql://localhost/customers --username
analytics --password analytics --table application
$ sqoop export --connect jdbc:mysql://localhost/customers --username
analytics --password analytics --table application_copy --export-dir
application
But as soon as I try to specify delimiters or use sequence files, I
got the following error during export:
...
java.lang.RuntimeException: java.io.IOException: WritableName can't
load class: application
at
org.apache.hadoop.io.SequenceFile$Reader.getValueClass(SequenceFile.java:1673)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1613)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1486)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
at
org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.initialize(SequenceFileRecordReader.java:50)
at
org.apache.sqoop.mapreduce.CombineShimRecordReader.initialize(CombineShimRecordReader.java:76)
at
org.apache.sqoop.mapreduce.CombineFileRecordReader.initialize(CombineFileRecordReader.java:62)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:522)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
Caused by: java.io.IOException: WritableName can't load class:
application
at org.apache.hadoop.io.WritableName.getClass(WritableName.java:73)
at
org.apache.hadoop.io.SequenceFile$Reader.getValueClass(SequenceFile.java:1671)
... 11 more
Caused by: java.lang.ClassNotFoundException: application
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819)
at org.apache.hadoop.io.WritableName.getClass(WritableName.java:71)
... 12 more
I guess that Hadoop tasks cannot find the .class generated, but I
cannot figure out how to configure them to find the .class
Thanks for your help!
--
*Vincent Barat*
*CTO
* logo
*Contact info *
vbarat@capptain.com <mailto:vbarat@capptain.com%20>
www.capptain.com <http://www.capptain.com>
Cell: +33 6 15 41 15 18
*Rennes Office *
Office: +33 2 99 65 69 13
10 rue Jean-Marie Duhamel
35000 Rennes
France
*Paris Office *
Office: +33 1 84 06 13 85
Fax: +33 9 57 72 20 18
18 rue Tronchet
75008 Paris
France
IMPORTANT NOTICE – UBIKOD and CAPPTAIN are registered trademarks of
UBIKOD S.A.R.L., all copyrights are reserved. The contents of this
email and attachments are confidential and may be subject to legal
privilege and/or protected by copyright. Copying or communicating
any part of it to others is prohibited and may be unlawful. If you
are not the intended recipient you must not use, copy, distribute or
rely on this email and should please return it immediately or notify
us by telephone. At present the integrity of email across the
Internet cannot be guaranteed. Therefore UBIKOD S.A.R.L. will not
accept liability for any claims arising as a result of the use of
this medium for transmissions by or to UBIKOD S.A.R.L.. UBIKOD
S.A.R.L. may exercise any of its rights under relevant law, to
monitor the content of all electronic communications. You should
therefore be aware that this communication and any responses might
have been monitored, and may be accessed by UBIKOD S.A.R.L. The
views expressed in this document are that of the individual and may
not necessarily constitute or imply its endorsement or
recommendation by UBIKOD S.A.R.L. The content of this electronic
mail may be subject to the confidentiality terms of a
"Non-Disclosure Agreement" (NDA).
Re: First tests does not work
Posted by Vincent Barat <vb...@capptain.com>.
Sqoop is the latest one (1.4.1)
Here is the full logs:
vbarat@lancelot:~$ sqoop export --verbose --connect
jdbc:mysql://localhost/customers --username analytics --password
analytics --table application_copy --export-dir application
Warning: $HADOOP_HOME is deprecated.
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated.
Please use org.apache.hadoop.log.metrics.EventCounter in all the
log4j.properties files.
12/08/08 12:38:39 WARN tool.BaseSqoopTool: Setting your password on
the command-line is insecure. Consider using -P instead.
12/08/08 12:38:39 INFO manager.MySQLManager: Preparing to use a
MySQL streaming resultset.
12/08/08 12:38:39 INFO tool.CodeGenTool: Beginning code generation
12/08/08 12:38:40 INFO manager.SqlManager: Executing SQL statement:
SELECT t.* FROM `application_copy` AS t LIMIT 1
12/08/08 12:38:40 INFO orm.CompilationManager: HADOOP_HOME is
/Users/vbarat/workspace/third-party/hadoop/1.0.0/hadoop-1.0.0/libexec/..
Note:
/tmp/sqoop-vbarat/compile/e35b7f394407b1e006ed2f72919a5b96/application_copy.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
12/08/08 12:38:41 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-vbarat/compile/e35b7f394407b1e006ed2f72919a5b96/application_copy.jar
12/08/08 12:38:41 INFO mapreduce.ExportJobBase: Beginning export of
application_copy
12/08/08 12:38:41 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java
classes where applicable
12/08/08 12:38:42 INFO input.FileInputFormat: Total input paths to
process : 1
12/08/08 12:38:42 INFO input.FileInputFormat: Total input paths to
process : 1
12/08/08 12:38:42 INFO filecache.TrackerDistributedCacheManager:
Creating avro-1.5.3.jar in
/tmp/hadoop-vbarat/mapred/local/archive/-2932763054757947696_501233603_1499633154/file/Users/vbarat/sqoop/lib/avro-1.5.3.jar-work-4282786485976927657
with rwxr-xr-x
12/08/08 12:38:42 INFO filecache.TrackerDistributedCacheManager:
Extracting
/tmp/hadoop-vbarat/mapred/local/archive/-2932763054757947696_501233603_1499633154/file/Users/vbarat/sqoop/lib/avro-1.5.3.jar-work-4282786485976927657/avro-1.5.3.jar
to
/tmp/hadoop-vbarat/mapred/local/archive/-2932763054757947696_501233603_1499633154/file/Users/vbarat/sqoop/lib/avro-1.5.3.jar-work-4282786485976927657
12/08/08 12:38:42 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/avro-1.5.3.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-2932763054757947696_501233603_1499633154/file/Users/vbarat/sqoop/lib/avro-1.5.3.jar
12/08/08 12:38:42 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/avro-1.5.3.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-2932763054757947696_501233603_1499633154/file/Users/vbarat/sqoop/lib/avro-1.5.3.jar
12/08/08 12:38:42 INFO filecache.TrackerDistributedCacheManager:
Creating sqoop-1.4.1-incubating.jar in
/tmp/hadoop-vbarat/mapred/local/archive/3778677005306021355_336917897_1499633154/file/Users/vbarat/sqoop/sqoop-1.4.1-incubating.jar-work--2516220286540258561
with rwxr-xr-x
12/08/08 12:38:42 INFO filecache.TrackerDistributedCacheManager:
Extracting
/tmp/hadoop-vbarat/mapred/local/archive/3778677005306021355_336917897_1499633154/file/Users/vbarat/sqoop/sqoop-1.4.1-incubating.jar-work--2516220286540258561/sqoop-1.4.1-incubating.jar
to
/tmp/hadoop-vbarat/mapred/local/archive/3778677005306021355_336917897_1499633154/file/Users/vbarat/sqoop/sqoop-1.4.1-incubating.jar-work--2516220286540258561
12/08/08 12:38:42 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/sqoop-1.4.1-incubating.jar as
/tmp/hadoop-vbarat/mapred/local/archive/3778677005306021355_336917897_1499633154/file/Users/vbarat/sqoop/sqoop-1.4.1-incubating.jar
12/08/08 12:38:42 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/sqoop-1.4.1-incubating.jar as
/tmp/hadoop-vbarat/mapred/local/archive/3778677005306021355_336917897_1499633154/file/Users/vbarat/sqoop/sqoop-1.4.1-incubating.jar
12/08/08 12:38:42 INFO filecache.TrackerDistributedCacheManager:
Creating commons-io-1.4.jar in
/tmp/hadoop-vbarat/mapred/local/archive/-1047902349077498192_1670508342_1499633154/file/Users/vbarat/sqoop/lib/commons-io-1.4.jar-work-7815310819134386113
with rwxr-xr-x
12/08/08 12:38:42 INFO filecache.TrackerDistributedCacheManager:
Extracting
/tmp/hadoop-vbarat/mapred/local/archive/-1047902349077498192_1670508342_1499633154/file/Users/vbarat/sqoop/lib/commons-io-1.4.jar-work-7815310819134386113/commons-io-1.4.jar
to
/tmp/hadoop-vbarat/mapred/local/archive/-1047902349077498192_1670508342_1499633154/file/Users/vbarat/sqoop/lib/commons-io-1.4.jar-work-7815310819134386113
12/08/08 12:38:42 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/commons-io-1.4.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-1047902349077498192_1670508342_1499633154/file/Users/vbarat/sqoop/lib/commons-io-1.4.jar
12/08/08 12:38:42 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/commons-io-1.4.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-1047902349077498192_1670508342_1499633154/file/Users/vbarat/sqoop/lib/commons-io-1.4.jar
12/08/08 12:38:42 INFO filecache.TrackerDistributedCacheManager:
Creating snappy-java-1.0.3.2.jar in
/tmp/hadoop-vbarat/mapred/local/archive/-3840060138833921867_-636960180_1499633154/file/Users/vbarat/sqoop/lib/snappy-java-1.0.3.2.jar-work--9215474448556779521
with rwxr-xr-x
12/08/08 12:38:42 INFO filecache.TrackerDistributedCacheManager:
Extracting
/tmp/hadoop-vbarat/mapred/local/archive/-3840060138833921867_-636960180_1499633154/file/Users/vbarat/sqoop/lib/snappy-java-1.0.3.2.jar-work--9215474448556779521/snappy-java-1.0.3.2.jar
to
/tmp/hadoop-vbarat/mapred/local/archive/-3840060138833921867_-636960180_1499633154/file/Users/vbarat/sqoop/lib/snappy-java-1.0.3.2.jar-work--9215474448556779521
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/snappy-java-1.0.3.2.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-3840060138833921867_-636960180_1499633154/file/Users/vbarat/sqoop/lib/snappy-java-1.0.3.2.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/snappy-java-1.0.3.2.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-3840060138833921867_-636960180_1499633154/file/Users/vbarat/sqoop/lib/snappy-java-1.0.3.2.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Creating avro-ipc-1.5.3.jar in
/tmp/hadoop-vbarat/mapred/local/archive/1366776222975634171_-1447973678_1499633154/file/Users/vbarat/sqoop/lib/avro-ipc-1.5.3.jar-work--7543141046611921070
with rwxr-xr-x
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Extracting
/tmp/hadoop-vbarat/mapred/local/archive/1366776222975634171_-1447973678_1499633154/file/Users/vbarat/sqoop/lib/avro-ipc-1.5.3.jar-work--7543141046611921070/avro-ipc-1.5.3.jar
to
/tmp/hadoop-vbarat/mapred/local/archive/1366776222975634171_-1447973678_1499633154/file/Users/vbarat/sqoop/lib/avro-ipc-1.5.3.jar-work--7543141046611921070
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/avro-ipc-1.5.3.jar as
/tmp/hadoop-vbarat/mapred/local/archive/1366776222975634171_-1447973678_1499633154/file/Users/vbarat/sqoop/lib/avro-ipc-1.5.3.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/avro-ipc-1.5.3.jar as
/tmp/hadoop-vbarat/mapred/local/archive/1366776222975634171_-1447973678_1499633154/file/Users/vbarat/sqoop/lib/avro-ipc-1.5.3.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Creating jackson-mapper-asl-1.7.3.jar in
/tmp/hadoop-vbarat/mapred/local/archive/-7668694286571775301_1527514939_1499633154/file/Users/vbarat/sqoop/lib/jackson-mapper-asl-1.7.3.jar-work-2213573873056933696
with rwxr-xr-x
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Extracting
/tmp/hadoop-vbarat/mapred/local/archive/-7668694286571775301_1527514939_1499633154/file/Users/vbarat/sqoop/lib/jackson-mapper-asl-1.7.3.jar-work-2213573873056933696/jackson-mapper-asl-1.7.3.jar
to
/tmp/hadoop-vbarat/mapred/local/archive/-7668694286571775301_1527514939_1499633154/file/Users/vbarat/sqoop/lib/jackson-mapper-asl-1.7.3.jar-work-2213573873056933696
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/jackson-mapper-asl-1.7.3.jar
as
/tmp/hadoop-vbarat/mapred/local/archive/-7668694286571775301_1527514939_1499633154/file/Users/vbarat/sqoop/lib/jackson-mapper-asl-1.7.3.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/jackson-mapper-asl-1.7.3.jar
as
/tmp/hadoop-vbarat/mapred/local/archive/-7668694286571775301_1527514939_1499633154/file/Users/vbarat/sqoop/lib/jackson-mapper-asl-1.7.3.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Creating jopt-simple-3.2.jar in
/tmp/hadoop-vbarat/mapred/local/archive/-7644862055077885339_-556670303_1499633154/file/Users/vbarat/sqoop/lib/jopt-simple-3.2.jar-work-3696150933132609508
with rwxr-xr-x
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Extracting
/tmp/hadoop-vbarat/mapred/local/archive/-7644862055077885339_-556670303_1499633154/file/Users/vbarat/sqoop/lib/jopt-simple-3.2.jar-work-3696150933132609508/jopt-simple-3.2.jar
to
/tmp/hadoop-vbarat/mapred/local/archive/-7644862055077885339_-556670303_1499633154/file/Users/vbarat/sqoop/lib/jopt-simple-3.2.jar-work-3696150933132609508
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/jopt-simple-3.2.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-7644862055077885339_-556670303_1499633154/file/Users/vbarat/sqoop/lib/jopt-simple-3.2.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/jopt-simple-3.2.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-7644862055077885339_-556670303_1499633154/file/Users/vbarat/sqoop/lib/jopt-simple-3.2.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Creating ant-contrib-1.0b3.jar in
/tmp/hadoop-vbarat/mapred/local/archive/1275338773934859493_-921207253_1499633154/file/Users/vbarat/sqoop/lib/ant-contrib-1.0b3.jar-work-7001092805904936654
with rwxr-xr-x
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Extracting
/tmp/hadoop-vbarat/mapred/local/archive/1275338773934859493_-921207253_1499633154/file/Users/vbarat/sqoop/lib/ant-contrib-1.0b3.jar-work-7001092805904936654/ant-contrib-1.0b3.jar
to
/tmp/hadoop-vbarat/mapred/local/archive/1275338773934859493_-921207253_1499633154/file/Users/vbarat/sqoop/lib/ant-contrib-1.0b3.jar-work-7001092805904936654
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/ant-contrib-1.0b3.jar as
/tmp/hadoop-vbarat/mapred/local/archive/1275338773934859493_-921207253_1499633154/file/Users/vbarat/sqoop/lib/ant-contrib-1.0b3.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/ant-contrib-1.0b3.jar as
/tmp/hadoop-vbarat/mapred/local/archive/1275338773934859493_-921207253_1499633154/file/Users/vbarat/sqoop/lib/ant-contrib-1.0b3.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Creating paranamer-2.3.jar in
/tmp/hadoop-vbarat/mapred/local/archive/-360082940192906969_-1088729262_1499633154/file/Users/vbarat/sqoop/lib/paranamer-2.3.jar-work--7598966335764832217
with rwxr-xr-x
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Extracting
/tmp/hadoop-vbarat/mapred/local/archive/-360082940192906969_-1088729262_1499633154/file/Users/vbarat/sqoop/lib/paranamer-2.3.jar-work--7598966335764832217/paranamer-2.3.jar
to
/tmp/hadoop-vbarat/mapred/local/archive/-360082940192906969_-1088729262_1499633154/file/Users/vbarat/sqoop/lib/paranamer-2.3.jar-work--7598966335764832217
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/paranamer-2.3.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-360082940192906969_-1088729262_1499633154/file/Users/vbarat/sqoop/lib/paranamer-2.3.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/paranamer-2.3.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-360082940192906969_-1088729262_1499633154/file/Users/vbarat/sqoop/lib/paranamer-2.3.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Creating avro-mapred-1.5.3.jar in
/tmp/hadoop-vbarat/mapred/local/archive/-2302110616373110307_-190867999_1499633154/file/Users/vbarat/sqoop/lib/avro-mapred-1.5.3.jar-work-7692567895133566489
with rwxr-xr-x
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Extracting
/tmp/hadoop-vbarat/mapred/local/archive/-2302110616373110307_-190867999_1499633154/file/Users/vbarat/sqoop/lib/avro-mapred-1.5.3.jar-work-7692567895133566489/avro-mapred-1.5.3.jar
to
/tmp/hadoop-vbarat/mapred/local/archive/-2302110616373110307_-190867999_1499633154/file/Users/vbarat/sqoop/lib/avro-mapred-1.5.3.jar-work-7692567895133566489
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/avro-mapred-1.5.3.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-2302110616373110307_-190867999_1499633154/file/Users/vbarat/sqoop/lib/avro-mapred-1.5.3.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/avro-mapred-1.5.3.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-2302110616373110307_-190867999_1499633154/file/Users/vbarat/sqoop/lib/avro-mapred-1.5.3.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Creating ant-eclipse-1.0-jvm1.2.jar in
/tmp/hadoop-vbarat/mapred/local/archive/-4863571760386286209_-1718446501_1499633154/file/Users/vbarat/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar-work--9214778226358281619
with rwxr-xr-x
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Extracting
/tmp/hadoop-vbarat/mapred/local/archive/-4863571760386286209_-1718446501_1499633154/file/Users/vbarat/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar-work--9214778226358281619/ant-eclipse-1.0-jvm1.2.jar
to
/tmp/hadoop-vbarat/mapred/local/archive/-4863571760386286209_-1718446501_1499633154/file/Users/vbarat/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar-work--9214778226358281619
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-4863571760386286209_-1718446501_1499633154/file/Users/vbarat/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-4863571760386286209_-1718446501_1499633154/file/Users/vbarat/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Creating mysql-connector-java-5.1.15-bin.jar in
/tmp/hadoop-vbarat/mapred/local/archive/-7101302404897703277_-1291091659_92613978/file/Users/vbarat/sqoop/lib/mysql-connector-java-5.1.15-bin.jar-work--728170204507736459
with rwxr-xr-x
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Extracting
/tmp/hadoop-vbarat/mapred/local/archive/-7101302404897703277_-1291091659_92613978/file/Users/vbarat/sqoop/lib/mysql-connector-java-5.1.15-bin.jar-work--728170204507736459/mysql-connector-java-5.1.15-bin.jar
to
/tmp/hadoop-vbarat/mapred/local/archive/-7101302404897703277_-1291091659_92613978/file/Users/vbarat/sqoop/lib/mysql-connector-java-5.1.15-bin.jar-work--728170204507736459
12/08/08 12:38:43 INFO filecache.TrackerDistributedCacheManager:
Cached
file:///Users/vbarat/sqoop/lib/mysql-connector-java-5.1.15-bin.jar
as
/tmp/hadoop-vbarat/mapred/local/archive/-7101302404897703277_-1291091659_92613978/file/Users/vbarat/sqoop/lib/mysql-connector-java-5.1.15-bin.jar
12/08/08 12:38:44 INFO filecache.TrackerDistributedCacheManager:
Cached
file:///Users/vbarat/sqoop/lib/mysql-connector-java-5.1.15-bin.jar
as
/tmp/hadoop-vbarat/mapred/local/archive/-7101302404897703277_-1291091659_92613978/file/Users/vbarat/sqoop/lib/mysql-connector-java-5.1.15-bin.jar
12/08/08 12:38:44 INFO filecache.TrackerDistributedCacheManager:
Creating jackson-core-asl-1.7.3.jar in
/tmp/hadoop-vbarat/mapred/local/archive/-7954365880444089298_-1051443559_1499633154/file/Users/vbarat/sqoop/lib/jackson-core-asl-1.7.3.jar-work--8102607283327795925
with rwxr-xr-x
12/08/08 12:38:44 INFO filecache.TrackerDistributedCacheManager:
Extracting
/tmp/hadoop-vbarat/mapred/local/archive/-7954365880444089298_-1051443559_1499633154/file/Users/vbarat/sqoop/lib/jackson-core-asl-1.7.3.jar-work--8102607283327795925/jackson-core-asl-1.7.3.jar
to
/tmp/hadoop-vbarat/mapred/local/archive/-7954365880444089298_-1051443559_1499633154/file/Users/vbarat/sqoop/lib/jackson-core-asl-1.7.3.jar-work--8102607283327795925
12/08/08 12:38:44 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/jackson-core-asl-1.7.3.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-7954365880444089298_-1051443559_1499633154/file/Users/vbarat/sqoop/lib/jackson-core-asl-1.7.3.jar
12/08/08 12:38:44 INFO filecache.TrackerDistributedCacheManager:
Cached file:///Users/vbarat/sqoop/lib/jackson-core-asl-1.7.3.jar as
/tmp/hadoop-vbarat/mapred/local/archive/-7954365880444089298_-1051443559_1499633154/file/Users/vbarat/sqoop/lib/jackson-core-asl-1.7.3.jar
12/08/08 12:38:44 INFO mapred.JobClient: Running job: job_local_0001
12/08/08 12:38:44 INFO mapred.Task: Using ResourceCalculatorPlugin
: null
12/08/08 12:38:44 WARN mapred.FileOutputCommitter: Output path is
null in cleanup
12/08/08 12:38:44 WARN mapred.LocalJobRunner: job_local_0001
java.lang.RuntimeException: java.io.IOException: WritableName can't
load class: application
at
org.apache.hadoop.io.SequenceFile$Reader.getValueClass(SequenceFile.java:1673)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1613)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1486)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
at
org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.initialize(SequenceFileRecordReader.java:50)
at
org.apache.sqoop.mapreduce.CombineShimRecordReader.initialize(CombineShimRecordReader.java:76)
at
org.apache.sqoop.mapreduce.CombineFileRecordReader.initialize(CombineFileRecordReader.java:62)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:522)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
Caused by: java.io.IOException: WritableName can't load class:
application
at org.apache.hadoop.io.WritableName.getClass(WritableName.java:73)
at
org.apache.hadoop.io.SequenceFile$Reader.getValueClass(SequenceFile.java:1671)
... 11 more
Caused by: java.lang.ClassNotFoundException: application
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819)
at org.apache.hadoop.io.WritableName.getClass(WritableName.java:71)
... 12 more
12/08/08 12:38:45 INFO mapred.JobClient: map 0% reduce 0%
12/08/08 12:38:45 INFO mapred.JobClient: Job complete: job_local_0001
12/08/08 12:38:45 INFO mapred.JobClient: Counters: 0
12/08/08 12:38:45 INFO mapreduce.ExportJobBase: Transferred 0 bytes
in 3,9557 seconds (0 bytes/sec)
12/08/08 12:38:45 INFO mapreduce.ExportJobBase: Exported 0 records.
12/08/08 12:38:45 ERROR tool.ExportTool: Error during export: Export
job failed!
Le 08/08/12 12:36, Jarek Jarcec Cecho a écrit :
> Hi Vincent,
> would you mind sharing used sqoop version and entire log that was generated with --verbose flag?
>
> Jarcec
>
> On Wed, Aug 08, 2012 at 12:31:08PM +0200, Vincent Barat wrote:
>> Hi,
>>
>> I'm currently starting evaluating Sqoop, and I've trouble doing a
>> simple import export. I use a local Hadoop cluster and LySQL server.
>>
>> This works fine:
>>
>> $ sqoop import --connect jdbc:mysql://localhost/customers --username
>> analytics --password analytics --table application
>> $ sqoop export --connect jdbc:mysql://localhost/customers --username
>> analytics --password analytics --table application_copy --export-dir
>> application
>>
>> But as soon as I try to specify delimiters or use sequence files, I
>> got the following error during export:
>> ...
>>
>> java.lang.RuntimeException: java.io.IOException: WritableName can't
>> load class: application
>> at org.apache.hadoop.io.SequenceFile$Reader.getValueClass(SequenceFile.java:1673)
>> at
>> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1613)
>> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1486)
>> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
>> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
>> at org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.initialize(SequenceFileRecordReader.java:50)
>> at org.apache.sqoop.mapreduce.CombineShimRecordReader.initialize(CombineShimRecordReader.java:76)
>> at org.apache.sqoop.mapreduce.CombineFileRecordReader.initialize(CombineFileRecordReader.java:62)
>> at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:522)
>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
>> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
>> Caused by: java.io.IOException: WritableName can't load class:
>> application
>> at org.apache.hadoop.io.WritableName.getClass(WritableName.java:73)
>> at org.apache.hadoop.io.SequenceFile$Reader.getValueClass(SequenceFile.java:1671)
>> ... 11 more
>> Caused by: java.lang.ClassNotFoundException: application
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:247)
>> at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819)
>> at org.apache.hadoop.io.WritableName.getClass(WritableName.java:71)
>> ... 12 more
>>
>>
>> I guess that Hadoop tasks cannot find the .class generated, but I
>> cannot figure out how to configure them to find the .class
>>
>> Thanks for your help!
>>
>>
>> --
>> *Vincent Barat*
>> *CTO
>> * logo
>> *Contact info *
>> vbarat@capptain.com <mailto:vbarat@capptain.com%20>
>> www.capptain.com <http://www.capptain.com>
>> Cell: +33 6 15 41 15 18
>> *Rennes Office *
>> Office: +33 2 99 65 69 13
>> 10 rue Jean-Marie Duhamel
>> 35000 Rennes
>> France
>> *Paris Office *
>> Office: +33 1 84 06 13 85
>> Fax: +33 9 57 72 20 18
>> 18 rue Tronchet
>> 75008 Paris
>> France
>>
>> IMPORTANT NOTICE – UBIKOD and CAPPTAIN are registered trademarks of
>> UBIKOD S.A.R.L., all copyrights are reserved. The contents of this
>> email and attachments are confidential and may be subject to legal
>> privilege and/or protected by copyright. Copying or communicating
>> any part of it to others is prohibited and may be unlawful. If you
>> are not the intended recipient you must not use, copy, distribute or
>> rely on this email and should please return it immediately or notify
>> us by telephone. At present the integrity of email across the
>> Internet cannot be guaranteed. Therefore UBIKOD S.A.R.L. will not
>> accept liability for any claims arising as a result of the use of
>> this medium for transmissions by or to UBIKOD S.A.R.L.. UBIKOD
>> S.A.R.L. may exercise any of its rights under relevant law, to
>> monitor the content of all electronic communications. You should
>> therefore be aware that this communication and any responses might
>> have been monitored, and may be accessed by UBIKOD S.A.R.L. The
>> views expressed in this document are that of the individual and may
>> not necessarily constitute or imply its endorsement or
>> recommendation by UBIKOD S.A.R.L. The content of this electronic
>> mail may be subject to the confidentiality terms of a
>> "Non-Disclosure Agreement" (NDA).
--
*Vincent Barat*
*CTO
* logo
*Contact info *
vbarat@capptain.com <mailto:vbarat@capptain.com%20>
www.capptain.com <http://www.capptain.com>
Cell: +33 6 15 41 15 18
*Rennes Office *
Office: +33 2 99 65 69 13
10 rue Jean-Marie Duhamel
35000 Rennes
France
*Paris Office *
Office: +33 1 84 06 13 85
Fax: +33 9 57 72 20 18
18 rue Tronchet
75008 Paris
France
IMPORTANT NOTICE – UBIKOD and CAPPTAIN are registered trademarks of
UBIKOD S.A.R.L., all copyrights are reserved. The contents of this
email and attachments are confidential and may be subject to legal
privilege and/or protected by copyright. Copying or communicating
any part of it to others is prohibited and may be unlawful. If you
are not the intended recipient you must not use, copy, distribute or
rely on this email and should please return it immediately or notify
us by telephone. At present the integrity of email across the
Internet cannot be guaranteed. Therefore UBIKOD S.A.R.L. will not
accept liability for any claims arising as a result of the use of
this medium for transmissions by or to UBIKOD S.A.R.L.. UBIKOD
S.A.R.L. may exercise any of its rights under relevant law, to
monitor the content of all electronic communications. You should
therefore be aware that this communication and any responses might
have been monitored, and may be accessed by UBIKOD S.A.R.L. The
views expressed in this document are that of the individual and may
not necessarily constitute or imply its endorsement or
recommendation by UBIKOD S.A.R.L. The content of this electronic
mail may be subject to the confidentiality terms of a
"Non-Disclosure Agreement" (NDA).
Re: First tests does not work
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Vincent,
would you mind sharing used sqoop version and entire log that was generated with --verbose flag?
Jarcec
On Wed, Aug 08, 2012 at 12:31:08PM +0200, Vincent Barat wrote:
> Hi,
>
> I'm currently starting evaluating Sqoop, and I've trouble doing a
> simple import export. I use a local Hadoop cluster and LySQL server.
>
> This works fine:
>
> $ sqoop import --connect jdbc:mysql://localhost/customers --username
> analytics --password analytics --table application
> $ sqoop export --connect jdbc:mysql://localhost/customers --username
> analytics --password analytics --table application_copy --export-dir
> application
>
> But as soon as I try to specify delimiters or use sequence files, I
> got the following error during export:
> ...
>
> java.lang.RuntimeException: java.io.IOException: WritableName can't
> load class: application
> at org.apache.hadoop.io.SequenceFile$Reader.getValueClass(SequenceFile.java:1673)
> at
> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1613)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1486)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
> at org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.initialize(SequenceFileRecordReader.java:50)
> at org.apache.sqoop.mapreduce.CombineShimRecordReader.initialize(CombineShimRecordReader.java:76)
> at org.apache.sqoop.mapreduce.CombineFileRecordReader.initialize(CombineFileRecordReader.java:62)
> at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:522)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
> Caused by: java.io.IOException: WritableName can't load class:
> application
> at org.apache.hadoop.io.WritableName.getClass(WritableName.java:73)
> at org.apache.hadoop.io.SequenceFile$Reader.getValueClass(SequenceFile.java:1671)
> ... 11 more
> Caused by: java.lang.ClassNotFoundException: application
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819)
> at org.apache.hadoop.io.WritableName.getClass(WritableName.java:71)
> ... 12 more
>
>
> I guess that Hadoop tasks cannot find the .class generated, but I
> cannot figure out how to configure them to find the .class
>
> Thanks for your help!
>
>
> --
> *Vincent Barat*
> *CTO
> * logo
> *Contact info *
> vbarat@capptain.com <mailto:vbarat@capptain.com%20>
> www.capptain.com <http://www.capptain.com>
> Cell: +33 6 15 41 15 18
> *Rennes Office *
> Office: +33 2 99 65 69 13
> 10 rue Jean-Marie Duhamel
> 35000 Rennes
> France
> *Paris Office *
> Office: +33 1 84 06 13 85
> Fax: +33 9 57 72 20 18
> 18 rue Tronchet
> 75008 Paris
> France
>
> IMPORTANT NOTICE – UBIKOD and CAPPTAIN are registered trademarks of
> UBIKOD S.A.R.L., all copyrights are reserved. The contents of this
> email and attachments are confidential and may be subject to legal
> privilege and/or protected by copyright. Copying or communicating
> any part of it to others is prohibited and may be unlawful. If you
> are not the intended recipient you must not use, copy, distribute or
> rely on this email and should please return it immediately or notify
> us by telephone. At present the integrity of email across the
> Internet cannot be guaranteed. Therefore UBIKOD S.A.R.L. will not
> accept liability for any claims arising as a result of the use of
> this medium for transmissions by or to UBIKOD S.A.R.L.. UBIKOD
> S.A.R.L. may exercise any of its rights under relevant law, to
> monitor the content of all electronic communications. You should
> therefore be aware that this communication and any responses might
> have been monitored, and may be accessed by UBIKOD S.A.R.L. The
> views expressed in this document are that of the individual and may
> not necessarily constitute or imply its endorsement or
> recommendation by UBIKOD S.A.R.L. The content of this electronic
> mail may be subject to the confidentiality terms of a
> "Non-Disclosure Agreement" (NDA).