You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by xeonmailinglist-gmail <xe...@gmail.com> on 2015/04/27 23:36:03 UTC

how to merge sequence files?

Hi,

I have several directories that contain several files of 
|SequenceFileOutputFormat| with |org.apache.hadoop.io.Text| as key and 
value. I want to merge all these files into one.

I have looked to the join example [1], but it is not working. How I 
merge SequenceFiles?

|+ /root/Programs/hadoop/bin/hadoop jar /root/Programs/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar join -inFormat org.apache.hadoop.mapred.SequenceFileInputFormat -outFormat org.apache.hadoop.mapred.SequenceFileOutputFormat -outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text /wiki-output,/wiki-output2 /aggregate-output
15/04/27 17:34:47 INFO client.RMProxy: Connecting to ResourceManager at /172.16.100.11:8040
java.lang.ClassCastException: class org.apache.hadoop.mapred.SequenceFileOutputFormat
         at java.lang.Class.asSubclass(Class.java:3168)
         at org.apache.hadoop.examples.Join.run(Join.java:112)
         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
         at org.apache.hadoop.examples.Join.main(Join.java:173)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         at java.lang.reflect.Method.invoke(Method.java:622)
         at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
         at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
         at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         at java.lang.reflect.Method.invoke(Method.java:622)
         at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
         at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
|

—

​