You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Aaron Baff <Aa...@telescope.tv> on 2011/10/25 07:33:00 UTC

Writing an Hbase Result object out to SequenceFileOutputFormat

So, I'm trying to write out an Hbase Result object (same one I get from my TableMapper) to a SequenceFileOutputFormat from my Reducer as the value, but I'm getting an error when it's trying to get a serializer. It looks like the SerializationFactory can't find a Serialization (only one listed in the Job's configuration in io.serializations is WritableSerialization) that will accept the Result object. Which is funny, because from the source code of the WritableSerialization.accept(), it uses Writable.class.isAssignableFrom(c). When I run that manually and pass it Result.class, it returns true.


When I use TextOutputFormat, it outputs fine, which I'm guessing is because it just calls .toString() on the Key & Value, so doesn't need to mess with the Serialization stuff. So, any ideas or hints to try and get this working?

In case it matters, I'm on CDH3u1.

    java.lang.NullPointerException
        at org.apache.hadoop.io.serializer.SerializationFactory.getSerializer(SerializationFactory.java:73)
        at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:908)
        at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:843)
        at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:393)
        at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:354)
        at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:427)
        at org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getRecordWriter(SequenceFileOutputFormat.java:61)
        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:559)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:414)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
        at org.apache.hadoop.mapred.Child.main(Child.java:264)