Adding serialization support to storm-hdfs

[Mike Thomsen <mi...@gmail.com>]

I sent this pull request and haven't heard anything about it.

https://github.com/apache/storm/pull/351

The current storm-hdfs only supports Writables which means that developers
who use Avro or Java serialization are going to have to implement something
on top of the abstract bolt as things currently stand.

Not trying to be pushy here, I just haven't heard anything and my team
needs this in order to scrap our current approach to getting data into HDFS
from Storm. After making that change, adding serialization support was as
simple as subclassing the sequence file bolt and adding this for the
prepare method:

    public void doPrepare(Map conf, TopologyContext topologyContext,
OutputCollector collector) throws IOException {
        avroKeySchema = new Schema.Parser().parse(avroKeySchemaStr);
        avroValueSchema = new Schema.Parser().parse(avroValueSchemaStr);
        AvroSerialization.setKeyWriterSchema(hdfsConfig, avroKeySchema)
        AvroSerialization.setValueWriterSchema(hdfsConfig, avroValueSchema)

        super.doPrepare(conf, topologyContext, collector)
    }

Thanks,

Mike