[jira] [Commented] (AVRO-1243) Support all compression codecs

["Ted Malaska (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-1243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573598#comment-13573598 ] 

Ted Malaska commented on AVRO-1243:
-----------------------------------

Hmm.  Now I understand why this is difficult.  All the hadoop compression codec use the org.apache.hadoop.io.compress.CompressionCodec interface.  

And there are no maven dependancies in the Avro maven module for Hadoop, so this interface will not be there.

But there are dependancies in the avro-mapred module for Hadoop.  So I will try something that I think will work in terms of execution but because I'm new to Avro I'm not sure it will fit in with the Avro spirt.

I will try to get my patch in today and I will be interested to see what the Avro community feedback will be.



                
> Support all compression codecs
> ------------------------------
>
>                 Key: AVRO-1243
>                 URL: https://issues.apache.org/jira/browse/AVRO-1243
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>    Affects Versions: 1.7.3
>            Reporter: Ted Malaska
>            Priority: Minor
>
> I may be reading this wrong but at this time org.apache.avro.file.CodecFactory only supports null, deflate, and snappy compression codecs.
> I would like to change the fromString method to use Class.forName(codec).newInstance(); after the codec was not found in the REGISTERED map but before the AvroRuntimeException is thrown. 
> Here are some of my supporting thoughts
> 1. This should not interduce much slowness because it will only be called initialize.
> 2. This will allow for support for GZip, BZip2, and LZO with out adding more dependances to the maven pom file.
> 3. This will allow for a future Jiri I would like to do that would allow AvroOutputFormat to be able to use the following configs: mapred.output.compress and mapred.output.compression.codec

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira