You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Doug Cutting (JIRA)" <ji...@apache.org> on 2011/03/24 17:10:05 UTC
[jira] [Created] (AVRO-788) add Snappy compression
add Snappy compression
----------------------
Key: AVRO-788
URL: https://issues.apache.org/jira/browse/AVRO-788
Project: Avro
Issue Type: New Feature
Components: java, spec
Reporter: Doug Cutting
Google has released it's Snappy compression codec.
https://code.google.com/p/snappy/
Avro should support this for data file compression.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-788) add Snappy compression
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-788:
------------------------------
Fix Version/s: 1.5.1
Assignee: Doug Cutting
> add Snappy compression
> ----------------------
>
> Key: AVRO-788
> URL: https://issues.apache.org/jira/browse/AVRO-788
> Project: Avro
> Issue Type: New Feature
> Components: java, spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.5.1
>
> Attachments: AVRO-788.patch
>
>
> Google has released it's Snappy compression codec.
> https://code.google.com/p/snappy/
> Avro should support this for data file compression.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-788) add Snappy compression
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-788:
------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I committed this.
> add Snappy compression
> ----------------------
>
> Key: AVRO-788
> URL: https://issues.apache.org/jira/browse/AVRO-788
> Project: Avro
> Issue Type: New Feature
> Components: java, spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.5.1
>
> Attachments: AVRO-788.patch, AVRO-788.patch
>
>
> Google has released it's Snappy compression codec.
> https://code.google.com/p/snappy/
> Avro should support this for data file compression.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-788) add Snappy compression
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-788:
------------------------------
Status: Patch Available (was: Open)
Any reason not to commit this soon?
> add Snappy compression
> ----------------------
>
> Key: AVRO-788
> URL: https://issues.apache.org/jira/browse/AVRO-788
> Project: Avro
> Issue Type: New Feature
> Components: java, spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.5.1
>
> Attachments: AVRO-788.patch, AVRO-788.patch
>
>
> Google has released it's Snappy compression codec.
> https://code.google.com/p/snappy/
> Avro should support this for data file compression.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-788) add Snappy compression
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-788:
------------------------------
Attachment: AVRO-788.patch
Here's a patch that implements this.
> add Snappy compression
> ----------------------
>
> Key: AVRO-788
> URL: https://issues.apache.org/jira/browse/AVRO-788
> Project: Avro
> Issue Type: New Feature
> Components: java, spec
> Reporter: Doug Cutting
> Attachments: AVRO-788.patch
>
>
> Google has released it's Snappy compression codec.
> https://code.google.com/p/snappy/
> Avro should support this for data file compression.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-788) add Snappy compression
Posted by "Scott Carey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016188#comment-13016188 ]
Scott Carey commented on AVRO-788:
----------------------------------
+1 Straightforward, looks good.
I wonder about the maven scope. Should this be 'provided' scope due to its use of JNI? See http://code.google.com/p/snappy-java/ towards the bottom. A maven user can override the current scope as needed or exclude the jar, so this isn't critical and can be changed easily later.
> add Snappy compression
> ----------------------
>
> Key: AVRO-788
> URL: https://issues.apache.org/jira/browse/AVRO-788
> Project: Avro
> Issue Type: New Feature
> Components: java, spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.5.1
>
> Attachments: AVRO-788.patch, AVRO-788.patch
>
>
> Google has released it's Snappy compression codec.
> https://code.google.com/p/snappy/
> Avro should support this for data file compression.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-788) add Snappy compression
Posted by "Scott Carey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016193#comment-13016193 ]
Scott Carey commented on AVRO-788:
----------------------------------
+1 OK, tests pass. They embed compiled libraries for Mac, Windows, and Linux. So this feature is available only for those platforms at this time.
> add Snappy compression
> ----------------------
>
> Key: AVRO-788
> URL: https://issues.apache.org/jira/browse/AVRO-788
> Project: Avro
> Issue Type: New Feature
> Components: java, spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.5.1
>
> Attachments: AVRO-788.patch, AVRO-788.patch
>
>
> Google has released it's Snappy compression codec.
> https://code.google.com/p/snappy/
> Avro should support this for data file compression.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-788) add Snappy compression
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-788:
------------------------------
Attachment: AVRO-788.patch
Here's a new version of the patch that adds support for Snappy to Avro's MapReduce API and also updates the spec, adding Snappy as an optional codec for now.
> add Snappy compression
> ----------------------
>
> Key: AVRO-788
> URL: https://issues.apache.org/jira/browse/AVRO-788
> Project: Avro
> Issue Type: New Feature
> Components: java, spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.5.1
>
> Attachments: AVRO-788.patch, AVRO-788.patch
>
>
> Google has released it's Snappy compression codec.
> https://code.google.com/p/snappy/
> Avro should support this for data file compression.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-788) add Snappy compression
Posted by "Scott Carey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016191#comment-13016191 ]
Scott Carey commented on AVRO-788:
----------------------------------
Hold off on my +1. I have one other concern. What JNI library is embedded here? Linux? Windows? Do users have to compile their own?
http://code.google.com/p/snappy-java/source/browse/INSTALL
I'm going to check this more thoroughly on my mac.
> add Snappy compression
> ----------------------
>
> Key: AVRO-788
> URL: https://issues.apache.org/jira/browse/AVRO-788
> Project: Avro
> Issue Type: New Feature
> Components: java, spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.5.1
>
> Attachments: AVRO-788.patch, AVRO-788.patch
>
>
> Google has released it's Snappy compression codec.
> https://code.google.com/p/snappy/
> Avro should support this for data file compression.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-788) add Snappy compression
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016189#comment-13016189 ]
Doug Cutting commented on AVRO-788:
-----------------------------------
I've thus far avoided learning too much about Maven, so don't have an opinion.
> add Snappy compression
> ----------------------
>
> Key: AVRO-788
> URL: https://issues.apache.org/jira/browse/AVRO-788
> Project: Avro
> Issue Type: New Feature
> Components: java, spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.5.1
>
> Attachments: AVRO-788.patch, AVRO-788.patch
>
>
> Google has released it's Snappy compression codec.
> https://code.google.com/p/snappy/
> Avro should support this for data file compression.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-788) add Snappy compression
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13010744#comment-13010744 ]
Harsh J Chouraria commented on AVRO-788:
----------------------------------------
There is also an acknowledged issue ticket for adding Java support. Link for reference: https://code.google.com/p/snappy/issues/detail?id=15
> add Snappy compression
> ----------------------
>
> Key: AVRO-788
> URL: https://issues.apache.org/jira/browse/AVRO-788
> Project: Avro
> Issue Type: New Feature
> Components: java, spec
> Reporter: Doug Cutting
>
> Google has released it's Snappy compression codec.
> https://code.google.com/p/snappy/
> Avro should support this for data file compression.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira