You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Joe Stein <cr...@gmail.com> on 2011/11/05 18:39:09 UTC
Snappy Compression
Hi folks, I am about to create a new JIRA ticket to add Snappy compression
codec. I would like to contribute this myself if that is ok.
I will modify CompressionUtils, CompressionCodec and CompressionUtilsTests
and upload a patch to that ticket once I am done.
There is a good chance I can get to this before Monday but not sure yet and
I expect this being my first toe dip changing the code to take a bit more
time than I expect.
I will be at Hadoop World NYC this week if anyone is around and wants to
put names to faces, let me know. Cheers!
/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/
Re: Snappy Compression
Posted by Joe Stein <cr...@gmail.com>.
Neha, JIRA created, I don't have access to assign the ticket to myself but
if someone else does and wants to-do so please go ahead.
I will try the performance tests patched in KAFKA-176 once I am done and
post the results.
Looking forward to getting my feet wet with this ticket so I can be most
fruitful while helping with replication the week after next.
Cheers!
/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/
On Sat, Nov 5, 2011 at 2:06 PM, Neha Narkhede <ne...@gmail.com>wrote:
> Hi Joe,
>
> Its great to hear you want to contribute the Snappy codec. Please file
> a JIRA and assign it to yourself. Also, wondering if you can try these
> new performance tests on your patch to compare GZIP vs Snappy
> performance - https://issues.apache.org/jira/browse/KAFKA-176
>
> Particularly, producer and consumer throughput with GZIP and Snappy.
>
> Thanks,
> Neha
>
> On Sat, Nov 5, 2011 at 10:39 AM, Joe Stein <cr...@gmail.com> wrote:
> > Hi folks, I am about to create a new JIRA ticket to add Snappy
> compression
> > codec. I would like to contribute this myself if that is ok.
> >
> > I will modify CompressionUtils, CompressionCodec and
> CompressionUtilsTests
> > and upload a patch to that ticket once I am done.
> >
> > There is a good chance I can get to this before Monday but not sure yet
> and
> > I expect this being my first toe dip changing the code to take a bit more
> > time than I expect.
> >
> > I will be at Hadoop World NYC this week if anyone is around and wants to
> > put names to faces, let me know. Cheers!
> >
> > /*
> > Joe Stein
> > http://www.linkedin.com/in/charmalloc
> > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > */
> >
>
Re: Snappy Compression
Posted by Neha Narkhede <ne...@gmail.com>.
Hi Joe,
Its great to hear you want to contribute the Snappy codec. Please file
a JIRA and assign it to yourself. Also, wondering if you can try these
new performance tests on your patch to compare GZIP vs Snappy
performance - https://issues.apache.org/jira/browse/KAFKA-176
Particularly, producer and consumer throughput with GZIP and Snappy.
Thanks,
Neha
On Sat, Nov 5, 2011 at 10:39 AM, Joe Stein <cr...@gmail.com> wrote:
> Hi folks, I am about to create a new JIRA ticket to add Snappy compression
> codec. I would like to contribute this myself if that is ok.
>
> I will modify CompressionUtils, CompressionCodec and CompressionUtilsTests
> and upload a patch to that ticket once I am done.
>
> There is a good chance I can get to this before Monday but not sure yet and
> I expect this being my first toe dip changing the code to take a bit more
> time than I expect.
>
> I will be at Hadoop World NYC this week if anyone is around and wants to
> put names to faces, let me know. Cheers!
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> */
>
Re: Snappy Compression
Posted by JG Homan <jg...@gmail.com>.
I'm here, wearing a linkedin shirt today and an Apache Kafka shirt tomorrow. Come up and say hi!
On Nov 5, 2011, at 4:14 PM, Jay Kreps <ja...@gmail.com> wrote:
> I think Jakob Homan a Kafka comitter from LinkedIn will be in Hadoop world, as will Richard Park who built LinkedIn's Kafka/Hadoop data load processes.
>
> -Jay
>
> On Sat, Nov 5, 2011 at 10:39 AM, Joe Stein <cr...@gmail.com> wrote:
> Hi folks, I am about to create a new JIRA ticket to add Snappy compression
> codec. I would like to contribute this myself if that is ok.
>
> I will modify CompressionUtils, CompressionCodec and CompressionUtilsTests
> and upload a patch to that ticket once I am done.
>
> There is a good chance I can get to this before Monday but not sure yet and
> I expect this being my first toe dip changing the code to take a bit more
> time than I expect.
>
> I will be at Hadoop World NYC this week if anyone is around and wants to
> put names to faces, let me know. Cheers!
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> */
>
Re: Snappy Compression
Posted by Jay Kreps <ja...@gmail.com>.
I think Jakob Homan a Kafka comitter from LinkedIn will be in Hadoop world,
as will Richard Park who built LinkedIn's Kafka/Hadoop data load processes.
-Jay
On Sat, Nov 5, 2011 at 10:39 AM, Joe Stein <cr...@gmail.com> wrote:
> Hi folks, I am about to create a new JIRA ticket to add Snappy compression
> codec. I would like to contribute this myself if that is ok.
>
> I will modify CompressionUtils, CompressionCodec and CompressionUtilsTests
> and upload a patch to that ticket once I am done.
>
> There is a good chance I can get to this before Monday but not sure yet and
> I expect this being my first toe dip changing the code to take a bit more
> time than I expect.
>
> I will be at Hadoop World NYC this week if anyone is around and wants to
> put names to faces, let me know. Cheers!
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> */
>