You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Jonathan Haddad <jo...@jonhaddad.com> on 2019/03/02 02:41:45 UTC
Re: CASSANDRA-14482
Hey all,
I finally got around to doing some testing. Nothing too crazy, I had it
run on my laptop while I did other things around the house.
Test 1: Inserting Random Data in a K/V table, 10 million inserts
LZ4 compression rate: 0.909857609644112
ZStd: 0.6136099401596449
Test 2: Inserting fairly compressable text data into K/V table: 20 million
inserts
LZ4: 0.6950432247957942
ZStd: 0.4797311577845362
Definitely worth it at first glance. More testing to come.
Jon
On Sun, Feb 17, 2019 at 8:46 PM dinesh.joshi@yahoo.com.INVALID
<di...@yahoo.com.invalid> wrote:
> Thanks all for your input. The consensus is to go forward with this ticket.
> Dinesh
>
> On Friday, February 15, 2019, 12:54:20 PM PST, Sumanth Pasupuleti <
> spasupuleti@netflix.com.INVALID> wrote:
>
> +1
>
> On Fri, Feb 15, 2019 at 12:14 PM Dikang Gu <di...@gmail.com> wrote:
>
> > +1
> >
> > On Fri, Feb 15, 2019 at 10:27 AM Vinay Chella <vi...@gmail.com>
> > wrote:
> >
> > > We have been using Zstd compressor across different products/services
> > here
> > > and have seen significant improvements, getting this in 4.0 would be a
> > big
> > > win.
> > >
> > > +1
> > >
> > > Thanks,
> > > Vinay Chella
> > >
> > >
> > > On Fri, Feb 15, 2019 at 10:19 AM Jeff Jirsa <jj...@gmail.com> wrote:
> > >
> > > > +1
> > > >
> > > > --
> > > > Jeff Jirsa
> > > >
> > > >
> > > > > On Feb 15, 2019, at 9:35 AM, Jonathan Ellis <jb...@gmail.com>
> > wrote:
> > > > >
> > > > > IMO "add a new compression class that has demonstrable benefits to
> > > Sushma
> > > > > and Joseph" is sufficiently noninvasive that we should allow it
> into
> > > 4.0.
> > > > >
> > > > > On Fri, Feb 15, 2019 at 10:48 AM Dinesh Joshi
> > > > > <di...@yahoo.com.invalid> wrote:
> > > > >
> > > > >> Hey folks,
> > > > >>
> > > > >> Just wanted to get a pulse on whether we can proceed with ZStd
> > > support.
> > > > >> The consensus on the ticket was that it’s a very valuable addition
> > > > without
> > > > >> any risk of destabilizing 4.0. It’s ready to go if there aren’t
> any
> > > > >> objections.
> > > > >>
> > > > >> Dinesh
> > > > >>
> > > > >>
> > ---------------------------------------------------------------------
> > > > >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > >> For additional commands, e-mail: dev-help@cassandra.apache.org
> > > > >>
> > > > >>
> > > > >
> > > > > --
> > > > > Jonathan Ellis
> > > > > co-founder, http://www.datastax.com
> > > > > @spyced
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > > >
> > > >
> > >
> >
> >
> > --
> > Dikang
> >
--
Jon Haddad
http://www.rustyrazorblade.com
twitter: rustyrazorblade
Re: CASSANDRA-14482
Posted by Jonathan Haddad <jo...@jonhaddad.com>.
I plan on doing some more thorough testing, it’s just a matter of finding
time.
On Fri, Mar 1, 2019 at 7:17 PM Dinesh Joshi <dj...@icloud.com.invalid>
wrote:
> Thanks for testing this out Jon. Apart from the compression ratio could
> you also test the insertion rate and GC? 14482 allows you to use full range
> of Zstd compression levels.
>
> Dinesh
>
> > On Mar 1, 2019, at 6:41 PM, Jonathan Haddad <jo...@jonhaddad.com> wrote:
> >
> > Hey all,
> >
> > I finally got around to doing some testing. Nothing too crazy, I had it
> > run on my laptop while I did other things around the house.
> >
> > Test 1: Inserting Random Data in a K/V table, 10 million inserts
> >
> > LZ4 compression rate: 0.909857609644112
> > ZStd: 0.6136099401596449
> >
> > Test 2: Inserting fairly compressable text data into K/V table: 20
> million
> > inserts
> > LZ4: 0.6950432247957942
> > ZStd: 0.4797311577845362
> >
> > Definitely worth it at first glance. More testing to come.
> >
> > Jon
> >
> >
> >
> >
> > On Sun, Feb 17, 2019 at 8:46 PM dinesh.joshi@yahoo.com.INVALID
> > <di...@yahoo.com.invalid> wrote:
> >
> >> Thanks all for your input. The consensus is to go forward with this
> ticket.
> >> Dinesh
> >>
> >> On Friday, February 15, 2019, 12:54:20 PM PST, Sumanth Pasupuleti <
> >> spasupuleti@netflix.com.INVALID> wrote:
> >>
> >> +1
> >>
> >>> On Fri, Feb 15, 2019 at 12:14 PM Dikang Gu <di...@gmail.com> wrote:
> >>>
> >>> +1
> >>>
> >>> On Fri, Feb 15, 2019 at 10:27 AM Vinay Chella <vinaykumarcse@gmail.com
> >
> >>> wrote:
> >>>
> >>>> We have been using Zstd compressor across different products/services
> >>> here
> >>>> and have seen significant improvements, getting this in 4.0 would be a
> >>> big
> >>>> win.
> >>>>
> >>>> +1
> >>>>
> >>>> Thanks,
> >>>> Vinay Chella
> >>>>
> >>>>
> >>>>> On Fri, Feb 15, 2019 at 10:19 AM Jeff Jirsa <jj...@gmail.com>
> wrote:
> >>>>>
> >>>>> +1
> >>>>>
> >>>>> --
> >>>>> Jeff Jirsa
> >>>>>
> >>>>>
> >>>>>> On Feb 15, 2019, at 9:35 AM, Jonathan Ellis <jb...@gmail.com>
> >>> wrote:
> >>>>>>
> >>>>>> IMO "add a new compression class that has demonstrable benefits to
> >>>> Sushma
> >>>>>> and Joseph" is sufficiently noninvasive that we should allow it
> >> into
> >>>> 4.0.
> >>>>>>
> >>>>>> On Fri, Feb 15, 2019 at 10:48 AM Dinesh Joshi
> >>>>>> <di...@yahoo.com.invalid> wrote:
> >>>>>>
> >>>>>>> Hey folks,
> >>>>>>>
> >>>>>>> Just wanted to get a pulse on whether we can proceed with ZStd
> >>>> support.
> >>>>>>> The consensus on the ticket was that it’s a very valuable addition
> >>>>> without
> >>>>>>> any risk of destabilizing 4.0. It’s ready to go if there aren’t
> >> any
> >>>>>>> objections.
> >>>>>>>
> >>>>>>> Dinesh
> >>>>>>>
> >>>>>>>
> >>> ---------------------------------------------------------------------
> >>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Jonathan Ellis
> >>>>>> co-founder, http://www.datastax.com
> >>>>>> @spyced
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> Dikang
> >>>
> >
> >
> >
> > --
> > Jon Haddad
> > http://www.rustyrazorblade.com
> > twitter: rustyrazorblade
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
> --
Jon Haddad
http://www.rustyrazorblade.com
twitter: rustyrazorblade
Re: CASSANDRA-14482
Posted by Dinesh Joshi <dj...@icloud.com.INVALID>.
Thanks for testing this out Jon. Apart from the compression ratio could you also test the insertion rate and GC? 14482 allows you to use full range of Zstd compression levels.
Dinesh
> On Mar 1, 2019, at 6:41 PM, Jonathan Haddad <jo...@jonhaddad.com> wrote:
>
> Hey all,
>
> I finally got around to doing some testing. Nothing too crazy, I had it
> run on my laptop while I did other things around the house.
>
> Test 1: Inserting Random Data in a K/V table, 10 million inserts
>
> LZ4 compression rate: 0.909857609644112
> ZStd: 0.6136099401596449
>
> Test 2: Inserting fairly compressable text data into K/V table: 20 million
> inserts
> LZ4: 0.6950432247957942
> ZStd: 0.4797311577845362
>
> Definitely worth it at first glance. More testing to come.
>
> Jon
>
>
>
>
> On Sun, Feb 17, 2019 at 8:46 PM dinesh.joshi@yahoo.com.INVALID
> <di...@yahoo.com.invalid> wrote:
>
>> Thanks all for your input. The consensus is to go forward with this ticket.
>> Dinesh
>>
>> On Friday, February 15, 2019, 12:54:20 PM PST, Sumanth Pasupuleti <
>> spasupuleti@netflix.com.INVALID> wrote:
>>
>> +1
>>
>>> On Fri, Feb 15, 2019 at 12:14 PM Dikang Gu <di...@gmail.com> wrote:
>>>
>>> +1
>>>
>>> On Fri, Feb 15, 2019 at 10:27 AM Vinay Chella <vi...@gmail.com>
>>> wrote:
>>>
>>>> We have been using Zstd compressor across different products/services
>>> here
>>>> and have seen significant improvements, getting this in 4.0 would be a
>>> big
>>>> win.
>>>>
>>>> +1
>>>>
>>>> Thanks,
>>>> Vinay Chella
>>>>
>>>>
>>>>> On Fri, Feb 15, 2019 at 10:19 AM Jeff Jirsa <jj...@gmail.com> wrote:
>>>>>
>>>>> +1
>>>>>
>>>>> --
>>>>> Jeff Jirsa
>>>>>
>>>>>
>>>>>> On Feb 15, 2019, at 9:35 AM, Jonathan Ellis <jb...@gmail.com>
>>> wrote:
>>>>>>
>>>>>> IMO "add a new compression class that has demonstrable benefits to
>>>> Sushma
>>>>>> and Joseph" is sufficiently noninvasive that we should allow it
>> into
>>>> 4.0.
>>>>>>
>>>>>> On Fri, Feb 15, 2019 at 10:48 AM Dinesh Joshi
>>>>>> <di...@yahoo.com.invalid> wrote:
>>>>>>
>>>>>>> Hey folks,
>>>>>>>
>>>>>>> Just wanted to get a pulse on whether we can proceed with ZStd
>>>> support.
>>>>>>> The consensus on the ticket was that it’s a very valuable addition
>>>>> without
>>>>>>> any risk of destabilizing 4.0. It’s ready to go if there aren’t
>> any
>>>>>>> objections.
>>>>>>>
>>>>>>> Dinesh
>>>>>>>
>>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Jonathan Ellis
>>>>>> co-founder, http://www.datastax.com
>>>>>> @spyced
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Dikang
>>>
>
>
>
> --
> Jon Haddad
> http://www.rustyrazorblade.com
> twitter: rustyrazorblade
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org