You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Eric Czech <er...@nextbigsound.com> on 2012/09/13 06:49:35 UTC
Changing bloom filter false positive ratio
Hi everyone,
I'm running into heap pressure issues and I seem to have traced the
problem to very large bloom filters. The bloom_filter_fp_chance is
set to the default value on all my column families but I'd like to try
changing it on some of them. Can I just change that value through the
cli and restart or are there any concerns I should have before trying
to tweak that parameter?
My cassandra version is 1.0.8.
Thank you!
Re: Changing bloom filter false positive ratio
Posted by Peter Schuller <pe...@infidyne.com>.
> I have a hunch that the SSTable selection based on the Min and Max keys in
> ColumnFamilyStore.markReferenced() means that a higher false positive has
> less of an impact.
>
> it's just a hunch, i've not tested it.
For leveled compaction, yes. For non-leveled, I can't see how it would
since each sstable will effectively cover almost the entire range
(since you're effectively spraying random tokens at it, unless clients
are writing data in md5 order).
(Maybe it's different for ordered partitioning though.)
--
/ Peter Schuller (@scode, http://worldmodscode.wordpress.com)
Re: Changing bloom filter false positive ratio
Posted by aaron morton <aa...@thelastpickle.com>.
I have a hunch that the SSTable selection based on the Min and Max keys in ColumnFamilyStore.markReferenced() means that a higher false positive has less of an impact.
it's just a hunch, i've not tested it.
Cheers
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 14/09/2012, at 4:52 AM, Peter Schuller <pe...@infidyne.com> wrote:
>> changing it on some of them. Can I just change that value through the
>> cli and restart or are there any concerns I should have before trying
>> to tweak that parameter?
>
> You can change it, you don't have to restart. It will take effect on
> new sstables - flushed and compacted, not on pre-existing sstables.
>
> Keep in mind that you can get to about half the size of the default
> (iirc with about 10% fp chance). Beyond that you have extremely
> diminishing returns.
>
> --
> / Peter Schuller (@scode, http://worldmodscode.wordpress.com)
Re: Changing bloom filter false positive ratio
Posted by Eric Czech <er...@nextbigsound.com>.
Thanks Peter.
On Thu, Sep 13, 2012 at 12:52 PM, Peter Schuller
<pe...@infidyne.com> wrote:
>> changing it on some of them. Can I just change that value through the
>> cli and restart or are there any concerns I should have before trying
>> to tweak that parameter?
>
> You can change it, you don't have to restart. It will take effect on
> new sstables - flushed and compacted, not on pre-existing sstables.
>
> Keep in mind that you can get to about half the size of the default
> (iirc with about 10% fp chance). Beyond that you have extremely
> diminishing returns.
>
> --
> / Peter Schuller (@scode, http://worldmodscode.wordpress.com)
Re: Changing bloom filter false positive ratio
Posted by Peter Schuller <pe...@infidyne.com>.
> changing it on some of them. Can I just change that value through the
> cli and restart or are there any concerns I should have before trying
> to tweak that parameter?
You can change it, you don't have to restart. It will take effect on
new sstables - flushed and compacted, not on pre-existing sstables.
Keep in mind that you can get to about half the size of the default
(iirc with about 10% fp chance). Beyond that you have extremely
diminishing returns.
--
/ Peter Schuller (@scode, http://worldmodscode.wordpress.com)