You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Kai Wang <de...@gmail.com> on 2016/05/03 18:47:13 UTC

Bloom filter memory usage disparity

Hi,

I have a table on 3-node cluster. I notice bloom filter memory usage are
very different on one of the node. For a given table, I checked
CassandraMetricsRegistry$JmxGauge.[table]_BloomFilterOffHeapMemoryUsed.Value.
2 of 3 nodes show 1.5GB while the other shows 2.5 GB.

What could be the reason?

That table is using LCS.
bloom_filter_fp_chance=0.1
That table has about 16M keys and 140GB of data.

Thanks.

Re: Bloom filter memory usage disparity

Posted by Jeff Jirsa <je...@crowdstrike.com>.
Even with the same data, bloom filter is based on sstables. If your compaction behaves differently on 2 nodes than the third, your bloom filter RAM usage may be different.


From:  Kai Wang
Reply-To:  "user@cassandra.apache.org"
Date:  Tuesday, May 17, 2016 at 8:02 PM
To:  "user@cassandra.apache.org"
Subject:  Re: Bloom filter memory usage disparity

Alain,

Thanks for replying.

I am using C* 2.2.4. 
Yes the table is RF=3. 
I changed bloom_filter_fp_chance from 0.01 to 0.1 a couple of months ago.


On Tue, May 17, 2016 at 11:05 AM, Alain RODRIGUEZ <ar...@gmail.com> wrote:
Hi, we would need more information here (if you did not solve it yet). 

What is your Cassandra version?
Does this 3 node cluster use a Replication Factor of 3?
Did you change the bloom_filter_fp_chance recently?

That table has about 16M keys and 140GB of data.

Is that the total value or per node? In any case, we need the data size for the 3 nodes to understand.

It might have been a temporary situation, but in this case you would know by now.

C*heers,


2016-05-03 18:47 GMT+02:00 Kai Wang <de...@gmail.com>:
Hi,

I have a table on 3-node cluster. I notice bloom filter memory usage are very different on one of the node. For a given table, I checked CassandraMetricsRegistry$JmxGauge.[table]_BloomFilterOffHeapMemoryUsed.Value. 2 of 3 nodes show 1.5GB while the other shows 2.5 GB.

What could be the reason?

That table is using LCS. 
bloom_filter_fp_chance=0.1
That table has about 16M keys and 140GB of data.

Thanks.

Re: Bloom filter memory usage disparity

Posted by Kai Wang <de...@gmail.com>.
Alain,

Thanks for replying.

I am using C* 2.2.4.
Yes the table is RF=3.
I changed bloom_filter_fp_chance from 0.01 to 0.1 a couple of months ago.


On Tue, May 17, 2016 at 11:05 AM, Alain RODRIGUEZ <ar...@gmail.com>
wrote:

> Hi, we would need more information here (if you did not solve it yet).
>
> What is your Cassandra version?
> Does this 3 node cluster use a Replication Factor of 3?
> Did you change the bloom_filter_fp_chance recently?
>
> That table has about 16M keys and 140GB of data.
>>
>
> Is that the total value or per node? In any case, we need the data size
> for the 3 nodes to understand.
>
> It might have been a temporary situation, but in this case you would know
> by now.
>
> C*heers,
>
>
> 2016-05-03 18:47 GMT+02:00 Kai Wang <de...@gmail.com>:
>
>> Hi,
>>
>> I have a table on 3-node cluster. I notice bloom filter memory usage are
>> very different on one of the node. For a given table, I checked
>> CassandraMetricsRegistry$JmxGauge.[table]_BloomFilterOffHeapMemoryUsed.Value.
>> 2 of 3 nodes show 1.5GB while the other shows 2.5 GB.
>>
>> What could be the reason?
>>
>> That table is using LCS.
>> bloom_filter_fp_chance=0.1
>> That table has about 16M keys and 140GB of data.
>>
>> Thanks.
>>
>
>

Re: Bloom filter memory usage disparity

Posted by Alain RODRIGUEZ <ar...@gmail.com>.
Hi, we would need more information here (if you did not solve it yet).

What is your Cassandra version?
Does this 3 node cluster use a Replication Factor of 3?
Did you change the bloom_filter_fp_chance recently?

That table has about 16M keys and 140GB of data.
>

Is that the total value or per node? In any case, we need the data size for
the 3 nodes to understand.

It might have been a temporary situation, but in this case you would know
by now.

C*heers,


2016-05-03 18:47 GMT+02:00 Kai Wang <de...@gmail.com>:

> Hi,
>
> I have a table on 3-node cluster. I notice bloom filter memory usage are
> very different on one of the node. For a given table, I checked
> CassandraMetricsRegistry$JmxGauge.[table]_BloomFilterOffHeapMemoryUsed.Value.
> 2 of 3 nodes show 1.5GB while the other shows 2.5 GB.
>
> What could be the reason?
>
> That table is using LCS.
> bloom_filter_fp_chance=0.1
> That table has about 16M keys and 140GB of data.
>
> Thanks.
>