You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jaroslaw Grabowski (Jira)" <ji...@apache.org> on 2020/05/26 10:33:00 UTC
[jira] [Updated] (CASSANDRA-15834) Bloom filter false positive rate
calculation does not take into account true negatives
[ https://issues.apache.org/jira/browse/CASSANDRA-15834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jaroslaw Grabowski updated CASSANDRA-15834:
-------------------------------------------
Description:
The bloom filter false positive ratio is [currently computed|https://github.com/apache/cassandra/blob/ded62076e7fdfd1cfdcf96447489ea607ca796a0/src/java/org/apache/cassandra/metrics/TableMetrics.java#L738] as:
{{bf_fp_ratio = false_positive_count / (false_positive_count + true_positive_count)}}
However, this calculation doesn't take into account true negatives (false negatives never happen on bloom filters).
In a situation where there are 1000 reads for non existing rows, and there are 10 false positives, the bloom filter false positive ratio will be wrongly calculated as 10/10 = 1.0, while it should be 10/1000 = 0.01.
We should update the calculation to:
{{bf_fp_ratio = false_positive_count / #bf_queries}}
was:
The bloom filter false positive ratio is [currently computed|https://github.com/apache/cassandra/blob/ded62076e7fdfd1cfdcf96447489ea607ca796a0/src/java/org/apache/cassandra/metrics/TableMetrics.java#L738] as:
{\{bf_fp_ratio = false_positive_count / (false_positive_count + true_positive_count)}}
However, this calculation doesn't take into account true negatives (false negatives never happen on bloom filters).
In a situation where there are 1000 reads for non existing rows, and there are 10 false positives, the bloom filter false positive ratio will be wrongly calculated as 10/10 = 1.0, while it should be 10/1000 = 0.01.
We should update the calculation to:
{\{bf_fp_ratio = false_positive_count / #bf_queries}}
> Bloom filter false positive rate calculation does not take into account true negatives
> --------------------------------------------------------------------------------------
>
> Key: CASSANDRA-15834
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15834
> Project: Cassandra
> Issue Type: Bug
> Reporter: Jaroslaw Grabowski
> Priority: Normal
>
> The bloom filter false positive ratio is [currently computed|https://github.com/apache/cassandra/blob/ded62076e7fdfd1cfdcf96447489ea607ca796a0/src/java/org/apache/cassandra/metrics/TableMetrics.java#L738] as:
> {{bf_fp_ratio = false_positive_count / (false_positive_count + true_positive_count)}}
> However, this calculation doesn't take into account true negatives (false negatives never happen on bloom filters).
> In a situation where there are 1000 reads for non existing rows, and there are 10 false positives, the bloom filter false positive ratio will be wrongly calculated as 10/10 = 1.0, while it should be 10/1000 = 0.01.
> We should update the calculation to:
> {{bf_fp_ratio = false_positive_count / #bf_queries}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org