You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jay Zhuang (JIRA)" <ji...@apache.org> on 2018/01/08 00:10:00 UTC
[jira] [Commented] (CASSANDRA-9067) BloomFilter serialization
format should not change byte ordering
[ https://issues.apache.org/jira/browse/CASSANDRA-9067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16315523#comment-16315523 ]
Jay Zhuang commented on CASSANDRA-9067:
---------------------------------------
Seems the byte re-ordering is because it needs to backward compatible with the older on-heap BloomFilter file. Removing that does improve the performance: [9067-test|https://github.com/cooldoger/cassandra/tree/9067-test]
Here is the microbench test result before and after the test (The test {{[Add bf bench test|https://github.com/cooldoger/cassandra/commit/0e82a30e852279176e78f036b07a7000cf6f7e12]}} basically serialize and deserialize a BF with {{numElemsInK}} elements):
{noformat}
NoFix:
[java] Benchmark (numElemsInK) Mode Cnt Score Error Units
[java] BloomFilterSerializerBench.serializationTest 10 avgt 4 1863.613 ± 286.654 us/op
[java] BloomFilterSerializerBench.serializationTest 100 avgt 4 15027.986 ± 1021.473 us/op
[java] BloomFilterSerializerBench.serializationTest 1024 avgt 4 143462.679 ± 4833.878 us/op
[java] BloomFilterSerializerBench.serializationTest 10240 avgt 4 1413625.791 ± 72691.584 us/op
[java] BloomFilterSerializerBench.serializationTest 102400 avgt 4 14219387.793 ± 1243380.279 us/op
WithFix:
[java] Benchmark (numElemsInK) Mode Cnt Score Error Units
[java] BloomFilterSerializerBench.serializationTest 10 avgt 4 609.689 ± 312.061 us/op
[java] BloomFilterSerializerBench.serializationTest 100 avgt 4 2509.484 ± 65.843 us/op
[java] BloomFilterSerializerBench.serializationTest 1024 avgt 4 21010.078 ± 4246.935 us/op
[java] BloomFilterSerializerBench.serializationTest 10240 avgt 4 203592.239 ± 28953.074 us/op
[java] BloomFilterSerializerBench.serializationTest 102400 avgt 4 2076461.208 ± 96848.833 us/op
{noformat}
For 10K elements, it has about 3 times improvement, for 100M, it's about 7 times.
But as the BloomFilter file doesn't have version support, how could we migrate the existing format to the new one?
> BloomFilter serialization format should not change byte ordering
> ----------------------------------------------------------------
>
> Key: CASSANDRA-9067
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9067
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Benedict
> Assignee: Jay Zhuang
> Priority: Minor
> Fix For: 4.x
>
>
> As a follow-up to CASSANDRA-9066 and CASSANDRA-9060, it appears we do some unnecessary byte swapping during the serialization of bloom filters, which makes the logic slower and harder to follow. We should either perform them more efficiently (using Long.reverseBytes) or, preferably, eliminate the conversion altogether since it does not appear to serve any purpose.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org