You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@datasketches.apache.org by "freakyzoidberg (via GitHub)" <gi...@apache.org> on 2023/12/19 20:04:40 UTC
[PR] Switch Murmur3 to faster implementation [datasketches-go]
freakyzoidberg opened a new pull request, #3:
URL: https://github.com/apache/datasketches-go/pull/3
Switch Murmur3 dependency to https://github.com/twmb/murmur3 as it yield better performance than the current one.
```
pkg: github.com/apache/datasketches-go/hll
│ before.bench │ after.bench │
│ sec/op │ sec/op vs base │
HLLDataSketch/lgK4_HLL4_uint-10 13.365n ± 1% 9.237n ± 2% -30.88% (p=0.000 n=10)
HLLDataSketch/lgK16_HLL4_uint-10 13.535n ± 1% 9.496n ± 1% -29.84% (p=0.000 n=10)
HLLDataSketch/lgK21_HLL4_uint-10 18.96n ± 1% 13.04n ± 1% -31.25% (p=0.000 n=10)
HLLDataSketch/lgK4_HLL6_uint-10 13.545n ± 1% 9.361n ± 0% -30.89% (p=0.000 n=10)
HLLDataSketch/lgK16_HLL6_uint-10 13.560n ± 1% 9.504n ± 1% -29.91% (p=0.000 n=10)
HLLDataSketch/lgK21_HLL6_uint-10 18.55n ± 0% 12.76n ± 1% -31.23% (p=0.000 n=10)
HLLDataSketch/lgK4_HLL8_uint-10 12.595n ± 1% 8.440n ± 1% -32.99% (p=0.000 n=10)
HLLDataSketch/lgK16_HLL8_uint-10 12.670n ± 1% 8.460n ± 1% -33.23% (p=0.000 n=10)
HLLDataSketch/lgK21_HLL8_uint-10 17.34n ± 1% 11.28n ± 1% -34.97% (p=0.000 n=10)
HLLDataSketch/lgK4_HLL4_slice-10 13.445n ± 0% 9.716n ± 1% -27.74% (p=0.000 n=10)
HLLDataSketch/lgK16_HLL4_slice-10 13.670n ± 1% 9.731n ± 1% -28.82% (p=0.000 n=10)
HLLDataSketch/lgK21_HLL4_slice-10 19.09n ± 0% 13.32n ± 3% -30.21% (p=0.000 n=10)
HLLDataSketch/lgK4_HLL6_slice-10 13.570n ± 0% 9.803n ± 1% -27.76% (p=0.000 n=10)
HLLDataSketch/lgK16_HLL6_slice-10 13.770n ± 1% 9.826n ± 0% -28.64% (p=0.000 n=10)
HLLDataSketch/lgK21_HLL6_slice-10 18.85n ± 1% 12.81n ± 1% -32.06% (p=0.000 n=10)
HLLDataSketch/lgK4_HLL8_slice-10 12.765n ± 1% 8.835n ± 8% -30.79% (p=0.000 n=10)
HLLDataSketch/lgK16_HLL8_slice-10 12.880n ± 1% 8.870n ± 1% -31.13% (p=0.000 n=10)
HLLDataSketch/lgK21_HLL8_slice-10 17.69n ± 1% 11.54n ± 1% -34.72% (p=0.000 n=10)
HLLDataSketch/lgK4_HLL8_union-10 26.76n ± 1% 22.68n ± 1% -15.21% (p=0.000 n=10)
HLLDataSketch/lgK16_HLL8_union-10 26.60n ± 1% 22.68n ± 1% -14.76% (p=0.000 n=10)
HLLDataSketch/lgK21_HLL8_union-10 26.50n ± 1% 22.68n ± 0% -14.43% (p=0.000 n=10)
geomean 16.10n 11.45n -28.87%
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] Switch Murmur3 to faster implementation [datasketches-go]
Posted by "freakyzoidberg (via GitHub)" <gi...@apache.org>.
freakyzoidberg commented on PR #3:
URL: https://github.com/apache/datasketches-go/pull/3#issuecomment-1863561041
Checked a few hash output for equality.
I am actually thinking of checking back in the serialised output test data, to test for possible change like this.
Anyhow, in this case, crc32 for the various generations match with both implementations
```
❯ sdiff <(crc32 ./go_generated_files/*) <(crc32 ./go_generated_files_old/*)
422a5893 ./go_generated_files/hll4_n0_go.sk | 422a5893 ./go_generated_files_old/hll4_n0_go.sk
dcb6ae94 ./go_generated_files/hll4_n1000000_go.sk | dcb6ae94 ./go_generated_files_old/hll4_n1000000_go.sk
415450fd ./go_generated_files/hll4_n100000_go.sk | 415450fd ./go_generated_files_old/hll4_n100000_go.sk
9942a074 ./go_generated_files/hll4_n10000_go.sk | 9942a074 ./go_generated_files_old/hll4_n10000_go.sk
44466b15 ./go_generated_files/hll4_n1000_go.sk | 44466b15 ./go_generated_files_old/hll4_n1000_go.sk
cee02cc2 ./go_generated_files/hll4_n100_go.sk | cee02cc2 ./go_generated_files_old/hll4_n100_go.sk
f6ac7c4e ./go_generated_files/hll4_n10_go.sk | f6ac7c4e ./go_generated_files_old/hll4_n10_go.sk
f39fce1b ./go_generated_files/hll4_n1_go.sk | f39fce1b ./go_generated_files_old/hll4_n1_go.sk
45479c8a ./go_generated_files/hll6_n0_go.sk | 45479c8a ./go_generated_files_old/hll6_n0_go.sk
8f48e75f ./go_generated_files/hll6_n1000000_go.sk | 8f48e75f ./go_generated_files_old/hll6_n1000000_go.sk
cf60fc77 ./go_generated_files/hll6_n100000_go.sk | cf60fc77 ./go_generated_files_old/hll6_n100000_go.sk
2a06d1cb ./go_generated_files/hll6_n10000_go.sk | 2a06d1cb ./go_generated_files_old/hll6_n10000_go.sk
96f9cd30 ./go_generated_files/hll6_n1000_go.sk | 96f9cd30 ./go_generated_files_old/hll6_n1000_go.sk
b37ef4b7 ./go_generated_files/hll6_n100_go.sk | b37ef4b7 ./go_generated_files_old/hll6_n100_go.sk
1567d7bd ./go_generated_files/hll6_n10_go.sk | 1567d7bd ./go_generated_files_old/hll6_n10_go.sk
061f68db ./go_generated_files/hll6_n1_go.sk | 061f68db ./go_generated_files_old/hll6_n1_go.sk
4cf1d0a1 ./go_generated_files/hll8_n0_go.sk | 4cf1d0a1 ./go_generated_files_old/hll8_n0_go.sk
151be68f ./go_generated_files/hll8_n1000000_go.sk | 151be68f ./go_generated_files_old/hll8_n1000000_go.sk
c1170afe ./go_generated_files/hll8_n100000_go.sk | c1170afe ./go_generated_files_old/hll8_n100000_go.sk
3c8bcc28 ./go_generated_files/hll8_n10000_go.sk | 3c8bcc28 ./go_generated_files_old/hll8_n10000_go.sk
935c33e5 ./go_generated_files/hll8_n1000_go.sk | 935c33e5 ./go_generated_files_old/hll8_n1000_go.sk
35dd9c28 ./go_generated_files/hll8_n100_go.sk | 35dd9c28 ./go_generated_files_old/hll8_n100_go.sk
ea4a2de9 ./go_generated_files/hll8_n10_go.sk | ea4a2de9 ./go_generated_files_old/hll8_n10_go.sk
c3ef85da ./go_generated_files/hll8_n1_go.sk | c3ef85da ./go_generated_files_old/hll8_n1_go.sk
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] Switch Murmur3 to faster implementation [datasketches-go]
Posted by "AlexanderSaydakov (via GitHub)" <gi...@apache.org>.
AlexanderSaydakov merged PR #3:
URL: https://github.com/apache/datasketches-go/pull/3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] Switch Murmur3 to faster implementation [datasketches-go]
Posted by "AlexanderSaydakov (via GitHub)" <gi...@apache.org>.
AlexanderSaydakov commented on PR #3:
URL: https://github.com/apache/datasketches-go/pull/3#issuecomment-1863545076
Have you tried to test compatibility with C++ and Java? Even a slightest difference in hash function can make sketches incompatible (disjoint sets despite of the same input)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org