You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Michael McCandless (Jira)" <ji...@apache.org> on 2020/06/14 18:40:00 UTC
[jira] [Commented] (LUCENE-9403) tune
BufferedChecksum.DEFAULT_BUFFERSIZE
[ https://issues.apache.org/jira/browse/LUCENE-9403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17135263#comment-17135263 ]
Michael McCandless commented on LUCENE-9403:
--------------------------------------------
+1 looks compelling!
> tune BufferedChecksum.DEFAULT_BUFFERSIZE
> ----------------------------------------
>
> Key: LUCENE-9403
> URL: https://issues.apache.org/jira/browse/LUCENE-9403
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Robert Muir
> Priority: Major
> Attachments: LUCENE-9403.patch
>
>
> This is currently set to 256 bytes, so that's the amount of data we pass to crc.update() at once.
> I tried different sizes with https://github.com/benalexau/hash-bench and JDK14:
> {noformat}
> HashBench.withArray crc32-jre 256 avgt 5 81.349 ± 8.364 ns/op
> HashBench.withArray crc32-jre 512 avgt 5 95.204 ± 10.057 ns/op
> HashBench.withArray crc32-jre 1024 avgt 5 120.081 ± 8.471 ns/op
> HashBench.withArray crc32-jre 2048 avgt 5 173.505 ± 8.857 ns/op
> HashBench.withArray crc32-jre 8192 avgt 5 487.721 ± 11.435 ns/op
> {noformat}
> based on this let's bump the buffersize from 256 to 1024? I think we want to avoid huge buffers but still keep the CPU overhead low. It only impacts ChecksumIndexInputs (e.g. speed of checkIntegrity() calls at merge) because IndexOutputs do not need this buffer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org