You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Andrey Yarovoy (Jira)" <ji...@apache.org> on 2026/04/27 17:41:00 UTC
[jira] [Created] (HDDS-15128) switch default checksum to CRC32C since its providing better throughput on JRE17+
Andrey Yarovoy created HDDS-15128:
-------------------------------------
Summary: switch default checksum to CRC32C since its providing better throughput on JRE17+
Key: HDDS-15128
URL: https://issues.apache.org/jira/browse/HDDS-15128
Project: Apache Ozone
Issue Type: Bug
Reporter: Andrey Yarovoy
According to the benchmark JVM native CRC32C provide slightly better throughout vs CRC32 on JRE17+. It would also make it consistent with HDFS default settings.
{noformat}
Benchmark (checksumSize) (crcImpl) Mode Cnt Score Error Units
BenchMarkCRCStreaming.runCRC 512 pureCRC32 thrpt 5 14.731 ± 0.214 ops/s
BenchMarkCRCStreaming.runCRC 512 pureCRC32C thrpt 5 14.427 ± 0.199 ops/s
BenchMarkCRCStreaming.runCRC 512 hadoopCRC32C thrpt 5 24.204 ± 0.349 ops/s
BenchMarkCRCStreaming.runCRC 512 hadoopCRC32 thrpt 5 26.560 ± 0.450 ops/s
BenchMarkCRCStreaming.runCRC 512 zipCRC32 thrpt 5 284.283 ± 13.557 ops/s
BenchMarkCRCStreaming.runCRC 512 zipCRC32C thrpt 5 283.479 ± 6.288 ops/s
BenchMarkCRCStreaming.runCRC 1024 pureCRC32 thrpt 5 14.462 ± 0.223 ops/s
BenchMarkCRCStreaming.runCRC 1024 pureCRC32C thrpt 5 14.801 ± 0.289 ops/s
BenchMarkCRCStreaming.runCRC 1024 hadoopCRC32C thrpt 5 24.162 ± 0.382 ops/s
BenchMarkCRCStreaming.runCRC 1024 hadoopCRC32 thrpt 5 25.943 ± 0.406 ops/s
BenchMarkCRCStreaming.runCRC 1024 zipCRC32 thrpt 5 266.230 ± 13.065 ops/s
BenchMarkCRCStreaming.runCRC 1024 zipCRC32C thrpt 5 254.955 ± 10.738 ops/s
BenchMarkCRCStreaming.runCRC 2048 pureCRC32 thrpt 5 14.411 ± 0.217 ops/s
BenchMarkCRCStreaming.runCRC 2048 pureCRC32C thrpt 5 14.752 ± 0.244 ops/s
BenchMarkCRCStreaming.runCRC 2048 hadoopCRC32C thrpt 5 21.074 ± 0.371 ops/s
BenchMarkCRCStreaming.runCRC 2048 hadoopCRC32 thrpt 5 22.141 ± 0.360 ops/s
BenchMarkCRCStreaming.runCRC 2048 zipCRC32 thrpt 5 268.253 ± 10.607 ops/s
BenchMarkCRCStreaming.runCRC 2048 zipCRC32C thrpt 5 259.033 ± 12.080 ops/s
BenchMarkCRCStreaming.runCRC 4096 pureCRC32 thrpt 5 14.817 ± 0.257 ops/s
BenchMarkCRCStreaming.runCRC 4096 pureCRC32C thrpt 5 14.455 ± 0.227 ops/s
BenchMarkCRCStreaming.runCRC 4096 hadoopCRC32C thrpt 5 21.138 ± 0.362 ops/s
BenchMarkCRCStreaming.runCRC 4096 hadoopCRC32 thrpt 5 21.984 ± 0.352 ops/s
BenchMarkCRCStreaming.runCRC 4096 zipCRC32 thrpt 5 280.286 ± 8.329 ops/s
BenchMarkCRCStreaming.runCRC 4096 zipCRC32C thrpt 5 274.376 ± 12.497 ops/s
BenchMarkCRCStreaming.runCRC 32768 pureCRC32 thrpt 5 17.933 ± 0.269 ops/s
BenchMarkCRCStreaming.runCRC 32768 pureCRC32C thrpt 5 17.933 ± 0.268 ops/s
BenchMarkCRCStreaming.runCRC 32768 hadoopCRC32C thrpt 5 21.276 ± 0.289 ops/s
BenchMarkCRCStreaming.runCRC 32768 hadoopCRC32 thrpt 5 21.953 ± 0.327 ops/s
BenchMarkCRCStreaming.runCRC 32768 zipCRC32 thrpt 5 270.707 ± 20.560 ops/s
BenchMarkCRCStreaming.runCRC 32768 zipCRC32C thrpt 5 276.425 ± 8.271 ops/s
BenchMarkCRCStreaming.runCRC 1048576 pureCRC32 thrpt 5 17.579 ± 0.294 ops/s
BenchMarkCRCStreaming.runCRC 1048576 pureCRC32C thrpt 5 17.937 ± 0.283 ops/s
BenchMarkCRCStreaming.runCRC 1048576 hadoopCRC32C thrpt 5 21.141 ± 0.352 ops/s
BenchMarkCRCStreaming.runCRC 1048576 hadoopCRC32 thrpt 5 21.948 ± 0.379 ops/s
BenchMarkCRCStreaming.runCRC 1048576 zipCRC32 thrpt 5 277.069 ± 7.166 ops/s
BenchMarkCRCStreaming.runCRC 1048576 zipCRC32C thrpt 5 280.083 ± 7.043 ops/s{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org