You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Andrey Yarovoy (Jira)" <ji...@apache.org> on 2026/04/27 17:41:00 UTC

[jira] [Created] (HDDS-15128) switch default checksum to CRC32C since its providing better throughput on JRE17+

Andrey Yarovoy created HDDS-15128:
-------------------------------------

             Summary: switch default checksum to CRC32C since its providing better throughput on JRE17+
                 Key: HDDS-15128
                 URL: https://issues.apache.org/jira/browse/HDDS-15128
             Project: Apache Ozone
          Issue Type: Bug
            Reporter: Andrey Yarovoy


According to the benchmark JVM native CRC32C provide slightly better throughout vs CRC32 on JRE17+. It would also make it consistent with HDFS default settings.
{noformat}
Benchmark                     (checksumSize)     (crcImpl)   Mode  Cnt    Score    Error  Units
BenchMarkCRCStreaming.runCRC             512     pureCRC32  thrpt    5   14.731 ±  0.214  ops/s
BenchMarkCRCStreaming.runCRC             512    pureCRC32C  thrpt    5   14.427 ±  0.199  ops/s
BenchMarkCRCStreaming.runCRC             512  hadoopCRC32C  thrpt    5   24.204 ±  0.349  ops/s
BenchMarkCRCStreaming.runCRC             512   hadoopCRC32  thrpt    5   26.560 ±  0.450  ops/s
BenchMarkCRCStreaming.runCRC             512      zipCRC32  thrpt    5  284.283 ± 13.557  ops/s
BenchMarkCRCStreaming.runCRC             512     zipCRC32C  thrpt    5  283.479 ±  6.288  ops/s
BenchMarkCRCStreaming.runCRC            1024     pureCRC32  thrpt    5   14.462 ±  0.223  ops/s
BenchMarkCRCStreaming.runCRC            1024    pureCRC32C  thrpt    5   14.801 ±  0.289  ops/s
BenchMarkCRCStreaming.runCRC            1024  hadoopCRC32C  thrpt    5   24.162 ±  0.382  ops/s
BenchMarkCRCStreaming.runCRC            1024   hadoopCRC32  thrpt    5   25.943 ±  0.406  ops/s
BenchMarkCRCStreaming.runCRC            1024      zipCRC32  thrpt    5  266.230 ± 13.065  ops/s
BenchMarkCRCStreaming.runCRC            1024     zipCRC32C  thrpt    5  254.955 ± 10.738  ops/s
BenchMarkCRCStreaming.runCRC            2048     pureCRC32  thrpt    5   14.411 ±  0.217  ops/s
BenchMarkCRCStreaming.runCRC            2048    pureCRC32C  thrpt    5   14.752 ±  0.244  ops/s
BenchMarkCRCStreaming.runCRC            2048  hadoopCRC32C  thrpt    5   21.074 ±  0.371  ops/s
BenchMarkCRCStreaming.runCRC            2048   hadoopCRC32  thrpt    5   22.141 ±  0.360  ops/s
BenchMarkCRCStreaming.runCRC            2048      zipCRC32  thrpt    5  268.253 ± 10.607  ops/s
BenchMarkCRCStreaming.runCRC            2048     zipCRC32C  thrpt    5  259.033 ± 12.080  ops/s
BenchMarkCRCStreaming.runCRC            4096     pureCRC32  thrpt    5   14.817 ±  0.257  ops/s
BenchMarkCRCStreaming.runCRC            4096    pureCRC32C  thrpt    5   14.455 ±  0.227  ops/s
BenchMarkCRCStreaming.runCRC            4096  hadoopCRC32C  thrpt    5   21.138 ±  0.362  ops/s
BenchMarkCRCStreaming.runCRC            4096   hadoopCRC32  thrpt    5   21.984 ±  0.352  ops/s
BenchMarkCRCStreaming.runCRC            4096      zipCRC32  thrpt    5  280.286 ±  8.329  ops/s
BenchMarkCRCStreaming.runCRC            4096     zipCRC32C  thrpt    5  274.376 ± 12.497  ops/s
BenchMarkCRCStreaming.runCRC           32768     pureCRC32  thrpt    5   17.933 ±  0.269  ops/s
BenchMarkCRCStreaming.runCRC           32768    pureCRC32C  thrpt    5   17.933 ±  0.268  ops/s
BenchMarkCRCStreaming.runCRC           32768  hadoopCRC32C  thrpt    5   21.276 ±  0.289  ops/s
BenchMarkCRCStreaming.runCRC           32768   hadoopCRC32  thrpt    5   21.953 ±  0.327  ops/s
BenchMarkCRCStreaming.runCRC           32768      zipCRC32  thrpt    5  270.707 ± 20.560  ops/s
BenchMarkCRCStreaming.runCRC           32768     zipCRC32C  thrpt    5  276.425 ±  8.271  ops/s
BenchMarkCRCStreaming.runCRC         1048576     pureCRC32  thrpt    5   17.579 ±  0.294  ops/s
BenchMarkCRCStreaming.runCRC         1048576    pureCRC32C  thrpt    5   17.937 ±  0.283  ops/s
BenchMarkCRCStreaming.runCRC         1048576  hadoopCRC32C  thrpt    5   21.141 ±  0.352  ops/s
BenchMarkCRCStreaming.runCRC         1048576   hadoopCRC32  thrpt    5   21.948 ±  0.379  ops/s
BenchMarkCRCStreaming.runCRC         1048576      zipCRC32  thrpt    5  277.069 ±  7.166  ops/s
BenchMarkCRCStreaming.runCRC         1048576     zipCRC32C  thrpt    5  280.083 ±  7.043  ops/s{noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org