[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

["Adrien Grand (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13771671#comment-13771671 ] 

Adrien Grand commented on CASSANDRA-5862:
-----------------------------------------

I just learned about this issue and was wondering if you had considered XXHash (http://code.google.com/p/xxhash/) for checksuming. I am a little biased since I wrote the JNI bindings and Java ports of XXHash but the benchmarks show interesting results compared to other hash/checksum implementations: http://jpountz.github.io/lz4-java/1.2.0/xxhash-benchmark/. Just beware that depending on the size of the input, the fastest impl of XXHash is not always the same: for large inputs (> 1024 bytes), the JNI one is faster while on smaller input (<= 1024 bytes), the Java port using the Unsafe API is faster, probably because of the JNI overhead.

You can find the sources of the benchmark at https://github.com/jpountz/jvm-checksum-benchmark.
                
> Switch to adler checksum for sstables
> -------------------------------------
>
>                 Key: CASSANDRA-5862
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Jonathan Ellis
>            Assignee: T Jake Luciani
>             Fix For: 2.0.1
>
>         Attachments: 5862.txt
>
>
> Adler is significantly faster than CRC32: http://java-performance.info/java-crc32-and-adler32/
> (Adler is weaker for short inputs, so we should leave the commitlog alone, as it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira