[jira] [Commented] (HADOOP-9601) Support native CRC on byte arrays

["Trevor Robinson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-9601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13757097#comment-13757097 ] 

Trevor Robinson commented on HADOOP-9601:
-----------------------------------------

While unaligned loads should be avoided whenever possible for portability and performance, ARMv6 and later (which includes low-end ARM11 implementations such as Raspberry Pi) do [support unaligned word and half-word accesses in most cases|http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0301h/Cdfejcbh.html] by default. In other cases, the [Linux kernel will trap and fix the access|http://lxr.free-electrons.com/source/arch/arm/mm/alignment.c?a=arm] by default (with the obvious performance penalty). The specifics are complicated ([no atomicity|http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0301h/Cdffhdje.html], [no kernel support for unaligned floats|http://jsolano.net/2012/09/06/arm-unaligned-data-access-and-floating-point-in-linux/], etc.), but the point is that this fix is likely to benefit ARM even if it must initially do unaligned 32-bit loads. Linux also provides unaligned access support on other architectures, such as PowerPC, though the overhead may be higher.

                
> Support native CRC on byte arrays
> ---------------------------------
>
>                 Key: HADOOP-9601
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9601
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: performance, util
>    Affects Versions: 3.0.0
>            Reporter: Todd Lipcon
>            Assignee: Gopal V
>              Labels: perfomance
>         Attachments: HADOOP-9601-bench.patch, HADOOP-9601-rebase+benchmark.patch, HADOOP-9601-trunk-rebase-2.patch, HADOOP-9601-trunk-rebase.patch, HADOOP-9601-WIP-01.patch, HADOOP-9601-WIP-02.patch
>
>
> When we first implemented the Native CRC code, we only did so for direct byte buffers, because these correspond directly to native heap memory and thus make it easy to access via JNI. We'd generally assumed that accessing byte[] arrays from JNI was not efficient enough, but now that I know more about JNI I don't think that's true -- we just need to make sure that the critical sections where we lock the buffers are short.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira