[jira] [Commented] (HADOOP-10846) DataChecksum#calculateChunkedSums not working for PPC when buffers not backed by array

["Binglin Chang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-10846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344759#comment-14344759 ] 

Binglin Chang commented on HADOOP-10846:
----------------------------------------

Hi Ayappan, thanks for the patch, when validating on macosx, got compile error like:

{code}
     [exec] Building C object CMakeFiles/hadoop.dir/main/native/src/org/apache/hadoop/util/bulk_crc32.c.o
     [exec] /usr/bin/cc  -Dhadoop_EXPORTS -g -/Volumes/SSD/projects/hadoop-trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/util/bulk_crc32.c:3Wall -O2 -D_REENTRANT -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -isysroot /Applications/Xcode.app/Contents/Develo4:10: fatal error: 'byteswap.h' file not found
     [exec] #include <byteswap.h>
     [exec]          ^
     [exec] 1 error generated.
     [exec] make[2]: *** [CMakeFiles/hadoop.dir/main/native/src/org/apache/hadoop/util/bulk_crc32.c.o] Error 1
     [exec] make[1]: *** [CMakeFiles/hadoop.dir/all] Error 2
     [exec] make: *** [all] Error 2
{code}

I think a more standard way of handling byteorder stuff is needed(not only in this jira), like google did in many open sourced code bases:

https://github.com/google/flatbuffers/blob/master/include/flatbuffers/flatbuffers.h

 

> DataChecksum#calculateChunkedSums not working for PPC when buffers not backed by array
> --------------------------------------------------------------------------------------
>
>                 Key: HADOOP-10846
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10846
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: util
>    Affects Versions: 2.4.1, 2.5.2
>         Environment: PowerPC platform
>            Reporter: Jinghui Wang
>            Assignee: Ayappan
>         Attachments: HADOOP-10846-v1.patch, HADOOP-10846-v2.patch, HADOOP-10846-v3.patch, HADOOP-10846-v4.patch, HADOOP-10846.patch
>
>
> Got the following exception when running Hadoop on Power PC. The implementation for computing checksum when the data buffer and checksum buffer are not backed by arrays.
> 13/09/16 04:06:57 ERROR security.UserGroupInformation: PriviledgedActionException as:biadmin (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(java.io.IOException): org.apache.hadoop.fs.ChecksumException: Checksum error



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)