You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by Alan Burlison <Al...@oracle.com> on 2015/06/25 17:03:48 UTC

MapReduce's Client's use of bswap and bswap64

hadoop-mapreduce-client makes extensive use of bswap and bswap64, 
defined in primitives.h. There are multiple issues with both the 
definition and use of these functions:

1. There's an ARM-specific implementation of bswap but the default case 
assumes every other platform is x86. There is no platform-independent 
fallback.

2. bswap64 has a platform-independent fallback but it relies on the 
implementation of bswap, so in effect it too assumes all platforms other 
than ARM are x86.

3. All the uses of bswap are unconditional, so the byte-swapping is done 
on all platforms irrespective of endianness. I'm not entirely why the 
byte swapping is being done in the first place, my guess is it mainly 
length fields associated with streams that are being sent over the wire. 
I'm struggling to see how unconditional byte swaps will work across all 
platforms, irrespective of endianness.

My suspicion is that the right thing to do is to replace all the uses of 
bswap and bswap64 with htonl/ntohl and htonll/ntonll or equivalent. Is 
that correct?

-- 
Alan Burlison
--