You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by Alan Burlison <Al...@oracle.com> on 2015/06/25 17:03:48 UTC
MapReduce's Client's use of bswap and bswap64
hadoop-mapreduce-client makes extensive use of bswap and bswap64,
defined in primitives.h. There are multiple issues with both the
definition and use of these functions:
1. There's an ARM-specific implementation of bswap but the default case
assumes every other platform is x86. There is no platform-independent
fallback.
2. bswap64 has a platform-independent fallback but it relies on the
implementation of bswap, so in effect it too assumes all platforms other
than ARM are x86.
3. All the uses of bswap are unconditional, so the byte-swapping is done
on all platforms irrespective of endianness. I'm not entirely why the
byte swapping is being done in the first place, my guess is it mainly
length fields associated with streams that are being sent over the wire.
I'm struggling to see how unconditional byte swaps will work across all
platforms, irrespective of endianness.
My suspicion is that the right thing to do is to replace all the uses of
bswap and bswap64 with htonl/ntohl and htonll/ntonll or equivalent. Is
that correct?
--
Alan Burlison
--