[GitHub] [spark] mundaym commented on pull request #29762: [SPARK-32892][CORE][SQL] Fix hash functions on big-endian platforms.

[GitBox <gi...@apache.org>]

mundaym commented on pull request #29762:
URL: https://github.com/apache/spark/pull/29762#issuecomment-693262630


   In an ideal world we'd probably consolidate platform endianness handling into the Platform class by adding methods that perform the necessary conversions. I'm not sure if that would be a reasonable to add to the Platform class API? We could, for example, have overloads for getters and setters for at least the primitive types that take a byte order argument so we could make calls like `Platform.getInt(base, offset, ByteOrder.LITTLE_ENDIAN)`. The implementation would be trivial:
   
   ```java
   public static int getInt(Object object, long offset, ByteOrder order) {
     int x = _UNSAFE.getInt(object, offset);
     return ByteOrder.nativeOrder().equals(order) ? x : Integer.reverseBytes(x);
   }
   ```
   
   That would make changes like this one far cleaner. Though it might be slightly more work (and therefore be slightly riskier performance-wise) for the JIT compiler to optimize well than the proposed inline conversions in this PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org