You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/09/22 03:45:51 UTC

[GitHub] [spark] mkaravel edited a comment on pull request #34056: [SPARK-36811][SQL] Add SQL functions for the BINARY data type for AND, OR, XOR, and NOT

mkaravel edited a comment on pull request #34056:
URL: https://github.com/apache/spark/pull/34056#issuecomment-924225882


   > Any particular reasons to not use the [BitSet](https://docs.oracle.com/javase/8/docs/api/java/util/BitSet.html) from JDK?
   
   The way that the `BitSet` class works makes it unsuitable to use here:
   * The `and`, `or`, and `xor` in the class produce a Bitset whose size (in bytes) seems to be the minimum of the sizes of the two inputs.
   * Trailing zeros in the computed `BitSet` (when using `and`, `or`, or `xor`) seem to be discarded after the operation.
   * For different sized inputs the bits in `BitSet` are aligned to the left (most significant bit) which does not have the same semantics as the functions implemented in this PR.
   
   For reference, here is the implementation that I used for implementing `bitwiseAnd`:
   ```java
   public static byte[] bitwiseAnd(byte[] bytes1, byte[] bytes2) {
     BitSet bs1 = BitSet.valueOf(bytes1);
     BitSet bs2 = BitSet.valueOf(bytes2);
     bs1.and(bs2);
     return bs1.toByteArray();
   }
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org