You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/10/19 13:53:03 UTC

[GitHub] [spark] mkaravel commented on a change in pull request #34154: [SPARK-37047][SQL] Add lpad and rpad functions for binary strings

mkaravel commented on a change in pull request #34154:
URL: https://github.com/apache/spark/pull/34154#discussion_r731891091



##########
File path: common/unsafe/src/main/java/org/apache/spark/unsafe/types/ByteArray.java
##########
@@ -101,4 +101,82 @@ public static long getPrefix(byte[] bytes) {
     }
     return result;
   }
+
+  // Helper method for implementing `lpad` and `rpad`.
+  // If the padding pattern's length is 0, return the first `len` bytes of the input
+  // binary string if it longer than `len` bytes, or a copy of the binary string, otherwise.
+  protected static byte[] padWithEmptyPattern(byte[] bytes, int len) {
+    len = Math.min(bytes.length, len);
+    final byte[] result = new byte[len];
+    Platform.copyMemory(bytes, Platform.BYTE_ARRAY_OFFSET, result, Platform.BYTE_ARRAY_OFFSET, len);
+    return result;
+  }
+
+  // Helper method for implementing `lpad` and `rpad`.
+  // Fills the resulting binary string with the pattern. The resulting binary string
+  // is passed as the first argument and it is filled from position `firstPos` (inclusive)
+  // to position `beyondPos` (not inclusive).
+  protected static void fillWithPattern(byte[] result, int firstPos, int beyondPos, byte[] pad) {
+    for (int pos = firstPos; pos < beyondPos; pos += pad.length) {
+      final int jMax = Math.min(pad.length, beyondPos - pos);
+      for (int j = 0; j < jMax; ++j) {
+        result[pos + j] = (byte) pad[j];
+      }
+    }
+  }
+
+  // Left-pads the input binary string using the provided padding pattern.
+  // In the special case that the padding pattern is empty, the resulting binary string
+  // contains the first `len` bytes of the input if they exist, or is a copy of the input
+  // binary stringkm otherwise.
+  // For padding patterns with positive byte length, the resulting binary string's byte length is
+  // equal to `len`. If the input binary string is not less than `len` bytes, its first `len` bytes
+  // are returned. Otherwise, the remaining missing bytes are filled in with the provided pattern.
+  public static byte[] lpad(byte[] bytes, int len, byte[] pad) {

Review comment:
       Yes. We adopt the same behavior as for character strings:
   ```
   scala> spark.sql("select lpad('abc', 10, '12')").show
   +-----------------+
   |lpad(abc, 10, 12)|
   +-----------------+
   |       1212121abc|
   +-----------------+
   
   
   scala> spark.sql("select rpad('abc', 10, '12')").show
   +-----------------+
   |rpad(abc, 10, 12)|
   +-----------------+
   |       abc1212121|
   +-----------------+
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org