[GitHub] [spark] cloud-fan commented on a diff in pull request #37631: [SPARK-40194][SQL] SPLIT function on empty regex should truncate trailing empty string.

[GitBox <gi...@apache.org>]

cloud-fan commented on code in PR #37631:
URL: https://github.com/apache/spark/pull/37631#discussion_r953400215


##########
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java:
##########
@@ -1000,6 +1000,17 @@ public static UTF8String concatWs(UTF8String separator, UTF8String... inputs) {
   }
 
   public UTF8String[] split(UTF8String pattern, int limit) {
+    // This is a special case for converting string into array of symbols without a trailing empty
+    // string. E.g. `"hello".split("", 0) => ["h", "e", "l", "l", "o"].
+    // Note that negative limit will preserve a trailing empty string.
+    if (limit == 0 && numBytes() != 0 && pattern.numBytes() == 0) {
+      byte[] input = getBytes();
+      UTF8String[] result = new UTF8String[numBytes];
+      for (int i = 0; i < numBytes; i++) {

Review Comment:
   are we sure it's one byte per character?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org