[I] rpad/lpad panic (Option::unwrap on None) when the length column contains NULL [datafusion-comet]

["peterxcli (via GitHub)" <gi...@apache.org>]

peterxcli opened a new issue, #5672:
URL: https://github.com/apache/datafusion-comet/issues/5672

   ### Describe the bug
   
   `spark_read_side_padding_internal` (`native/spark-expr/src/static_invoke/char_varchar_utils/read_side_padding.rs:225`) iterates the length array with `let length = length.unwrap();`, so any NULL in the length column panics with `called Option::unwrap() on a None value` and fails the task with a `CometNativeException`. Spark's `StringRPad` / `StringLPad` are `NullIntolerant` and return NULL for that row.
   
   The path is default-on: `CometStringRPad` / `CometStringLPad` only require the pad to be a literal, so `rpad(s, len_col)` runs natively. Both the 2-arg `[Array, Array]` arm (lines 144-162) and the 3-arg `[Array, Array, Scalar]` arm (163-183) reach the unwrap.
   
   ### Steps to reproduce
   
   ```sql
   CREATE TABLE t USING parquet AS SELECT * FROM VALUES ('abc', 5), ('abc', CAST(NULL AS INT)) AS t(s, l);
   SELECT rpad(s, l) FROM t;      -- same for lpad(s, l) and rpad(s, l, 'x')
   ```
   
   Spark: `abc  `, `NULL`. Comet: task fails — `called Option::unwrap() on a None value` at `read_side_padding.rs:225`.
   
   Unit level: `spark_rpad(["abc","abc"], lengths=[Some(4), None], " ")` panics.
   
   ### Expected behavior
   
   NULL for every row whose length is NULL, like Spark.
   
   ### Proposed solution
   
   Replace the unwrap with a match that appends null (and continues) when either the string or the length is null, in both arms. Add a unit test with a NULL length and a NULL-length row to `string_rpad.sql` / `testStringPadding`.
   
   ### Additional context
   
   The fuzz test cannot catch this because `FuzzDataGenerator` never generates NULL integers (#5389). The unwrap was introduced in #2630; #2096 was a different, earlier `rpad` panic (non-literal length hitting the unsupported-arguments arm).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] rpad/lpad panic (Option::unwrap on None) when the length column contains NULL [datafusion-comet]

["sunchao (via GitHub)" <gi...@apache.org>]

sunchao closed issue #5672: rpad/lpad panic (Option::unwrap on None) when the length column contains NULL
URL: https://github.com/apache/datafusion-comet/issues/5672


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org