[I] Native sequence raises "Unreachable code reached." where Spark 4.0.5, 4.1.4 and 4.2.1 return the sequence (SPARK-58821) [datafusion-comet]

["andygrove (via GitHub)" <gi...@apache.org>]

andygrove opened a new issue, #6769:
URL: https://github.com/apache/datafusion-comet/issues/6769

   ### Describe the bug
   
   Spark 4.0.5, 4.1.4 and 4.2.1 include SPARK-58821. When `stop - start` overflows `Long` but the
   sequence is short enough to allocate, `Sequence` now returns the recomputed length instead of
   raising `internalError("Unreachable code reached.")`. Comet's native `sequence` kernel
   (`array_funcs/sequence.rs`) reproduces the old error for parity, so on those releases Comet fails a
   query that Spark answers.
   
   ### Steps to reproduce
   
   ```sql
   SELECT sequence(-9223372036854775808L, 9223372036854775807L, 9223372036854775807L)
   ```
   
   Against the staged 4.1.4-rc2, 4.0.5-rc1 and 4.2.1-rc1 artifacts, `expressions/array/sequence.sql`
   fails because Spark no longer raises for its `expect_error(Unreachable code reached)` case. `main`
   at f80042f78 fails the same way.
   
   ### Expected behavior
   
   Return the sequence on releases with SPARK-58821, and keep the error on earlier ones. The version
   can be resolved in Scala and passed to the kernel.
   
   ### Additional context
   
   Found by running the array suites against the Spark release candidates for #6563.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org