You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "LuciferYang (via GitHub)" <gi...@apache.org> on 2026/08/24 16:37:46 UTC
[PR] [GLUTEN-12863][VL][TEST] Re-enable six GlutenTryCastSuite cases excluded on Spark 3.4+ [gluten]
LuciferYang opened a new pull request, #12886:
URL: https://github.com/apache/gluten/pull/12886
## What changes are proposed in this pull request?
`VeloxTestSettings` excludes six cases from `GlutenTryCastSuite` on Spark 3.4, 3.5, 4.0 and 4.1. All six pass once the exclusions are removed, so this deletes them. Unlike the neighbouring entries in that block, none of the six carries a comment saying what fails.
```scala
.exclude("ANSI mode: Throw exception on casting out-of-range value to byte type")
.exclude("ANSI mode: Throw exception on casting out-of-range value to short type")
.exclude("ANSI mode: Throw exception on casting out-of-range value to int type")
.exclude("ANSI mode: Throw exception on casting out-of-range value to long type")
.exclude("cast from invalid string to numeric should throw NumberFormatException")
.exclude("SPARK-26218: Fix the corner case of codegen when casting float to Integer")
```
Why they were likely added: on Spark 3.3 these cases live in `AnsiCastSuiteBase`, and `TryCastSuite` builds them through `TryCast`, a 3.3-only expression class. Gluten's expression mapping is keyed by class, `Sig(expClass: Class[_], name: String)`, and there is no `Sig[TryCast]`, so a plan containing `TryCast` cannot be offloaded on 3.3 and falls back to vanilla Spark. A 3.3 run logs `Skipping evaluation - Nonempty inputRow cannot be converted to DataFrame` ten times for this suite, and Gluten's own try_cast validation is gated the same way with `testWithMinSparkVersion("try_cast", "3.4")` in `ScalarFunctionsValidateSuite`. Spark 3.4 replaced `TryCast` with `Cast(child, dataType, timeZoneId, EvalMode.TRY)`, which Gluten does offload (`Spark34Shims.withTryEvalMode`, `UnaryExpressionTransformer`, `VeloxSparkPlanExecApi`), so the cases went from never reaching Velox to genuinely exercising it. Whatever failed at that point passes now.
Fixes #12863's one actionable finding. That issue asks whether removing `gluten-ut/spark33` under #12807 would drop coverage that only runs on 3.3. The inventory behind it produced 17 candidate cases; these six are the only ones that pass once enabled. The rest either fail in a way consistent with the reasons written beside them, or are ClickHouse-only and need a CH build to check. So the answer to #12863 is that nothing is lost by removing the module, and the one thing worth doing is re-enabling these six.
## How was this patch tested?
Draft on purpose: CI on all four versions is the check that matters here, not my laptop.
Removing the six from `gluten-ut/spark34` and running the suite against Spark 3.4 locally takes it from 72 to 78 cases with no new failures, and all six report as passed. The delta is what makes that readable: before the change these cases do not run at all, so they cannot appear in a baseline failure set, and afterwards they run and pass.
The same local run also fails `Gluten - data type casting` and `Gluten - cast string to timestamp`, and a companion run of `GlutenParquetIOSuite` fails five more cases. All of those are pre-existing, timezone or parquet related, and green in CI, so this machine deviates from CI and a local result is not authoritative on its own. Hence the draft.
`test-compile` passes on `-Pspark-3.4`, `-Pspark-3.5 -Pscala-2.13`, `-Pspark-4.0 -Pscala-2.13` and `-Pspark-4.1 -Pscala-2.13`, each with `-Pbackends-velox -Pspark-ut -Pjava-17`, and `spotless:check` is clean.
If CI red-flags any of the six on a particular version, the intent is to put the exclusion back for that version with the failure written next to it, rather than to drop the whole change.
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude claude-opus-5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12863][VL][TEST] Re-enable six GlutenTryCastSuite cases excluded on Spark 3.4+ [gluten]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #12886:
URL: https://github.com/apache/gluten/pull/12886#issuecomment-5398326941
Test first
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12863][VL][TEST] Re-enable six GlutenTryCastSuite cases excluded on Spark 3.4+ [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #12886:
URL: https://github.com/apache/gluten/pull/12886#issuecomment-5398511923
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12863][VL][TEST] Re-enable six GlutenTryCastSuite cases excluded on Spark 3.4+ [gluten]
Posted by "jackylee-ch (via GitHub)" <gi...@apache.org>.
jackylee-ch merged PR #12886:
URL: https://github.com/apache/gluten/pull/12886
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12863][VL][TEST] Re-enable six GlutenTryCastSuite cases excluded on Spark 3.4+ [gluten]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #12886:
URL: https://github.com/apache/gluten/pull/12886#issuecomment-5404885798
cc @zhouyuan @jackylee-ch @philo-he
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12863][VL][TEST] Re-enable six GlutenTryCastSuite cases excluded on Spark 3.4+ [gluten]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #12886:
URL: https://github.com/apache/gluten/pull/12886#issuecomment-5407375155
Thank you @jackylee-ch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org