You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "akshaytayal (via GitHub)" <gi...@apache.org> on 2026/10/12 03:27:41 UTC
[PR] [GLUTEN-13177][VL] Refresh the Spark 4.2 SQLQueryTest golden files [gluten]
akshaytayal opened a new pull request, #13245:
URL: https://github.com/apache/gluten/pull/13245
## What changes are proposed in this pull request?
`GlutenSQLQueryTestSuite` reads its inputs and expected results from `gluten-ut/spark42/src/test/resources/backends-velox/sql-tests`, which were copied from the spark41 module (Spark 4.1.1). Spark 4.2.0 changed 31 of these files, so #13163 removed them from the allowlists. This PR refreshes them:
- Replaces the 27 `SUPPORTED_SQL_QUERY_LIST` files with their Spark 4.2.0 versions (inputs and results). They were unmodified Spark 4.1.1 copies.
- Three-way merges the 4 `OVERWRITE_SQL_QUERY_LIST` files (`charvarchar`, `count`, `cte`, `postgreSQL/window_part3`), so Gluten's own edits (the extra `set spark.sql.optimizer.excludedRules=...` statements and their outputs) carry over unchanged onto the Spark 4.2.0 versions.
- Restores the 31 files to `SUPPORTED_SQL_QUERY_LIST` / `OVERWRITE_SQL_QUERY_LIST` at their spark41 positions.
The 15 TPC-DS plan-stability tests in #13177 will follow in this PR.
Part of #13177.
## How was this patch tested?
Spark 4.2 UT CI (`GlutenSQLQueryTestSuite` runs in `spark-test-spark42-slow (extended)`).
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: GitHub Copilot CLI (Claude Opus 5.5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-13177][VL] Refresh the Spark 4.2 SQLQueryTest golden files: postgreSQL/int4, postgreSQL/int8, try_aggregates [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13245:
URL: https://github.com/apache/gluten/pull/13245#issuecomment-6186667876
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-13177][VL] Refresh the Spark 4.2 SQLQueryTest golden files and TPC-DS approved plans [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13245:
URL: https://github.com/apache/gluten/pull/13245#issuecomment-6178834918
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-13177][VL] Refresh the Spark 4.2 SQLQueryTest golden files and TPC-DS approved plans [gluten]
Posted by "weiting-chen (via GitHub)" <gi...@apache.org>.
weiting-chen commented on code in PR #13245:
URL: https://github.com/apache/gluten/pull/13245#discussion_r4271688203
##########
gluten-ut/spark42/src/test/scala/org/apache/gluten/utils/velox/VeloxSQLQueryTestSettings.scala:
##########
@@ -152,11 +162,15 @@ object VeloxSQLQueryTestSettings extends SQLQueryTestSettings {
"postgreSQL/boolean.sql",
"postgreSQL/case.sql",
"postgreSQL/comments.sql",
+ "postgreSQL/create_view.sql",
"postgreSQL/date.sql",
"postgreSQL/float4.sql",
+ "postgreSQL/float8.sql",
"postgreSQL/groupingsets.sql",
"postgreSQL/insert.sql",
"postgreSQL/int2.sql",
+ "postgreSQL/int4.sql",
+ "postgreSQL/int8.sql",
Review Comment:
**Refresh the Java 21 shadow goldens for the newly enabled overflow tests**
**Target Location:** `gluten-ut/spark42/src/test/scala/org/apache/gluten/utils/velox/VeloxSQLQueryTestSettings.scala:173,194`; corresponding `results/postgreSQL/int8.sql.out.java21` and `results/try_aggregates.sql.out.java21` under `gluten-ut/spark42/src/test/resources/backends-velox/sql-tests`.
**Problem:** These newly enabled cases still select Spark 4.1-era expected output on JDK 21/25. `GlutenSQLQueryTestSuite` prefers an existing `.out.java21` file whenever `Utils.isJavaVersionAtLeast21`; this PR refreshes the ordinary `.out` files but leaves both higher-priority companions unchanged. They expect `"long overflow"` where Spark 4.2 produces `"overflow"`, so these restored cases fail golden comparison on those supported JDK profiles. This is a test-compatibility defect, not a production wrong-result claim. Confidence: high; the mismatch is source-proven, not a locally executed JDK 21 failure.
**Evidence:**
```scala
// Newly added allowlist entries:
"postgreSQL/int8.sql",
// ...
"try_aggregates.sql",
```
The [result-file selector](https://github.com/apache/gluten/blob/56d9f1c54b64a683f11ddf067be4f36c2f3ac1da/gluten-ut/spark42/src/test/scala/org/apache/spark/sql/GlutenSQLQueryTestSuite.scala#L170-L173) selects the stale companions. Their `"message" : "long overflow"` values remain at lines 398/921/961/1001 (`int8`) and 160/368 (`try_aggregates`). Spark 4.2's [arithmeticOverflowError](https://github.com/apache/spark/blob/32f7299601108917fb01920a54e084595b7b3bf8/sql/api/src/main/scala/org/apache/spark/sql/errors/ExecutionErrors.scala#L118-L143) explicitly canonicalizes `\w+ overflow` to `overflow`; the exact upstream Java 21 goldens contain that normalized value. The passing SQL CI XML records JDK 17, which selects the ordinary files and does not exercise this branch.
**Suggested Fix:** Refresh both `.out.java21` resources from Spark `32f7299601108917fb01920a54e084595b7b3bf8`, just as for their ordinary counterparts. The complete substantive delta is these six parameter replacements:
```diff
- "message" : "long overflow"
+ "message" : "overflow"
```
Preserve the other Java-specific output rather than deleting the overrides. Run the two restored SQL cases on JDK 21 with golden generation unset; include alternate result-file suffixes in the import/provenance check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-13177][VL] Refresh the Spark 4.2 SQLQueryTest golden files and TPC-DS approved plans [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13245:
URL: https://github.com/apache/gluten/pull/13245#issuecomment-6180550829
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-13177][VL] Refresh the Spark 4.2 SQLQueryTest golden files [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13245:
URL: https://github.com/apache/gluten/pull/13245#issuecomment-6117210328
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-13177][VL] Refresh the Spark 4.2 SQLQueryTest golden files and TPC-DS approved plans [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13245:
URL: https://github.com/apache/gluten/pull/13245#issuecomment-6117355912
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-13177][VL] Refresh the Spark 4.2 SQLQueryTest golden files: postgreSQL/int4, postgreSQL/int8, try_aggregates [gluten]
Posted by "rui-mo (via GitHub)" <gi...@apache.org>.
rui-mo merged PR #13245:
URL: https://github.com/apache/gluten/pull/13245
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org