You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2026/07/08 16:14:21 UTC
(spark) branch master updated: [SPARK-57775][SQL][FOLLOWUP] Document the grand-total behavior change in the migration guide
This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 68d6ad186c43 [SPARK-57775][SQL][FOLLOWUP] Document the grand-total behavior change in the migration guide
68d6ad186c43 is described below
commit 68d6ad186c436296c4e6b0851851c696130b2774
Author: Uros Bojanic <22...@users.noreply.github.com>
AuthorDate: Wed Jul 8 09:14:12 2026 -0700
[SPARK-57775][SQL][FOLLOWUP] Document the grand-total behavior change in the migration guide
### What changes were proposed in this pull request?
Follow-up to #56891: add a SQL migration guide entry for the behavior change: `GROUP BY GROUPING SETS (())` and empty `CUBE()` / `ROLLUP()` now return one row over empty input.
### Why are the changes needed?
The behavior change was requested to be documented in the migration guide as part of https://github.com/apache/spark/pull/56891#pullrequestreview-4649357419.
### Does this PR introduce _any_ user-facing change?
No, docs only change.
### How was this patch tested?
N/A, docs only change.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #57121 from uros-b/SPARK-57775-migration-guide.
Lead-authored-by: Uros Bojanic <22...@users.noreply.github.com>
Co-authored-by: Uros Bojanic <ur...@databricks.com>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
docs/sql-migration-guide.md | 1 +
1 file changed, 1 insertion(+)
diff --git a/docs/sql-migration-guide.md b/docs/sql-migration-guide.md
index 460a4d4219bf..a0a7057e51f7 100644
--- a/docs/sql-migration-guide.md
+++ b/docs/sql-migration-guide.md
@@ -27,6 +27,7 @@ license: |
- Since Spark 4.3, zero-length files are skipped during Parquet schema inference instead of failing with a `FAILED_READ_FILE.CANNOT_READ_FILE_FOOTER` error.
- Since Spark 4.3, the configuration key `spark.sql.sources.v2.bucketing.allowJoinKeysSubsetOfPartitionKeys.enabled` has been renamed to `spark.sql.sources.v2.bucketing.allowKeysSubsetOfPartitionKeys.enabled` to reflect that it now applies to storage-partitioned joins, aggregates, and windows. The old key continues to work as an alias.
- Since Spark 4.3, the Spark Thrift Server rejects setting JVM system properties through the `set:system:` session configuration overlay (for example, in a JDBC connection string). To restore the previous behavior, set `spark.sql.legacy.hive.thriftServer.allowSettingSystemProperties` to `true`.
+- Since Spark 4.3, `GROUP BY GROUPING SETS (())` (and the equivalent empty `GROUP BY CUBE()` / `GROUP BY ROLLUP()`) is treated as a grand total and returns one row over empty input, matching an aggregation with no `GROUP BY` clause; previously it returned no rows. To restore the previous behavior, set `spark.sql.analyzer.lowerEmptyGroupingSetToGlobalAggregate.enabled` to `false`.
## Upgrading from Spark SQL 4.1 to 4.2
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org