You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "developer-rpai (via GitHub)" <gi...@apache.org> on 2026/09/25 04:45:25 UTC
[PR] fix: use database rollups for pivot table subtotals in Actual Values mode [superset]
developer-rpai opened a new pull request, #44631:
URL: https://github.com/apache/superset/pull/44631
## Summary
Fixes #44625.
SIP-216 (#41184) stopped the live pivot-table view from re-aggregating already-aggregated cells for subtotals/totals, by issuing `GROUPING SETS` queries and reading the database-computed rollups. The scheduled-report/alert/export path (`pivot_table_v2` in `superset/charts/client_processing.py`) was left behind: `_apply_rollup_totals` was only called under `if percent_mode and rollup_levels`, so in the default **Actual Values** mode subtotals were still a mean-of-means for Average, nunique-of-counts for Count Unique, etc. — wrong numbers delivered to executives on schedule.
## Change
- `pivot_df` now applies `_apply_rollup_totals` in Actual Values mode too, whenever rollup levels are present. A total the database did not compute keeps its previous leaf-derived value (graceful fallback).
- The currency-context pass (`__currency_context__` aggfunc) is excluded: its cells hold currency-code sets, not metric values, so it keeps its leaf-derived unions.
## Test
- New regression tests in `tests/unit_tests/charts/test_client_processing.py`:
- `test_pivot_table_v2_actual_values_uses_database_rollups` — synthetic `GROUPING SETS` frame (leaves 10/20, DB rollups 21/18/19): totals now read 21/18/19 instead of 20/15/15; leaf cells unchanged.
- `test_pivot_table_v2_actual_values_falls_back_without_rollups` — no `GROUPING SETS` levels present: the old pandas reduction is kept (15).
- Executed a pandas-only repro of `pivot_table_v2` against current master before the fix (bug reproduced: mean-of-means 15.0 where the DB rollup is 18) and after (fixed). Re-ran the existing percent-mode rollup tests' assertions (divide-by-DB-rollups, nested subtotals, NULL-rollup-stays-blank, min/max reducers, leaf-only pivoting) — all pass.
- Confirmed identical output on plain (non-grouping-sets) data across 8 form-data variants (Sum/Average/Median/Minimum, percent modes, transpose/combine, metrics-on-rows) before vs after the change.
- Note: the full pytest module needs the repo's dev environment, which isn't available in my sandbox — I ran the assertions through a stub harness loading the real module. CI will be the final word; happy to iterate on feedback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix: use database rollups for pivot table subtotals in Actual Values mode [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas merged PR #44631:
URL: https://github.com/apache/superset/pull/44631
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix: use database rollups for pivot table subtotals in Actual Values mode [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #44631:
URL: https://github.com/apache/superset/pull/44631#issuecomment-5826942205
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/44631?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 57.14%. Comparing base ([`599026a`](https://app.codecov.io/gh/apache/superset/commit/599026a0e5a82f9728a333aea0e07b6a7bf8f771?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`ab97569`](https://app.codecov.io/gh/apache/superset/commit/ab97569347dd634ec276ece2728f4b90c0bf111d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/superset/pull/44631?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [superset/charts/client\_processing.py](https://app.codecov.io/gh/apache/superset/pull/44631?src=pr&el=tree&filepath=superset%2Fcharts%2Fclient_processing.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvY2hhcnRzL2NsaWVudF9wcm9jZXNzaW5nLnB5) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/44631?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (599026a) and HEAD (ab97569). Click for more details.
>
> <details><summary>HEAD has 86 uploads less than BASE</summary>
>
>| Flag | BASE (599026a) | HEAD (ab97569) |
>|------|------|------|
>|python|68|2|
>|presto|11|1|
>|hive|11|1|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #44631 +/- ##
===========================================
- Coverage 81.10% 57.14% -23.96%
===========================================
Files 2955 2955
Lines 178385 178385
Branches 41336 41336
===========================================
- Hits 144680 101941 -42739
- Misses 30996 75461 +44465
+ Partials 2709 983 -1726
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/44631/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/44631/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `36.80% <0.00%> (ø)` | |
| [mysql](https://app.codecov.io/gh/apache/superset/pull/44631/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [postgres](https://app.codecov.io/gh/apache/superset/pull/44631/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/44631/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.70% <0.00%> (ø)` | |
| [python](https://app.codecov.io/gh/apache/superset/pull/44631/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.75% <0.00%> (-46.64%)` | :arrow_down: |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/44631/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unit](https://app.codecov.io/gh/apache/superset/pull/44631/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/44631?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix: use database rollups for pivot table subtotals in Actual Values mode [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44631:
URL: https://github.com/apache/superset/pull/44631#issuecomment-5826895406
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org