You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "FrancescoCastaldi (via GitHub)" <gi...@apache.org> on 2026/10/06 22:34:49 UTC
[PR] fix(pandas): prevent duplicate column labels in mixed overwrite and rename postprocessing [superset]
FrancescoCastaldi opened a new pull request, #45037:
URL: https://github.com/apache/superset/pull/45037
### SUMMARY
Fixes #45017.
When a post-processing columns mapping both renames and overwrites columns (e.g. {'y': 'y', 'z': 'z2'}), _append_columns produced duplicated column labels and leaked unmapped columns from append_df.
This change:
1. Separates the overwrite logic from the append logic in _append_columns by delegating to _transfer_column.
2. Selectively transfers only explicitly mapped columns from append_df, discarding unwanted extra columns.
3. Preserves base_df immutability via defensive copy and safely handles both single-level and MultiIndex hierarchies.
### TESTING INSTRUCTIONS
Run the pandas postprocessing unit tests:
pytest tests/unit_tests/pandas_postprocessing/test_utils.py
All tests pass, confirming zero duplicate column labels and expected Series return types.
### ADDITIONAL INFORMATION
- [x] Has associated issue: fixes #45017
- [x] Changes UI: No
- [x] Requires DB Migration: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(pandas): prevent duplicate column labels in mixed overwrite and rename postprocessing [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on PR #45037:
URL: https://github.com/apache/superset/pull/45037#issuecomment-6026789929
## CodeAnt PR Risk: Medium Risk
- The PR needs attention before merging because the MultiIndex transfer path can fail for differing column hierarchy depths.
- When a mapped source has more levels than the base columns, `_transfer_column` builds source-depth tuples using the base column names, which can raise a `ValueError`.
**Assessed commit:** `dcc3a4282fd6`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(pandas): prevent duplicate column labels in mixed overwrite and rename postprocessing [superset]
Posted by "FrancescoCastaldi (via GitHub)" <gi...@apache.org>.
FrancescoCastaldi closed pull request #45037: fix(pandas): prevent duplicate column labels in mixed overwrite and rename postprocessing
URL: https://github.com/apache/superset/pull/45037
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(pandas): prevent duplicate column labels in mixed overwrite and rename postprocessing [superset]
Posted by "SEPURI-SAI-KRISHNA (via GitHub)" <gi...@apache.org>.
SEPURI-SAI-KRISHNA commented on PR #45037:
URL: https://github.com/apache/superset/pull/45037#issuecomment-6030287867
Flagging an overlap rather than anything about the code here: #45018 has been open against #45017 since yesterday morning, about fifteen hours before this one, and its checks are green.
It covers the same two defects and also adds end-to-end coverage in `test_cum.py` and `test_geography.py` next to the `test_utils.py` cases.
Posting so a maintainer does not end up reviewing both.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(pandas): prevent duplicate column labels in mixed overwrite and rename postprocessing [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #45037:
URL: https://github.com/apache/superset/pull/45037#issuecomment-6026968874
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/45037?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `6.89655%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 66.13%. Comparing base ([`a5ffac5`](https://app.codecov.io/gh/apache/superset/commit/a5ffac58f44527607a7937c01b4b21eb989ebf49?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`dcc3a42`](https://app.codecov.io/gh/apache/superset/commit/dcc3a4282fd6fdca66f7cec049b8530512d95d81?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 10 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/superset/pull/45037?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [superset/utils/pandas\_postprocessing/utils.py](https://app.codecov.io/gh/apache/superset/pull/45037?src=pr&el=tree&filepath=superset%2Futils%2Fpandas_postprocessing%2Futils.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvdXRpbHMvcGFuZGFzX3Bvc3Rwcm9jZXNzaW5nL3V0aWxzLnB5) | 6.89% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/45037?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (a5ffac5) and HEAD (dcc3a42). Click for more details.
>
> <details><summary>HEAD has 4 uploads less than BASE</summary>
>
>| Flag | BASE (a5ffac5) | HEAD (dcc3a42) |
>|------|------|------|
>|python|7|3|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #45037 +/- ##
===========================================
- Coverage 82.47% 66.13% -16.35%
===========================================
Files 3002 3004 +2
Lines 186172 186537 +365
Branches 43128 43199 +71
===========================================
- Hits 153550 123364 -30186
- Misses 29867 60484 +30617
+ Partials 2755 2689 -66
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/45037/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/45037/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `36.25% <6.89%> (-0.02%)` | :arrow_down: |
| [mysql](https://app.codecov.io/gh/apache/superset/pull/45037/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [postgres](https://app.codecov.io/gh/apache/superset/pull/45037/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/45037/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.11% <6.89%> (-0.03%)` | :arrow_down: |
| [python](https://app.codecov.io/gh/apache/superset/pull/45037/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.00% <6.89%> (-31.32%)` | :arrow_down: |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/45037/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `54.88% <6.89%> (-0.08%)` | :arrow_down: |
| [unit](https://app.codecov.io/gh/apache/superset/pull/45037/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/45037?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(pandas): prevent duplicate column labels in mixed overwrite and rename postprocessing [superset]
Posted by "FrancescoCastaldi (via GitHub)" <gi...@apache.org>.
FrancescoCastaldi commented on PR #45037:
URL: https://github.com/apache/superset/pull/45037#issuecomment-6035988936
Thanks @SEPURI-SAI-KRISHNA for the heads up! I missed #45018 when submitting this. Since your PR was opened earlier and already provides thorough end-to-end coverage in test_cum.py and test_geography.py, I am closing this one to avoid splitting maintainer review effort. Best regards!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(pandas): prevent duplicate column labels in mixed overwrite and rename postprocessing [superset]
Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #45037:
URL: https://github.com/apache/superset/pull/45037#issuecomment-6026717631
### <span aria-hidden="true">✅</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | dcc3a4282fd6fdca66f7cec049b8530512d95d81 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/6ac5778a7af17e00086ee7ff |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-45037--superset-docs-preview.netlify.app](https://deploy-preview-45037--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
|<span aria-hidden="true">🤖</span> Make changes | [Run an agent on this branch](https://app.netlify.com/projects/superset-docs-preview/agent-runs#DctRDkAwDADQu_TbYhJU3MAxautowtYYCRF3t7_3817QI-16wghQwXxQdGtxkLuWnC82bWcbNErRUzY7qUpcjL90E0enpFiWZ93SM_nyenId4kBIoUG21g49M4YA3w8) |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-45037--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org