You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "33modeling (via GitHub)" <gi...@apache.org> on 2026/10/19 01:31:46 UTC
[PR] fix(csv): preserve cell values with duplicate labels [superset]
33modeling opened a new pull request, #45441:
URL: https://github.com/apache/superset/pull/45441
### SUMMARY
CSV export can overwrite distinct cells when row labels or column names repeat. `df_to_escaped_csv` assigns each escaped string through `.at`, which selects every matching label when either axis is non-unique. Escaping headers can also turn two originally distinct column names into the same label.
Address each cell by row and column position instead. The regression tests cover repeated labels on either axis, duplicate MultiIndex labels, escaped-header collisions, nullable integers and mixed object values, with pandas copy-on-write both enabled and disabled.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
For `pd.DataFrame({"value": ["first", "=1+1"]}, index=[7, 7])`, exported without the index:
```text
Before (copy-on-write disabled):
value
first
first
After:
value
first
'=1+1
```
With copy-on-write enabled, the original code instead exports the escaped formula twice. The corrected output preserves both cells in either mode.
### TESTING INSTRUCTIONS
```bash
pytest tests/unit_tests/utils/csv_tests.py tests/unit_tests/utils/excel_tests.py tests/unit_tests/charts/test_client_processing.py -q
pre-commit run
pylint --rcfile=.pylintrc --load-plugins=superset.extensions.pylint --reports=no superset/utils/csv.py
```
- All 14 new regression cases fail against the original implementation.
- With the fix: 28 CSV tests pass; the three export-related files above total 150 passing tests.
- Staged-file pre-commit checks, including mypy and Ruff, pass. Direct Pylint reports no findings.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(csv): preserve cell values with duplicate labels [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #45441:
URL: https://github.com/apache/superset/pull/45441#issuecomment-6259264419
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/45441?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 64.93%. Comparing base ([`4f49d24`](https://app.codecov.io/gh/apache/superset/commit/4f49d24151d1583b225de5df0983e29d8375ca80?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6c53e57`](https://app.codecov.io/gh/apache/superset/commit/6c53e57f69f8045d68ead55fa50ea3edf74fa92a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #45441 +/- ##
==========================================
- Coverage 65.17% 64.93% -0.24%
==========================================
Files 3048 3048
Lines 199705 199705
Branches 46596 46596
==========================================
- Hits 130148 129684 -464
- Misses 66627 67086 +459
- Partials 2930 2935 +5
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/45441/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/45441/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.97% <0.00%> (ø)` | |
| [mysql](https://app.codecov.io/gh/apache/superset/pull/45441/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [postgres](https://app.codecov.io/gh/apache/superset/pull/45441/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/45441/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `36.73% <0.00%> (ø)` | |
| [python](https://app.codecov.io/gh/apache/superset/pull/45441/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `52.85% <100.00%> (-0.42%)` | :arrow_down: |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/45441/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `52.73% <100.00%> (ø)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/45441?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(csv): preserve cell values with duplicate labels [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #45441:
URL: https://github.com/apache/superset/pull/45441#discussion_r4354944589
##########
superset/utils/csv.py:
##########
@@ -102,18 +102,14 @@ def escape_values(v: Any) -> Union[str, Any]:
# Escape csv headers
df = df.rename(columns=escape_values)
- # Escape csv values. Iterate by index label (via ``items``) rather than by
- # positional offset so the escaped value is written back to the correct row
- # even when the DataFrame has a non-default index (e.g. the flattened
- # MultiIndex produced by pivot_table_v2 post-processing). Pairing positional
- # indices with the label-based ``.at`` accessor would otherwise create
- # phantom rows and corrupt the output. Only string cells are reassigned, so
- # the dtype of mixed object columns (e.g. nullable integers) is preserved.
- for name, column in df.items():
+ # Address cells by position: row and column labels need not be unique, and
+ # escaping column names can turn distinct labels into duplicates. Only
+ # string cells are reassigned to preserve numeric and nullable dtypes.
+ for column_index, (_, column) in enumerate(df.items()):
if pd.api.types.is_string_dtype(column.dtype):
Review Comment:
✅ **CodeAnt verified this suggestion was addressed in subsequent commits and marked this thread resolved** as of `15547a0`.
Added a `CategoricalDtype` branch that maps `escape_values` over the column before CSV export, escaping string-valued categorical cells.
<sub>If that's not right, unresolve this thread and CodeAnt will leave it open.</sub>
<!-- codeant-auto-resolve-reply -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(csv): preserve cell values with duplicate labels [superset]
Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #45441:
URL: https://github.com/apache/superset/pull/45441#issuecomment-6257770066
### <span aria-hidden="true">✅</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | 6c53e57f69f8045d68ead55fa50ea3edf74fa92a |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/6ad57302da1379029e17a0ee |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-45441--superset-docs-preview.netlify.app](https://deploy-preview-45441--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
|<span aria-hidden="true">🤖</span> Make changes | [Run an agent on this branch](https://app.netlify.com/projects/superset-docs-preview/agent-runs#q1YqKMrPLShRslJS0lFKKkrMS84AstMyK3STi8t0U0oLcjKTE0tSdXMSk1JzioFqUlILcvIrPVOAqswSU0zNjQ2MUhINjc0tDYwsUw3NEw1SU5VqAQ) |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-45441--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(csv): preserve cell values with duplicate labels [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on PR #45441:
URL: https://github.com/apache/superset/pull/45441#issuecomment-6257831968
## CodeAnt PR Risk: Medium Risk
- The PR needs attention before merging because string-valued categorical cells can bypass formula escaping in CSV exports.
- At superset/utils/csv.py, the dtype check may skip categorical cells such as “=1+1”, leaving formulas executable in spreadsheet software.
**Assessed commit:** `6c53e57f69f8`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(csv): preserve cell values with duplicate labels [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #45441:
URL: https://github.com/apache/superset/pull/45441#discussion_r4309674374
##########
superset/utils/csv.py:
##########
@@ -102,18 +102,14 @@ def escape_values(v: Any) -> Union[str, Any]:
# Escape csv headers
df = df.rename(columns=escape_values)
- # Escape csv values. Iterate by index label (via ``items``) rather than by
- # positional offset so the escaped value is written back to the correct row
- # even when the DataFrame has a non-default index (e.g. the flattened
- # MultiIndex produced by pivot_table_v2 post-processing). Pairing positional
- # indices with the label-based ``.at`` accessor would otherwise create
- # phantom rows and corrupt the output. Only string cells are reassigned, so
- # the dtype of mixed object columns (e.g. nullable integers) is preserved.
- for name, column in df.items():
+ # Address cells by position: row and column labels need not be unique, and
+ # escaping column names can turn distinct labels into duplicates. Only
+ # string cells are reassigned to preserve numeric and nullable dtypes.
+ for column_index, (_, column) in enumerate(df.items()):
if pd.api.types.is_string_dtype(column.dtype):
Review Comment:
**Suggestion:** A string-valued `CategoricalDtype` fails this check, so a cell such as `=1+1` skips `escape_value` and is exported as an executable spreadsheet formula.
**Assessment:** 🟠 `Major` · 🔁 `Occurrence: Rarely` · 🏷️ `Security`
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=2ed2191604374adeae974a6d730f2867&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=2ed2191604374adeae974a6d730f2867&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/utils/csv.py
**Line:** 109:109
**Comment:**
*Security: A string-valued `CategoricalDtype` fails this check, so a cell such as `=1+1` skips `escape_value` and is exported as an executable spreadsheet formula.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F45441&comment_hash=9e2feeb59edc27e5d446766a11e2e0c7b3636c116201043b005de7d43520156a&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F45441&comment_hash=9e2feeb59edc27e5d446766a11e2e0c7b3636c116201043b005de7d43520156a&reaction=dislike'>👎</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(csv): preserve cell values with duplicate labels [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #45441:
URL: https://github.com/apache/superset/pull/45441#discussion_r4408809247
##########
superset/utils/csv.py:
##########
@@ -102,18 +102,14 @@ def escape_values(v: Any) -> Union[str, Any]:
# Escape csv headers
df = df.rename(columns=escape_values)
- # Escape csv values. Iterate by index label (via ``items``) rather than by
- # positional offset so the escaped value is written back to the correct row
- # even when the DataFrame has a non-default index (e.g. the flattened
- # MultiIndex produced by pivot_table_v2 post-processing). Pairing positional
- # indices with the label-based ``.at`` accessor would otherwise create
- # phantom rows and corrupt the output. Only string cells are reassigned, so
- # the dtype of mixed object columns (e.g. nullable integers) is preserved.
- for name, column in df.items():
+ # Address cells by position: row and column labels need not be unique, and
+ # escaping column names can turn distinct labels into duplicates. Only
+ # string cells are reassigned to preserve numeric and nullable dtypes.
+ for column_index, (_, column) in enumerate(df.items()):
if pd.api.types.is_string_dtype(column.dtype):
Review Comment:
✅ **CodeAnt verified this suggestion was addressed in subsequent commits and marked this thread resolved** as of `3031a92`.
Added a `CategoricalDtype` branch that maps `escape_values` over the column before CSV export, escaping string-valued categorical cells.
<sub>If that's not right, unresolve this thread and CodeAnt will leave it open.</sub>
<!-- codeant-auto-resolve-reply -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org