You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "rusackas (via GitHub)" <gi...@apache.org> on 2026/08/03 23:10:47 UTC
[PR] test(charts): pin pivoted CSV export bytes/str crash (#32370) [superset]
rusackas opened a new pull request, #42735:
URL: https://github.com/apache/superset/pull/42735
### SUMMARY
Test-only PR pinning the bug reported in #32370 ("Error 505 when exporting pivoted .csv with special characters").
Traced the actual mechanism: `QueryContextProcessor.get_data` encodes the query's `data` to **bytes** for any CSV `result_format` (`superset/common/query_context_processor.py`), and that's exactly the payload `ChartDataRestApi._send_chart_response` hands to `apply_client_processing` for a `resultType=post_processed` request (i.e. "Export to pivoted .csv"). `apply_client_processing`'s CSV branch feeds that straight into `StringIO(data)`, which only accepts `str`:
```
TypeError: initial_value must be str or None, not bytes
```
This reproduces for **every** pivoted CSV export, not only the special-character metric labels the issue focuses on — I independently verified metric labels containing `%`, `\`, `|` round-trip fine through `apply_client_processing` when fed as `str` (matching how the existing unit tests in this file construct their input), so the special characters aren't actually the trigger. The existing tests never caught this because they hand-construct `data` as a plain `str`, which doesn't match what the real query pipeline produces for CSV result_format. The new test uses `bytes` (matching production) with a `%`-prefixed metric label, so it still pins the originally reported symptom.
Root cause fix is not included here — this PR only adds the regression test (red on master), per the TDD-PR pattern: a future fix PR should decode `data` to `str` before parsing (or have `apply_client_processing` handle both `bytes` and `str`), then this test should go green.
### TESTING INSTRUCTIONS
```
pytest tests/unit_tests/charts/test_client_processing.py::test_apply_client_processing_csv_format_bytes_data
```
Currently fails with `TypeError: initial_value must be str or None, not bytes` on master.
### ADDITIONAL INFORMATION
- [x] Has associated issue: #32370
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] test(charts): pin pivoted CSV export bytes/str crash (#32370) [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42735:
URL: https://github.com/apache/superset/pull/42735#issuecomment-5172708322
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): decode CSV bytes payload before feeding pandas StringIO (#32370) [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42735:
URL: https://github.com/apache/superset/pull/42735#issuecomment-5286012061
**Bito Automatic Review Skipped – PR Already Merged**<br/><br/>Bito scheduled an automatic review for this pull request, but the review was skipped because this PR was merged before the review could be run.<br/>No action is needed if you didn't intend to review it. To get a review, you can type `/review` in a comment and save it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): decode CSV bytes payload before feeding pandas StringIO (#32370) [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe merged PR #42735:
URL: https://github.com/apache/superset/pull/42735
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] test(charts): pin pivoted CSV export bytes/str crash (#32370) [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #42735:
URL: https://github.com/apache/superset/pull/42735#issuecomment-5172820765
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/42735?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 64.76%. Comparing base ([`25ab961`](https://app.codecov.io/gh/apache/superset/commit/25ab96188f1bea54029345ac79f4cc3b05a4d1d4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`ba6fe16`](https://app.codecov.io/gh/apache/superset/commit/ba6fe167d61a855607d92cd2d253364bff319b50?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #42735 +/- ##
==========================================
- Coverage 65.57% 64.76% -0.81%
==========================================
Files 2818 2818
Lines 160038 160038
Branches 36557 36557
==========================================
- Hits 104938 103646 -1292
- Misses 53054 54354 +1300
+ Partials 2046 2038 -8
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/42735/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/42735/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.08% <ø> (ø)` | |
| [mysql](https://app.codecov.io/gh/apache/superset/pull/42735/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [postgres](https://app.codecov.io/gh/apache/superset/pull/42735/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/42735/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `39.96% <ø> (ø)` | |
| [python](https://app.codecov.io/gh/apache/superset/pull/42735/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `57.67% <ø> (-1.62%)` | :arrow_down: |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/42735/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `57.54% <ø> (ø)` | |
| [unit](https://app.codecov.io/gh/apache/superset/pull/42735/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/42735?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): decode CSV bytes payload before feeding pandas StringIO (#32370) [superset]
Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #42735:
URL: https://github.com/apache/superset/pull/42735#issuecomment-5203290934
### <span aria-hidden="true">✅</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | c595c2c2c0f07b127d8221da509ce2a7e2192eb9 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/6a745dba9c133e000875d3e2 |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-42735--superset-docs-preview.netlify.app](https://deploy-preview-42735--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
|<span aria-hidden="true">🤖</span> Make changes | [Run an agent on this branch](https://app.netlify.com/projects/superset-docs-preview/agent-runs#DctRCsIwDADQu-TbYF2cVW_gMbIksMK0oakDEe9u_97P-4K3-vQOd4ADLI1fsg531WOJeBvSRDmhl712U5TYMdyk8IaycouR1Hyrn4eOduF8nnXhm5yILKV0zbOSTfD7Aw) |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-42735--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): decode CSV bytes payload before feeding pandas StringIO (#32370) [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on code in PR #42735:
URL: https://github.com/apache/superset/pull/42735#discussion_r3709705398
##########
tests/unit_tests/charts/test_client_processing.py:
##########
@@ -2200,6 +2200,51 @@ def test_apply_client_processing_csv_format_escapes_formula_values():
assert "\n=SUM(1+1)" not in processed["queries"][0]["data"]
+def test_apply_client_processing_csv_format_bytes_data():
+ """
+ Regression for #32370: "Export to pivoted .csv" fails with a 505 error.
+
+ For a real ``resultType=post_processed``/``resultFormat=csv`` request,
+ the query's ``data`` is CSV *bytes* by the time it reaches
+ ``apply_client_processing`` --
+ ``QueryContextProcessor.get_data`` encodes it to bytes for any CSV
+ result_format (``superset/common/query_context_processor.py``), and
+ that's the exact payload ``ChartDataRestApi._send_chart_response`` hands
+ to ``apply_client_processing`` for a POST_PROCESSED result. The CSV
+ branch here passes that straight into ``StringIO(data)``, which raises
+ ``TypeError: initial_value must be str or None, not bytes`` -- a crash
+ that reproduces for every pivoted CSV export, not only the reporter's
+ special-character metric labels (this test uses one anyway, to also
+ pin the originally reported symptom).
+ """
+
+ result = {
+ "queries": [
+ {
+ "result_format": ChartDataResultFormat.CSV,
+ "data": b"name,% of total\nA,1\nB,2\n",
+ }
+ ]
+ }
+ form_data = {
+ "datasource": "19__table",
+ "viz_type": "pivot_table_v2",
+ "slice_id": 69,
+ "groupbyColumns": [],
+ "groupbyRows": ["name"],
+ "metrics": ["% of total"],
+ "metricsLayout": "COLUMNS",
+ "aggregateFunction": "Sum",
+ "rowOrder": "key_a_to_z",
+ "colOrder": "key_a_to_z",
+ "result_format": "csv",
+ "result_type": "post_processed",
+ }
+
+ processed = apply_client_processing(result, form_data)
+ assert "% of total" in processed["queries"][0]["data"]
Review Comment:
<!-- Bito Reply -->
The suggestion provided by the reviewer is correct and addresses the identified regression. Applying the suggested bytes-decode guard ensures that the CSV data is properly converted to a string before being passed to `StringIO`, preventing the `TypeError` crash.
**superset/charts/client_processing.py**
```
if isinstance(data, bytes):
data = data.decode("utf-8")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] test(charts): pin pivoted CSV export bytes/str crash (#32370) [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on code in PR #42735:
URL: https://github.com/apache/superset/pull/42735#discussion_r3708318202
##########
tests/unit_tests/charts/test_client_processing.py:
##########
@@ -2200,6 +2200,51 @@ def test_apply_client_processing_csv_format_escapes_formula_values():
assert "\n=SUM(1+1)" not in processed["queries"][0]["data"]
+def test_apply_client_processing_csv_format_bytes_data():
+ """
+ Regression for #32370: "Export to pivoted .csv" fails with a 505 error.
+
+ For a real ``resultType=post_processed``/``resultFormat=csv`` request,
+ the query's ``data`` is CSV *bytes* by the time it reaches
+ ``apply_client_processing`` --
+ ``QueryContextProcessor.get_data`` encodes it to bytes for any CSV
+ result_format (``superset/common/query_context_processor.py``), and
+ that's the exact payload ``ChartDataRestApi._send_chart_response`` hands
+ to ``apply_client_processing`` for a POST_PROCESSED result. The CSV
+ branch here passes that straight into ``StringIO(data)``, which raises
+ ``TypeError: initial_value must be str or None, not bytes`` -- a crash
+ that reproduces for every pivoted CSV export, not only the reporter's
+ special-character metric labels (this test uses one anyway, to also
+ pin the originally reported symptom).
+ """
+
+ result = {
+ "queries": [
+ {
+ "result_format": ChartDataResultFormat.CSV,
+ "data": b"name,% of total\nA,1\nB,2\n",
+ }
+ ]
+ }
+ form_data = {
+ "datasource": "19__table",
+ "viz_type": "pivot_table_v2",
+ "slice_id": 69,
+ "groupbyColumns": [],
+ "groupbyRows": ["name"],
+ "metrics": ["% of total"],
+ "metricsLayout": "COLUMNS",
+ "aggregateFunction": "Sum",
+ "rowOrder": "key_a_to_z",
+ "colOrder": "key_a_to_z",
+ "result_format": "csv",
+ "result_type": "post_processed",
+ }
+
+ processed = apply_client_processing(result, form_data)
+ assert "% of total" in processed["queries"][0]["data"]
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>Test correct, production fix needed</b></div>
<div id="fix">
This regression test (lines 2203–2245) is correctly written and will correctly validate the production fix once the missing bytes-decode guard is added at `superset/charts/client_processing.py` line ~383. The crash (`TypeError: initial_value must be str or None, not bytes`) is confirmed: `QueryContextProcessor.get_data` encodes CSV output to bytes before passing it to `apply_client_processing`, and the CSV branch at line 383 passes it directly to `StringIO(data)` with no bytes check. Adding `if isinstance(data, bytes): data = data.decode("utf-8")` (after the existing str-strip at line 365, before the `pd.read_csv` call at line 382) will make the test pass.
</div>
</div>
<small><i>Code Review Run #85cd80</i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): decode CSV bytes payload before feeding pandas StringIO (#32370) [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42735:
URL: https://github.com/apache/superset/pull/42735#issuecomment-5205131939
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): decode CSV bytes payload before feeding pandas StringIO (#32370) [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on code in PR #42735:
URL: https://github.com/apache/superset/pull/42735#discussion_r3709704817
##########
tests/unit_tests/charts/test_client_processing.py:
##########
@@ -2200,6 +2200,51 @@ def test_apply_client_processing_csv_format_escapes_formula_values():
assert "\n=SUM(1+1)" not in processed["queries"][0]["data"]
+def test_apply_client_processing_csv_format_bytes_data():
+ """
+ Regression for #32370: "Export to pivoted .csv" fails with a 505 error.
+
+ For a real ``resultType=post_processed``/``resultFormat=csv`` request,
+ the query's ``data`` is CSV *bytes* by the time it reaches
+ ``apply_client_processing`` --
+ ``QueryContextProcessor.get_data`` encodes it to bytes for any CSV
+ result_format (``superset/common/query_context_processor.py``), and
+ that's the exact payload ``ChartDataRestApi._send_chart_response`` hands
+ to ``apply_client_processing`` for a POST_PROCESSED result. The CSV
+ branch here passes that straight into ``StringIO(data)``, which raises
+ ``TypeError: initial_value must be str or None, not bytes`` -- a crash
+ that reproduces for every pivoted CSV export, not only the reporter's
+ special-character metric labels (this test uses one anyway, to also
+ pin the originally reported symptom).
+ """
+
+ result = {
+ "queries": [
+ {
+ "result_format": ChartDataResultFormat.CSV,
+ "data": b"name,% of total\nA,1\nB,2\n",
+ }
+ ]
+ }
+ form_data = {
+ "datasource": "19__table",
+ "viz_type": "pivot_table_v2",
+ "slice_id": 69,
+ "groupbyColumns": [],
+ "groupbyRows": ["name"],
+ "metrics": ["% of total"],
+ "metricsLayout": "COLUMNS",
+ "aggregateFunction": "Sum",
+ "rowOrder": "key_a_to_z",
+ "colOrder": "key_a_to_z",
+ "result_format": "csv",
+ "result_type": "post_processed",
+ }
+
+ processed = apply_client_processing(result, form_data)
+ assert "% of total" in processed["queries"][0]["data"]
Review Comment:
Fixed in the next commit, the CSV branch now decodes bytes before hitting `StringIO`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): decode CSV bytes payload before feeding pandas StringIO (#32370) [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42735:
URL: https://github.com/apache/superset/pull/42735#issuecomment-5174264420
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): decode CSV bytes payload before feeding pandas StringIO (#32370) [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #42735:
URL: https://github.com/apache/superset/pull/42735#discussion_r3727161617
##########
superset/charts/client_processing.py:
##########
@@ -379,8 +379,17 @@ def apply_client_processing( # noqa: C901
# reports to avoid unwanted conversions
# This allows users to control which values should be treated as null/NA
na_values = current_app.config["REPORTS_CSV_NA_NAMES"]
+ # QueryContextProcessor.get_data encodes CSV `data` to bytes using
+ # the configured CSV_EXPORT encoding (default utf-8), matching
+ # the encoding SQL Lab's own CSV export uses -- decode with that
+ # same encoding rather than assuming `data` is already a `str`.
+ csv_data = (
+ data.decode(csv_export_config.get("encoding", "utf-8"))
+ if isinstance(data, bytes)
+ else data
+ )
Review Comment:
Whitespace-only CSV bytes still bypass the existing no-data guard because decoding happens after the `str.strip()` check; for example, a columnless frame is serialized as a truthy BOM/newline payload and then reaches `pd.read_csv`, which raises `EmptyDataError`. Should we decode CSV bytes before the strip/empty check and extend the existing no-data parametrization with the bytes forms?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): decode CSV bytes payload before feeding pandas StringIO (#32370) [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on code in PR #42735:
URL: https://github.com/apache/superset/pull/42735#discussion_r3727851707
##########
superset/charts/client_processing.py:
##########
@@ -379,8 +379,17 @@ def apply_client_processing( # noqa: C901
# reports to avoid unwanted conversions
# This allows users to control which values should be treated as null/NA
na_values = current_app.config["REPORTS_CSV_NA_NAMES"]
+ # QueryContextProcessor.get_data encodes CSV `data` to bytes using
+ # the configured CSV_EXPORT encoding (default utf-8), matching
+ # the encoding SQL Lab's own CSV export uses -- decode with that
+ # same encoding rather than assuming `data` is already a `str`.
+ csv_data = (
+ data.decode(csv_export_config.get("encoding", "utf-8"))
+ if isinstance(data, bytes)
+ else data
+ )
Review Comment:
Good catch — fixed, decoding CSV bytes before the strip/empty check now, and added `b""`/`b"\n"` to the no-data test parametrization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org