You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "aipiJuancho (via GitHub)" <gi...@apache.org> on 2026/09/25 00:09:37 UTC
[PR] fix(charts): stream large CSV exports without executing the query first [superset]
aipiJuancho opened a new pull request, #44624:
URL: https://github.com/apache/superset/pull/44624
### SUMMARY
When the frontend decides to stream a CSV export (CSV, not a pivot, row count ≥ `CSV_STREAMING_ROW_THRESHOLD`), it sends `expected_rows` and expects the rows to start flowing. On the backend, however, `ChartDataRestApi._get_data_response` still calls `command.execute()` first: the full query runs and every row is materialized in memory, and only then does `_send_chart_response` notice the export should be streamed, discard that result and hand the query context to `StreamingCSVExportCommand`, which **runs the same query again**.
For large exports that first pass is long and completely silent (no byte is sent to the client), so load balancers and reverse proxies drop the idle connection and the download never starts. It also executes the query twice and holds the whole result in a web worker for nothing.
This PR short-circuits that case: when `expected_rows` is present and ≥ `CSV_STREAMING_ROW_THRESHOLD`, the format is CSV and the result type is `FULL`, `_get_data_response` goes straight to `_create_streaming_csv_response` without executing the command first. Everything else (JSON, XLSX, post-processed results such as pivot tables, CSV below the threshold or without `expected_rows`) keeps the regular path.
Design notes:
- **Permissions:** the export permission check that `_send_chart_response` applies (`can_csv`, or `can_export_data` with `GRANULAR_EXPORT_CONTROLS`) is extracted into `_has_export_permission()` and used by both paths, so they cannot drift apart. Datasource access is still enforced by `command.validate()` in the endpoint and again by `StreamingCSVExportCommand.validate()`.
- **Same eligibility as the frontend:** `expected_rows` is only sent by `useStreamingExport` after the dashboard/explore code decides to stream with the same threshold read from the server config, and the backend re-checks that threshold.
- **Nothing is lost by skipping execution:** the streaming command builds its SQL from the same query context; dashboard filter context is applied to the query context before the command is created; the acting-user handling from #44425 lives inside `StreamingCSVExportCommand` and is unchanged.
Real-world impact: we have been running this same change as a runtime patch on a 6.1.0 deployment (AWS ALB in front, idle timeout 60 s at the time). A 380–400k-row table export from a dashboard never completed — minutes of silence, then `TimeoutError` on the gunicorn write and a dropped DB connection. With the change, the same export (402,749 rows, 139.6 MB) completes in 17–25 s, and the query runs once.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A (backend only).
### TESTING INSTRUCTIONS
Unit tests (added to `tests/unit_tests/charts/test_chart_data_api.py`):
- a large CSV export with `expected_rows` streams without calling `command.execute()`;
- below the threshold, without `expected_rows`, `POST_PROCESSED` and XLSX all keep executing through the regular path;
- the up-front streaming path returns 403 without the export permission, for both `can_csv` and `can_export_data` (`GRANULAR_EXPORT_CONTROLS`).
The three tests covering the new behavior fail on current `master` and pass with this change; the four regular-path tests pass on both. `tests/unit_tests/charts`, `tests/unit_tests/commands/chart` and `tests/unit_tests/common` pass (743 passed, 2 xfailed).
Manual:
1. Set `CSV_STREAMING_ROW_THRESHOLD` low (e.g. `1000`) and use a table chart with more rows than that.
2. From the dashboard, choose *Download → Export to .CSV*.
3. The download starts right away; the logs show a single execution of the chart query (from the streaming command) instead of two.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
A note on the tests: `security_manager` is a proxy, so a plain `patch("superset.charts.data.api.security_manager")` turns `can_access` into an async mock whose truthy coroutine grants every permission. The new tests patch it with `new_callable=MagicMock` so a denied permission is actually exercised.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): stream large CSV exports without executing the query first [superset]
Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #44624:
URL: https://github.com/apache/superset/pull/44624#issuecomment-5852574278
### <span aria-hidden="true">✅</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | 228757f718d56cf8b764f37391db68b39ec74973 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/6ab898be77120e0008525e57 |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-44624--superset-docs-preview.netlify.app](https://deploy-preview-44624--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
|<span aria-hidden="true">🤖</span> Make changes | [Run an agent on this branch](https://app.netlify.com/projects/superset-docs-preview/agent-runs#DcrRDkAgGAbQd_mvNbGleAOPUfnQRrXEmHl33Z2L81JMYY-ZBqKKTNLersWzu2t7XOzICXp3fmEGc0hguGHP7IIve0LcwjNO5XfaqF4ZSNm0HJxzJVoBIen7AQ) |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-44624--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): stream large CSV exports without executing the query first [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44624:
URL: https://github.com/apache/superset/pull/44624#issuecomment-5824394630
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): stream large CSV exports without executing the query first [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #44624:
URL: https://github.com/apache/superset/pull/44624#discussion_r4099771139
##########
superset/charts/data/api.py:
##########
@@ -655,6 +649,24 @@ def _get_data_response(
slice_: Slice | None = None,
) -> Response:
"""Get data response and optionally log is_cached information."""
+ query_context = command.query_context
+ if self._should_stream_before_execution(query_context, expected_rows):
+ # The client already decided to stream this export, so skip the
+ # regular execution: it would run the full query and hold every row
+ # in memory only to discard the result, and the streaming command
+ # runs the query again anyway. On large exports that first pass is
+ # long enough for proxies to drop the idle connection before the
+ # first byte is sent.
+ if not self._has_export_permission():
+ return self.response_403()
+ return self._create_streaming_csv_response(
+ {"query_context": query_context},
+ form_data,
+ filename=filename,
+ expected_rows=expected_rows,
+ slice_=slice_ or query_context.slice_,
+ )
Review Comment:
**Suggestion:** Skipping `command.execute()` also skips its `ChartDataQueryFailedError` and cache-error mapping, so failed large exports no longer return the normal chart-data 4xx response.
**Assessment:** 🟠 `Major` · 🔁 `Occurrence: Sometimes` · 🏷️ `Api mismatch`
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=ffc91a3ffc88493f9c2d741717cb3cf2&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=ffc91a3ffc88493f9c2d741717cb3cf2&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/charts/data/api.py
**Line:** 662:668
**Comment:**
*Api Mismatch: Skipping `command.execute()` also skips its `ChartDataQueryFailedError` and cache-error mapping, so failed large exports no longer return the normal chart-data 4xx response.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F44624&comment_hash=8ffdc02ae3417af8e82b4ce656f1fae7d5813e090ccc46d0a6b0c305ce5d1243&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F44624&comment_hash=8ffdc02ae3417af8e82b4ce656f1fae7d5813e090ccc46d0a6b0c305ce5d1243&reaction=dislike'>👎</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): stream large CSV exports without executing the query first [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #44624:
URL: https://github.com/apache/superset/pull/44624#issuecomment-5824717130
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/44624?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `95.23810%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 77.53%. Comparing base ([`599026a`](https://app.codecov.io/gh/apache/superset/commit/599026a0e5a82f9728a333aea0e07b6a7bf8f771?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`8893bcd`](https://app.codecov.io/gh/apache/superset/commit/8893bcd6ca9c6a47fc30c0d9ee472dd67d5f28bf?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/superset/pull/44624?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [superset/charts/data/api.py](https://app.codecov.io/gh/apache/superset/pull/44624?src=pr&el=tree&filepath=superset%2Fcharts%2Fdata%2Fapi.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvY2hhcnRzL2RhdGEvYXBpLnB5) | 95.23% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/superset/pull/44624?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #44624 +/- ##
==========================================
- Coverage 81.10% 77.53% -3.57%
==========================================
Files 2955 2955
Lines 178385 178402 +17
Branches 41336 41341 +5
==========================================
- Hits 144676 138326 -6350
- Misses 31000 37231 +6231
- Partials 2709 2845 +136
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/44624/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/44624/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `36.79% <19.04%> (-0.01%)` | :arrow_down: |
| [mysql](https://app.codecov.io/gh/apache/superset/pull/44624/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [postgres](https://app.codecov.io/gh/apache/superset/pull/44624/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/44624/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.70% <28.57%> (+<0.01%)` | :arrow_up: |
| [python](https://app.codecov.io/gh/apache/superset/pull/44624/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `78.51% <95.23%> (-6.89%)` | :arrow_down: |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/44624/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unit](https://app.codecov.io/gh/apache/superset/pull/44624/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `77.67% <95.23%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/44624?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): stream large CSV exports without executing the query first [superset]
Posted by "hainenber (via GitHub)" <gi...@apache.org>.
hainenber merged PR #44624:
URL: https://github.com/apache/superset/pull/44624
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): stream large CSV exports without executing the query first [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44624:
URL: https://github.com/apache/superset/pull/44624#issuecomment-5824451185
<!-- Bito Reply -->
The flagged issue is correct. By bypassing `command.execute()` in the new streaming path, the code skips the error handling logic (such as `ChartDataQueryFailedError` and cache-error mapping) that is normally triggered during execution.
To resolve this, you should wrap the streaming call in a `try-except` block that mirrors the error handling in the standard execution path.
### Proposed Fix
In `superset/charts/data/api.py`, update the `_get_data_response` method to include error handling for the streaming path:
```python
if not self._has_export_permission():
return self.response_403()
try:
return self._create_streaming_csv_response(
{"query_context": query_context},
form_data,
filename=filename,
expected_rows=expected_rows,
slice_=slice_ or query_context.slice_,
)
except ChartDataQueryFailedError as exc:
return self.response_400(message=str(exc))
```
Would you like me to fetch all other comments on this PR to validate and implement fixes for them as well?
**superset/charts/data/api.py**
```
try:
return self._create_streaming_csv_response(
{"query_context": query_context},
form_data,
filename=filename,
expected_rows=expected_rows,
slice_=slice_ or query_context.slice_,
)
except ChartDataQueryFailedError as exc:
return self.response_400(message=str(exc))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): stream large CSV exports without executing the query first [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44624:
URL: https://github.com/apache/superset/pull/44624#issuecomment-5853773725
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): stream large CSV exports without executing the query first [superset]
Posted by "hainenber (via GitHub)" <gi...@apache.org>.
hainenber commented on code in PR #44624:
URL: https://github.com/apache/superset/pull/44624#discussion_r4113826777
##########
superset/charts/data/api.py:
##########
@@ -655,6 +649,24 @@ def _get_data_response(
slice_: Slice | None = None,
) -> Response:
"""Get data response and optionally log is_cached information."""
+ query_context = command.query_context
+ if self._should_stream_before_execution(query_context, expected_rows):
+ # The client already decided to stream this export, so skip the
+ # regular execution: it would run the full query and hold every row
+ # in memory only to discard the result, and the streaming command
+ # runs the query again anyway. On large exports that first pass is
+ # long enough for proxies to drop the idle connection before the
+ # first byte is sent.
+ if not self._has_export_permission():
+ return self.response_403()
+ return self._create_streaming_csv_response(
+ {"query_context": query_context},
+ form_data,
+ filename=filename,
+ expected_rows=expected_rows,
+ slice_=slice_ or query_context.slice_,
+ )
Review Comment:
This is valid. @aipiJuancho can you help bringing this code block under the try-catch section below?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): stream large CSV exports without executing the query first [superset]
Posted by "aipiJuancho (via GitHub)" <gi...@apache.org>.
aipiJuancho commented on code in PR #44624:
URL: https://github.com/apache/superset/pull/44624#discussion_r4114055657
##########
superset/charts/data/api.py:
##########
@@ -655,6 +649,24 @@ def _get_data_response(
slice_: Slice | None = None,
) -> Response:
"""Get data response and optionally log is_cached information."""
+ query_context = command.query_context
+ if self._should_stream_before_execution(query_context, expected_rows):
+ # The client already decided to stream this export, so skip the
+ # regular execution: it would run the full query and hold every row
+ # in memory only to discard the result, and the streaming command
+ # runs the query again anyway. On large exports that first pass is
+ # long enough for proxies to drop the idle connection before the
+ # first byte is sent.
+ if not self._has_export_permission():
+ return self.response_403()
+ return self._create_streaming_csv_response(
+ {"query_context": query_context},
+ form_data,
+ filename=filename,
+ expected_rows=expected_rows,
+ slice_=slice_ or query_context.slice_,
+ )
Review Comment:
Good catch — moved the streaming block under the existing try/except in 228757f, so failures raised synchronously while building the streaming response (`validate()` -> `raise_for_access()`, and the SQL compilation inside `run()`) now go through the same 422/400 mapping as the regular path. Added a regression test covering both exception types.
While at it, also addressed a related finding from the automated review: the shortcut now requires a single-query context (`len(queries) == 1`) — `StreamingCSVExportCommand` only exports `queries[0]`, while the regular path bundles multi-query results into a zip, so those contexts now keep the regular path (also tested).
Unit suite green: 750 passed / 2 pre-existing xfails across `charts` + `commands/chart` + `common`; ruff clean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org