You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "villebro (via GitHub)" <gi...@apache.org> on 2026/09/08 22:55:54 UTC
[PR] feat(sqllab): settle async queries via GTF task.status realtime push [superset]
villebro opened a new pull request, #44032:
URL: https://github.com/apache/superset/pull/44032
### SUMMARY
Child PR 5 of the [SQL execution on GTF epic](https://github.com/apache/superset/pull/43928) (umbrella `villebro/sqllab-gtf`).
Adds a low-latency realtime path for async SQL Lab queries — they settle via the GTF `task.status` websocket push instead of waiting up to 2s for the next `/query/updated_since` poll. The poll is unchanged and remains the correctness backstop; the websocket is a pure accelerator. Progress still rides the poll (`task.status` is terminal-only).
**Generalized per-tab delivery.** Chart-data's per-tab consumer subscription policy (ref-count the browser tabs watching a task and route its terminal `task.status` fanout to exactly those tabs) is extracted into a reusable `PerTabConsumerPolicy` in `superset/tasks/subscription.py`. `ChartQueryConsumerPolicy` becomes a thin subclass (chart-data behavior unchanged), and the SQL Lab task `run_sql_lab_query` registers the shared policy so its completion is delivered only to the originating tab. Any future task type gets tab-isolated delivery for free.
**Task descriptor in the execute 202.** `ExecuteSqlCommand.submit_async` captures a pre-schedule status cursor (the recovery watermark) plus the scheduled task's UUID and returns an `async_job` `{task_id, cursor, tab_id?}`; the `/execute/` endpoint surfaces it in the 202 (`QueryExecutionResponseSchema` extended). The cursor is captured before the task is created so a completion can't slip in ahead of the client's waiter.
**Frontend.** `runQuery` advertises the per-tab id (`getTabId`) on execute so the backend can route the push back, and stores the 202's `async_job.task_id` on the query (new `taskId` field + `SET_QUERY_TASK_ID`). `QueryAutoRefresh` subscribes to the generic `task.status` realtime topic and triggers an immediate refresh when a currently-running query's tracked task reaches a terminal status; `task.status` messages for untracked task ids (e.g. chart-data) are ignored. The 2s poll is untouched as the backstop.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A — behavior-preserving latency improvement. With `WEBSOCKET_ENABLE` on, an async query settles on its terminal `task.status` push; with it off (or a missed push), the existing poll still settles it.
### TESTING INSTRUCTIONS
Backend: `pytest tests/unit_tests/tasks/test_subscription.py tests/unit_tests/commands/sql_lab/test_execute.py tests/unit_tests/daos/test_tasks.py`. Frontend: `npm run test -- src/SqlLab/components/QueryAutoRefresh/QueryAutoRefresh.test.tsx src/SqlLab/actions/sqlLab.test.ts`. End-to-end (dev instance with `GLOBAL_TASK_FRAMEWORK` + `WEBSOCKET_ENABLE`): run an async query and confirm it settles promptly via the websocket; disable the websocket and confirm it still settles via the 2s poll.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [x] Required feature flags: `GLOBAL_TASK_FRAMEWORK` (async SQL Lab execution); `WEBSOCKET_ENABLE` + `WEBSOCKET_URL` for the realtime push (poll fallback otherwise)
- [x] Changes UI
- [ ] Includes DB Migration
- [x] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): settle async queries via GTF task.status realtime push [superset]
Posted by "villebro (via GitHub)" <gi...@apache.org>.
villebro merged PR #44032:
URL: https://github.com/apache/superset/pull/44032
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): settle async queries via GTF task.status realtime push [superset]
Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #44032:
URL: https://github.com/apache/superset/pull/44032#issuecomment-5593022519
### <span aria-hidden="true">✅</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | ae2f641e1c7bb2caaa62460b520fb3c54cadca10 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/6aa0927a8316890008ce189a |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-44032--superset-docs-preview.netlify.app](https://deploy-preview-44032--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
|<span aria-hidden="true">🤖</span> Make changes | [Run an agent on this branch](https://app.netlify.com/projects/superset-docs-preview/agent-runs#DctRDoMgDADQu_RbsuqigjfwGEXKZlIBKzFZlt19_L2f94Wi-SgVFoAOvFLa3s33LsJe8-M6RcibV40GRxM1p8opGGWSuh_cTuAi-bOGtiYidMNM9tlP1iGi3bi3juD3Bw) |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-44032--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): settle async queries via GTF task.status realtime push [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #44032:
URL: https://github.com/apache/superset/pull/44032#issuecomment-5593084505
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/44032?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `40.00000%` with `36 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 57.44%. Comparing base ([`470c507`](https://app.codecov.io/gh/apache/superset/commit/470c507e6a29b15eb2b282922a995568532656d0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`ae2f641`](https://app.codecov.io/gh/apache/superset/commit/ae2f641e1c7bb2caaa62460b520fb3c54cadca10?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/superset/pull/44032?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [superset/tasks/subscription.py](https://app.codecov.io/gh/apache/superset/pull/44032?src=pr&el=tree&filepath=superset%2Ftasks%2Fsubscription.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvdGFza3Mvc3Vic2NyaXB0aW9uLnB5) | 36.36% | [21 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/44032?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [superset/commands/sql\_lab/execute.py](https://app.codecov.io/gh/apache/superset/pull/44032?src=pr&el=tree&filepath=superset%2Fcommands%2Fsql_lab%2Fexecute.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvY29tbWFuZHMvc3FsX2xhYi9leGVjdXRlLnB5) | 10.00% | [9 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/44032?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [superset/sqllab/api.py](https://app.codecov.io/gh/apache/superset/pull/44032?src=pr&el=tree&filepath=superset%2Fsqllab%2Fapi.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvc3FsbGFiL2FwaS5weQ==) | 33.33% | [4 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/44032?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [superset/tasks/sql\_queries.py](https://app.codecov.io/gh/apache/superset/pull/44032?src=pr&el=tree&filepath=superset%2Ftasks%2Fsql_queries.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvdGFza3Mvc3FsX3F1ZXJpZXMucHk=) | 0.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/44032?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (470c507) and HEAD (ae2f641). Click for more details.
>
> <details><summary>HEAD has 4 uploads less than BASE</summary>
>
>| Flag | BASE (470c507) | HEAD (ae2f641) |
>|------|------|------|
>|python|6|2|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## villebro/sqllab-gtf #44032 +/- ##
========================================================
- Coverage 79.87% 57.44% -22.43%
========================================================
Files 2916 2916
Lines 170607 170625 +18
Branches 39522 39524 +2
========================================================
- Hits 136268 98016 -38252
- Misses 31832 71681 +39849
+ Partials 2507 928 -1579
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/44032/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/44032/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.52% <40.00%> (+<0.01%)` | :arrow_up: |
| [mysql](https://app.codecov.io/gh/apache/superset/pull/44032/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [postgres](https://app.codecov.io/gh/apache/superset/pull/44032/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/44032/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `39.39% <40.00%> (+<0.01%)` | :arrow_up: |
| [python](https://app.codecov.io/gh/apache/superset/pull/44032/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `39.45% <40.00%> (-44.88%)` | :arrow_down: |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/44032/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unit](https://app.codecov.io/gh/apache/superset/pull/44032/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/44032?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): settle async queries via GTF task.status realtime push [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44032:
URL: https://github.com/apache/superset/pull/44032#issuecomment-5593006714
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org