You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "villebro (via GitHub)" <gi...@apache.org> on 2026/09/05 14:11:29 UTC

[PR] refactor(sql): share CTAS/limit helpers + add cancel seam to the execution feature [superset]

villebro opened a new pull request, #43929:
URL: https://github.com/apache/superset/pull/43929

   ### SUMMARY
   
   Child PR 1 of the [SQL execution on GTF epic](https://github.com/apache/superset/pull/43928) (umbrella `villebro/sqllab-gtf`).
   
   Lays the foundation for running SQL Lab and the unified SQL execution API on GTF, without changing any runtime behavior yet:
   
   - **superset-core contract** — extend `QueryOptions` with the fields the SQL Lab execution path needs (`select_as_cta`, `ctas_method`, `tmp_table_name`, `tmp_schema_name`, `expand_data`, `store_results`, `existing_query_id`, `caller_owns_timeout`) and `QueryResult` with `results_key` / `limiting_factor`. All additive and defaulted — the MCP `execute()` caller is unaffected.
   - **Share `apply_ctas` / `apply_limit`** — moved from `superset/sql_lab.py` into `superset/sql/execution/executor.py` so the unified `SQLExecutor` and the classic SQL Lab path use one implementation (`sql_lab.py` now imports them). Immediate de-duplication; no logic change.
   - **Engine-cancel seam** — call `notify_cursor(cursor)` in the shared `execute_sql_with_cursor` loop before the blocking execute, so a GTF task wrapping this feature can capture an engine cancel id — the same seam `Database.get_df` already uses for chart-data. A no-op when no cancellation sink is registered (e.g. the MCP sync path).
   - **`SQLExecutor` hooks** — `caller_owns_timeout` (skip the in-process `SIGALRM` timeout when a GTF `@task` timeout owns it, on the async path) and `existing_query_id` (execute against the SQL-Lab-created `Query` row instead of creating a minimal one).
   
   These hooks are wired but not yet exercised by a new caller; they are consumed by the subsequent child PRs (SQL-Lab-complete executor, then the GTF SQL task).
   
   ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
   
   N/A — no user-facing change.
   
   ### TESTING INSTRUCTIONS
   
   `pytest tests/unit_tests/sql/execution/ tests/unit_tests/sql_lab_test.py` — 148 existing tests pass unchanged (the `apply_ctas`/`apply_limit` move and the cancel seam are behavior-preserving; the MCP `execute()` path is unaffected). New behavior for the added options is tested in the child PRs that consume them.
   
   ### ADDITIONAL INFORMATION
   
   - [ ] Has associated issue:
   - [ ] Required feature flags:
   - [ ] Changes UI
   - [ ] Includes DB Migration
   - [x] Introduces new feature or API
   - [ ] Removes existing feature or API
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [PR] refactor(sql): share CTAS/limit helpers + add cancel seam to the execution feature [superset]

Posted by "villebro (via GitHub)" <gi...@apache.org>.
villebro commented on PR #43929:
URL: https://github.com/apache/superset/pull/43929#issuecomment-5552563337

   /review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [PR] refactor(sql): share CTAS/limit helpers + add cancel seam to the execution feature [superset]

Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #43929:
URL: https://github.com/apache/superset/pull/43929#issuecomment-5552383449

   ## [Codecov](https://app.codecov.io/gh/apache/superset/pull/43929?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :warning: Please [upload](https://docs.codecov.com/docs/codecov-uploader) report for BASE (`villebro/sqllab-gtf@802dfe3`). [Learn more](https://docs.codecov.io/docs/error-reference?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#section-missing-base-commit) about missing BASE report.
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@                  Coverage Diff                   @@
   ##             villebro/sqllab-gtf   #43929   +/-   ##
   ======================================================
     Coverage                       ?   79.84%           
   ======================================================
     Files                          ?     2918           
     Lines                          ?   170860           
     Branches                       ?    39461           
   ======================================================
     Hits                           ?   136423           
     Misses                         ?    31918           
     Partials                       ?     2519           
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/superset/pull/43929/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [superset-extensions-cli](https://app.codecov.io/gh/apache/superset/pull/43929/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `90.57% <ø> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   </details>
   
   [:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/43929?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [PR] refactor(sql): share CTAS/limit helpers + add cancel seam to the execution feature [superset]

Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43929:
URL: https://github.com/apache/superset/pull/43929#issuecomment-5552377122

   AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [PR] refactor(sql): share CTAS/limit helpers + add cancel seam to the execution feature [superset]

Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #43929:
URL: https://github.com/apache/superset/pull/43929#discussion_r3940889055


##########
superset-core/src/superset_core/queries/types.py:
##########
@@ -83,6 +85,31 @@ class QueryOptions:
     # Dry run option
     dry_run: bool = False  # Return transformed SQL without executing
 
+    # CREATE TABLE/VIEW AS SELECT. When ``select_as_cta`` is set the last (CTAS)
+    # or sole (CVAS) SELECT is rewritten to materialize into
+    # ``tmp_schema_name.tmp_table_name`` using ``ctas_method`` ("TABLE"/"VIEW").
+    select_as_cta: bool = False
+    ctas_method: str | None = None
+    tmp_table_name: str | None = None
+    tmp_schema_name: str | None = None

Review Comment:
   **Suggestion:** `select_as_cta` and its CTAS fields are exposed through the public options contract, but `SQLExecutor` never applies them, so requested CTAS or CVAS executions run as ordinary queries. [api mismatch]
   
   **Assessment:** 🟠 `Major` · 🔁 `Occurrence: Sometimes`
   
   [![Use CodeAnt Skill](https://new-codeant-butcket.s3.us-west-1.amazonaws.com/badges/use-codeant-skill-flat-v2.svg)](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [![Fix in Cursor](https://new-codeant-butcket.s3.us-west-1.amazonaws.com/badges/fix-in-cursor-flat.svg)](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=81db98dddbae41b5a6649ab2fd7fafb2&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [![Fix in VSCode Claude](https://new-codeant-butcket.s3.us-west-1.amazonaws.com/badges/fix-in-vscode-claude-flat.svg)](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=81db98dddbae41b5a6649ab2fd7fafb2&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
   <details>
   <summary><b>Prompt for AI Agent 🤖 </b></summary>
   
   ```mdx
   This is a comment left during a code review.
   
   **Path:** superset-core/src/superset_core/queries/types.py
   **Line:** 91:94
   **Comment:**
   	*Api Mismatch: `select_as_cta` and its CTAS fields are exposed through the public options contract, but `SQLExecutor` never applies them, so requested CTAS or CVAS executions run as ordinary queries.
   
   Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
   Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
   ```
   </details>
   <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43929&comment_hash=c180f23adeca524494af0ce9e1d0499a8b4d2291758d3fb41f7f775b71ecfd14&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43929&comment_hash=c180f23adeca524494af0ce9e1d0499a8b4d2291758d3fb41f7f775b71ecfd14&reaction=dislike'>👎</a>



##########
superset-core/src/superset_core/queries/types.py:
##########
@@ -83,6 +85,31 @@ class QueryOptions:
     # Dry run option
     dry_run: bool = False  # Return transformed SQL without executing
 
+    # CREATE TABLE/VIEW AS SELECT. When ``select_as_cta`` is set the last (CTAS)
+    # or sole (CVAS) SELECT is rewritten to materialize into
+    # ``tmp_schema_name.tmp_table_name`` using ``ctas_method`` ("TABLE"/"VIEW").
+    select_as_cta: bool = False
+    ctas_method: str | None = None
+    tmp_table_name: str | None = None
+    tmp_schema_name: str | None = None
+
+    # Result handling. ``expand_data`` expands nested columns (Presto/Trino).
+    # ``store_results`` persists the serialized result to the results backend and
+    # returns its handle on ``QueryResult.results_key`` (the SQL Lab path);
+    # otherwise results are returned inline as DataFrames on each StatementResult.
+    expand_data: bool = False
+    store_results: bool = False

Review Comment:
   **Suggestion:** `store_results` is exposed as a supported option, but the executor still returns inline DataFrames and never populates `QueryResult.results_key`, silently ignoring requests for backend storage. [api mismatch]
   
   **Assessment:** 🟠 `Major` · 🔁 `Occurrence: Sometimes`
   
   [![Use CodeAnt Skill](https://new-codeant-butcket.s3.us-west-1.amazonaws.com/badges/use-codeant-skill-flat-v2.svg)](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [![Fix in Cursor](https://new-codeant-butcket.s3.us-west-1.amazonaws.com/badges/fix-in-cursor-flat.svg)](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=24c025652bc84c479aa57e86e12af112&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [![Fix in VSCode Claude](https://new-codeant-butcket.s3.us-west-1.amazonaws.com/badges/fix-in-vscode-claude-flat.svg)](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=24c025652bc84c479aa57e86e12af112&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
   <details>
   <summary><b>Prompt for AI Agent 🤖 </b></summary>
   
   ```mdx
   This is a comment left during a code review.
   
   **Path:** superset-core/src/superset_core/queries/types.py
   **Line:** 101:101
   **Comment:**
   	*Api Mismatch: `store_results` is exposed as a supported option, but the executor still returns inline DataFrames and never populates `QueryResult.results_key`, silently ignoring requests for backend storage.
   
   Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
   Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
   ```
   </details>
   <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43929&comment_hash=591e32fcd567336dd3a933bb662e714b1ce05369e68548395dc8f7e7cfd66980&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43929&comment_hash=591e32fcd567336dd3a933bb662e714b1ce05369e68548395dc8f7e7cfd66980&reaction=dislike'>👎</a>



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [PR] refactor(sql): share CTAS/limit helpers + add cancel seam to the execution feature [superset]

Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43929:
URL: https://github.com/apache/superset/pull/43929#issuecomment-5552564160

   AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [PR] refactor(sql): share CTAS/limit helpers + add cancel seam to the execution feature [superset]

Posted by "villebro (via GitHub)" <gi...@apache.org>.
villebro merged PR #43929:
URL: https://github.com/apache/superset/pull/43929


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org