You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "dpgaspar (via GitHub)" <gi...@apache.org> on 2026/04/10 13:56:15 UTC
[PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
dpgaspar opened a new pull request, #39278:
URL: https://github.com/apache/superset/pull/39278
### SUMMARY
Upgrade Flask from 2.3.3 to 3.1.3 while keeping SQLAlchemy on 1.4.x. This is achieved by using **flask-sqlalchemy 3.0.5**, the last version that supports `sqlalchemy>=1.4.18` (3.1.0+ requires SQLAlchemy 2.x).
**Version changes:**
| Package | Before | After |
|---------|--------|-------|
| flask | 2.3.3 | 3.1.3 |
| flask-sqlalchemy | 2.5.1 | 3.0.5 |
| flask-babel | 3.1.0 | 4.0.0 |
**Why flask-babel 4.0.0?** Version 3.1.0 imports `locked_cached_property` from `flask.helpers`, which was removed in Flask 3.0.
**Code changes:**
- Replace deprecated `from flask_sqlalchemy import BaseQuery` β `from sqlalchemy.orm import Query`
- Replace deprecated `engine.execute()` β `with engine.connect() as conn: conn.execute()` pattern across 7 files
- Remove dead `SQLALCHEMY_COMMIT_ON_TEARDOWN` code in Celery teardown (config key never set, removed in FSA 3.0)
- Pin `flask-sqlalchemy>=3.0.0, <3.1` in pyproject.toml to prevent accidental upgrade to 3.1+ (which requires SA 2.x)
- Update test mocks for new connection context manager pattern
**All other Flask extensions** (flask-appbuilder 5.2.1, flask-login, flask-cors, flask-limiter, etc.) are already compatible with Flask 3.x at their current versions. FAB has a built-in compatibility layer that auto-detects FSA 2.x vs 3.x.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A β no UI changes
### TESTING INSTRUCTIONS
- Verify Superset starts correctly with Flask 3.x
- Verify Celery workers start and execute tasks
- Run unit tests: `pytest tests/unit_tests/`
- Verify db engine specs that use `engine.connect()` pattern (Databricks, Hive, Impala, Doris)
- Verify `superset db upgrade` and `superset init` work
### ADDITIONAL INFORMATION
- [x] Has associated issue: [sc-103602](https://app.shortcut.com/preset/story/103602)
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
π€ Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed pull request #39278: chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat)
URL: https://github.com/apache/superset/pull/39278
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #39278:
URL: https://github.com/apache/superset/pull/39278#issuecomment-4224249129
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #39278:
URL: https://github.com/apache/superset/pull/39278#issuecomment-4224259923
### <span aria-hidden="true">β
</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">π¨</span> Latest commit | 592ff926f5dead52ebb010efd07fc2e2e5f0023f |
|<span aria-hidden="true">π</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/69d901802934de00088b8646 |
|<span aria-hidden="true">π</span> Deploy Preview | [https://deploy-preview-39278--superset-docs-preview.netlify.app](https://deploy-preview-39278--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">π±</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-39278--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #39278:
URL: https://github.com/apache/superset/pull/39278#discussion_r3064744276
##########
superset/db_engine_specs/hive.py:
##########
@@ -223,7 +223,8 @@ def df_to_sql(
catalog=table.catalog,
schema=table.schema,
) as engine:
- engine.execute(f"DROP TABLE IF EXISTS {str(table)}")
+ with engine.connect() as conn:
+ conn.execute(text(f"DROP TABLE IF EXISTS {str(table)}"))
Review Comment:
**Suggestion:** `conn.execute()` runs inside an implicit transaction on SQLAlchemy 1.4+, and this block closes the connection without committing. That can roll back the `DROP TABLE` statement, so `if_exists="replace"` may fail because the old table still exists. Commit explicitly (or use `engine.begin()`). [logic error]
<details>
<summary><b>Severity Level:</b> Major β οΈ</summary>
```mdx
- β Hive "replace" uploads may error on existing table metadata.
- β οΈ CSV upload workflow to Hive less reliable for existing tables.
```
</details>
```suggestion
conn.commit()
```
<details>
<summary><b>Steps of Reproduction β
</b></summary>
```mdx
1. Configure a Hive database in Superset (Database model `get_sqla_engine` at
`superset/models/core.py:427-546`) with file uploads enabled and CSV-to-Hive upload
settings (`CSV_TO_HIVE_UPLOAD_DIRECTORY_FUNC` and `CSV_TO_HIVE_UPLOAD_S3_BUCKET`) in
`superset/config.py:5-28` so uploads target Hive.
2. Call the file-upload API `POST /api/v1/database/<pk>/upload` implemented in
`superset/databases/api.py:1760-1835` with `type="csv"` and `table_name` pointing to an
already-existing Hive table, and set the upload option `already_exists="replace"` so
`UploadCommand.run()` in `superset/commands/database/uploaders/base.py:57-70` calls
`database.db_engine_spec.df_to_sql(database, Table(...), df, to_sql_kwargs={"if_exists":
"replace", ...})`.
3. For a Hive database, `database.db_engine_spec` is `HiveEngineSpec`, so
`HiveEngineSpec.df_to_sql()` in `superset/db_engine_specs/hive.py:22-72` executes; with
`to_sql_kwargs["if_exists"] == "replace"` and the table existing, the branch at
`superset/db_engine_specs/hive.py:61-68` (diff lines 221-227) runs, opening an engine via
`cls.get_engine(...)` and then a connection with `with engine.connect() as conn:` and
executing `conn.execute(text(f"DROP TABLE IF EXISTS {str(table)}"))` at lines 226-227
without any explicit commit or `engine.begin()`.
4. Under SQLAlchemy 1.4's transactional `Connection` semantics, `engine.connect()` starts
an implicit transaction; when the `with engine.connect()` block exits at
`superset/db_engine_specs/hive.py:61-68` without `conn.commit()` or an `engine.begin()`
context, that transaction is rolled back, so the `DROP TABLE` DDL may be undone. On
backends that require an explicit commit for DDL (which is the normal SQLAlchemy
behavior), the later `CREATE TABLE` block in the same method sees the original table still
present and can raise a "table already exists" error instead of correctly replacing it for
the upload request.
```
</details>
<details>
<summary><b>Prompt for AI Agent π€ </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/db_engine_specs/hive.py
**Line:** 228:228
**Comment:**
*Logic Error: `conn.execute()` runs inside an implicit transaction on SQLAlchemy 1.4+, and this block closes the connection without committing. That can roll back the `DROP TABLE` statement, so `if_exists="replace"` may fail because the old table still exists. Commit explicitly (or use `engine.begin()`).
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39278&comment_hash=c1cd70e40751d44531a4cc428e26b46641a93004f2c95e1e251dadde2ea7111e&reaction=like'>π</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39278&comment_hash=c1cd70e40751d44531a4cc428e26b46641a93004f2c95e1e251dadde2ea7111e&reaction=dislike'>π</a>
##########
superset/db_engine_specs/hive.py:
##########
@@ -249,22 +250,25 @@ def _get_hive_type(dtype: np.dtype[Any]) -> str:
catalog=table.catalog,
schema=table.schema,
) as engine:
- engine.execute(
- text(
- f"""
- CREATE TABLE {str(table)} ({schema_definition})
- STORED AS PARQUET
- LOCATION :location
- """
- ),
- location=upload_to_s3(
- filename=file.name,
- upload_prefix=app.config["CSV_TO_HIVE_UPLOAD_DIRECTORY_FUNC"](
- database, g.user, table.schema
+ with engine.connect() as conn:
+ conn.execute(
+ text(
+ f"""
+ CREATE TABLE {str(table)} ({schema_definition})
+ STORED AS PARQUET
+ LOCATION :location
+ """
),
- table=table,
- ),
- )
+ {
+ "location": upload_to_s3(
+ filename=file.name,
+ upload_prefix=app.config[
+ "CSV_TO_HIVE_UPLOAD_DIRECTORY_FUNC"
+ ](database, g.user, table.schema),
+ table=table,
+ ),
+ },
+ )
Review Comment:
**Suggestion:** The `CREATE TABLE` execution is also done on a plain connection with no commit, so the table creation can be rolled back when the context exits. This can cause upload flow to appear successful while the table is not actually created. [logic error]
<details>
<summary><b>Severity Level:</b> Critical π¨</summary>
```mdx
- β Hive CSV upload may succeed but table remains absent.
- β οΈ Users cannot query uploaded data via missing Hive table.
```
</details>
```suggestion
conn.commit()
```
<details>
<summary><b>Steps of Reproduction β
</b></summary>
```mdx
1. Configure a Hive database in Superset with CSV upload enabled using the S3
configuration in `superset/config.py:5-28` and register it so it appears in the Database
API (`superset/databases/api.py`).
2. Upload a new CSV/Excel/columnar file to that Hive database via `POST
/api/v1/database/<pk>/upload` implemented in `superset/databases/api.py:1760-1835` (for
example from the "Upload a CSV" UI or via direct API), providing a `table_name` that does
not yet exist in Hive; this causes `UploadCommand.run()` in
`superset/commands/database/uploaders/base.py:57-70` to read the file and call
`database.db_engine_spec.df_to_sql(database, Table(table=table_name, schema=schema), df,
to_sql_kwargs={"if_exists": "fail" or "replace", ...})`.
3. Because the database engine spec for Hive is `HiveEngineSpec`,
`HiveEngineSpec.df_to_sql()` in `superset/db_engine_specs/hive.py:22-72` executes: after
computing `schema_definition` and writing the DataFrame to a Parquet temp file
(`superset/db_engine_specs/hive.py:44-47`), it opens an engine context via
`cls.get_engine(...)` and then, at `superset/db_engine_specs/hive.py:49-72` (diff lines
252-271), it does `with engine.connect() as conn:` and calls `conn.execute(text("""CREATE
TABLE {str(table)} ({schema_definition}) STORED AS PARQUET LOCATION :location"""),
{"location": upload_to_s3(...)})` without committing the connection.
4. On SQLAlchemy 1.4+, that `engine.connect()` call creates a transactional `Connection`;
because there is no `conn.commit()` and no `engine.begin()` context, when the `with
engine.connect()` block ends the implicit transaction is rolled back, so the `CREATE
TABLE` DDL is not persisted on backends that require explicit commit for DDL. The upload
handler in `superset/databases/api.py:1760-1835` then returns HTTP 201 "OK" (no exception
is raised from `df_to_sql`), but the Hive external table never actually appears in the
database even though the Parquet data was uploaded to S3 by `upload_to_s3()` at
`superset/db_engine_specs/hive.py:1-30`.
```
</details>
<details>
<summary><b>Prompt for AI Agent π€ </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/db_engine_specs/hive.py
**Line:** 272:272
**Comment:**
*Logic Error: The `CREATE TABLE` execution is also done on a plain connection with no commit, so the table creation can be rolled back when the context exits. This can cause upload flow to appear successful while the table is not actually created.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39278&comment_hash=6e43fa8cfae20f95eaf33688af9ecb16056c82bd5b980e738db9cb7c111a42e0&reaction=like'>π</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39278&comment_hash=6e43fa8cfae20f95eaf33688af9ecb16056c82bd5b980e738db9cb7c111a42e0&reaction=dislike'>π</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #39278:
URL: https://github.com/apache/superset/pull/39278#issuecomment-5188013028
@dpgaspar scratch my gevent theory from June, think I found the real cause. It's not session scoping at all, it's a gap in Flask-AppBuilder's own FSA-3.x shim: `base_legacy.py` overrides `create_session()`/`get_bind()` to bind Superset's models onto FAB's own `declarative_base()`, but `base.py` (the FSA 3.x version) doesn't do any of that anymore, and `SQLA(app)` never passes `model_class=`. So under FSA 3.x, Superset's models and `db.session`'s bind resolution are looking at two disconnected registries, which lines up with the NoneType breadth I saw reproducing this in #42542 (dashboards, security, tasks, all hitting the same mismatched seam).
You'd know this better than I would since you wrote FAB. Does that match your read, and is wiring `model_class=` through when `base.py` builds `SQLA(app)` enough, or is there more to it?
We're about to pick step 6 back up on #40273 now that the driver-compat prep landed in #42542, so this is converging with that work. Worth coordinating rather than duplicating effort.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #39278:
URL: https://github.com/apache/superset/pull/39278#discussion_r3064718211
##########
tests/integration_tests/reports/commands_tests.py:
##########
@@ -127,7 +127,7 @@ def get_bcctarget_from_report_schedule(report_schedule: ReportSchedule) -> list[
]
-def get_error_logs_query(report_schedule: ReportSchedule) -> BaseQuery:
+def get_error_logs_query(report_schedule: ReportSchedule) -> Query:
return (
Review Comment:
**Suggestion:** Add a docstring to the new helper function so the new definition complies with the docstring requirement for newly introduced functions. [custom_rule]
**Severity Level:** Minor β οΈ
```suggestion
"""Return the error log query for a report schedule."""
```
<details>
<summary><b>Why it matters? π€ </b></summary>
The new helper function in the final file has no docstring, which violates the stated rule for newly introduced functions. The improved code adds a valid Python docstring without changing behavior, so it is syntactically correct and executable.
</details>
<details>
<summary><b>Prompt for AI Agent π€ </b></summary>
```mdx
This is a comment left during a code review.
**Path:** tests/integration_tests/reports/commands_tests.py
**Line:** 131:131
**Comment:**
*Custom Rule: Add a docstring to the new helper function so the new definition complies with the docstring requirement for newly introduced functions.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39278&comment_hash=7842ff88212ed2bdb47824a8c3548bd01cba2b854c6816f27f4466e5c78088a6&reaction=like'>π</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39278&comment_hash=7842ff88212ed2bdb47824a8c3548bd01cba2b854c6816f27f4466e5c78088a6&reaction=dislike'>π</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on code in PR #39278:
URL: https://github.com/apache/superset/pull/39278#discussion_r3064856047
##########
pyproject.toml:
##########
@@ -47,8 +47,9 @@ dependencies = [
"cron-descriptor",
"cryptography>=42.0.4, <47.0.0",
"deprecation>=2.1.0, <2.2.0",
- "flask>=2.2.5, <4.0.0",
+ "flask>=3.0.0, <4.0.0",
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>Extension Compatibility Risk</b></div>
<div id="fix">
Updating Flask to >=3.0.0 requires flask-login >=0.6.3, as versions <0.6.3 are incompatible due to Werkzeug 3.0 breaking changes that affect session management. The current pin >=0.6.0 may install incompatible versions, potentially causing login failures.
</div>
<details>
<summary>
<b>Code suggestion</b>
</summary>
<blockquote>Check the AI-generated fix before applying</blockquote>
<div id="code">
```
- "flask-login>=0.6.0, < 1.0",
+ "flask-login>=0.6.3, < 1.0",
```
</div>
</details>
</div>
<small><i>Code Review Run #1f23e7</i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
##########
superset/cli/test_db.py:
##########
@@ -121,11 +121,13 @@ def test_datetime(console: Console, engine: Engine) -> None:
console.print("Inserting timestamp value...")
insert_stmt = insert(table).values(ts=now)
- engine.execute(insert_stmt)
+ with engine.connect() as conn:
+ conn.execute(insert_stmt)
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>Incorrect transaction handling for insert</b></div>
<div id="fix">
The replacement of deprecated engine.execute() with engine.connect() for the insert statement fails to commit the transaction, unlike the original code. This will cause the test to fail as the subsequent select won't find the inserted row. Use engine.begin() instead to auto-commit on success.
</div>
<details>
<summary>
<b>Code suggestion</b>
</summary>
<blockquote>Check the AI-generated fix before applying</blockquote>
<div id="code">
````suggestion
with engine.begin() as conn:
conn.execute(insert_stmt)
````
</div>
</details>
</div>
<small><i>Code Review Run #1f23e7</i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #39278:
URL: https://github.com/apache/superset/pull/39278#discussion_r3064723191
##########
superset/cli/test_db.py:
##########
@@ -121,11 +121,13 @@ def test_datetime(console: Console, engine: Engine) -> None:
console.print("Inserting timestamp value...")
insert_stmt = insert(table).values(ts=now)
- engine.execute(insert_stmt)
+ with engine.connect() as conn:
Review Comment:
**Suggestion:** The insert is executed inside `engine.connect()` without an explicit commit, so on transactional backends (like PostgreSQL) the transaction is rolled back when the context exits and the subsequent read can return no row. Use a transactional context that commits on success. [logic error]
<details>
<summary><b>Severity Level:</b> Major β οΈ</summary>
```mdx
- β `superset test-db` fails datetime test on PostgreSQL.
- β οΈ Database diagnostics CLI misreports timestamp roundtrip capability.
```
</details>
```suggestion
with engine.begin() as conn:
```
<details>
<summary><b>Steps of Reproduction β
</b></summary>
```mdx
1. Invoke the `test_db` CLI command defined at `superset/cli/test_db.py:135-143` with a
PostgreSQL URI, e.g. `superset test-db postgresql://user:pass@host:5432/dbname`, so
`test_db()` constructs the engine and calls `test_database_connectivity(console, engine)`
at lines 171-175.
2. Inside `test_database_connectivity` (`superset/cli/test_db.py:75-106`), after basic
connectivity checks, the code retrieves engine-specific tests via
`registry.get_tests(engine.dialect.name)` at line 99 and iterates them at lines 100-105,
calling `test(console, engine)` for each when `engine.dialect.name` is `"postgresql"`.
3. The `test_datetime` function registered for `"sqlite"` and `"postgresql"` at
`superset/cli/test_db.py:104-105` is invoked; it creates the `test` table with a `ts`
`DateTime` column at lines 109-114 and calls `md.create_all(engine)` at line 117 to create
the table.
4. Still in `test_datetime`, it prepares `insert_stmt = insert(table).values(ts=now)` at
line 123, then executes the insert inside `with engine.connect() as conn:` at lines
124-125 without calling `conn.commit()` or using a transactional context; on PostgreSQL
this opens a transaction that is rolled back when the connection is closed at context
exit.
5. Next, `test_datetime` builds `select_stmt = select(table)` at line 128 and opens a new
connection `with engine.connect() as conn:` at line 129, executing `row =
conn.execute(select_stmt).fetchone()` at line 130; because the prior insert was rolled
back, `fetchone()` returns `None`, and `assert row[0] == now` at line 131 raises
`TypeError: 'NoneType' object is not subscriptable`, causing the `superset test-db`
command to fail for PostgreSQL backends.
```
</details>
<details>
<summary><b>Prompt for AI Agent π€ </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/cli/test_db.py
**Line:** 124:124
**Comment:**
*Logic Error: The insert is executed inside `engine.connect()` without an explicit commit, so on transactional backends (like PostgreSQL) the transaction is rolled back when the context exits and the subsequent read can return no row. Use a transactional context that commits on success.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39278&comment_hash=28a94b42104f65a6f27c978ba423cbef0e7ff2addc1acbaef931ecbb0d38defc&reaction=like'>π</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39278&comment_hash=28a94b42104f65a6f27c978ba423cbef0e7ff2addc1acbaef931ecbb0d38defc&reaction=dislike'>π</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #39278:
URL: https://github.com/apache/superset/pull/39278#discussion_r3080391876
##########
tests/integration_tests/security_tests.py:
##########
@@ -111,6 +112,7 @@ def delete_schema_perm(view_menu_name: str) -> None:
return None
+@pytest.mark.usefixtures("load_world_bank_dashboard_with_slices_class_scope")
Review Comment:
**Suggestion:** Using a class-scoped dashboard fixture here conflicts with existing method-level uses of the function-scoped dashboard fixture in the same test class. Those method-level teardowns can delete the shared dashboard early, and the class fixture finalizer then tries to delete an already-removed object, causing teardown failures and flaky tests. Use the same function-scoped fixture at class level to avoid double-cleanup of shared state. [logic error]
<details>
<summary><b>Severity Level:</b> Critical π¨</summary>
```mdx
- β TestRolePermission teardown crashes when deleting already-removed dashboard.
- β οΈ CI integration test suite fails, blocking merges and releases.
```
</details>
```suggestion
@pytest.mark.usefixtures("load_world_bank_dashboard_with_slices")
```
<details>
<summary><b>Steps of Reproduction β
</b></summary>
```mdx
1. Run a single test from the class using the dashboard fixture, for example:
`pytest
tests/integration_tests/security_tests.py::TestRolePermission::test_gamma_user_schema_access_to_dashboards`.
The class `TestRolePermission` at `tests/integration_tests/security_tests.py:116` is
decorated with
`@pytest.mark.usefixtures("load_world_bank_dashboard_with_slices_class_scope")` at line
115.
2. Before any test method runs, pytest sets up the class-scoped fixture
`load_world_bank_dashboard_with_slices_class_scope` defined in
`tests/integration_tests/fixtures/world_bank_dashboard.py:92-98`. This fixture calls
`create_dashboard_for_loaded_data()` at `world_bank_dashboard.py:100-106`, which in
turn calls
`_create_world_bank_dashboard()` and `dashboard_utils.create_dashboard()` at
`tests/integration_tests/dashboard_utils.py:84-90`. Because `create_dashboard()`
returns an existing
dashboard when one with slug `"world_health"` already exists, the fixture stores a
specific `dash_id_to_delete`.
3. For `test_gamma_user_schema_access_to_dashboards` at
`tests/integration_tests/security_tests.py:1228-1235`, pytest also applies the
function-scoped fixture
`load_world_bank_dashboard_with_slices` (decorator at line 1228), defined at
`world_bank_dashboard.py:75-80`. That fixture again calls
`create_dashboard_for_loaded_data()`, reusing the
same `"world_health"` dashboard, then on teardown calls `_cleanup(dash_id_to_delete,
slices_ids_to_delete)`
at `world_bank_dashboard.py:143-148`, which deletes the dashboard and its slices from
the database.
4. After all tests in `TestRolePermission` complete, pytest tears down the class-scoped
fixture
`load_world_bank_dashboard_with_slices_class_scope` (world_bank_dashboard.py:93-97).
Its finalizer again calls
`_cleanup()` with the original `dash_id_to_delete`, but that dashboard was already
removed by the
function-scoped fixture. `db.session.query(Dashboard).filter_by(id=dash_id).first()` at
line 144 returns `None`,
and `db.session.delete(dash)` at line 145 is invoked with `dash is None`, which in
SQLAlchemy 1.4 raises an
exception (unmapped instance), causing teardown errors and making the test run fail.
```
</details>
<details>
<summary><b>Prompt for AI Agent π€ </b></summary>
```mdx
This is a comment left during a code review.
**Path:** tests/integration_tests/security_tests.py
**Line:** 115:115
**Comment:**
*Logic Error: Using a class-scoped dashboard fixture here conflicts with existing method-level uses of the function-scoped dashboard fixture in the same test class. Those method-level teardowns can delete the shared dashboard early, and the class fixture finalizer then tries to delete an already-removed object, causing teardown failures and flaky tests. Use the same function-scoped fixture at class level to avoid double-cleanup of shared state.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39278&comment_hash=13b343e2ec84b437d28efaa83b64deafe545cfa7dcdb2c8659c79e70ba2b02b0&reaction=like'>π</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39278&comment_hash=13b343e2ec84b437d28efaa83b64deafe545cfa7dcdb2c8659c79e70ba2b02b0&reaction=dislike'>π</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #39278:
URL: https://github.com/apache/superset/pull/39278#issuecomment-4290576327
I updated the branch which should fix the code cove problem. If you want to review and accept changes for many of the bot comments, that would be cool. I'm happy to give it a thorough review after that. I just sent Co-Pilot to add some more comments to the pile.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #39278:
URL: https://github.com/apache/superset/pull/39278#discussion_r3064737600
##########
superset/cli/test_db.py:
##########
@@ -121,11 +121,13 @@ def test_datetime(console: Console, engine: Engine) -> None:
console.print("Inserting timestamp value...")
insert_stmt = insert(table).values(ts=now)
- engine.execute(insert_stmt)
+ with engine.connect() as conn:
+ conn.execute(insert_stmt)
console.print("Reading timestamp value...")
select_stmt = select(table)
- row = engine.execute(select_stmt).fetchone()
+ with engine.connect() as conn:
+ row = conn.execute(select_stmt).fetchone()
Review Comment:
**π Architect Review β HIGH**
The test_datetime helper inserts via `with engine.connect() as conn: conn.execute(insert_stmt)` and then reads in a separate `engine.connect()` context, but never commits a transaction; on backends where uncommitted work is rolled back on connection close (the default for most SQLAlchemy dialects), the SELECT runs on a new connection and will not see the inserted row, causing this diagnostic command to fail under normal use.
**Suggestion:** Perform the insert (and optionally the read) within an explicit transactional context, e.g. `with engine.begin() as conn: conn.execute(insert_stmt)` so that the insert is committed before the subsequent SELECT, regardless of backend autocommit behavior.
##########
superset/utils/mock_data.py:
##########
@@ -202,11 +202,12 @@ def add_data(
table = DBTable(table_name, metadata, *column_objects)
metadata.create_all(engine)
- if not append:
- engine.execute(table.delete())
+ with engine.connect() as conn:
+ if not append:
+ conn.execute(table.delete())
- data = generate_data(columns, num_rows)
- engine.execute(table.insert(), data)
+ data = generate_data(columns, num_rows)
+ conn.execute(table.insert(), data)
Review Comment:
**π Architect Review β HIGH**
The add_data helper now issues DELETE/INSERT using `with engine.connect() as conn: ... conn.execute(...)` without any explicit commit; on typical SQLAlchemy backends, the pending transaction is rolled back when the connection context exits, so mock data generation and table clearing can silently have no effect despite reporting success.
**Suggestion:** Wrap the delete/insert sequence in an explicit transactional context, e.g. `with engine.begin() as conn:` or equivalent, so that generated data and table clearing are durably committed across dialects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #39278:
URL: https://github.com/apache/superset/pull/39278#issuecomment-4245271425
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #39278:
URL: https://github.com/apache/superset/pull/39278#issuecomment-4251362101
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #39278:
URL: https://github.com/apache/superset/pull/39278#issuecomment-4224320465
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/39278?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 66.46%. Comparing base ([`b1d69f5`](https://app.codecov.io/gh/apache/superset/commit/b1d69f5b3987adcef0b8a0853e5c1d633efa9b5e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`592ff92`](https://app.codecov.io/gh/apache/superset/commit/592ff926f5dead52ebb010efd07fc2e2e5f0023f?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #39278 +/- ##
==========================================
+ Coverage 64.33% 66.46% +2.12%
==========================================
Files 2553 1841 -712
Lines 132461 74737 -57724
Branches 30720 23784 -6936
==========================================
- Hits 85218 49671 -35547
+ Misses 45757 25066 -20691
+ Partials 1486 0 -1486
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/39278/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Ξ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/39278/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [mysql](https://app.codecov.io/gh/apache/superset/pull/39278/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [postgres](https://app.codecov.io/gh/apache/superset/pull/39278/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/39278/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [python](https://app.codecov.io/gh/apache/superset/pull/39278/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `100.00% <ΓΈ> (+37.73%)` | :arrow_up: |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/39278/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unit](https://app.codecov.io/gh/apache/superset/pull/39278/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `100.00% <ΓΈ> (ΓΈ)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/superset/pull/39278?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #39278:
URL: https://github.com/apache/superset/pull/39278#discussion_r3119309116
##########
superset/cli/test_db.py:
##########
@@ -121,11 +121,13 @@ def test_datetime(console: Console, engine: Engine) -> None:
console.print("Inserting timestamp value...")
insert_stmt = insert(table).values(ts=now)
- engine.execute(insert_stmt)
+ with engine.connect() as conn:
+ conn.execute(insert_stmt)
console.print("Reading timestamp value...")
select_stmt = select(table)
- row = engine.execute(select_stmt).fetchone()
+ with engine.connect() as conn:
+ row = conn.execute(select_stmt).fetchone()
Review Comment:
`engine.connect()` starts an implicit transaction; because this block never commits, the INSERT can be rolled back when the connection closes, causing the subsequent SELECT to not see the row (notably on PostgreSQL and SQLite). Use `engine.begin()` for the insert/select (or explicitly `conn.commit()` after the insert) to preserve the old `engine.execute()` autocommit behavior.
##########
superset/utils/mock_data.py:
##########
@@ -202,11 +202,12 @@ def add_data(
table = DBTable(table_name, metadata, *column_objects)
metadata.create_all(engine)
- if not append:
- engine.execute(table.delete())
+ with engine.connect() as conn:
+ if not append:
+ conn.execute(table.delete())
- data = generate_data(columns, num_rows)
- engine.execute(table.insert(), data)
+ data = generate_data(columns, num_rows)
+ conn.execute(table.insert(), data)
Review Comment:
`with engine.connect() as conn:` opens an implicit transaction. Since thereβs no commit, both the DELETE (when `append=False`) and the INSERT are liable to be rolled back when the connection closes, so the generated data may not actually persist. Use `engine.begin()` here (or call `conn.commit()` after the writes) to ensure data is committed.
##########
superset/db_engine_specs/hive.py:
##########
@@ -223,7 +223,8 @@ def df_to_sql(
catalog=table.catalog,
schema=table.schema,
) as engine:
- engine.execute(f"DROP TABLE IF EXISTS {str(table)}")
+ with engine.connect() as conn:
+ conn.execute(text(f"DROP TABLE IF EXISTS {str(table)}"))
Review Comment:
This `DROP TABLE` is executed inside an implicit transaction created by `engine.connect()`, but the transaction is never committed. On transactional backends this can be rolled back on connection close, changing behavior vs legacy `engine.execute()` and potentially leaving the table in place. Prefer `engine.begin()` for DDL/DML here (or explicitly commit).
##########
tests/integration_tests/conftest.py:
##########
@@ -213,12 +215,14 @@ def setup_presto_if_needed():
database = get_example_database()
with database.get_sqla_engine() as engine:
drop_from_schema(engine, CTAS_SCHEMA_NAME)
- engine.execute(f"DROP SCHEMA IF EXISTS {CTAS_SCHEMA_NAME}")
- engine.execute(f"CREATE SCHEMA {CTAS_SCHEMA_NAME}")
+ with engine.connect() as conn:
+ conn.execute(text(f"DROP SCHEMA IF EXISTS {CTAS_SCHEMA_NAME}"))
+ conn.execute(text(f"CREATE SCHEMA {CTAS_SCHEMA_NAME}"))
drop_from_schema(engine, ADMIN_SCHEMA_NAME)
- engine.execute(f"DROP SCHEMA IF EXISTS {ADMIN_SCHEMA_NAME}")
- engine.execute(f"CREATE SCHEMA {ADMIN_SCHEMA_NAME}")
+ with engine.connect() as conn:
+ conn.execute(text(f"DROP SCHEMA IF EXISTS {ADMIN_SCHEMA_NAME}"))
+ conn.execute(text(f"CREATE SCHEMA {ADMIN_SCHEMA_NAME}"))
Review Comment:
These schema DDL statements are executed inside `engine.connect()` without a commit. On transactional engines this can be rolled back on connection close, which may cause the Presto/Hive setup to intermittently fail to recreate schemas. Prefer `engine.begin()` for the DROP/CREATE SCHEMA blocks (or explicitly commit).
##########
superset/extensions/__init__.py:
##########
@@ -135,7 +136,11 @@ def init_app(self, app: Flask) -> None:
cache_manager = CacheManager()
celery_app = celery.Celery()
csrf = CSRFProtect()
-db = get_sqla_class()()
+# Use thread-based session scoping (like Flask-SQLAlchemy 2.x) instead of
+# app-context scoping (Flask-SQLAlchemy 3.x default). This is needed because
+# Superset code and tests pass SQLAlchemy objects across app context boundaries,
+# which breaks with app-context-scoped sessions.
+db = get_sqla_class()(session_options={"scopefunc": threading.get_ident})
Review Comment:
Using `threading.get_ident` as the SQLAlchemy session `scopefunc` is not greenlet-safe: when running with gevent workers (documented for Superset), multiple concurrent greenlets share the same thread ident and will therefore share the same scoped session, risking cross-request state bleed and data corruption. Consider using SQLAlchemyβs greenlet-aware ident (e.g. `sqlalchemy.util.concurrency.get_ident`) or another scope function that isolates greenlets as well as threads.
##########
superset/db_engine_specs/hive.py:
##########
@@ -249,22 +250,25 @@ def _get_hive_type(dtype: np.dtype[Any]) -> str:
catalog=table.catalog,
schema=table.schema,
) as engine:
- engine.execute(
- text(
- f"""
- CREATE TABLE {str(table)} ({schema_definition})
- STORED AS PARQUET
- LOCATION :location
- """
- ),
- location=upload_to_s3(
- filename=file.name,
- upload_prefix=app.config["CSV_TO_HIVE_UPLOAD_DIRECTORY_FUNC"](
- database, g.user, table.schema
+ with engine.connect() as conn:
+ conn.execute(
+ text(
+ f"""
+ CREATE TABLE {str(table)} ({schema_definition})
+ STORED AS PARQUET
+ LOCATION :location
+ """
),
- table=table,
- ),
- )
+ {
+ "location": upload_to_s3(
+ filename=file.name,
+ upload_prefix=app.config[
+ "CSV_TO_HIVE_UPLOAD_DIRECTORY_FUNC"
+ ](database, g.user, table.schema),
+ table=table,
+ ),
+ },
+ )
Review Comment:
The `CREATE TABLE ... LOCATION` statement is executed via `engine.connect()` without an explicit commit. With SQLAlchemy 1.4 this runs in an implicit transaction and may be rolled back on connection close, which would prevent the table from being created on transactional engines. Use `engine.begin()` for this block (or commit on the connection) to keep behavior consistent with the previous `engine.execute()` usage.
##########
tests/integration_tests/conftest.py:
##########
@@ -146,14 +147,15 @@ def setup_sample_data() -> Any:
def drop_from_schema(engine: Engine, schema_name: str):
- schemas = engine.execute(f"SHOW SCHEMAS").fetchall() # noqa: F541
- if schema_name not in [s[0] for s in schemas]:
- # schema doesn't exist
- return
- tables_or_views = engine.execute(f"SHOW TABLES in {schema_name}").fetchall()
- for tv in tables_or_views:
- engine.execute(f"DROP TABLE IF EXISTS {schema_name}.{tv[0]}")
- engine.execute(f"DROP VIEW IF EXISTS {schema_name}.{tv[0]}")
+ with engine.connect() as conn:
+ schemas = conn.execute(text("SHOW SCHEMAS")).fetchall()
+ if schema_name not in [s[0] for s in schemas]:
+ # schema doesn't exist
+ return
+ tables_or_views = conn.execute(text(f"SHOW TABLES in {schema_name}")).fetchall()
+ for tv in tables_or_views:
+ conn.execute(text(f"DROP TABLE IF EXISTS {schema_name}.{tv[0]}"))
+ conn.execute(text(f"DROP VIEW IF EXISTS {schema_name}.{tv[0]}"))
Review Comment:
`drop_from_schema` now performs DROP statements via `engine.connect()` but never commits. For transactional backends this can roll back the DROPs when the connection closes, potentially leaving schema objects behind and making the test setup flaky. Use `engine.begin()` for the DROP loop (or commit explicitly) to ensure the cleanup is persisted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #39278:
URL: https://github.com/apache/superset/pull/39278#issuecomment-5188383422
Update: tested the `model_class=` theory directly (apache/superset#42789, draft, closed now). `db.Model is FABModel` came back `True` locally, confirming the registries do get unified, but the same CI failures happened anyway, so that wasn't the real cause.
Pulled the actual failure logs this time instead of guessing from the summary. The real symptom is sharper than "NoneType has no attribute X" suggested: `db.session.bind` is `None` outright at the point of failure (`db.session.bind.dialect.name` blowing up in `soft_delete_tests.py`), and separately a `db.session.query(Dashboard).filter_by(slug="births").first()` right after a fixture set the dashboard up returns `None`. That's a session with no resolved engine, or one that can't see data a different context wrote. Circles back to FSA 3.0's actual changelog line, "the session is scoped to the current app context instead of the thread," more directly than the registry theory did.
Sorry for the runaround. Not going to keep guessing in public, will dig into where `db.session.bind` resolution actually breaks down before saying more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #39278:
URL: https://github.com/apache/superset/pull/39278#issuecomment-4246104842
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on code in PR #39278:
URL: https://github.com/apache/superset/pull/39278#discussion_r3080859086
##########
tests/integration_tests/conftest.py:
##########
@@ -146,14 +147,17 @@ def setup_sample_data() -> Any:
def drop_from_schema(engine: Engine, schema_name: str):
- schemas = engine.execute(f"SHOW SCHEMAS").fetchall() # noqa: F541
- if schema_name not in [s[0] for s in schemas]:
- # schema doesn't exist
- return
- tables_or_views = engine.execute(f"SHOW TABLES in {schema_name}").fetchall()
- for tv in tables_or_views:
- engine.execute(f"DROP TABLE IF EXISTS {schema_name}.{tv[0]}")
- engine.execute(f"DROP VIEW IF EXISTS {schema_name}.{tv[0]}")
+ with engine.connect() as conn:
+ schemas = conn.execute(text("SHOW SCHEMAS")).fetchall()
+ if schema_name not in [s[0] for s in schemas]:
+ # schema doesn't exist
+ return
+ tables_or_views = conn.execute(
+ text(f"SHOW TABLES in {schema_name}")
+ ).fetchall()
+ for tv in tables_or_views:
+ conn.execute(text(f"DROP TABLE IF EXISTS {schema_name}.{tv[0]}"))
+ conn.execute(text(f"DROP VIEW IF EXISTS {schema_name}.{tv[0]}"))
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>Transaction management issue</b></div>
<div id="fix">
Missing explicit transaction management. DDL statements should use `with engine.begin() as conn:` for proper auto-commit on success, auto-rollback on exception. Current `engine.connect()` pattern lacks explicit transaction control.
</div>
<details>
<summary>
<b>Code suggestion</b>
</summary>
<blockquote>Check the AI-generated fix before applying</blockquote>
<div id="code">
```
--- tests/integration_tests/conftest.py
+++ tests/integration_tests/conftest.py
@@ -147,7 +147,7 @@
def drop_from_schema(engine: Engine, schema_name: str):
- with engine.connect() as conn:
+ with engine.begin() as conn:
schemas = conn.execute(text("SHOW SCHEMAS")).fetchall()
if schema_name not in [s[0] for s in schemas]:
# schema doesn't exist
```
</div>
</details>
</div>
<details>
<summary><b>Citations</b></summary>
<ul>
<li>
Rule Violated: <a href="https://github.com/apache/superset/blob/1acec92/.cursor/rules/dev-standard.mdc#L28">dev-standard.mdc:28</a>
</li>
</ul>
</details>
<small><i>Code Review Run #c99afc</i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] chore: upgrade Flask to 3.x (with SQLAlchemy 1.4 compat) [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #39278:
URL: https://github.com/apache/superset/pull/39278#issuecomment-4224840950
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org