You are viewing a plain text version of this content. The canonical link for it is here.

Posted to notifications@superset.apache.org by "sfirke (via GitHub)" <gi...@apache.org> on 2026/10/19 15:46:13 UTC

[I] SQL Lab schema/table lists fail with "timeout must be an int or datetime.timedelta, got 'str'" on databases with legacy string cache timeouts (7.0.0rc2) [superset]

sfirke opened a new issue, #45460:
URL: https://github.com/apache/superset/issues/45460

### Bug description

After upgrading to 7.0.0rc2, the schema list in SQL Lab fails to load for some of our existing database connections. Picking the database in the "Select Database and Schema" panel shows a "DB engine Error" with:

```
timeout must be an int or datetime.timedelta, got 'str'
```

The affected connections are the ones that have a **Schema cache timeout** / **Table cache timeout** set under Advanced → Performance. Their `extra` JSON holds those values as **strings**:

```json
"metadata_cache_timeout": {"schema_cache_timeout": "3600", "table_cache_timeout": "600"}
```

Before 6.0 the database modal saved these fields as strings. That was reported in #28735 and fixed for new saves in #28737, but rows saved before the fix were never converted. They kept working on Redis because cachelib 0.13 passed the value through and Redis accepted `"3600"`.

7.0 bumps cachelib from 0.13.0 to 0.17.0 (#43875). cachelib 0.17's `BaseCache._to_seconds` now raises `TypeError` for any non-int/timedelta timeout, on every cache backend. `memoized_func` passes `database.schema_cache_timeout` straight through to `cache.set()`, so the whole `/api/v1/database/<pk>/schemas/` request fails after the schemas were fetched successfully. `table_cache_timeout` goes through the same path, so table lists should fail the same way.

The new `extra` validation in `superset/databases/schemas.py` rejects string timeouts on save. It does not touch values already stored, and no migration converts them. Any install upgrading with legacy values will hit this.

**How to reproduce**

1. On a 7.0.0rc2 instance, give a database a legacy-style string timeout. This is what a connection saved from the pre-6.0 UI looks like. For example, in the metadata DB:
```sql
UPDATE dbs
SET extra = '{"metadata_cache_timeout": {"schema_cache_timeout": "3600", "table_cache_timeout": "600"}}'
WHERE id = <id>;
```
2. Open SQL Lab and select that database in the left-hand panel.
3. See the "DB engine Error: timeout must be an int or datetime.timedelta, got 'str'" message, with no schema list.

**Expected behavior**

The schema list loads. Superset should either convert stored timeouts to integers when it reads them (e.g. in the `Database.schema_cache_timeout` / `table_cache_timeout` / `catalog_cache_timeout` properties in `superset/models/core.py`, or in `memoized_func`), falling back to the default timeout when a value can't be parsed, or ship a data migration that rewrites string values in `dbs.extra` to integers.

**Workaround**

I edited the databases, cleared and re-entered the Schema/Table cache timeout values under Advanced → Performance, and saved. The current UI stores them as numbers. You have to re-enter them: saving after changing only another field sends the old string values back, and the new validation rejects them.

### Screenshots/recordings

### Superset version

master / latest-dev

### Python version

3.11

### Node version

### Browser

Chrome

### Additional context

This is 7.0.0rc2

### Checklist

- [x] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [x] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] SQL Lab schema/table lists fail with "timeout must be an int or datetime.timedelta, got 'str'" on databases with legacy string cache timeouts (7.0.0rc2) [superset]

Posted by "sushu0 (via GitHub)" <gi...@apache.org>.

sushu0 commented on issue #45460:
URL: https://github.com/apache/superset/issues/45460#issuecomment-6271948203

   Understood. I'll hold off on further implementation and on opening a PR for this issue until that discussion is settled. I have a local regression that reproduces the reported cachelib TypeError and a read-time compatibility candidate, but neither has been pushed or submitted. Thanks for clarifying the preferred sequence.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] SQL Lab schema/table lists fail with "timeout must be an int or datetime.timedelta, got 'str'" on databases with legacy string cache timeouts (7.0.0rc2) [superset]

Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.

sadpandajoe commented on issue #45460:
URL: https://github.com/apache/superset/issues/45460#issuecomment-6272808681

   @sfirke i think it depends, is it going to be the next migration we just haven't pulled in yet. If there are other migrations before this I would say don't do it with the migrations as that will mess up the alembic chain when cherry-picking.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] SQL Lab schema/table lists fail with "timeout must be an int or datetime.timedelta, got 'str'" on databases with legacy string cache timeouts (7.0.0rc2) [superset]

Posted by "sushu0 (via GitHub)" <gi...@apache.org>.

sushu0 commented on issue #45460:
URL: https://github.com/apache/superset/issues/45460#issuecomment-6271843110

I'd like to take this on with Codex assistance and add regression coverage against the actual database model and cache backend.

My preference is a defensive conversion at the Database metadata-timeout read boundary: normalize legacy integer strings for catalog/schema/table timeouts, preserve integer zero, and use the existing default-timeout path for values that cannot be interpreted as a valid timeout. The current strict validation for new saves would remain in place. This would handle existing connections without rewriting dbs.extra or requiring a data migration, and avoids changing timeout handling for unrelated callers of memoized_func.

I will prepare and test that focused change locally while the migration-versus-conversion question is discussed here. The regression should exercise a real cache write, since merely asserting the property value would miss the cachelib failure. If a migration is preferred, I can adapt the implementation before submitting it for review.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] SQL Lab schema/table lists fail with "timeout must be an int or datetime.timedelta, got 'str'" on databases with legacy string cache timeouts (7.0.0rc2) [superset]

Posted by "sfirke (via GitHub)" <gi...@apache.org>.

sfirke commented on issue #45460:
URL: https://github.com/apache/superset/issues/45460#issuecomment-6271916747

   Let's hold off @sushu0, first I'd like to see discussion here re: whether the way to go is a DB migration or not. Once the community settles on that, I'm fine to have you implement.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] SQL Lab schema/table lists fail with "timeout must be an int or datetime.timedelta, got 'str'" on databases with legacy string cache timeouts (7.0.0rc2) [superset]

Posted by "sfirke (via GitHub)" <gi...@apache.org>.

sfirke commented on issue #45460:
URL: https://github.com/apache/superset/issues/45460#issuecomment-6271267299

   That is a mix of me and Claude Code, I manually validated that deleting and re-typing the values caused the schemas to list successfully. The fix should be easy but before anyone sends a PR, do we prefer a migration or a defensive conversion check?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] SQL Lab schema/table lists fail with "timeout must be an int or datetime.timedelta, got 'str'" on databases with legacy string cache timeouts (7.0.0rc2) [superset]

Posted by "sfirke (via GitHub)" <gi...@apache.org>.

sfirke commented on issue #45460:
URL: https://github.com/apache/superset/issues/45460#issuecomment-6423140925

   Okay @sushu0 let's go with the migration approach. Are you comfortable with that + do you still want to work on this? If so go ahead.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] SQL Lab schema/table lists fail with "timeout must be an int or datetime.timedelta, got 'str'" on databases with legacy string cache timeouts (7.0.0rc2) [superset]

Posted by "sushu0 (via GitHub)" <gi...@apache.org>.

sushu0 commented on issue #45460:
URL: https://github.com/apache/superset/issues/45460#issuecomment-6445471092

Implemented the agreed migration approach in https://github.com/apache/superset/pull/45790.

It follows master's current Alembic head (`d623a0cb6bb0`) and converts only the three known metadata timeout keys' non-negative integer strings, including zero. Invalid/unrelated data are preserved, including high-precision JSON numbers. Downgrade leaves the compatible integer values in place rather than guessing their original types.

Local validation run with Codex assistance: 45 related project tests passed, including actual SQLite rows, Alembic operations, the catalog/schema/table cachelib failure before upgrade and successful cache writes afterward, mixed-row preservation, idempotence and transaction rollback. `superset db heads` reports one new head, `9670ab7f6bc6`. Details and platform limitations are in the PR; I have not chosen a 7.0 RC cherry-pick order.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] SQL Lab schema/table lists fail with "timeout must be an int or datetime.timedelta, got 'str'" on databases with legacy string cache timeouts (7.0.0rc2) [superset]

Posted by "villebro (via GitHub)" <gi...@apache.org>.

villebro closed issue #45460: SQL Lab schema/table lists fail with "timeout must be an int or datetime.timedelta, got 'str'" on databases with legacy string cache timeouts (7.0.0rc2)
URL: https://github.com/apache/superset/issues/45460


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] SQL Lab schema/table lists fail with "timeout must be an int or datetime.timedelta, got 'str'" on databases with legacy string cache timeouts (7.0.0rc2) [superset]

Posted by "sfirke (via GitHub)" <gi...@apache.org>.

sfirke commented on issue #45460:
URL: https://github.com/apache/superset/issues/45460#issuecomment-6274710573

   @sadpandajoe good point, I guess if this could make it into the next RC for 7.0.0 then @villebro would have control of order he pulls the cherries and could align the migration order. Do you like migration as the solution if we can get that right?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org