You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "uplsh580 (via GitHub)" <gi...@apache.org> on 2026/08/11 16:17:07 UTC
[PR] Recover the revoked-token check from a stale DB connection (fixes idle-timeout 500) [airflow]
uplsh580 opened a new pull request, #71444:
URL: https://github.com/apache/airflow/pull/71444
Recover the JWT revoked-token check from a stale pooled DB connection so the first authenticated request after an idle period no longer returns HTTP 500.
## Problem
With MySQL as the metadata DB and the FAB auth manager, the first authenticated request after the api-server sits idle longer than MySQL's `wait_timeout` fails with HTTP 500. The exception is an `OperationalError` (MySQL error 4031, *"The client was disconnected by the server because of inactivity"*) raised from `RevokedToken.is_revoked` inside `BaseAuthManager.get_user_from_token`. The immediately following request succeeds, because the failed query resets the pooled connection — so in production it looks like intermittent 500s after overnight idle periods.
### Root cause
1. The FAB auth manager shares core's scoped `settings.Session`. A prior request (e.g. `FabAuthManager.deserialize_user`, or FAB's Flask views) can leave that scoped session bound to a connection the database later drops on idle timeout. Because the connection is never returned to the pool, there is **no checkout event** — `pool_pre_ping` / `pool_recycle` cannot help.
2. This exact failure mode was reported in #62903 and fixed by #62919, which added a discard-and-retry recovery to `FabAuthManager.deserialize_user`.
3. But 3.2.0 introduced the JWT revocation check (#61339 / AIP-84): `RevokedToken.is_revoked` in `get_user_from_token` now runs **before** `deserialize_user`, so it is the first thing to touch the poisoned scoped session — and it had no recovery logic. The 500 that #62919 fixed is back, just raised one step earlier in the auth path.
## Fix
Give the revoked-token check the same recovery `deserialize_user` already has: on `SQLAlchemyError`, discard the scoped session (`settings.Session.remove()`) and retry the check once on a fresh connection. The recovery is factored into a small `BaseAuthManager._is_token_revoked` helper so `get_user_from_token` stays readable.
This is the per-call-site variant proposed in the issue (the one that has been running in production for several weeks and eliminated these 500s). It is minimal and mirrors the established #62919 pattern rather than introducing a new session-management abstraction. A persistent DB error still surfaces — only a single transient stale-connection error is absorbed.
## Tests
Added two regression tests in `test_base_auth_manager.py`. Both raise a 4031-style `OperationalError` on the first `is_revoked` call and assert the scoped session is discarded (`settings.Session.remove()`) and the check is retried exactly once:
- retry returns *not revoked* → the request is served transparently;
- retry returns *revoked* → `InvalidTokenError("Token has been revoked")` is still raised.
related: #71395
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Recover the revoked-token check from a stale DB connection (fixes idle-timeout 500) [airflow]
Posted by "uplsh580 (via GitHub)" <gi...@apache.org>.
uplsh580 commented on code in PR #71444:
URL: https://github.com/apache/airflow/pull/71444#discussion_r3766808742
##########
airflow-core/src/airflow/api_fastapi/auth/managers/base_auth_manager.py:
##########
@@ -161,15 +164,45 @@ async def get_user_from_token(self, token: str) -> BaseUser:
log.error("JWT token is not valid: %s", e)
raise e
- if (jti := payload.get("jti")) and RevokedToken.is_revoked(jti):
- raise InvalidTokenError("Token has been revoked")
+ if jti := payload.get("jti"):
+ if self._is_token_revoked(jti):
Review Comment:
Thanks, applied
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Recover the revoked-token check from a stale DB connection (fixes idle-timeout 500) [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #71444:
URL: https://github.com/apache/airflow/pull/71444#issuecomment-5881291514
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Recover the revoked-token check from a stale DB connection (fixes idle-timeout 500) [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #71444: Recover the revoked-token check from a stale DB connection (fixes idle-timeout 500)
URL: https://github.com/apache/airflow/pull/71444
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Recover the revoked-token check from a stale DB connection (fixes idle-timeout 500) [airflow]
Posted by "uplsh580 (via GitHub)" <gi...@apache.org>.
uplsh580 commented on code in PR #71444:
URL: https://github.com/apache/airflow/pull/71444#discussion_r3759960287
##########
airflow-core/newsfragments/71444.bugfix.rst:
##########
@@ -0,0 +1 @@
+Recover the JWT revoked-token check from a stale pooled database connection. On MySQL the first authenticated request after an idle period could fail with HTTP 500 (``OperationalError`` 4031) because ``RevokedToken.is_revoked`` ran on a scoped session left bound to a dropped connection; it now discards that session and retries once on a fresh connection, matching the recovery ``FabAuthManager.deserialize_user`` already had.
Review Comment:
Thanks, removed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Recover the revoked-token check from a stale DB connection (fixes idle-timeout 500) [airflow]
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on code in PR #71444:
URL: https://github.com/apache/airflow/pull/71444#discussion_r3759765414
##########
airflow-core/newsfragments/71444.bugfix.rst:
##########
@@ -0,0 +1 @@
+Recover the JWT revoked-token check from a stale pooled database connection. On MySQL the first authenticated request after an idle period could fail with HTTP 500 (``OperationalError`` 4031) because ``RevokedToken.is_revoked`` ran on a scoped session left bound to a dropped connection; it now discards that session and retries once on a fresh connection, matching the recovery ``FabAuthManager.deserialize_user`` already had.
Review Comment:
No need to have this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Recover the revoked-token check from a stale DB connection (fixes idle-timeout 500) [airflow]
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on code in PR #71444:
URL: https://github.com/apache/airflow/pull/71444#discussion_r3766735009
##########
airflow-core/src/airflow/api_fastapi/auth/managers/base_auth_manager.py:
##########
@@ -161,15 +164,45 @@ async def get_user_from_token(self, token: str) -> BaseUser:
log.error("JWT token is not valid: %s", e)
raise e
- if (jti := payload.get("jti")) and RevokedToken.is_revoked(jti):
- raise InvalidTokenError("Token has been revoked")
+ if jti := payload.get("jti"):
+ if self._is_token_revoked(jti):
Review Comment:
```suggestion
if jti := payload.get("jti") and self._is_token_revoked(jti):
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org