You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "GrumpyCat51 (via GitHub)" <gi...@apache.org> on 2025/04/28 09:56:15 UTC
[I] JWT token and validation not working [airflow]
GrumpyCat51 opened a new issue, #49871:
URL: https://github.com/apache/airflow/issues/49871
### Official Helm Chart version
1.16.0 (latest released)
### Apache Airflow version
3.0.0
### Kubernetes Version
Minikube v1.35.0
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### What happened
After downloading the newest chart for airflow 3.0.0, I've made no changes and deployed it through argocd on a local minikube on WSL. I then get the following warning message:
```
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
2025-04-28 09:27:25 [warning ] `api_auth/jwt_secret` was empty, using a generated one for now. Please set this in your config [airflow.api_fastapi.auth.tokens]
[2025-04-28T09:27:25.164+0000] {api_server_command.py:92} INFO - Running the uvicorn with:
```
followed by the error
```
[2025-04-28T09:27:45.295+0000] {_core.py:294} DEBUG - registered factory <function _jwt_generator at 0x7f3d57a1f920> for service type airflow.api_fastapi.auth.tokens.JWTGenerator
Stack (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/lib/python3.12/multiprocessing/spawn.py", line 122, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "/usr/local/lib/python3.12/multiprocessing/spawn.py", line 135, in _main
return self._bootstrap(parent_sentinel)
File "/usr/local/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/local/lib/python3.12/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/airflow/.local/lib/python3.12/site-packages/uvicorn/_subprocess.py", line 80, in subprocess_started
target(sockets=sockets)
File "/home/airflow/.local/lib/python3.12/site-packages/uvicorn/supervisors/multiprocess.py", line 63, in target
return self.real_target(sockets)
File "/home/airflow/.local/lib/python3.12/site-packages/uvicorn/server.py", line 66, in run
return asyncio.run(self.serve(sockets=sockets))
File "/usr/local/lib/python3.12/asyncio/runners.py", line 195, in run
return runner.run(main)
File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
File "/home/airflow/.local/lib/python3.12/site-packages/uvicorn/lifespan/on.py", line 86, in main
await app(scope, self.receive, self.send)
File "/home/airflow/.local/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
await super().__call__(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/applications.py", line 112, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/errors.py", line 152, in __call__
await self.app(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 19, in __call__
await self.app(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/cors.py", line 77, in __call__
await self.app(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/base.py", line 100, in __call__
await self.app(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 48, in __call__
await self.app(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/routing.py", line 714, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/routing.py", line 723, in app
await self.lifespan(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/routing.py", line 692, in lifespan
async with self.lifespan_context(app) as maybe_state:
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
File "/home/airflow/.local/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
async with original_context(app) as maybe_original_state:
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
File "/home/airflow/.local/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
async with original_context(app) as maybe_original_state:
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/app.py", line 62, in lifespan
await stack.enter_async_context(
File "/usr/local/lib/python3.12/contextlib.py", line 659, in enter_async_context
result = await _enter(cm)
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
File "/home/airflow/.local/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
async with original_context(app) as maybe_original_state:
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
File "/home/airflow/.local/lib/python3.12/site-packages/svcs/fastapi.py", line 66, in __call__
async with self.registry, cm(app, self.registry) as state:
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/execution_api/app.py", line 91, in lifespan
registry.register_factory(JWTGenerator, _jwt_generator)
File "/home/airflow/.local/lib/python3.12/site-packages/svcs/_core.py", line 294, in register_factory
log.debug(
ERROR: Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/routing.py", line 692, in lifespan
async with self.lifespan_context(app) as maybe_state:
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
async with original_context(app) as maybe_original_state:
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
async with original_context(app) as maybe_original_state:
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/app.py", line 62, in lifespan
await stack.enter_async_context(
File "/usr/local/lib/python3.12/contextlib.py", line 659, in enter_async_context
result = await _enter(cm)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
async with original_context(app) as maybe_original_state:
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/svcs/fastapi.py", line 66, in __call__
async with self.registry, cm(app, self.registry) as state:
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/execution_api/app.py", line 93, in lifespan
registry.register_value(JWTValidator, _jwt_validator(), ping=JWTValidator.status)
^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/execution_api/app.py", line 64, in _jwt_validator
**get_sig_validation_args(make_secret_key_if_needed=False),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/auth/tokens.py", line 578, in get_sig_validation_args
return {"secret_key": get_signing_key("api_auth", "jwt_secret", make_secret_key_if_needed)}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/auth/tokens.py", line 535, in get_signing_key
raise ValueError(f"The value {section}/{key} must be set!")
ValueError: The value api_auth/jwt_secret must be set!
ERROR: Application startup failed. Exiting.
INFO: Stopping parent process [7]
```
which prevents the API server from being started.
Then I've added one to the config section
```
config:
...
api_auth:
jwt_secret: '2fn4z0DVXsj3z8p1qU5nfA'
logging:
logging_level: 'DEBUG'
fab_logging_level: 'DEBUG'
```
now the api server starts up as expected and shows this line in the logs
```
[2025-04-28T09:44:52.475+0000] {_core.py:339} DEBUG - registered value JWTValidator(jwks=None, issuer=None, required_claims=frozenset({'iat', 'exp', 'aud'}), audience=['urn:airflow.apache.org:task'], algorithm=['HS512'], leeway=10) for service type airflow.api_fastapi.auth.tokens.JWTValidator
```
Then I try to login with the default admin/admin credentials, but I'm now stuck in an endless loop of 401 Unauthorized
```
INFO: 10.244.2.90:34112 - "GET / HTTP/1.1" 200 OK
INFO: 10.244.2.90:34112 - "GET /static/assets/index-DVWpfKkn.js HTTP/1.1" 200 OK
INFO: 10.244.2.90:34112 - "GET /ui/config HTTP/1.1" 401 Unauthorized
INFO: 10.244.2.90:34112 - "GET /api/v2/auth/login?next=http%3A%2F%2Fairflow.local%2F HTTP/1.1" 307 Temporary Redirect
INFO: 10.244.2.90:34112 - "GET /auth/login/?next=http://airflow.local/ HTTP/1.1" 200 OK
INFO: 10.244.2.90:34112 - "GET /auth/static/appbuilder/css/bootstrap.min.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:42252 - "GET /auth/static/appbuilder/css/fontawesome/fontawesome.min.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:42264 - "GET /auth/static/appbuilder/css/fontawesome/solid.min.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:42268 - "GET /auth/static/appbuilder/css/fontawesome/brands.min.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:34112 - "GET /auth/static/appbuilder/css/bootstrap-datepicker/bootstrap-datepicker3.min.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:42252 - "GET /auth/static/appbuilder/css/select2/select2.min.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:42260 - "GET /auth/static/appbuilder/css/fontawesome/regular.min.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:42264 - "GET /auth/static/appbuilder/css/select2/select2-bootstrap.min.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:42268 - "GET /auth/static/appbuilder/css/flags/flags16.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:34112 - "GET /auth/static/appbuilder/css/ab.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:42252 - "GET /auth/static/dist/airflowDefaultTheme.feec4a4075c2f3d6ae01.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:42284 - "GET /auth/static/appbuilder/css/fontawesome/v4-shims.min.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:42260 - "GET /auth/static/dist/materialIcons.57390fa60d8f61175334.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:42264 - "GET /auth/static/dist/main.edb2d40dfbbc537916e3.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:42268 - "GET /auth/static/dist/loadingDots.48ab7d5b04e66f2686b0.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:34112 - "GET /auth/static/dist/jquery-ui.min.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:42284 - "GET /auth/static/appbuilder/js/jquery-latest.js HTTP/1.1" 200 OK
INFO: 10.244.2.90:42268 - "GET /auth/static/appbuilder/js/ab_actions.js HTTP/1.1" 200 OK
INFO: 10.244.2.90:42252 - "GET /auth/static/dist/flash.137b30cff85b5588e661.css HTTP/1.1" 200 OK
INFO: 10.244.2.90:34112 - "GET /auth/static/appbuilder/js/bootstrap.min.js HTTP/1.1" 200 OK
INFO: 10.244.2.90:42260 - "GET /auth/static/appbuilder/js/bootstrap-datepicker/bootstrap-datepicker.min.js HTTP/1.1" 200 OK
INFO: 10.244.2.90:42264 - "GET /auth/static/appbuilder/js/ab_filters.js HTTP/1.1" 200 OK
INFO: 10.244.2.90:42268 - "GET /auth/static/appbuilder/js/ab.js HTTP/1.1" 200 OK
INFO: 10.244.2.90:42284 - "GET /auth/static/appbuilder/js/select2/select2.min.js HTTP/1.1" 200 OK
INFO: 10.244.2.90:42252 - "GET /auth/static/dist/moment.624b1f00ba723d39ce06.js HTTP/1.1" 200 OK
INFO: 10.244.2.90:34112 - "GET /auth/static/dist/main.edb2d40dfbbc537916e3.js HTTP/1.1" 200 OK
INFO: 10.244.2.90:42260 - "GET /auth/static/dist/jquery-ui.min.js HTTP/1.1" 200 OK
INFO: 10.244.2.90:42260 - "GET /auth/static/appbuilder/css/webfonts/fa-solid-900.woff2 HTTP/1.1" 200 OK
INFO: 10.244.2.90:42260 - "GET /auth/static/pin_32.png HTTP/1.1" 404 Not Found
INFO: 10.244.0.1:45996 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 10.244.0.1:46010 - "GET /api/v2/version HTTP/1.1" 200 OK
[2025-04-28T09:48:19.326+0000] {override.py:1423} INFO - Updated user admin user
INFO: 10.244.2.90:34112 - "POST /auth/login/?next=http://airflow.local/ HTTP/1.1" 302 Found
INFO: 10.244.2.90:34112 - "GET / HTTP/1.1" 200 OK
INFO: 10.244.2.90:42260 - "GET /ui/config HTTP/1.1" 401 Unauthorized
INFO: 10.244.2.90:34112 - "GET /api/v2/auth/login?next=http%3A%2F%2Fairflow.local%2F HTTP/1.1" 307 Temporary Redirect
INFO: 10.244.2.90:34112 - "GET /auth/login/?next=http://airflow.local/ HTTP/1.1" 302 Found
INFO: 10.244.2.90:42260 - "GET /auth/ HTTP/1.1" 302 Found
INFO: 10.244.2.90:42252 - "GET / HTTP/1.1" 200 OK
INFO: 10.244.2.90:42252 - "GET /ui/config HTTP/1.1" 401 Unauthorized
INFO: 10.244.2.90:42252 - "GET /api/v2/auth/login?next=http%3A%2F%2Fairflow.local%2F HTTP/1.1" 307 Temporary Redirect
INFO: 10.244.2.90:42252 - "GET /auth/login/?next=http://airflow.local/ HTTP/1.1" 302 Found
INFO: 10.244.2.90:42252 - "GET /auth/ HTTP/1.1" 302 Found
INFO: 10.244.2.90:42268 - "GET / HTTP/1.1" 200 OK
```
I've then tried to use the new `airflow.api_fastapi.auth.managers.simple.simple_auth_manager.SimpleAuthManager`, set through the same configs as above, with `simple_auth_manager_all_admins` set to `True` as this should avoid any verification, but I'm still stuck in the same Unauthorized loop
### What you think should happen instead
I'm not quite sure what went wrong (I suspect the JWT part), but I'd expect the default settings to work.
### How to reproduce
Start up the helm chart in a minikube cluster and add the ingress from https://github.com/apache/airflow/pull/49727
### Anything else
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] JWT token and validation not working [airflow]
Posted by "GrumpyCat51 (via GitHub)" <gi...@apache.org>.
GrumpyCat51 commented on issue #49871:
URL: https://github.com/apache/airflow/issues/49871#issuecomment-2835154150
Thanks @tirkarthi, updating to apache-airflow-providers-fab==2.0.2rc2 has resolved the issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] JWT token and validation not working [airflow]
Posted by "paolofacchinetti (via GitHub)" <gi...@apache.org>.
paolofacchinetti commented on issue #49871:
URL: https://github.com/apache/airflow/issues/49871#issuecomment-2834975317
hey @GrumpyCat51 FYI it's a known issue that you can mitigate by setting `AIRFLOW__API_AUTH__JWT_SECRET` manually, as shown here: #49646
fix is yet to be implemented
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] JWT token and validation not working [airflow]
Posted by "tirkarthi (via GitHub)" <gi...@apache.org>.
tirkarthi commented on issue #49871:
URL: https://github.com/apache/airflow/issues/49871#issuecomment-2835127158
This seems to be relevant to #49495 and below comment might be helpful.
https://github.com/apache/airflow/issues/49495#issuecomment-2829973372
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] JWT token and validation not working [airflow]
Posted by "bdsoha (via GitHub)" <gi...@apache.org>.
bdsoha commented on issue #49871:
URL: https://github.com/apache/airflow/issues/49871#issuecomment-2838705439
The JWT secret token is bundled in #49923
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] JWT token and validation not working [airflow]
Posted by "GrumpyCat51 (via GitHub)" <gi...@apache.org>.
GrumpyCat51 commented on issue #49871:
URL: https://github.com/apache/airflow/issues/49871#issuecomment-2835012691
> hey [@GrumpyCat51](https://github.com/GrumpyCat51) FYI it's a known issue that you can mitigate by setting `AIRFLOW__API_AUTH__JWT_SECRET` manually, as shown here: [#49646](https://github.com/apache/airflow/issues/49646)
>
> fix is yet to be implemented
I've added this through
```
api_auth:
jwt_secret: '2fn4z0DVXsj3z8p1qU5nfA'
```
but still always run into the 401 Unauthorized further down. I'm trying to investigate where the error comes from through adding more logging (and disabling certain steps)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] JWT token and validation not working [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #49871:
URL: https://github.com/apache/airflow/issues/49871#issuecomment-2834668669
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] JWT token and validation not working [airflow]
Posted by "GrumpyCat51 (via GitHub)" <gi...@apache.org>.
GrumpyCat51 closed issue #49871: JWT token and validation not working
URL: https://github.com/apache/airflow/issues/49871
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] JWT token and validation not working [airflow]
Posted by "OleksandrBrukhalskyi (via GitHub)" <gi...@apache.org>.
OleksandrBrukhalskyi commented on issue #49871:
URL: https://github.com/apache/airflow/issues/49871#issuecomment-2877034291
Hi, I also was facing this issue in `3.0.1` version. This [issue](https://github.com/apache/airflow/issues/49646) helped me in my case!
To fix this I followed these steps:
- I generated a token using this command in my server: `openssl rand -base64 16`
- I have added a new environment variable into docker compose file: `AIRFLOW__API_AUTH__JWT__SECRET: <your_genearted_token>`
I hope it will help to resolve this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org