You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "kurtrwall (via GitHub)" <gi...@apache.org> on 2025/01/02 18:43:08 UTC
[I] Airflow Celery CLI broken [airflow]
kurtrwall opened a new issue, #45364:
URL: https://github.com/apache/airflow/issues/45364
### Apache Airflow version
2.10.4
### If "Other Airflow 2 version" selected, which one?
2.10.3, 2.10.2
### What happened?
I'm not entirely sure of the situation because it's a little weird. When I try to install `apache-airflow`, it will always gives the version `2.10.3` for versions `2.10.2-4`. Additionally, it seems like there was a bug that's fixed on the `main` branch that breaks the `airflow celery` CLI like so:
```
worker-1 |
worker-1 | Traceback (most recent call last):
worker-1 | File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/celery/cli/celery_command.py", line 45, in _run_command_with_daemon_option
worker-1 | from airflow.cli.commands.local_commands.daemon_utils import run_command_with_daemon_option
worker-1 | ModuleNotFoundError: No module named 'airflow.cli.commands.local_commands'
worker-1 |
worker-1 | During handling of the above exception, another exception occurred:
worker-1 |
worker-1 | Traceback (most recent call last):
worker-1 | File "/home/airflow/.local/bin/airflow", line 8, in <module>
worker-1 | sys.exit(main())
worker-1 | ^^^^^^
worker-1 | File "/home/airflow/.local/lib/python3.12/site-packages/airflow/__main__.py", line 62, in main
worker-1 | args.func(args)
worker-1 | File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/cli_config.py", line 49, in command
worker-1 | return func(*args, **kwargs)
worker-1 | ^^^^^^^^^^^^^^^^^^^^^
worker-1 | File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/cli.py", line 116, in wrapper
worker-1 | return f(*args, **kwargs)
worker-1 | ^^^^^^^^^^^^^^^^^^
worker-1 | File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/celery/cli/celery_command.py", line 61, in wrapper
worker-1 | providers_configuration_loaded(func)(*args, **kwargs)
worker-1 | File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/providers_configuration_loader.py", line 55, in wrapped_function
worker-1 | return func(*args, **kwargs)
worker-1 | ^^^^^^^^^^^^^^^^^^^^^
worker-1 | File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/celery/cli/celery_command.py", line 234, in worker
worker-1 | _run_command_with_daemon_option(
worker-1 | File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/celery/cli/celery_command.py", line 51, in _run_command_with_daemon_option
worker-1 | raise AirflowOptionalProviderFeatureException(
worker-1 | airflow.exceptions.AirflowOptionalProviderFeatureException: Failed to import run_command_with_daemon_option. This feature is only available in Airflow versions >= 2.8.0
```
The offending code seems to be repaired with [this PR](https://github.com/apache/airflow/pull/45255), but, and here's another weird part, if you browse the codebase at the aforementioned versions (tags), the offending code doesn't exist. To me, it seems like somehow "bad" versions of the code were packaged and uploaded to PyPI.
### What you think should happen instead?
- installing the `apache-airflow` at versions `2.10.2` or `2.10.4` should install that version, not `2.10.3`
- the code at the version should be the same as what's on GitHub at that version's tag
- running `airflow celery worker` CLI command should work
### How to reproduce
1. `pip install apache-airflow==2.10.4`
2. `airflow celery worker`
### Operating System
OSX
### Versions of Apache Airflow Providers
apache-airflow-providers-celery==3.8.5
### Deployment
Docker-Compose
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow Celery CLI broken [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #45364:
URL: https://github.com/apache/airflow/issues/45364#issuecomment-2568209809
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow Celery CLI broken [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #45364:
URL: https://github.com/apache/airflow/issues/45364#issuecomment-2568784717
Yes. This PR has been merged 4 days ago but fixed celery provider has not yet been relased - see the changelog:
https://airflow.apache.org/docs/apache-airflow-providers-celery/stable/changelog.html
Look at the devlist announcements about the next release when we announce it - there will be - as usual "status of testing" issue created in here and you will be asked to test if the new RC candidate solves the issue.
Looking forward for your testing and confirming it @kurtrwall @ncrocfer. Marking it as closed and linking to the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow Celery CLI broken [airflow]
Posted by "ncrocfer (via GitHub)" <gi...@apache.org>.
ncrocfer commented on issue #45364:
URL: https://github.com/apache/airflow/issues/45364#issuecomment-2568535614
We are also experiencing this issue with the CeleryExecutor, which prevents any tasks from being executed. To resolve the problem, we rolled back to `apache-airflow-providers-celery==3.8.5`: https://github.com/opencve/opencve/pull/493
```
Python: 3.11.11
Airflow: 2.10.4
Celery Provider: 3.9.0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow Celery CLI broken [airflow]
Posted by "kurtrwall (via GitHub)" <gi...@apache.org>.
kurtrwall commented on issue #45364:
URL: https://github.com/apache/airflow/issues/45364#issuecomment-2568247121
Sorry if the description is confusing and ill-informed. I'm still not exactly sure what's happening or how the providers are built, but at least the celery CLI is fixed with `apache-airflow-providers-celery<3.9`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow Celery CLI broken [airflow]
Posted by "kurtrwall (via GitHub)" <gi...@apache.org>.
kurtrwall commented on issue #45364:
URL: https://github.com/apache/airflow/issues/45364#issuecomment-2715213185
I'll confirm that this issue seems to be resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow Celery CLI broken [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #45364: Airflow Celery CLI broken
URL: https://github.com/apache/airflow/issues/45364
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org