You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/11/13 00:43:46 UTC
[GitHub] [airflow] jorenby opened a new issue #19568: Error running worker after upgrade to 2.2
jorenby opened a new issue #19568:
URL: https://github.com/apache/airflow/issues/19568
### Apache Airflow version
2.2.1 (latest released)
### Operating System
Linux
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon @ file:///Users/joshuajorenby/Library/Caches/pypoetry/artifacts/46/cb/dc/5d78f8e7fa74661e7f8ae99d492b60ed2641b4e8872d51ca3f416ef043/apache_airflow_providers_amazon-1.4.0-py3-none-any.whl
apache-airflow-providers-ftp @ file:///Users/joshuajorenby/Library/Caches/pypoetry/artifacts/a5/13/da/bf14abc40193a1ee1b82bbd800e3ac230427d7684b9d40998ac3684bef/apache_airflow_providers_ftp-2.0.1-py3-none-any.whl
apache-airflow-providers-http @ file:///Users/joshuajorenby/Library/Caches/pypoetry/artifacts/22/4d/d8/afbd1e5175f52b087f61e8adba4fb9f3dd775eba59ab1a870dbfd06bbb/apache_airflow_providers_http-2.0.1-py3-none-any.whl
apache-airflow-providers-imap @ file:///Users/joshuajorenby/Library/Caches/pypoetry/artifacts/af/5d/de/21c10bfc7ac076a415dcc3fc909317547e77e38c005487552cf40ddd97/apache_airflow_providers_imap-2.0.1-py3-none-any.whl
apache-airflow-providers-postgres @ file:///Users/joshuajorenby/Library/Caches/pypoetry/artifacts/de/5e/a2/a263b7839be406099d7b464770ab18e4502c638fd37784f69e12b13c5f/apache_airflow_providers_postgres-1.0.2-py3-none-any.whl
apache-airflow-providers-slack @ file:///Users/joshuajorenby/Library/Caches/pypoetry/artifacts/38/a6/58/f0d07dbddf3241cfcb10dcc1b16391e6ba4120e1019d2d1d07b73ed84c/apache_airflow_providers_slack-4.1.0-py3-none-any.whl
apache-airflow-providers-sqlite @ file:///Users/joshuajorenby/Library/Caches/pypoetry/artifacts/ec/e6/a3/e0d81fef662ccf79609e7d2c4e4440839a464771fd2a002d252c9a401d/apache_airflow_providers_sqlite-2.0.1-py3-none-any.whl
### Deployment
Other Docker-based deployment
### Deployment details
_No response_
### What happened
We've been running 2.1.2 using CeleryExecutor without issue. After upgrading to 2.2.1, workers started failing with:
```
Traceback (most recent call last):
--
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/__main__.py", line 48, in main
args.func(args)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py", line 92, in wrapper
return f(*args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/celery_command.py", line 188, in worker
_run_worker(options=options, skip_serve_logs=skip_serve_logs)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/celery_command.py", line 94, in _run_worker
celery_app.worker_main(options)
File "/home/airflow/.local/lib/python3.8/site-packages/celery/app/base.py", line 365, in worker_main
return instantiate(
File "/home/airflow/.local/lib/python3.8/site-packages/celery/bin/base.py", line 283, in execute_from_commandline
self.maybe_patch_concurrency(argv)
File "/home/airflow/.local/lib/python3.8/site-packages/celery/bin/base.py", line 315, in maybe_patch_concurrency
maybe_patch_concurrency(argv, *pool_option)
File "/home/airflow/.local/lib/python3.8/site-packages/celery/__init__.py", line 143, in maybe_patch_concurrency
pool = _find_option_with_arg(argv, short_opts, long_opts)
File "/home/airflow/.local/lib/python3.8/site-packages/celery/__init__.py", line 95, in _find_option_with_arg
if arg.startswith('-'):
AttributeError: 'int' object has no attribute 'startswith'
```
### What you expected to happen
Workers to be able to function as they did before.
### How to reproduce
Changed dependency to apache-airflow 2.2.1 and set up the entry point to run `airflow db upgrade`. No other functional changes beside swapping out `max_active_tasks_per_dag` for `dag_concurrency` in the config.
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #19568: Error running worker after upgrade to 2.2
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #19568:
URL: https://github.com/apache/airflow/issues/19568#issuecomment-967746331
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #19568: Error running worker after upgrade to 2.2
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #19568:
URL: https://github.com/apache/airflow/issues/19568
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #19568: Error running worker after upgrade to 2.2
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #19568:
URL: https://github.com/apache/airflow/issues/19568#issuecomment-968367158
This is a celery incompatibility issue. Airflow 2.1.4 used celery 4 and 2.2.1 uses celery 5 and you seem to have still celery 4.
You know that you should install airflow with the right constraints https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org