You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "or-benjamin (via GitHub)" <gi...@apache.org> on 2026/07/10 13:20:12 UTC
[I] RabbitMQ transient_nonexcl_queues deprecation triggered by Celery pidbox — bump kombu to >= 5.7.0 [airflow]
or-benjamin opened a new issue, #69704:
URL: https://github.com/apache/airflow/issues/69704
### Under which category would you file this issue?
Airflow Core
### Apache Airflow version
3.3.0
### What happened and how to reproduce it?
With `CeleryExecutor` and a RabbitMQ 4.x broker, RabbitMQ reports usage of the
**deprecated feature `transient_nonexcl_queues`**:
> Deprecated features are being used ... transient_nonexcl_queues
The offending queue is Celery's **pidbox** (remote-control mailbox), which kombu
declares as **non-durable + non-exclusive** — exactly the queue shape RabbitMQ 4.x
deprecates and plans to remove in a future major version.
Task queues themselves are fine (they can be quorum), but a quorum queue must be
durable, so the pidbox cannot be converted via configuration — it's a kombu-side
declaration.
### What you think should happen instead?
The bundled/pinned **kombu** version should include the pidbox durability fix so the
deprecation is not triggered out of the box.
The fix landed in **kombu 5.7.0** (kombu#2237 → PR celery/kombu#2531: pidbox declared
durable). However, Airflow's constraints still pin **kombu 5.6.2**:
| Constraints | kombu | celery | amqp |
|---|---|---|---|
| constraints-3.2.2 (py3.12) | 5.6.2 | 5.6.3 | 5.3.1 |
| constraints-3.3.0 (py3.12) | 5.6.2 | 5.6.3 | 5.3.1 |
Request: allow/bump **kombu >= 5.7.0** (via `apache-airflow-providers-celery`
requirements / constraints) so the RabbitMQ `transient_nonexcl_queues` deprecation is
resolved without a workaround.
### Operating System
_No response_
### Deployment
None
### Apache Airflow Provider(s)
_No response_
### Versions of Apache Airflow Providers
_No response_
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### Anything else?
### How to reproduce
1. Deploy Airflow with `CeleryExecutor` against a RabbitMQ **4.x** broker.
2. Start scheduler + a Celery worker (remote control enabled — the default).
3. Observe RabbitMQ logs (or `rabbitmqctl list_deprecated_features --used`): the
`transient_nonexcl_queues` deprecated feature is flagged as used, caused by the
`*.pidbox` queue.
### Workarounds (for reference)
- RabbitMQ: `deprecated_features.permit.transient_nonexcl_queues = true` (temporary).
- Celery: `worker_enable_remote_control = False` (drops pidbox; disables remote
control / Flower inspect / revoke).
Neither is a real fix — kombu >= 5.7.0 is.
### References
- kombu#2237 — pidbox durable flag: https://github.com/celery/kombu/issues/2237
- RabbitMQ transient_nonexcl_queues: https://github.com/rabbitmq/rabbitmq-server/discussions/13161
- Celery quorum queues support: https://github.com/apache/airflow/discussions/35649
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] RabbitMQ transient_nonexcl_queues deprecation triggered by Celery pidbox — bump kombu to >= 5.7.0 [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #69704:
URL: https://github.com/apache/airflow/issues/69704#issuecomment-4936649707
Where do you see that Airflow limits the versions?
https://github.com/apache/airflow/blob/d19a68a152dfc1887450336e236206c4c96f56ef/providers/celery/pyproject.toml#L68
You don't have to use Airflow constraints. You can create your own.
Airflow constraints does **not** guarantee latest available versions. It guarantees only [reproducible install](https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#reproducible-airflow-installation).
PIP algorithm decides on the versions. Since you are likely not to use all the dependencies Airflow has sometimes generating your own constraints file just with the packages you need will give you a better result.
Unless there is a version which is broken completely and should be excluded entirely from our resolution options I don't see any task here for Airflow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] RabbitMQ transient_nonexcl_queues deprecation triggered by Celery pidbox — bump kombu to >= 5.7.0 [airflow]
Posted by "or-benjamin (via GitHub)" <gi...@apache.org>.
or-benjamin closed issue #69704: RabbitMQ transient_nonexcl_queues deprecation triggered by Celery pidbox — bump kombu to >= 5.7.0
URL: https://github.com/apache/airflow/issues/69704
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] RabbitMQ transient_nonexcl_queues deprecation triggered by Celery pidbox — bump kombu to >= 5.7.0 [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #69704:
URL: https://github.com/apache/airflow/issues/69704#issuecomment-4981390440
> The one caveat is that kombu 5.7.0 isn't released on PyPI yet — the latest release is 5.6.2. The pidbox durability fix (https://github.com/celery/kombu/pull/2531) is merged but still tagged for the 5.7.0 milestone, so there's nothing installable to bump to right now. Once 5.7.0 ships I'll just override it on my end.
That is a request for the Celery team. I have several times requested to expedite releases by Celery but it is not up to me :)
maybe @auvipy can share expected timelines?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] RabbitMQ transient_nonexcl_queues deprecation triggered by Celery pidbox — bump kombu to >= 5.7.0 [airflow]
Posted by "auvipy (via GitHub)" <gi...@apache.org>.
auvipy commented on issue #69704:
URL: https://github.com/apache/airflow/issues/69704#issuecomment-5971984673
v5.7alpha1 is already released on pypi. betas should be released tomorrow or early next week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] RabbitMQ transient_nonexcl_queues deprecation triggered by Celery pidbox — bump kombu to >= 5.7.0 [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #69704:
URL: https://github.com/apache/airflow/issues/69704#issuecomment-4935754106
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] RabbitMQ transient_nonexcl_queues deprecation triggered by Celery pidbox — bump kombu to >= 5.7.0 [airflow]
Posted by "or-benjamin (via GitHub)" <gi...@apache.org>.
or-benjamin commented on issue #69704:
URL: https://github.com/apache/airflow/issues/69704#issuecomment-4980233229
Thanks @eladkal, that clears it up — I had wrongly assumed the constraints file was mandatory.
The one caveat is that kombu 5.7.0 isn't released on PyPI yet — the latest release is 5.6.2. The pidbox durability fix (celery/kombu#2531) is merged but still tagged for the 5.7.0 milestone, so there's nothing installable to bump to right now. Once 5.7.0 ships I'll just override it on my end.
So I agree there's no action needed on the Airflow side. In the meantime I'll use the RabbitMQ-side workaround (deprecated_features.permit.transient_nonexcl_queues = true).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org