You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "UnightSun (via GitHub)" <gi...@apache.org> on 2023/08/28 07:41:43 UTC
[GitHub] [airflow] UnightSun opened a new issue, #33818: Support dispatching triggers to different triggerer instances.
UnightSun opened a new issue, #33818:
URL: https://github.com/apache/airflow/issues/33818
### Description
Let it say:
We have two celery workers: A and B, they cannot access to each other
worker A can access file A, but can not access file B,
and worker B can access file B, not file A.
With sensors, we can set workers A and B to different queue, and specify task's `queue` parameter.
But when we trying to use deferrable operator and triggers, we cannot specify which triggerer it runs on.
### Use case/motivation
`airflow triggerer` can take a parameter like celery worker's queue, and `Operator.defer` or `Operator.__init__` can specify that parameter, make tigger runs on that triggerer (or one of triggerers).
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #33818: Support dispatching triggers to different triggerer instances.
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #33818:
URL: https://github.com/apache/airflow/issues/33818#issuecomment-1695221739
Yes. I thin it's a good idea and necessary feature to complet multi-tenancy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support dispatching triggers to different triggerer instances. [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl closed issue #33818: Support dispatching triggers to different triggerer instances.
URL: https://github.com/apache/airflow/issues/33818
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support dispatching triggers to different triggerer instances. [airflow]
Posted by "avkirilishin (via GitHub)" <gi...@apache.org>.
avkirilishin commented on issue #33818:
URL: https://github.com/apache/airflow/issues/33818#issuecomment-1865255168
@UnightSun @potiuk please take a look at this [PR](https://github.com/apache/airflow/pull/36338)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org