[Proposal] Explicit re-schedule of sensors

[Stefan Seelmann <ma...@stefan-seelmann.de>]

Hi all,

I'd like to discuss a proposal to enable explicit re-scheduling of
sensors. I think there is demand for such a thing, in the last weeks
multiple people asked for it or mentioned workarounds.

I created a Jira [1] that describes the proposal and an initial PR [2].

Feedback welcomed :-)

Kind Regards,
Stefan

[1] https://issues.apache.org/jira/browse/AIRFLOW-2747
[2] https://github.com/apache/incubator-airflow/pull/3596

Re: [Proposal] Explicit re-schedule of sensors

["Driesprong, Fokko" <fo...@driesprong.frl>]

Thanks Stefan for picking this up. The sensors are in desperate need for
some redesign for the aforementioned reasons. Please note this ticket:
https://issues.apache.org/jira/browse/AIRFLOW-2001 It addresses the same
issue.

Regarding the open question. I would be reluctant for introducing new
states. Adding new states involves also changing/adding logic in the
scheduler. This scheduler is already far too complex right now. Maybe we
can also do something with the priority of the sensor, maybe lower the
priority once it has been polled, and came back with a negative state. In
such a strategy the other tasks will get priority over the sensors.

Cheers, Fokko

2018-07-12 14:53 GMT+02:00 Stefan Seelmann <ma...@stefan-seelmann.de>:

> Hi all,
>
> I'd like to discuss a proposal to enable explicit re-scheduling of
> sensors. I think there is demand for such a thing, in the last weeks
> multiple people asked for it or mentioned workarounds.
>
> I created a Jira [1] that describes the proposal and an initial PR [2].
>
> Feedback welcomed :-)
>
> Kind Regards,
> Stefan
>
> [1] https://issues.apache.org/jira/browse/AIRFLOW-2747
> [2] https://github.com/apache/incubator-airflow/pull/3596
>