You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "kaxil (via GitHub)" <gi...@apache.org> on 2025/06/09 19:15:10 UTC
[I] Investigate `Timetable` usage for client-server separation [airflow]
kaxil opened a new issue, #51549:
URL: https://github.com/apache/airflow/issues/51549
Analyze current timetable usage in scheduler vs DAG processor. Decide if timetables belong in core, task-sdk, or separate package (`airflow-commons`/`airflow-protocols`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `Timetable` usage for client-server separation [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on issue #51549:
URL: https://github.com/apache/airflow/issues/51549#issuecomment-2969784296
> Timetable is kind of the reverse of e.g. dag and asset. It allows customisation, but all the logis is only needed in the scheduler, not the worker nor the dag processor. The main implementation should therefore likely belong in core. However, non-core timetables (e.g. `EventsTimetable`) may belong in the standard provider instead.
Doesn't the dag processor use timetable to get the next run time though?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `Timetable` usage for client-server separation [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on issue #51549:
URL: https://github.com/apache/airflow/issues/51549#issuecomment-3226822679
Most of the timetable logic need to be in the scheduler. So we should kind of do the opposite of SerializedBaseOperator.
1. Existing tiemtable classes in core are scheduler-only.
2. Create timetable “stub” classes that only takes arguments without any scheduler logic.
3. The stubs are serialised, and deserialised in the scheduler into their “real” timetable class counterparts.
Custom subclasses are more problematic. I guess for now (Airflow 3.1) we can just make that one single subclass fulfill both roles as a stub and real timetable. Maybe at some point we need to have an Airflow Extension SDK for this, and other things plugin-wise that you can register into the scheduler.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `Timetable` usage for client-server separation [airflow]
Posted by "phanikumv (via GitHub)" <gi...@apache.org>.
phanikumv commented on issue #51549:
URL: https://github.com/apache/airflow/issues/51549#issuecomment-3195526758
Are we still targeting this for 3.1.0?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `Timetable` usage for client-server separation [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on issue #51549:
URL: https://github.com/apache/airflow/issues/51549#issuecomment-2996272488
If I use a Custom Timetable, does that code currently run in the Scheduler?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `Timetable` usage for client-server separation [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on issue #51549:
URL: https://github.com/apache/airflow/issues/51549#issuecomment-2999361543
So that has to be on Server side?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `Timetable` usage for client-server separation [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #51549:
URL: https://github.com/apache/airflow/issues/51549#issuecomment-3889149976
@uranusjr is there anything pending on this one? I think we have handled it already?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `Timetable` usage for client-server separation [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on issue #51549:
URL: https://github.com/apache/airflow/issues/51549#issuecomment-2969281200
Timetable is kind of the reverse of e.g. dag and asset. It allows customisation, but all the logis is only needed in the scheduler, not the worker nor the dag processor. The main implementation should therefore likely belong in core. However, non-core timetables (e.g. `EventsTimetable`) may belong in the standard provider instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `Timetable` usage for client-server separation [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #51549:
URL: https://github.com/apache/airflow/issues/51549#issuecomment-3912377676
This was handled by: https://github.com/apache/airflow/pull/58669
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `Timetable` usage for client-server separation [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on issue #51549:
URL: https://github.com/apache/airflow/issues/51549#issuecomment-3196269557
Yes ideally. @uranusjr ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `Timetable` usage for client-server separation [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #51549:
URL: https://github.com/apache/airflow/issues/51549#issuecomment-3615737792
@uranusjr have we merged all the changes w.r.t this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `Timetable` usage for client-server separation [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on issue #51549:
URL: https://github.com/apache/airflow/issues/51549#issuecomment-2979233282
Oh yes, it does, for the first ever run. I forgot about that. This can be changed if we need to though. The scheduler is responsible for calculating all later runs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `Timetable` usage for client-server separation [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on issue #51549:
URL: https://github.com/apache/airflow/issues/51549#issuecomment-2998754788
Yes it does.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `Timetable` usage for client-server separation [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh closed issue #51549: Investigate `Timetable` usage for client-server separation
URL: https://github.com/apache/airflow/issues/51549
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org