You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/09/09 19:38:28 UTC
[GitHub] [airflow] malthe edited a comment on pull request #17576: Add pre/post execution hooks
malthe edited a comment on pull request #17576:
URL: https://github.com/apache/airflow/pull/17576#issuecomment-916381389
@ashb a real example could be two data sources coming in at different times, each one needs some post-processing and it makes sense to do this immediately – because ultimately, both are used to form some output that should be available as soon as possible.
So in this case, we can't start processing the data before we know it's come in (let's assume that this is entirely based on time of day, you can't "sense" it). I think it's a fairly normal situation – and at least, in the workflows I'm seeing, the pattern is used all of the place. Is it "one workflow" (i.e. DAG) – I would say definitely because the dependencies are real.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org