[GitHub] [airflow] gdavoian commented on issue #28382: Implement all_done_min_one_success trigger rule

[GitBox <gi...@apache.org>]

gdavoian commented on issue #28382:
URL: https://github.com/apache/airflow/issues/28382#issuecomment-1353889741

   @potiuk your proposed solution sounds really good, thank you!
   
   I'd just like to remark that the data loading task doesn't necessarily have to know which data ingestion tasks actually failed/succeeded to make further decisions. E.g., it may use some kind of a pattern like `YYYY/mm/dd/**.json` to load the current/latest data from the data lake into the data warehouse (whatever file it was able to find matching the pattern). 
   
   So my idea was to just have a trigger rule handling such simple cases (of course, without introducing extra complexity). Yes, I agree that adding yet another auxiliary task to check the state of the upstream tasks isn't a big deal, but it's still a kind of logic that might arise again and again...
   
   Also, I believe the approach described above is pretty generic, so it can be used to imitate a lot of different behaviors, including but not limited to `none_failed`, `none_skipped`, `none_failed_min_one_success`. Since the latter ones actually have their own trigger rules, I don't see a reason why the `all_done_min_one_success` behavior doesn't deserve a trigger rule on its own :)
   
   I know that what I'm trying to describe covers a pretty simple use case, but I really think that good software should strive to make simple things even easier (and wrong things impossible!), and in this particular case, unfortunately, there isn't an easy way to achieve the desired behavior.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org