[GitHub] [airflow] potiuk commented on issue #31013: Datasets: Allow expiration time for datasets

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk commented on issue #31013:
URL: https://github.com/apache/airflow/issues/31013#issuecomment-1531395463

   Similar to #30974:
   
   Just a reminder on that one (I personally have no opinion enough to chime in) but - since this is a change How airflow architecture is designed around datasets and their dependencies, whatever is discussed here shoudl be brought to the devlist as proposal: https://airflow.apache.org/community/ -> people might not follow this discussion / feature request here and only a devlist discussion will make it into "implementable" change. I guess it's fine to disucss it here to get the idea and concepts hashed out, but whoever would likel to implement, it should be summarised there, and the actual decision making should happen on the devlist - possibly in the form of AIP (Airflow Improvement Proposal) if the scope of it will turn to be big. 
   
   For me this is at the border of feature/discussion. Usually feature is something that adds a small thing that does not change the internals, in this case we are talking about changing the ways how dependencies are resolved by scheduler, when the scheduler considers  them fulfilled, and when they are "eligible" for running, so something rather critical in terms of "core of airflow". I could see that one converted to discsussion, but I think those two issue could be discussed together as set of things to improve to datasets and brought as an AIP for discussion there (it's ok to discuss them here - but only when they are brought to devlist the might have a chance to have some decisions on those).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org