[GitHub] [airflow] potiuk commented on pull request #25489: Dag processor manager queue split (fixes SLAs)

[GitBox <gi...@apache.org>]

potiuk commented on PR #25489:
URL: https://github.com/apache/airflow/pull/25489#issuecomment-1205190941

   Thanks for so thorough description and your willingness to improve SLA (I really, really appreciate it as this is one of the things that we had on our backburner for quite a while).
   
   However, I think @argibbs this is the kind of change and desciption is such big and potential of breaking things is such big that it requires a devlist discussion IMHO to drag attention of people who should be dragged rather than dicussing it only in PR.  I know you wrote ("probably not make it in 2.3.4"), but I also think "probably not make it in 2.4.0 either" is a better assesment.
   
   I strongly suggest to make a small digest of the description you did - with most important parts of "why" and "how" extracted (less is more) and send it to the devlist https://airflow.apache.org/community/, with link to this PR and inviting people to express their thoughts.
   
   I think this is not going to be merged before 2.4 branch is cut. Too big risks, it needs to be likely very thoroughly tested and justified that we know what we are doing here) so there is stiil a lot of time to discuss it. 
   
   I also think some people who are now looking at finalizing [AIP-48](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-48+Data+Dependency+Management+and+Data+Driven+Scheduling) so that it is ready for prime time for 2.4 (and who should be part of the discussiosn), have no time and energy to look at it. And we definiitely want to avoid of adding too many potentially high-impact changes in one release. We've already added too much in 2.3.0 and I think we learned a lesson there.
   
   So regardless of what you propose it might take a while
   
   I strongly encourage you to describe the case/problem/solution in concise way and send it to the devlist. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org