[GitHub] [airflow] jedcunningham commented on pull request #15500: Handle kubernetes watcher stream disconnection

[GitBox <gi...@apache.org>]

jedcunningham commented on pull request #15500:
URL: https://github.com/apache/airflow/pull/15500#issuecomment-827885109


   >  we do a list and get back version A+5 (say) then we have "lost" the events A+1..A+4, i.e. we would never call `process_event()` for the events in this range.
   
   Unfortunately, right now we have that same problem as we (attempt to) rewatch from "0", a.k.a _any_ point in history. Per the docs, the latest is preferred so the behavior is likely pretty similar:
   https://kubernetes.io/docs/reference/using-api/api-concepts/#the-resourceversion-parameter
   
   I think the piece we are missing is a step to reconcile our state against what the list returns, e.g. detect we missed A+1..A+4 and action them. More work needs to be done to harden this area and this is the first step (and this alone doesn't buy us much other than setting us up for being more complete later).
   
   @ephraimbuddy and I will spend some time trying to reproduce this scenario on demand to make sure we are starting down the right path.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org