Blue-Green Deployment of Structured Streaming

["Cressy, Taylor" <Ta...@intuit.com>]

Hi all,

We are attempting to come up with a blue-green deployment strategy for our structured streaming job to minimize down time. The general flow would be:


  1.  Job A is currently streaming
  2.  Job B comes up and starts loading Job A state without starting its query.
  3.  Job B completes the loading of Job A state then informs Job A to gracefully shutdown.
  4.  Job A acknowledges its own graceful shutdown.
  5.  Job B becomes the active query

Has anyone had any success in doing something similar? Every attempt we have made has ended up in some form of race condition against the state files.

Regards,
Taylor Cressy