Detecting when a job is "caught up"

[Kurtis Walker via user <us...@flink.apache.org>]

I’ve got a Flink job that uses a HybridSource.  It reads a bunch of S3 files and then transitions to a Kafka Topic where it stays processing live data.  Everything gets written to a warehouse where users build and run reports.  In takes about 36 hours to read data from the beginning before it’s caught up to the live data.  I’m working on how to switch from V1 of my job to V2, without any down time for the user.  Ideally I can start V2, let it read all the data from the beginning again, write it to a V2 warehouse, and only when it’s caught up, stop the V1 job and notify my reporting engine to use V2 instead of V1.  Is there any way for Flink to trigger some function when V2 is caught up, so that my code could make the switch?  Even something that could be called when the Hybrid Source switches from S3 to Kafka would be close enough for my purposes.

Thanks,

Kurt