[GitHub] [beam] je-ik commented on issue #21897: [Feature Request]: Flink runner savepoint backward compatibility

[GitBox <gi...@apache.org>]

je-ik commented on issue #21897:
URL: https://github.com/apache/beam/issues/21897#issuecomment-1171041116

   What do you mean by backward compatible savepoint in Flink? Beam's Flink runner uses Flink's checkpoint/savepoint mechanism, thus it should be compatible under the same circumstances as native Flink. The problem is that Beam adds additional layer of abstraction that can cause state incompatibility - e.g. when a translation of a Beam PTransform changes, when a Coder changes, etc.
   There currently is not any mechanism that can transfer incompatible savepoints between versions, the only option is to run a "bootstrap" version of your job, that creates the state from some historical data.
   
   Reading the a savepoint and converting it to preserve compatibility across beam versions *could* be possible under some circumstances, but is not possible in general.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org