Updating Scala package names while preserving state

[Thomas Eckestad <Th...@NiraDynamics.se>]

Hi,

I would like to change the package name of our Scala code from com.company.foo.something to com.company.bar.something while preserving the state. How can I make a Savepoint from an application built with com.company.foo.something and make that Savepoint compatible with new code built from com.company.bar.something?

In a Savepoint directory from one of our Flink jobs. Doing: egrep com\.company\.bar produces a lot of hits. Could it be expected to work with just using sed to replace the strings? Or are there binary, non-text data, as well that needs to be updated?

We are currently on FLink 1.13.6.

Thanks,
Thomas
Thomas Eckestad
Systems Engineer
Development RSI

NIRA Dynamics AB
Wallenbergs gata 4
58330 Link?ping, Sweden
Mobile: +46 701 447 279
Thomas.Eckestad@NiraDynamics.se
www.niradynamics.se

Re: Updating Scala package names while preserving state

[Thomas Eckestad <Th...@NiraDynamics.se>]

My conclusions. First, I think it would be good to clarify the background. The class for which I changed the package/namespace is a POJO class which is part of the applications state. According to the official Flink documentation on state evolution:

Class name of the POJO type cannot change, including the namespace of the class.

So that is quite clear.

I also investigated using the state-processor-api to create a new Savepoint with updated namespace for the class, but that is not trivial, so decided to give up on that since the state of the application in question was not considered that important.

If using the State-processor-api for handling a namespace change that would require both the moved and the original class to be on the java class path when running the state processor, to enabled both de-serializing the state (old namespace)  and serializing the new state (new namespace). So it could have been done that way I guess.

I did not find any other option for migrating the state due to a namespace/package name change. Performing text replace with sed does not work.

On 7 Feb 2023, at 12:03, Thomas Eckestad <Th...@NiraDynamics.se>> wrote:

Hi,

I would like to change the package name of our Scala code from com.company.foo.something to com.company.bar.something while preserving the state. How can I make a Savepoint from an application built with com.company.foo.something and make that Savepoint compatible with new code built from com.company.bar.something?

In a Savepoint directory from one of our Flink jobs. Doing: egrep com\.company\.bar produces a lot of hits. Could it be expected to work with just using sed to replace the strings? Or are there binary, non-text data, as well that needs to be updated?

We are currently on FLink 1.13.6.

Thanks,
Thomas
Thomas Eckestad
Systems Engineer
Development RSI

NIRA Dynamics AB
Wallenbergs gata 4
58330 Link?ping, Sweden
Mobile: +46 701 447 279
Thomas.Eckestad@NiraDynamics.se<ma...@NiraDynamics.se>
www.niradynamics.se