Backward compatibility of compiled processes and live instances

[Alex Boisvert <bo...@intalio.com>]

Hi devs,

We gathered at ApacheCon this week and yesterday discussed the thorny issue
of long-term process instance compatibility.  The question, to put it
succinctly, is how to continue the execution of live instances of process
definitions compiled on previous versions of Ode into future versions of the
software, despite changes and improvements to the engine.

After discussing several alternatives, we are proposing the following,

1) For identification and tracking, we would encode an O-model version
number in the compiled BPEL process files;  existing .cbp files with no
version are assumed to be version 1.

2) We would keep and maintain different versions of the process definition
model concurrently in the source tree.   Each version would be kept in a
separate module with different package names to avoid name collision during
serialization/deserialization, as well as to support concurrent class
loading in the same engine.   It would be possible to compile processes into
a specific O-model version by selecting which compiler version to use.

3) For convenience, we would bundle together the "
org.apache.ode.bpel.runtime" package, the O-model and the compiler into a
single module.  This would simplify comparing and tracking changes between
versions.   (This would require some refactoring to handle compile
dependencies)

4) The latest version of the O-model in trunk would always be considered
under development and "unstable".  In other words, fair game for
modification.  Only after an official release would this model be supported
with respect to forward-compatibility, and at that point we would "freeze"
this version and create a new one for future development.   By default, the
engine would compile processes using the latest stable version.

5) At this point in time, version 1 is embodied by the O-model currently in
the 1.1 branch, and version 2 is what's under development in the trunk now.
We would therefore merge back the O-model from the 1.1 branch into the trunk
to support both concurrently.

6) To keep the number of supported O-model versions to a manageable level,
we anticipate to release new O-model versions every 6 months to a year.  The
actual frequency would depend on developer and user interest, as well as the
type of changes required for new features.  To strike a balance between
backward compatibility and enhancements/progress, the project would
concurrently maintain and provide support -- on the same Ode server instance
-- for a "window" of O-model versions covering approx. 2 years [1].  It's
anticipated that commercial vendors will be encouraged to provide
longer-term support ;)   Older O-model versions would be supported on older
versions of the server for a longer period, based on community involvement
and interest.

Feedback and questions are very welcome!

Alex, Assaf, Maciej, Matthieu

[1] We generally recommend breaking down long-running processes into shorter
ones in order to support agile process improvement and change.  It's also
wise to consider modeling long-running process instances as persistent
external entities, as this approach generally offers higher levels of data
and state manageability ( c.f. External Variables).

Re: Backward compatibility of compiled processes and live instances

[Tammo van Lessen <tv...@gmail.com>]

Hi guys,

Alex Boisvert wrote:
> Hi devs,
> 
> We gathered at ApacheCon this week and yesterday discussed the thorny issue
> of long-term process instance compatibility.  The question, to put it
> succinctly, is how to continue the execution of live instances of process
> definitions compiled on previous versions of Ode into future versions of the
> software, despite changes and improvements to the engine.
> 
> After discussing several alternatives, we are proposing the following,
...
+1, sounds absolutely reasonable!

Cheers,
  Tammo

Re: Backward compatibility of compiled processes and live instances

[Zubin Wadia <zw...@gmail.com>]

Sorry chaps, I bungled there... I "fast-scanned" the initial thread post!

But yes, it is a very interesting paper, one of my all-time favorities.

Cheers,

Zubin.


On 11/14/07, Alex Boisvert <bo...@intalio.com> wrote:
>
> Hi Zubin,
>
> Interesting paper although it touches on a related but different subject,
> which is how to design and version process definitions themselves.
>
> The article doesn't discuss how you upgrade from Websphere Process Server
> 6.1 to (hypothetically) version 7, or 8, or 9...   Ideally, you just stop
> version 6, install version 7 and restart.  Under the cover, however, the
> engine must be designed to handle older instances as-is, or migrate them
> to
> its newer internal model if its data model changed.
>
> In our case, we want people who are running process instances on Ode 1.1to
> be able to painlessly upgrade to 2.0.  But what about migrating live
> process
> instances from 1.1 to (hypothetically) 3.0, or 4.0, or 5.0 in 6-8 years?
> Will that be supported?   If so, how?   There are the questions we're
> trying
> to answer.
>
> alex
>
>
> On 11/14/07, Zubin Wadia <zw...@gmail.com> wrote:
> >
> > Just to contribute, this is the IBM perspective on it:
> >
> >
> >
> http://www3.software.ibm.com/ibmdl/pub/software/dw/wes/pdf/0602_brown-wps_versioning_dynamicity.pdf
> >
> > Cheers,
> >
> > Zubin.
> >
> >
> > On 11/14/07, Alex Boisvert <bo...@intalio.com> wrote:
> > >
> > > Hi devs,
> > >
> > > We gathered at ApacheCon this week and yesterday discussed the thorny
> > > issue
> > > of long-term process instance compatibility.  The question, to put it
> > > succinctly, is how to continue the execution of live instances of
> > process
> > > definitions compiled on previous versions of Ode into future versions
> of
> > > the
> > > software, despite changes and improvements to the engine.
> > >
> > > After discussing several alternatives, we are proposing the following,
> > >
> > > 1) For identification and tracking, we would encode an O-model version
> > > number in the compiled BPEL process files;  existing .cbp files with
> no
> > > version are assumed to be version 1.
> > >
> > > 2) We would keep and maintain different versions of the process
> > definition
> > > model concurrently in the source tree.   Each version would be kept in
> a
> > > separate module with different package names to avoid name collision
> > > during
> > > serialization/deserialization, as well as to support concurrent class
> > > loading in the same engine.   It would be possible to compile
> processes
> > > into
> > > a specific O-model version by selecting which compiler version to use.
> > >
> > > 3) For convenience, we would bundle together the "
> > > org.apache.ode.bpel.runtime" package, the O-model and the compiler
> into
> > a
> > > single module.  This would simplify comparing and tracking changes
> > between
> > > versions.   (This would require some refactoring to handle compile
> > > dependencies)
> > >
> > > 4) The latest version of the O-model in trunk would always be
> considered
> > > under development and "unstable".  In other words, fair game for
> > > modification.  Only after an official release would this model be
> > > supported
> > > with respect to forward-compatibility, and at that point we would
> > "freeze"
> > > this version and create a new one for future development.   By
> default,
> > > the
> > > engine would compile processes using the latest stable version.
> > >
> > > 5) At this point in time, version 1 is embodied by the O-model
> currently
> > > in
> > > the 1.1 branch, and version 2 is what's under development in the trunk
> > > now.
> > > We would therefore merge back the O-model from the 1.1 branch into the
> > > trunk
> > > to support both concurrently.
> > >
> > > 6) To keep the number of supported O-model versions to a manageable
> > level,
> > > we anticipate to release new O-model versions every 6 months to a
> > > year.  The
> > > actual frequency would depend on developer and user interest, as well
> as
> > > the
> > > type of changes required for new features.  To strike a balance
> between
> > > backward compatibility and enhancements/progress, the project would
> > > concurrently maintain and provide support -- on the same Ode server
> > > instance
> > > -- for a "window" of O-model versions covering approx. 2 years
> > [1].  It's
> > > anticipated that commercial vendors will be encouraged to provide
> > > longer-term support ;)   Older O-model versions would be supported on
> > > older
> > > versions of the server for a longer period, based on community
> > involvement
> > > and interest.
> > >
> > > Feedback and questions are very welcome!
> > >
> > > Alex, Assaf, Maciej, Matthieu
> > >
> > > [1] We generally recommend breaking down long-running processes into
> > > shorter
> > > ones in order to support agile process improvement and change.  It's
> > also
> > > wise to consider modeling long-running process instances as persistent
> > > external entities, as this approach generally offers higher levels of
> > data
> > > and state manageability ( c.f. External Variables).
> > >
> >
>

Re: Backward compatibility of compiled processes and live instances

[Alex Boisvert <bo...@intalio.com>]

Hi Zubin,

Interesting paper although it touches on a related but different subject,
which is how to design and version process definitions themselves.

The article doesn't discuss how you upgrade from Websphere Process Server
6.1 to (hypothetically) version 7, or 8, or 9...   Ideally, you just stop
version 6, install version 7 and restart.  Under the cover, however, the
engine must be designed to handle older instances as-is, or migrate them to
its newer internal model if its data model changed.

In our case, we want people who are running process instances on Ode 1.1 to
be able to painlessly upgrade to 2.0.  But what about migrating live process
instances from 1.1 to (hypothetically) 3.0, or 4.0, or 5.0 in 6-8 years?
Will that be supported?   If so, how?   There are the questions we're trying
to answer.

alex


On 11/14/07, Zubin Wadia <zw...@gmail.com> wrote:
>
> Just to contribute, this is the IBM perspective on it:
>
>
> http://www3.software.ibm.com/ibmdl/pub/software/dw/wes/pdf/0602_brown-wps_versioning_dynamicity.pdf
>
> Cheers,
>
> Zubin.
>
>
> On 11/14/07, Alex Boisvert <bo...@intalio.com> wrote:
> >
> > Hi devs,
> >
> > We gathered at ApacheCon this week and yesterday discussed the thorny
> > issue
> > of long-term process instance compatibility.  The question, to put it
> > succinctly, is how to continue the execution of live instances of
> process
> > definitions compiled on previous versions of Ode into future versions of
> > the
> > software, despite changes and improvements to the engine.
> >
> > After discussing several alternatives, we are proposing the following,
> >
> > 1) For identification and tracking, we would encode an O-model version
> > number in the compiled BPEL process files;  existing .cbp files with no
> > version are assumed to be version 1.
> >
> > 2) We would keep and maintain different versions of the process
> definition
> > model concurrently in the source tree.   Each version would be kept in a
> > separate module with different package names to avoid name collision
> > during
> > serialization/deserialization, as well as to support concurrent class
> > loading in the same engine.   It would be possible to compile processes
> > into
> > a specific O-model version by selecting which compiler version to use.
> >
> > 3) For convenience, we would bundle together the "
> > org.apache.ode.bpel.runtime" package, the O-model and the compiler into
> a
> > single module.  This would simplify comparing and tracking changes
> between
> > versions.   (This would require some refactoring to handle compile
> > dependencies)
> >
> > 4) The latest version of the O-model in trunk would always be considered
> > under development and "unstable".  In other words, fair game for
> > modification.  Only after an official release would this model be
> > supported
> > with respect to forward-compatibility, and at that point we would
> "freeze"
> > this version and create a new one for future development.   By default,
> > the
> > engine would compile processes using the latest stable version.
> >
> > 5) At this point in time, version 1 is embodied by the O-model currently
> > in
> > the 1.1 branch, and version 2 is what's under development in the trunk
> > now.
> > We would therefore merge back the O-model from the 1.1 branch into the
> > trunk
> > to support both concurrently.
> >
> > 6) To keep the number of supported O-model versions to a manageable
> level,
> > we anticipate to release new O-model versions every 6 months to a
> > year.  The
> > actual frequency would depend on developer and user interest, as well as
> > the
> > type of changes required for new features.  To strike a balance between
> > backward compatibility and enhancements/progress, the project would
> > concurrently maintain and provide support -- on the same Ode server
> > instance
> > -- for a "window" of O-model versions covering approx. 2 years
> [1].  It's
> > anticipated that commercial vendors will be encouraged to provide
> > longer-term support ;)   Older O-model versions would be supported on
> > older
> > versions of the server for a longer period, based on community
> involvement
> > and interest.
> >
> > Feedback and questions are very welcome!
> >
> > Alex, Assaf, Maciej, Matthieu
> >
> > [1] We generally recommend breaking down long-running processes into
> > shorter
> > ones in order to support agile process improvement and change.  It's
> also
> > wise to consider modeling long-running process instances as persistent
> > external entities, as this approach generally offers higher levels of
> data
> > and state manageability ( c.f. External Variables).
> >
>

Re: Backward compatibility of compiled processes and live instances

[Zubin Wadia <zw...@gmail.com>]

Just to contribute, this is the IBM perspective on it:

http://www3.software.ibm.com/ibmdl/pub/software/dw/wes/pdf/0602_brown-wps_versioning_dynamicity.pdf

Cheers,

Zubin.


On 11/14/07, Alex Boisvert <bo...@intalio.com> wrote:
>
> Hi devs,
>
> We gathered at ApacheCon this week and yesterday discussed the thorny
> issue
> of long-term process instance compatibility.  The question, to put it
> succinctly, is how to continue the execution of live instances of process
> definitions compiled on previous versions of Ode into future versions of
> the
> software, despite changes and improvements to the engine.
>
> After discussing several alternatives, we are proposing the following,
>
> 1) For identification and tracking, we would encode an O-model version
> number in the compiled BPEL process files;  existing .cbp files with no
> version are assumed to be version 1.
>
> 2) We would keep and maintain different versions of the process definition
> model concurrently in the source tree.   Each version would be kept in a
> separate module with different package names to avoid name collision
> during
> serialization/deserialization, as well as to support concurrent class
> loading in the same engine.   It would be possible to compile processes
> into
> a specific O-model version by selecting which compiler version to use.
>
> 3) For convenience, we would bundle together the "
> org.apache.ode.bpel.runtime" package, the O-model and the compiler into a
> single module.  This would simplify comparing and tracking changes between
> versions.   (This would require some refactoring to handle compile
> dependencies)
>
> 4) The latest version of the O-model in trunk would always be considered
> under development and "unstable".  In other words, fair game for
> modification.  Only after an official release would this model be
> supported
> with respect to forward-compatibility, and at that point we would "freeze"
> this version and create a new one for future development.   By default,
> the
> engine would compile processes using the latest stable version.
>
> 5) At this point in time, version 1 is embodied by the O-model currently
> in
> the 1.1 branch, and version 2 is what's under development in the trunk
> now.
> We would therefore merge back the O-model from the 1.1 branch into the
> trunk
> to support both concurrently.
>
> 6) To keep the number of supported O-model versions to a manageable level,
> we anticipate to release new O-model versions every 6 months to a
> year.  The
> actual frequency would depend on developer and user interest, as well as
> the
> type of changes required for new features.  To strike a balance between
> backward compatibility and enhancements/progress, the project would
> concurrently maintain and provide support -- on the same Ode server
> instance
> -- for a "window" of O-model versions covering approx. 2 years [1].  It's
> anticipated that commercial vendors will be encouraged to provide
> longer-term support ;)   Older O-model versions would be supported on
> older
> versions of the server for a longer period, based on community involvement
> and interest.
>
> Feedback and questions are very welcome!
>
> Alex, Assaf, Maciej, Matthieu
>
> [1] We generally recommend breaking down long-running processes into
> shorter
> ones in order to support agile process improvement and change.  It's also
> wise to consider modeling long-running process instances as persistent
> external entities, as this approach generally offers higher levels of data
> and state manageability ( c.f. External Variables).
>