Evolvable OModel

[Gary Brown <ga...@pi4tech.com>]

As discussed on the call today, we need a way to overcome the
constraints of the serialized OModel which is produced when compiling
the BPEL process.

Although a more flexible serialization mechanism may provide a
solution, as was also discussed, there will be cases where additional
logic would be required to determine how an older representation
should be evolved into a new representation.

Another approach would be to use the BPEL process (xml) directly, so
rather than using a binary representation compiled in advance, the
runtime would simply load the BPEL DOM.

If this required a change to the internal OModel mechanism, then it
may be more work than the proposed dynamic OModel idea, however if we
simply took the approach of "compile on load", then we get away from
the issue, which is the persisted serialized form. We also no longer
require any specific migration logic to move from older versions of
the persisted representation (dynamic or not).

So we could keep the existing OModel format, which can evolve as
required as long as the compiler is kept in step, and we should no
longer have any issues with long running processes.

This change could be introduced into the current 1.3.x trunk without
any backward compatibility issues.

The only trade off is the speed difference between (1) loading the
compiled representation versus (2) parsing the XML & compiling the
BPEL process. If this would need to be performed multiple times in the
same runtime, then a hash of the process could be used to cache the
compiled version so this is only performed once per process
definition.

Thoughts?

Regards
Gary

Re: Evolvable OModel

[Gary Brown <ga...@pi4tech.com>]

Oh well - back to the drawing board :)


On Tue, Mar 22, 2011 at 3:12 PM, Tammo van Lessen <tv...@gmail.com> wrote:
> Hi Shivaraj, hi Gary,
>
> yes and no: first, when compiling the process model, each OModel node
> contains a unique ID and this is currently a GUID. Thus, even if the
> BPEL code is unchanged, the OModel nodes will have different IDs after
> compilation. Second, these IDs are used when serializing the runtime
> state. The JACOB soup keeps references to the OModel. When serializing
> the state, these references are replaced by placeholder objects that
> store the GUID of the original node to avoid serializing the whole
> OModel object graph. This is an optimization to save time and space.
>
> Tammo
>
> On 22.03.2011 15:35, Gary Brown wrote:
>> Hi Shivaraj
>>
>> As far as I am aware this issue is not related to state - only the
>> static representation of the business process. The OModel simply
>> represents a binary compiled version of the BPEL process definition.
>> So all I am suggesting is rather than store the binary version, we
>> store the xml version, compile on first load, and the end result is an
>> up-to-date OModel, regardless of the age of the process instance.
>>
>> Regards
>> Gary
>>
>> On Tue, Mar 22, 2011 at 12:34 PM, Shivaraj Tenginakai
>> <ts...@gmail.com> wrote:
>>> Gary,
>>>
>>> Would it be possible to distinguish between execution optimization and
>>> state? As long as state information can be derived from one version to
>>> another, it should be fine to recompile (to achieve execution
>>> optimization).
>>>
>>> Thanks,
>>>
>>> Shivaraj
>>>
>>>
>>> On Tue, Mar 22, 2011 at 3:40 PM, Gary Brown <ga...@pi4tech.com> wrote:
>>>> As discussed on the call today, we need a way to overcome the
>>>> constraints of the serialized OModel which is produced when compiling
>>>> the BPEL process.
>>>>
>>>> Although a more flexible serialization mechanism may provide a
>>>> solution, as was also discussed, there will be cases where additional
>>>> logic would be required to determine how an older representation
>>>> should be evolved into a new representation.
>>>>
>>>> Another approach would be to use the BPEL process (xml) directly, so
>>>> rather than using a binary representation compiled in advance, the
>>>> runtime would simply load the BPEL DOM.
>>>>
>>>> If this required a change to the internal OModel mechanism, then it
>>>> may be more work than the proposed dynamic OModel idea, however if we
>>>> simply took the approach of "compile on load", then we get away from
>>>> the issue, which is the persisted serialized form. We also no longer
>>>> require any specific migration logic to move from older versions of
>>>> the persisted representation (dynamic or not).
>>>>
>>>> So we could keep the existing OModel format, which can evolve as
>>>> required as long as the compiler is kept in step, and we should no
>>>> longer have any issues with long running processes.
>>>>
>>>> This change could be introduced into the current 1.3.x trunk without
>>>> any backward compatibility issues.
>>>>
>>>> The only trade off is the speed difference between (1) loading the
>>>> compiled representation versus (2) parsing the XML & compiling the
>>>> BPEL process. If this would need to be performed multiple times in the
>>>> same runtime, then a hash of the process could be used to cache the
>>>> compiled version so this is only performed once per process
>>>> definition.
>>>>
>>>> Thoughts?
>>>>
>>>> Regards
>>>> Gary
>>>>
>>>
>
> --
> Tammo van Lessen - http://www.taval.de
>

Re: Evolvable OModel

[Shivaraj Tenginakai <ts...@gmail.com>]

Also, if we can get this done - it would be possible to plug and play
multiple engines. Each engine may be optimized for a different use
case (for example,  Long Running Processes vs. Short Processes).

Thanks,

Shivaraj


On Tue, Mar 22, 2011 at 9:50 PM, Gary Brown <ga...@pi4tech.com> wrote:
> Agreed - just need to use a reference that remains stable across
> compilations, but still uniquely identifies the relevant node in the
> DAG.
>
> Regards
> Gary
>
> On Tue, Mar 22, 2011 at 3:43 PM, Shivaraj Tenginakai
> <ts...@gmail.com> wrote:
>> Way I have seen this implemented is that we need to distinguish
>> between user state and ode state. ODE state represents the structure
>> of the DAG while the user state represents the value of the variables
>> and the position in the DAG .
>>
>> If we can achieve a clean separation between ODE state and user state,
>> then future evolution would become easy.
>>
>> Shivaraj
>>
>>
>> On Tue, Mar 22, 2011 at 8:42 PM, Tammo van Lessen <tv...@gmail.com> wrote:
>>> Hi Shivaraj, hi Gary,
>>>
>>> yes and no: first, when compiling the process model, each OModel node
>>> contains a unique ID and this is currently a GUID. Thus, even if the
>>> BPEL code is unchanged, the OModel nodes will have different IDs after
>>> compilation. Second, these IDs are used when serializing the runtime
>>> state. The JACOB soup keeps references to the OModel. When serializing
>>> the state, these references are replaced by placeholder objects that
>>> store the GUID of the original node to avoid serializing the whole
>>> OModel object graph. This is an optimization to save time and space.
>>>
>>> Tammo
>>>
>>> On 22.03.2011 15:35, Gary Brown wrote:
>>>> Hi Shivaraj
>>>>
>>>> As far as I am aware this issue is not related to state - only the
>>>> static representation of the business process. The OModel simply
>>>> represents a binary compiled version of the BPEL process definition.
>>>> So all I am suggesting is rather than store the binary version, we
>>>> store the xml version, compile on first load, and the end result is an
>>>> up-to-date OModel, regardless of the age of the process instance.
>>>>
>>>> Regards
>>>> Gary
>>>>
>>>> On Tue, Mar 22, 2011 at 12:34 PM, Shivaraj Tenginakai
>>>> <ts...@gmail.com> wrote:
>>>>> Gary,
>>>>>
>>>>> Would it be possible to distinguish between execution optimization and
>>>>> state? As long as state information can be derived from one version to
>>>>> another, it should be fine to recompile (to achieve execution
>>>>> optimization).
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Shivaraj
>>>>>
>>>>>
>>>>> On Tue, Mar 22, 2011 at 3:40 PM, Gary Brown <ga...@pi4tech.com> wrote:
>>>>>> As discussed on the call today, we need a way to overcome the
>>>>>> constraints of the serialized OModel which is produced when compiling
>>>>>> the BPEL process.
>>>>>>
>>>>>> Although a more flexible serialization mechanism may provide a
>>>>>> solution, as was also discussed, there will be cases where additional
>>>>>> logic would be required to determine how an older representation
>>>>>> should be evolved into a new representation.
>>>>>>
>>>>>> Another approach would be to use the BPEL process (xml) directly, so
>>>>>> rather than using a binary representation compiled in advance, the
>>>>>> runtime would simply load the BPEL DOM.
>>>>>>
>>>>>> If this required a change to the internal OModel mechanism, then it
>>>>>> may be more work than the proposed dynamic OModel idea, however if we
>>>>>> simply took the approach of "compile on load", then we get away from
>>>>>> the issue, which is the persisted serialized form. We also no longer
>>>>>> require any specific migration logic to move from older versions of
>>>>>> the persisted representation (dynamic or not).
>>>>>>
>>>>>> So we could keep the existing OModel format, which can evolve as
>>>>>> required as long as the compiler is kept in step, and we should no
>>>>>> longer have any issues with long running processes.
>>>>>>
>>>>>> This change could be introduced into the current 1.3.x trunk without
>>>>>> any backward compatibility issues.
>>>>>>
>>>>>> The only trade off is the speed difference between (1) loading the
>>>>>> compiled representation versus (2) parsing the XML & compiling the
>>>>>> BPEL process. If this would need to be performed multiple times in the
>>>>>> same runtime, then a hash of the process could be used to cache the
>>>>>> compiled version so this is only performed once per process
>>>>>> definition.
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> Regards
>>>>>> Gary
>>>>>>
>>>>>
>>>
>>> --
>>> Tammo van Lessen - http://www.taval.de
>>>
>>
>

Re: Evolvable OModel

[Gary Brown <ga...@pi4tech.com>]

Agreed - just need to use a reference that remains stable across
compilations, but still uniquely identifies the relevant node in the
DAG.

Regards
Gary

On Tue, Mar 22, 2011 at 3:43 PM, Shivaraj Tenginakai
<ts...@gmail.com> wrote:
> Way I have seen this implemented is that we need to distinguish
> between user state and ode state. ODE state represents the structure
> of the DAG while the user state represents the value of the variables
> and the position in the DAG .
>
> If we can achieve a clean separation between ODE state and user state,
> then future evolution would become easy.
>
> Shivaraj
>
>
> On Tue, Mar 22, 2011 at 8:42 PM, Tammo van Lessen <tv...@gmail.com> wrote:
>> Hi Shivaraj, hi Gary,
>>
>> yes and no: first, when compiling the process model, each OModel node
>> contains a unique ID and this is currently a GUID. Thus, even if the
>> BPEL code is unchanged, the OModel nodes will have different IDs after
>> compilation. Second, these IDs are used when serializing the runtime
>> state. The JACOB soup keeps references to the OModel. When serializing
>> the state, these references are replaced by placeholder objects that
>> store the GUID of the original node to avoid serializing the whole
>> OModel object graph. This is an optimization to save time and space.
>>
>> Tammo
>>
>> On 22.03.2011 15:35, Gary Brown wrote:
>>> Hi Shivaraj
>>>
>>> As far as I am aware this issue is not related to state - only the
>>> static representation of the business process. The OModel simply
>>> represents a binary compiled version of the BPEL process definition.
>>> So all I am suggesting is rather than store the binary version, we
>>> store the xml version, compile on first load, and the end result is an
>>> up-to-date OModel, regardless of the age of the process instance.
>>>
>>> Regards
>>> Gary
>>>
>>> On Tue, Mar 22, 2011 at 12:34 PM, Shivaraj Tenginakai
>>> <ts...@gmail.com> wrote:
>>>> Gary,
>>>>
>>>> Would it be possible to distinguish between execution optimization and
>>>> state? As long as state information can be derived from one version to
>>>> another, it should be fine to recompile (to achieve execution
>>>> optimization).
>>>>
>>>> Thanks,
>>>>
>>>> Shivaraj
>>>>
>>>>
>>>> On Tue, Mar 22, 2011 at 3:40 PM, Gary Brown <ga...@pi4tech.com> wrote:
>>>>> As discussed on the call today, we need a way to overcome the
>>>>> constraints of the serialized OModel which is produced when compiling
>>>>> the BPEL process.
>>>>>
>>>>> Although a more flexible serialization mechanism may provide a
>>>>> solution, as was also discussed, there will be cases where additional
>>>>> logic would be required to determine how an older representation
>>>>> should be evolved into a new representation.
>>>>>
>>>>> Another approach would be to use the BPEL process (xml) directly, so
>>>>> rather than using a binary representation compiled in advance, the
>>>>> runtime would simply load the BPEL DOM.
>>>>>
>>>>> If this required a change to the internal OModel mechanism, then it
>>>>> may be more work than the proposed dynamic OModel idea, however if we
>>>>> simply took the approach of "compile on load", then we get away from
>>>>> the issue, which is the persisted serialized form. We also no longer
>>>>> require any specific migration logic to move from older versions of
>>>>> the persisted representation (dynamic or not).
>>>>>
>>>>> So we could keep the existing OModel format, which can evolve as
>>>>> required as long as the compiler is kept in step, and we should no
>>>>> longer have any issues with long running processes.
>>>>>
>>>>> This change could be introduced into the current 1.3.x trunk without
>>>>> any backward compatibility issues.
>>>>>
>>>>> The only trade off is the speed difference between (1) loading the
>>>>> compiled representation versus (2) parsing the XML & compiling the
>>>>> BPEL process. If this would need to be performed multiple times in the
>>>>> same runtime, then a hash of the process could be used to cache the
>>>>> compiled version so this is only performed once per process
>>>>> definition.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> Regards
>>>>> Gary
>>>>>
>>>>
>>
>> --
>> Tammo van Lessen - http://www.taval.de
>>
>

Re: Evolvable OModel

[Shivaraj Tenginakai <ts...@gmail.com>]

Way I have seen this implemented is that we need to distinguish
between user state and ode state. ODE state represents the structure
of the DAG while the user state represents the value of the variables
and the position in the DAG .

If we can achieve a clean separation between ODE state and user state,
then future evolution would become easy.

Shivaraj


On Tue, Mar 22, 2011 at 8:42 PM, Tammo van Lessen <tv...@gmail.com> wrote:
> Hi Shivaraj, hi Gary,
>
> yes and no: first, when compiling the process model, each OModel node
> contains a unique ID and this is currently a GUID. Thus, even if the
> BPEL code is unchanged, the OModel nodes will have different IDs after
> compilation. Second, these IDs are used when serializing the runtime
> state. The JACOB soup keeps references to the OModel. When serializing
> the state, these references are replaced by placeholder objects that
> store the GUID of the original node to avoid serializing the whole
> OModel object graph. This is an optimization to save time and space.
>
> Tammo
>
> On 22.03.2011 15:35, Gary Brown wrote:
>> Hi Shivaraj
>>
>> As far as I am aware this issue is not related to state - only the
>> static representation of the business process. The OModel simply
>> represents a binary compiled version of the BPEL process definition.
>> So all I am suggesting is rather than store the binary version, we
>> store the xml version, compile on first load, and the end result is an
>> up-to-date OModel, regardless of the age of the process instance.
>>
>> Regards
>> Gary
>>
>> On Tue, Mar 22, 2011 at 12:34 PM, Shivaraj Tenginakai
>> <ts...@gmail.com> wrote:
>>> Gary,
>>>
>>> Would it be possible to distinguish between execution optimization and
>>> state? As long as state information can be derived from one version to
>>> another, it should be fine to recompile (to achieve execution
>>> optimization).
>>>
>>> Thanks,
>>>
>>> Shivaraj
>>>
>>>
>>> On Tue, Mar 22, 2011 at 3:40 PM, Gary Brown <ga...@pi4tech.com> wrote:
>>>> As discussed on the call today, we need a way to overcome the
>>>> constraints of the serialized OModel which is produced when compiling
>>>> the BPEL process.
>>>>
>>>> Although a more flexible serialization mechanism may provide a
>>>> solution, as was also discussed, there will be cases where additional
>>>> logic would be required to determine how an older representation
>>>> should be evolved into a new representation.
>>>>
>>>> Another approach would be to use the BPEL process (xml) directly, so
>>>> rather than using a binary representation compiled in advance, the
>>>> runtime would simply load the BPEL DOM.
>>>>
>>>> If this required a change to the internal OModel mechanism, then it
>>>> may be more work than the proposed dynamic OModel idea, however if we
>>>> simply took the approach of "compile on load", then we get away from
>>>> the issue, which is the persisted serialized form. We also no longer
>>>> require any specific migration logic to move from older versions of
>>>> the persisted representation (dynamic or not).
>>>>
>>>> So we could keep the existing OModel format, which can evolve as
>>>> required as long as the compiler is kept in step, and we should no
>>>> longer have any issues with long running processes.
>>>>
>>>> This change could be introduced into the current 1.3.x trunk without
>>>> any backward compatibility issues.
>>>>
>>>> The only trade off is the speed difference between (1) loading the
>>>> compiled representation versus (2) parsing the XML & compiling the
>>>> BPEL process. If this would need to be performed multiple times in the
>>>> same runtime, then a hash of the process could be used to cache the
>>>> compiled version so this is only performed once per process
>>>> definition.
>>>>
>>>> Thoughts?
>>>>
>>>> Regards
>>>> Gary
>>>>
>>>
>
> --
> Tammo van Lessen - http://www.taval.de
>

Re: Evolvable OModel

[Tammo van Lessen <tv...@gmail.com>]

Hi Shivaraj, hi Gary,

yes and no: first, when compiling the process model, each OModel node
contains a unique ID and this is currently a GUID. Thus, even if the
BPEL code is unchanged, the OModel nodes will have different IDs after
compilation. Second, these IDs are used when serializing the runtime
state. The JACOB soup keeps references to the OModel. When serializing
the state, these references are replaced by placeholder objects that
store the GUID of the original node to avoid serializing the whole
OModel object graph. This is an optimization to save time and space.

Tammo

On 22.03.2011 15:35, Gary Brown wrote:
> Hi Shivaraj
> 
> As far as I am aware this issue is not related to state - only the
> static representation of the business process. The OModel simply
> represents a binary compiled version of the BPEL process definition.
> So all I am suggesting is rather than store the binary version, we
> store the xml version, compile on first load, and the end result is an
> up-to-date OModel, regardless of the age of the process instance.
> 
> Regards
> Gary
> 
> On Tue, Mar 22, 2011 at 12:34 PM, Shivaraj Tenginakai
> <ts...@gmail.com> wrote:
>> Gary,
>>
>> Would it be possible to distinguish between execution optimization and
>> state? As long as state information can be derived from one version to
>> another, it should be fine to recompile (to achieve execution
>> optimization).
>>
>> Thanks,
>>
>> Shivaraj
>>
>>
>> On Tue, Mar 22, 2011 at 3:40 PM, Gary Brown <ga...@pi4tech.com> wrote:
>>> As discussed on the call today, we need a way to overcome the
>>> constraints of the serialized OModel which is produced when compiling
>>> the BPEL process.
>>>
>>> Although a more flexible serialization mechanism may provide a
>>> solution, as was also discussed, there will be cases where additional
>>> logic would be required to determine how an older representation
>>> should be evolved into a new representation.
>>>
>>> Another approach would be to use the BPEL process (xml) directly, so
>>> rather than using a binary representation compiled in advance, the
>>> runtime would simply load the BPEL DOM.
>>>
>>> If this required a change to the internal OModel mechanism, then it
>>> may be more work than the proposed dynamic OModel idea, however if we
>>> simply took the approach of "compile on load", then we get away from
>>> the issue, which is the persisted serialized form. We also no longer
>>> require any specific migration logic to move from older versions of
>>> the persisted representation (dynamic or not).
>>>
>>> So we could keep the existing OModel format, which can evolve as
>>> required as long as the compiler is kept in step, and we should no
>>> longer have any issues with long running processes.
>>>
>>> This change could be introduced into the current 1.3.x trunk without
>>> any backward compatibility issues.
>>>
>>> The only trade off is the speed difference between (1) loading the
>>> compiled representation versus (2) parsing the XML & compiling the
>>> BPEL process. If this would need to be performed multiple times in the
>>> same runtime, then a hash of the process could be used to cache the
>>> compiled version so this is only performed once per process
>>> definition.
>>>
>>> Thoughts?
>>>
>>> Regards
>>> Gary
>>>
>>

-- 
Tammo van Lessen - http://www.taval.de

Re: Evolvable OModel

[Gary Brown <ga...@pi4tech.com>]

Hi Shivaraj

As far as I am aware this issue is not related to state - only the
static representation of the business process. The OModel simply
represents a binary compiled version of the BPEL process definition.
So all I am suggesting is rather than store the binary version, we
store the xml version, compile on first load, and the end result is an
up-to-date OModel, regardless of the age of the process instance.

Regards
Gary

On Tue, Mar 22, 2011 at 12:34 PM, Shivaraj Tenginakai
<ts...@gmail.com> wrote:
> Gary,
>
> Would it be possible to distinguish between execution optimization and
> state? As long as state information can be derived from one version to
> another, it should be fine to recompile (to achieve execution
> optimization).
>
> Thanks,
>
> Shivaraj
>
>
> On Tue, Mar 22, 2011 at 3:40 PM, Gary Brown <ga...@pi4tech.com> wrote:
>> As discussed on the call today, we need a way to overcome the
>> constraints of the serialized OModel which is produced when compiling
>> the BPEL process.
>>
>> Although a more flexible serialization mechanism may provide a
>> solution, as was also discussed, there will be cases where additional
>> logic would be required to determine how an older representation
>> should be evolved into a new representation.
>>
>> Another approach would be to use the BPEL process (xml) directly, so
>> rather than using a binary representation compiled in advance, the
>> runtime would simply load the BPEL DOM.
>>
>> If this required a change to the internal OModel mechanism, then it
>> may be more work than the proposed dynamic OModel idea, however if we
>> simply took the approach of "compile on load", then we get away from
>> the issue, which is the persisted serialized form. We also no longer
>> require any specific migration logic to move from older versions of
>> the persisted representation (dynamic or not).
>>
>> So we could keep the existing OModel format, which can evolve as
>> required as long as the compiler is kept in step, and we should no
>> longer have any issues with long running processes.
>>
>> This change could be introduced into the current 1.3.x trunk without
>> any backward compatibility issues.
>>
>> The only trade off is the speed difference between (1) loading the
>> compiled representation versus (2) parsing the XML & compiling the
>> BPEL process. If this would need to be performed multiple times in the
>> same runtime, then a hash of the process could be used to cache the
>> compiled version so this is only performed once per process
>> definition.
>>
>> Thoughts?
>>
>> Regards
>> Gary
>>
>

Re: Evolvable OModel

[Shivaraj Tenginakai <ts...@gmail.com>]

Gary,

Would it be possible to distinguish between execution optimization and
state? As long as state information can be derived from one version to
another, it should be fine to recompile (to achieve execution
optimization).

Thanks,

Shivaraj


On Tue, Mar 22, 2011 at 3:40 PM, Gary Brown <ga...@pi4tech.com> wrote:
> As discussed on the call today, we need a way to overcome the
> constraints of the serialized OModel which is produced when compiling
> the BPEL process.
>
> Although a more flexible serialization mechanism may provide a
> solution, as was also discussed, there will be cases where additional
> logic would be required to determine how an older representation
> should be evolved into a new representation.
>
> Another approach would be to use the BPEL process (xml) directly, so
> rather than using a binary representation compiled in advance, the
> runtime would simply load the BPEL DOM.
>
> If this required a change to the internal OModel mechanism, then it
> may be more work than the proposed dynamic OModel idea, however if we
> simply took the approach of "compile on load", then we get away from
> the issue, which is the persisted serialized form. We also no longer
> require any specific migration logic to move from older versions of
> the persisted representation (dynamic or not).
>
> So we could keep the existing OModel format, which can evolve as
> required as long as the compiler is kept in step, and we should no
> longer have any issues with long running processes.
>
> This change could be introduced into the current 1.3.x trunk without
> any backward compatibility issues.
>
> The only trade off is the speed difference between (1) loading the
> compiled representation versus (2) parsing the XML & compiling the
> BPEL process. If this would need to be performed multiple times in the
> same runtime, then a hash of the process could be used to cache the
> compiled version so this is only performed once per process
> definition.
>
> Thoughts?
>
> Regards
> Gary
>