Counter-based continuation IDs (was Re: Load-testing flowscript webapps)

[Sylvain Wallez <sy...@apache.org>]

Reinhard Poetz wrote:

> Sylvain Wallez wrote:

<snip/>

>> My idea is to write a special implementation of ContinuationManager 
>> that would produce IDs based on a counter stored in a session 
>> (meaning each user has a different counter). That way, one can record 
>> a test scenario using JMeter in proxy mode, and replay it with no 
>> modifications.
>

<snip/>

> After some more thinking about this I remembered that we discussed to 
> put the continuations of a user into its session. This is also a 
> requirement to make Cocoon Flowscript apps capable of being clustered.
>
> <hint>Wouldn't this be nice feature</hint> :-)


As Gianugo says, there's a long way to go before being clusterizable 
(could be easier with javaflow than with JS flow though). But we can 
start the journey ;-)

Also, along with the simple request recording it allows, I see another 
advantage in this simpler continuation numbering scheme: better 
readability of produced pages. This will avoid these long hexadecimal 
strings that look ugly in the address bar, and allow better 
understanding of what's going on when developing applications (also 
useful when training people).

One could think of security problems such as continuation hijacking 
because of the predictability of continuation IDs (as opposed to the 
SecureRandom used today), but there's actually no problem since each 
session has its own continuation counter. An attacker would first have 
to hijack the session before accessing its continuations. And if the 
session is hijacked, were already doomed anyway.

So I will implement this new scheme and see how's life with simpler URLs ;-)

Sylvain

-- 
Sylvain Wallez                                  Anyware Technologies
http://www.apache.org/~sylvain           http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }

Re: Counter-based continuation IDs (was Re: Load-testing flowscript webapps)

[Sylvain Wallez <sy...@apache.org>]

Reinhard Poetz wrote:

> Sylvain Wallez wrote:


<snip/>

>> As Gianugo says, there's a long way to go before being clusterizable 
>> (could be easier with javaflow than with JS flow though). But we can 
>> start the journey ;-)
>
>
> What do you consider as the main blockers?


Well, as Gianugo said, serialization is the key here. And AFAICS, a 
Rhino continuation is linked to the compiled script it continues, and 
thus serializing a continuation is likely to pull out a large object graph.

Javaflow on the other hand will ontly serialize the stack frames and the 
objects involved in these frames, but not the bytecode of their classes!

But the most difficult point will be objects pointed to by the stack 
frames (function parameters and local variables) for which we do not 
have the equivalent of "transient" for class attributes. So either we 
must be very carefull about nulling them before a continuation is 
created, or implement in javaflow the equivalent to "catch(break)" and 
"catch(return)" that exists in rhino+cont.

Maybe this can be implemented using special exceptions such as 
ProgramHalted and ProgramRestored? Torsten, Stefan?

Sylvain

-- 
Sylvain Wallez                                  Anyware Technologies
http://www.apache.org/~sylvain           http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }

Re: Counter-based continuation IDs (was Re: Load-testing flowscript webapps)

[Reinhard Poetz <re...@apache.org>]

Vadim Gritsenko wrote:

> Reinhard Poetz wrote:
>
>> Vadim Gritsenko wrote:
>>
>>> Reinhard Poetz wrote:
>>>
>>>> Sylvain Wallez wrote:
>>>>
>>>>> As Gianugo says, there's a long way to go before being 
>>>>> clusterizable (could be easier with javaflow than with JS flow 
>>>>> though). But we can start the journey ;-)
>>>>
>>>>
>>>>
>>>> What do you consider as the main blockers?
>>>
>>>
>>>
>>> Here is Chris opinion:
>>>   http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=106704593729463
>>
>>
>>
>> Thank you.
>> I understand Chris' explanations except the part "A proper 
>> replication architecture would only propagate state changes between 
>> servers, not an entire copy of a state." What does it mean?
>
>
> He points out that transferring continuations as a whole in between 
> servers might be too expensive due to size of all local rhino 
> variables / java beans / etc used in continuation and suggests to copy 
> only objects which were modified from one continuation to another, to 
> save a bandwidth.
>
> Meaning, that if you have continuation A, which is already replicated 
> to two servers, and then continue its execution and end up with 
> continuation B, and during execution you have changed only variable X 
> (but not Y, Z), and session attribute X1 (but not Y1, Z1) - then you 
> need to copy only continuation B, variables X, and session attribute 
> X1 to the second server.
>
> Somehow I feel it is really hard to do this in general case. Suppose 
> that java bean X in continuation has a reference to Y. Then, during 
> copying of X, you will have to recognize that Y should not be copied - 
> but referenced; otherwise destination server will have two copies of 
> same java bean, Y... I wonder how j2ee vendors go around this issue 
> with session replication.


Yes I had the same idea that je22 container have to solve this problem 
too. The difference is that our various objects and flowscripts are put 
into single object in the session and this single object can be quite 
large. So we would need an extension that can inspect this object and do 
replication based on this replication. Doesn't sound easy and 
unfortunatly far beyond my knowledge :-(

-- 
Reinhard

Re: Counter-based continuation IDs (was Re: Load-testing flowscript webapps)

[Vadim Gritsenko <va...@reverycodes.com>]

Reinhard Poetz wrote:
> Vadim Gritsenko wrote:
> 
>> Reinhard Poetz wrote:
>>
>>> Sylvain Wallez wrote:
>>>
>>>> As Gianugo says, there's a long way to go before being clusterizable 
>>>> (could be easier with javaflow than with JS flow though). But we can 
>>>> start the journey ;-)
>>>
>>>
>>> What do you consider as the main blockers?
>>
>>
>> Here is Chris opinion:
>>   http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=106704593729463
> 
> 
> Thank you.
> I understand Chris' explanations except the part "A proper replication 
> architecture would only propagate state changes between servers, not an 
> entire copy of a state." What does it mean?

He points out that transferring continuations as a whole in between 
servers might be too expensive due to size of all local rhino variables 
/ java beans / etc used in continuation and suggests to copy only 
objects which were modified from one continuation to another, to save a 
bandwidth.

Meaning, that if you have continuation A, which is already replicated to 
two servers, and then continue its execution and end up with 
continuation B, and during execution you have changed only variable X 
(but not Y, Z), and session attribute X1 (but not Y1, Z1) - then you 
need to copy only continuation B, variables X, and session attribute X1 
to the second server.

Somehow I feel it is really hard to do this in general case. Suppose 
that java bean X in continuation has a reference to Y. Then, during 
copying of X, you will have to recognize that Y should not be copied - 
but referenced; otherwise destination server will have two copies of 
same java bean, Y... I wonder how j2ee vendors go around this issue with 
session replication.

Vadim

Re: Counter-based continuation IDs (was Re: Load-testing flowscript webapps)

[Reinhard Poetz <re...@apache.org>]

Vadim Gritsenko wrote:

> Reinhard Poetz wrote:
>
>> Sylvain Wallez wrote:
>>
>>> As Gianugo says, there's a long way to go before being clusterizable 
>>> (could be easier with javaflow than with JS flow though). But we can 
>>> start the journey ;-)
>>
>>
>>
>> What do you consider as the main blockers?
>
>
> Here is Chris opinion:
>   http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=106704593729463
>
>
> Vadim


Thank you.
I understand Chris' explanations except the part "A proper replication 
architecture would only propagate state changes between servers, not an 
entire copy of a state." What does it mean?

-- 
Reinhard

Re: Counter-based continuation IDs (was Re: Load-testing flowscript webapps)

[Vadim Gritsenko <va...@reverycodes.com>]

Reinhard Poetz wrote:

> Sylvain Wallez wrote:
> 
>> As Gianugo says, there's a long way to go before being clusterizable 
>> (could be easier with javaflow than with JS flow though). But we can 
>> start the journey ;-)
> 
> 
> What do you consider as the main blockers?

Here is Chris opinion:
   http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=106704593729463


Vadim

Re: Counter-based continuation IDs (was Re: Load-testing flowscript webapps)

[Reinhard Poetz <re...@apache.org>]

Sylvain Wallez wrote:

> Reinhard Poetz wrote:
>
>> Sylvain Wallez wrote:
>
>
> <snip/>
>
>>> My idea is to write a special implementation of ContinuationManager 
>>> that would produce IDs based on a counter stored in a session 
>>> (meaning each user has a different counter). That way, one can 
>>> record a test scenario using JMeter in proxy mode, and replay it 
>>> with no modifications.
>>
>>
>
> <snip/>
>
>> After some more thinking about this I remembered that we discussed to 
>> put the continuations of a user into its session. This is also a 
>> requirement to make Cocoon Flowscript apps capable of being clustered.
>>
>> <hint>Wouldn't this be nice feature</hint> :-)
>
>
>
> As Gianugo says, there's a long way to go before being clusterizable 
> (could be easier with javaflow than with JS flow though). But we can 
> start the journey ;-)


What do you consider as the main blockers?

>
> Also, along with the simple request recording it allows, I see another 
> advantage in this simpler continuation numbering scheme: better 
> readability of produced pages. This will avoid these long hexadecimal 
> strings that look ugly in the address bar, and allow better 
> understanding of what's going on when developing applications (also 
> useful when training people).
>
> One could think of security problems such as continuation hijacking 
> because of the predictability of continuation IDs (as opposed to the 
> SecureRandom used today), but there's actually no problem since each 
> session has its own continuation counter. An attacker would first have 
> to hijack the session before accessing its continuations. And if the 
> session is hijacked, were already doomed anyway.
>
> So I will implement this new scheme and see how's life with simpler 
> URLs ;-)
>
No objection ;-)

-- 
Reinhard