Re: [javaflow] Stack class

[Kohsuke Kawaguchi <kk...@kohsuke.org>]

Torsten Curdt wrote:
>> 3. Stack has a memory leak. Imagine a thread that suspends twice.  
>> If in the first suspend point a stack is very deep, the Stack  
>> object will have a lot of objects in it. When it suspends next,  
>> assume the stack frame is very shallow. When the new stack frame is  
>> captured into a Stack object, the region of the arrays that aren't  
>> used still point to objects written by the first Stack object. Can  
>> I change this?
> 
> Sorry, did not get that. Please elaborate.

There are two key characteristics in the current Stack class:

1. an array may get bigger, but it will never get smaller, even when a 
stack is copied, and even if the stack size shrinks.
2. popped objects won't be clobbered from an array.

Therefore, once an object is recorded to ostack or rstack, the only way 
for them to be removed is for the entry to be overwritten by another 
object, but this may never happen (the stack size may never grows back 
to the same size.) IOW, a Stack retain references to objects even after 
they are popped (this alone is OK, as popped objects are just copied 
into stack), and those unnecessary references get copied to successive 
Stack objects created from it. This causes memory leak for a long 
running "thread".

The fix is actually easy, which is to change the copy constructor of 
Stack so that it only copies the valid region of the parent array.



> Another idea is to record stack changes
> ..which would radically change the stack
> handling and the memory consumption.
> 
> Let's say you start with Continuation1
> and the full stack gets store in it.
> We now continue with Continuation2
> again we store the full stack!
> Instead we could store what has changed
> from the previous stack.
> 
> I am not totally sure whether this
> really makes sense and is worth the
> hassle ...would need some research.

The only algorithm I can think of to do this require a lot of additional 
local variables on each stack frame (6 ints to record the stack top 
while Continuation1 is being restored --- we can use this value while 
capturing Continuation2.)

I don't know if this cost is worth the benefit. If the memory foot print 
is a concern, maybe it's easier to capture all stack frames first and 
then find the common part later. I suspect we need more user experience.

It's also bit tricky because Stacks will share their tops, not their roots.

-- 
Kohsuke Kawaguchi

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org

Re: [javaflow] Stack class

[Kohsuke Kawaguchi <Ko...@Sun.COM>]

Torsten Curdt wrote:
> I know it's a bit tricky ...and
> whether it's worth the hassle is
> really the question.
> 
> But we can leave that for later ;)

Agreed.


-- 
Kohsuke Kawaguchi
Sun Microsystems                   kohsuke.kawaguchi@sun.com

Re: [javaflow] Stack class

[Torsten Curdt <tc...@apache.org>]

> There are two key characteristics in the current Stack class:
>
> 1. an array may get bigger, but it will never get smaller, even  
> when a stack is copied, and even if the stack size shrinks.
> 2. popped objects won't be clobbered from an array.
>
> Therefore, once an object is recorded to ostack or rstack, the only  
> way for them to be removed is for the entry to be overwritten by  
> another object, but this may never happen (the stack size may never  
> grows back to the same size.) IOW, a Stack retain references to  
> objects even after they are popped (this alone is OK, as popped  
> objects are just copied into stack), and those unnecessary  
> references get copied to successive Stack objects created from it.  
> This causes memory leak for a long running "thread".
>
> The fix is actually easy, which is to change the copy constructor  
> of Stack so that it only copies the valid region of the parent array.

Sure ...makes sense.
Good finding!

> The only algorithm I can think of to do this require a lot of  
> additional local variables on each stack frame (6 ints to record  
> the stack top while Continuation1 is being restored --- we can use  
> this value while capturing Continuation2.)
>
> I don't know if this cost is worth the benefit. If the memory foot  
> print is a concern, maybe it's easier to capture all stack frames  
> first and then find the common part later. I suspect we need more  
> user experience.
>
> It's also bit tricky because Stacks will share their tops, not  
> their roots.

I know it's a bit tricky ...and
whether it's worth the hassle is
really the question.

But we can leave that for later ;)

cheers
--
Torsten