Re: new mem-management for libsvn_fs?

[Karl Fogel <kf...@newton.ch.collab.net>]

Ben Collins-Sussman <su...@collab.net> writes:
> * every txn_body() routine changes signature:
> 
>      * txn_body(baton, trail) ==> txn_body(baton, trail, returnpool)
> 
>      * txn_body routines continue to let fs routines use trail->pool
>        for both scratchwork and return-data as usual, but before
>        exiting, makes *copies* of important data into the returnpool.
> 
>      * retry_txn() simply passes the main caller pool as the
>        'returnpool' to the txn_body routine.
> 
> * commit_txn() destroys trail->pool upon exit.
> 
> ==> only return-data is preserved;  all scratchwork is freed.

This sounds so sane, I could eat my socks.

> The real work here is going through every single txn_body_ routine,
> and teaching it to make 'deep' copies of return-data into the return
> pool.  Before I start this, I want Old Hundred Eyes to agree that this
> is a good approach.  :-)   cmpilato and I think it's the way to go.

These two eyes are crying with joy :-).

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: new mem-management for libsvn_fs?

[Karl Fogel <kf...@newton.ch.collab.net>]

Ben Collins-Sussman <su...@collab.net> writes:
> Actually, gstein has a much better method.
> 
> Get this:
> 
>   * create a 2nd pool in each trail called 'scratchpool'.
> 
>   * make sure that commit_trail() frees trail->scratchpool.
> 
>   * now slowly, leisurely, teach each dag function to use
>     trail->scratchpool for temporary work, and trail->pool for
>     return-data.

Wow, that is much better, yes.  Gstein, drinks on me at next
opportunity :-).

> This allows us to incrementally improve our memory usage, by rewriting
> dag routines one-at-a-time.... very carefully.

Amen.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: new mem-management for libsvn_fs?

[Branko Čibej <br...@xbc.nu>]

Ben Collins-Sussman wrote:

>Actually, gstein has a much better method.
>
>Get this:
>
>  * create a 2nd pool in each trail called 'scratchpool'.
>
>  * make sure that commit_trail() frees trail->scratchpool.
>
>  * now slowly, leisurely, teach each dag function to use
>    trail->scratchpool for temporary work, and trail->pool for
>    return-data.
>
>This allows us to incrementally improve our memory usage, by rewriting
>dag routines one-at-a-time.... very carefully.
>
>*This* solution makes mine eyes cry with joy.  :-)
>  
>
Greg wins, as usual. Many +1's from me!

    Brane


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: new mem-management for libsvn_fs?

[Nuutti Kotivuori <na...@iki.fi>]

Ben Collins-Sussman wrote:
> Karl Fogel <kf...@newton.ch.collab.net> writes:
>>> The real work here is going through every single txn_body_
>>> routine, and teaching it to make 'deep' copies of return-data
>>> into the return pool.  Before I start this, I want Old Hundred
>>> Eyes to agree that this is a good approach.  :-) cmpilato and I
>>> think it's the way to go.
>> 
>> These two eyes are crying with joy :-).
> 
> Actually, gstein has a much better method.
> 
> Get this:
> 
>   * create a 2nd pool in each trail called 'scratchpool'.
> 
>   * make sure that commit_trail() frees trail->scratchpool.
> 
>   * now slowly, leisurely, teach each dag function to use
>     trail->scratchpool for temporary work, and trail->pool for
>     return-data.
> 
> This allows us to incrementally improve our memory usage, by
> rewriting dag routines one-at-a-time.... very carefully.
> 
> *This* solution makes mine eyes cry with joy.  :-)

Actually - I was *just* yesterday thinking about these pool problems
and realized the exact same problem: return data should not exist in
the same pool as scratch data - because if it does and we still want
low memory usage, we get back to the "old" way where each function
allocating non-trivial amounts of temporary storage has to do a
subpool for non-return data and clear it at the end, which makes us
lose the benefits of the caller managed pool system.

So, I thought that perhaps two pools should be passed to each function
- but rejected that idea as too cumbersome :-)

But for these trails, it seems like a perfect fit. So yes, good stuff.

-- Naked

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: new mem-management for libsvn_fs?

[Ben Collins-Sussman <su...@collab.net>]

Karl Fogel <kf...@newton.ch.collab.net> writes:

> > The real work here is going through every single txn_body_ routine,
> > and teaching it to make 'deep' copies of return-data into the return
> > pool.  Before I start this, I want Old Hundred Eyes to agree that this
> > is a good approach.  :-)   cmpilato and I think it's the way to go.
> 
> These two eyes are crying with joy :-).

Actually, gstein has a much better method.

Get this:

  * create a 2nd pool in each trail called 'scratchpool'.

  * make sure that commit_trail() frees trail->scratchpool.

  * now slowly, leisurely, teach each dag function to use
    trail->scratchpool for temporary work, and trail->pool for
    return-data.

This allows us to incrementally improve our memory usage, by rewriting
dag routines one-at-a-time.... very carefully.

*This* solution makes mine eyes cry with joy.  :-)


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org