You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Matthew Hambley <ma...@aether.demon.co.uk> on 2005/07/05 07:06:32 UTC

Guaranteeing memory pool lifetime

As I understand the usage of memory pools in subversion all I can take for
granted is that the pool passed into a function will exist for the duration
of the call.  However I want to allocate some space which persists for the
lifetime of the client.  What pool should I use which has such a lifespan
and how would I access it?

-- 
(\/)atthew )-(ambley ---------------\ If something's worth doing it's worth
E-mail : matthew@aether.demon.co.uk  \ doing badly until you can learn to
Public key : C991137B                 \ do it well.
Web : http://www.aether.demon.co.uk/   \-----------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Guaranteeing memory pool lifetime

Posted by David James <ja...@gmail.com>.
On 05 Jul 2005 03:05:56 -0500, C. Michael Pilato <cm...@collab.net> wrote:
> Matthew Hambley <ma...@aether.demon.co.uk> writes:
> 
> > As I understand the usage of memory pools in subversion all I can take for
> > granted is that the pool passed into a function will exist for the duration
> > of the call.  However I want to allocate some space which persists for the
> > lifetime of the client.  What pool should I use which has such a lifespan
> > and how would I access it?
> 
> What you can take for granted is that any pool your code creates will
> not be destroyed by a Subversion API (unless the docstring for that
> API explicitly states so -- and I dunno of any that do).
Mike, Matthew, you have raised some interesting points which may
affect the SWIG bindings.

In the Ruby and Perl bindings, we automatically destroy pools when we
determine that they are no longer needed. I am hoping to also
implement this feature for Python. Is there an easy way to determine
whether a pool is OK to destroy? Currently, I am planning to destroy
pools as soon as all of the variables returned by functions which
access that pool have gone out of scope. Is this is a safe rule to
follow, in general?

Here is an example, in Ruby:
  ctx = Svn::Client::Context.new()
  ctx.copy(a,b)

When you call the Svn::Client::Context.new() function, we
automatically create a pool if one does not already exist. In the
above case, only one variable (ctx) is created by functions which use
the automatic pool. We therefore assume that it is safe to delete the
automatic pool when the "ctx" variable goes out of scope.

Cheers,

David


-- 
David James -- http://www.cs.toronto.edu/~james

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org


Re: Guaranteeing memory pool lifetime

Posted by "C. Michael Pilato" <cm...@collab.net>.
Matthew Hambley <ma...@aether.demon.co.uk> writes:

> As I understand the usage of memory pools in subversion all I can take for
> granted is that the pool passed into a function will exist for the duration
> of the call.  However I want to allocate some space which persists for the
> lifetime of the client.  What pool should I use which has such a lifespan
> and how would I access it?

What you can take for granted is that any pool your code creates will
not be destroyed by a Subversion API (unless the docstring for that
API explicitly states so -- and I dunno of any that do).  

An application generally keeps a single top-most pool around from
which to create short-lived subpools.  For example, this code from the
Subversion command-line client:

  /* Create our top-level pool.  Use a seperate mutexless allocator,
   * given this application is single threaded.
   */
  if (apr_allocator_create (&allocator))
    return EXIT_FAILURE;
  apr_allocator_max_free_set (allocator, SVN_ALLOCATOR_RECOMMENDED_MAX_FREE);
  pool = svn_pool_create_ex (NULL, allocator);
  apr_allocator_owner_set (allocator, pool);

The pool named "pool" here is the top-most pool used by the
command-line client.  It or subpools created from it using
"svn_pool_create (pool)" are passed to all the Subversion library
APIs.  Stuff that needs to live the during of the application is
allocated from that pool.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Guaranteeing memory pool lifetime

Posted by Greg Hudson <gh...@MIT.EDU>.
On Tue, 2005-07-05 at 20:41 +0100, Matthew Hambley wrote:
> > And how would you access the same map in subsequent calls?
> 
> Hold the hash as a static variable and call the function which contained it
> to retrieve the pointer.

Static variables are not thread-safe, and we cannot protect them with a
mutex because we do not have statically initialized mutexes.  We went
through this issue with character-set mappings and wound up with a new
function svn_utf_initialize() and poor performance if you don't call it.

The right answer is to create a context for MIME type mappings and make
the calling code pass the context to the guessing functions.  We
couldn't do that for character set translations because it would have
required churning almost our entire code base, but MIME type guessing is
probably only done in a few places.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Guaranteeing memory pool lifetime

Posted by Matthew Hambley <ma...@aether.demon.co.uk>.
In message <11...@equal-rites.mit.edu>
          Greg Hudson <gh...@MIT.EDU> wrote:

> On Tue, 2005-07-05 at 19:16 +0100, Matthew Hambley wrote:
> > I'm taking another look at issue 1233 and looking to take on board what
> > Helge Jensen was saying about caching the MIME map information.  This
> > requires that the map isn't freed between calls to the MIME type
> > guessing function.
> 
> And how would you access the same map in subsequent calls?

Hold the hash as a static variable and call the function which contained it
to retrieve the pointer.

-- 
(\/)atthew )-(ambley ---------------\ If something's worth doing it's worth
E-mail : matthew@aether.demon.co.uk  \ doing badly until you can learn to
Public key : C991137B                 \ do it well.
Web : http://www.aether.demon.co.uk/   \-----------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Guaranteeing memory pool lifetime

Posted by Greg Hudson <gh...@MIT.EDU>.
On Tue, 2005-07-05 at 19:16 +0100, Matthew Hambley wrote:
> I'm taking another look at issue 1233 and looking to take on board what
> Helge Jensen was saying about caching the MIME map information.  This
> requires that the map isn't freed between calls to the MIME type guessing
> function.

And how would you access the same map in subsequent calls?


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Guaranteeing memory pool lifetime

Posted by Matthew Hambley <ma...@aether.demon.co.uk>.
In message <11...@egyptian-gods.mit.edu>
          Greg Hudson <gh...@MIT.EDU> wrote:

> On Tue, 2005-07-05 at 08:06 +0100, Matthew Hambley wrote:
> > As I understand the usage of memory pools in subversion all I can take
> > for granted is that the pool passed into a function will exist for the
> > duration of the call.  However I want to allocate some space which
> > persists for the lifetime of the client.  What pool should I use which
> > has such a lifespan and how would I access it?
[snip]
> Why do you want to allocate space which persists for the lifetime of the
> client?

I'm taking another look at issue 1233 and looking to take on board what
Helge Jensen was saying about caching the MIME map information.  This
requires that the map isn't freed between calls to the MIME type guessing
function.

-- 
(\/)atthew )-(ambley ---------------\ If something's worth doing it's worth
E-mail : matthew@aether.demon.co.uk  \ doing badly until you can learn to
Public key : C991137B                 \ do it well.
Web : http://www.aether.demon.co.uk/   \-----------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Guaranteeing memory pool lifetime

Posted by Greg Hudson <gh...@MIT.EDU>.
On Tue, 2005-07-05 at 08:06 +0100, Matthew Hambley wrote:
> As I understand the usage of memory pools in subversion all I can take for
> granted is that the pool passed into a function will exist for the duration
> of the call.  However I want to allocate some space which persists for the
> lifetime of the client.  What pool should I use which has such a lifespan
> and how would I access it?

Pools in Subversion calls are also assumed to last as long as the caller
is interested in any data returned by the call.

The only way to get a pool with an unlimited lifespan is to call
svn_pool_create(NULL) to create a child of the global pool, and destroy
it explicitly when it is no longer needed.  That's how we handle
svn_error_t objects, for instance.

Why do you want to allocate space which persists for the lifetime of the
client?


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org