You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Bert Huijben <be...@qqmail.nl> on 2013/09/15 20:07:36 UTC
RE: svn commit: r1523465 - /subversion/trunk/subversion/svnserve/svnserve.c
> -----Original Message-----
> From: stefan2@apache.org [mailto:stefan2@apache.org]
> Sent: zondag 15 september 2013 19:47
> To: commits@subversion.apache.org
> Subject: svn commit: r1523465 -
> /subversion/trunk/subversion/svnserve/svnserve.c
>
> Author: stefan2
> Date: Sun Sep 15 17:46:36 2013
> New Revision: 1523465
>
> URL: http://svn.apache.org/r1523465
> Log:
> As it turns out, allocating memory from the OS in a multi-threaded
> environment is relatively costly. With APR pools, this happens
> every time we use a newly created root pool.
>
> Therefore, teach svnserve to recycle the connection pools, keeping
> those precious memory blocks allocated instead of disposing and
> re-allocating them.
Is this really the best way to do this?
Can't we create a subpool here? (Or do we also need multiple allocators, etc.)
In the implementation I see that the existing pools are re-used, but they are not *cleared* before re-use?
Shouldn't we at least release the used memory (and thate) when handing back the memory to the pool allocator?
Bert
RE: svn commit: r1523465 - /subversion/trunk/subversion/svnserve/svnserve.c
Posted by Bert Huijben <be...@qqmail.nl>.
Ah, I read the patch too fast.
In this snippet
[[
+/* Clear and release the given connection POOL.
+ */
+static void
+release_connection_pool(apr_pool_t *pool)
+{
+ svn_error_t *err;
+ svn_pool_clear(pool);
+
+ err = svn_mutex__lock(connection_pools_mutex);
+ if (err)
+ {
+ svn_error_clear(err);
+ svn_pool_destroy(pool);
+ }
+ else
+ {
+ APR_ARRAY_PUSH(connection_pools, apr_pool_t *) = pool;
+ svn_error_clear(svn_mutex__unlock(connection_pools_mutex,
+ SVN_NO_ERROR));
+ }
+}
+
]]
I automatically assumed the 'svn_pool_clear' was just a variable assignment,
because usually we have a white line before the first real code. instead of
just after the first line.
Maybe we should fix the whitespace here to follow our usual layout?
Bert
From: Stefan Fuhrmann [mailto:stefan.fuhrmann@wandisco.com]
Sent: zondag 15 september 2013 20:25
To: Bert Huijben
Cc: Subversion Development; commits@subversion.apache.org
Subject: Re: svn commit: r1523465 -
/subversion/trunk/subversion/svnserve/svnserve.c
On Sun, Sep 15, 2013 at 8:07 PM, Bert Huijben <bert@qqmail.nl
<ma...@qqmail.nl> > wrote:
> -----Original Message-----
> From: stefan2@apache.org <ma...@apache.org>
[mailto:stefan2@apache.org <ma...@apache.org> ]
> Sent: zondag 15 september 2013 19:47
> To: commits@subversion.apache.org <ma...@subversion.apache.org>
> Subject: svn commit: r1523465 -
> /subversion/trunk/subversion/svnserve/svnserve.c
>
> Author: stefan2
> Date: Sun Sep 15 17:46:36 2013
> New Revision: 1523465
>
> URL: http://svn.apache.org/r1523465
> Log:
> As it turns out, allocating memory from the OS in a multi-threaded
> environment is relatively costly. With APR pools, this happens
> every time we use a newly created root pool.
>
> Therefore, teach svnserve to recycle the connection pools, keeping
> those precious memory blocks allocated instead of disposing and
> re-allocating them.
Is this really the best way to do this?
Not sure. I'm open for suggestions.
Can't we create a subpool here? (Or do we also need multiple allocators,
etc.)
No. Those pools will be used concurrently by their
respective worker threads.
In the implementation I see that the existing pools are re-used, but they
are not *cleared* before re-use?
release_connection_pool() clears them.
Shouldn't we at least release the used memory (and thate) when handing back
the memory to the pool allocator?
-- Stefan^2.
RE: svn commit: r1523465 - /subversion/trunk/subversion/svnserve/svnserve.c
Posted by Bert Huijben <be...@qqmail.nl>.
Ah, I read the patch too fast.
In this snippet
[[
+/* Clear and release the given connection POOL.
+ */
+static void
+release_connection_pool(apr_pool_t *pool)
+{
+ svn_error_t *err;
+ svn_pool_clear(pool);
+
+ err = svn_mutex__lock(connection_pools_mutex);
+ if (err)
+ {
+ svn_error_clear(err);
+ svn_pool_destroy(pool);
+ }
+ else
+ {
+ APR_ARRAY_PUSH(connection_pools, apr_pool_t *) = pool;
+ svn_error_clear(svn_mutex__unlock(connection_pools_mutex,
+ SVN_NO_ERROR));
+ }
+}
+
]]
I automatically assumed the 'svn_pool_clear' was just a variable assignment,
because usually we have a white line before the first real code. instead of
just after the first line.
Maybe we should fix the whitespace here to follow our usual layout?
Bert
From: Stefan Fuhrmann [mailto:stefan.fuhrmann@wandisco.com]
Sent: zondag 15 september 2013 20:25
To: Bert Huijben
Cc: Subversion Development; commits@subversion.apache.org
Subject: Re: svn commit: r1523465 -
/subversion/trunk/subversion/svnserve/svnserve.c
On Sun, Sep 15, 2013 at 8:07 PM, Bert Huijben <bert@qqmail.nl
<ma...@qqmail.nl> > wrote:
> -----Original Message-----
> From: stefan2@apache.org <ma...@apache.org>
[mailto:stefan2@apache.org <ma...@apache.org> ]
> Sent: zondag 15 september 2013 19:47
> To: commits@subversion.apache.org <ma...@subversion.apache.org>
> Subject: svn commit: r1523465 -
> /subversion/trunk/subversion/svnserve/svnserve.c
>
> Author: stefan2
> Date: Sun Sep 15 17:46:36 2013
> New Revision: 1523465
>
> URL: http://svn.apache.org/r1523465
> Log:
> As it turns out, allocating memory from the OS in a multi-threaded
> environment is relatively costly. With APR pools, this happens
> every time we use a newly created root pool.
>
> Therefore, teach svnserve to recycle the connection pools, keeping
> those precious memory blocks allocated instead of disposing and
> re-allocating them.
Is this really the best way to do this?
Not sure. I'm open for suggestions.
Can't we create a subpool here? (Or do we also need multiple allocators,
etc.)
No. Those pools will be used concurrently by their
respective worker threads.
In the implementation I see that the existing pools are re-used, but they
are not *cleared* before re-use?
release_connection_pool() clears them.
Shouldn't we at least release the used memory (and thate) when handing back
the memory to the pool allocator?
-- Stefan^2.
Re: svn commit: r1523465 - /subversion/trunk/subversion/svnserve/svnserve.c
Posted by Stefan Fuhrmann <st...@wandisco.com>.
On Sun, Sep 15, 2013 at 8:07 PM, Bert Huijben <be...@qqmail.nl> wrote:
>
>
> > -----Original Message-----
> > From: stefan2@apache.org [mailto:stefan2@apache.org]
> > Sent: zondag 15 september 2013 19:47
> > To: commits@subversion.apache.org
> > Subject: svn commit: r1523465 -
> > /subversion/trunk/subversion/svnserve/svnserve.c
> >
> > Author: stefan2
> > Date: Sun Sep 15 17:46:36 2013
> > New Revision: 1523465
> >
> > URL: http://svn.apache.org/r1523465
> > Log:
> > As it turns out, allocating memory from the OS in a multi-threaded
> > environment is relatively costly. With APR pools, this happens
> > every time we use a newly created root pool.
> >
> > Therefore, teach svnserve to recycle the connection pools, keeping
> > those precious memory blocks allocated instead of disposing and
> > re-allocating them.
>
> Is this really the best way to do this?
>
Not sure. I'm open for suggestions.
> Can't we create a subpool here? (Or do we also need multiple allocators,
> etc.)
>
No. Those pools will be used concurrently by their
respective worker threads.
> In the implementation I see that the existing pools are re-used, but they
> are not *cleared* before re-use?
>
release_connection_pool() clears them.
> Shouldn't we at least release the used memory (and thate) when handing
> back the memory to the pool allocator?
>
-- Stefan^2.
Re: svn commit: r1523465 - /subversion/trunk/subversion/svnserve/svnserve.c
Posted by Stefan Fuhrmann <st...@wandisco.com>.
On Sun, Sep 15, 2013 at 8:07 PM, Bert Huijben <be...@qqmail.nl> wrote:
>
>
> > -----Original Message-----
> > From: stefan2@apache.org [mailto:stefan2@apache.org]
> > Sent: zondag 15 september 2013 19:47
> > To: commits@subversion.apache.org
> > Subject: svn commit: r1523465 -
> > /subversion/trunk/subversion/svnserve/svnserve.c
> >
> > Author: stefan2
> > Date: Sun Sep 15 17:46:36 2013
> > New Revision: 1523465
> >
> > URL: http://svn.apache.org/r1523465
> > Log:
> > As it turns out, allocating memory from the OS in a multi-threaded
> > environment is relatively costly. With APR pools, this happens
> > every time we use a newly created root pool.
> >
> > Therefore, teach svnserve to recycle the connection pools, keeping
> > those precious memory blocks allocated instead of disposing and
> > re-allocating them.
>
> Is this really the best way to do this?
>
Not sure. I'm open for suggestions.
> Can't we create a subpool here? (Or do we also need multiple allocators,
> etc.)
>
No. Those pools will be used concurrently by their
respective worker threads.
> In the implementation I see that the existing pools are re-used, but they
> are not *cleared* before re-use?
>
release_connection_pool() clears them.
> Shouldn't we at least release the used memory (and thate) when handing
> back the memory to the pool allocator?
>
-- Stefan^2.