You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by "TROY.LIU 劉春偉" <TR...@DELTAWW.COM.CN> on 2013/05/30 03:44:58 UTC
apr_palloc is not thread safe
Dear all,
In our practice, we found two threads get same address returned by apr_palloc. It will happen about one hour later after our server starts.
We are using apr 1.4.5, the issue still exists in the latest subversion.
APR_DECLARE(void *) apr_palloc(apr_pool_t *pool, apr_size_t in_size)
{
…..
/* If the active node has enough bytes left, use it. */
if (size <= node_free_space(active)) {
mem = active->first_avail;
active->first_avail += size;
return mem;
}
……
}
Best Regards
Chunwei Liu
________________________________
*************************************************************************
This email message, including any attachments, is for the sole
use of the intended recipient(s) and may contain confidential and
privileged information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient, please
contact the sender by reply e-mail and destroy all copies of the original
message. [Delta Electronics, INC. China]
*************************************************************************
Re: apr_palloc is not thread safe
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 30, 2013 at 9:44 AM, TROY.LIU 劉春偉 <TR...@deltaww.com.cn> wrote:
> Dear all,
>
> In our practice, we found two threads get same address returned by
> apr_palloc. It will happen about one hour later after our server starts.
>
Please check whether you compile apr with thread support.
答复: apr_palloc is not thread safe
Posted by "TROY.LIU 劉春偉" <TR...@DELTAWW.COM.CN>.
It was my fault to misunderstood apr pool.
My ill design was "each thread will create a condition variable with a global pool".
Temporarily I put my apr_thread_cond_create into mutex lock, and my server has been running well for hours.
Next I prepare to let each thread create its own pool.
Thanks for your explanation.
-----邮件原件-----
发件人: Graham Leggett [mailto:minfrin@sharp.fm]
发送时间: 2013年5月31日 19:07
收件人: TROY.LIU 劉春偉
抄送: Philip Martin; dev@apr.apache.org; modules-dev@httpd.apache.org
主题: Re: apr_palloc is not thread safe
On 31 May 2013, at 6:15 AM, TROY.LIU 劉春偉 <TR...@DELTAWW.COM.CN> wrote:
> Thanks for reply, I got it. Now the problem is my APIs does not expose apr_pool_t to user like apr, so my APIs are not thread safe.
The typical way you use a pool is that pools are arranged logically in a hierarchy. So a process would have a pool, then one subpool of the process pool for each thread, and then one subpool per unit of work (for example a connection handled by the thread).
It is not the job of pools to dictate to you how you should arrange your pools, that is your job. Trying to add mutexes to an API that doesn't need to be threadsafe is a waste of resources, so it is left up to you to decide how your pool usage remains threadsafe.
In the above example, there is no need for any locks, because the per thread pool would only be used by that thread. You might have a pool representing a connection, and you might allow that connection to be handled by any thread, but as long as just one thread handles that connection at any given time you get away with not needing locks still.
It is only when the pool is attached to something that does span threads, such as a database connection pool, that you start needing locks, in which case use the mutex APIs to use the locks you need.
Again, APR gives you choice: do you use an in-process thread lock, or a machine wide process lock, which you choose is up to your application's needs.
Regards,
Graham
--
*************************************************************************
This email message, including any attachments, is for the sole
use of the intended recipient(s) and may contain confidential and
privileged information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient, please
contact the sender by reply e-mail and destroy all copies of the original
message. [Delta Electronics, INC. China]
*************************************************************************
Re: apr_palloc is not thread safe
Posted by Graham Leggett <mi...@sharp.fm>.
On 31 May 2013, at 6:15 AM, TROY.LIU 劉春偉 <TR...@DELTAWW.COM.CN> wrote:
> Thanks for reply, I got it. Now the problem is my APIs does not expose apr_pool_t to user like apr, so my APIs are not thread safe.
The typical way you use a pool is that pools are arranged logically in a hierarchy. So a process would have a pool, then one subpool of the process pool for each thread, and then one subpool per unit of work (for example a connection handled by the thread).
It is not the job of pools to dictate to you how you should arrange your pools, that is your job. Trying to add mutexes to an API that doesn't need to be threadsafe is a waste of resources, so it is left up to you to decide how your pool usage remains threadsafe.
In the above example, there is no need for any locks, because the per thread pool would only be used by that thread. You might have a pool representing a connection, and you might allow that connection to be handled by any thread, but as long as just one thread handles that connection at any given time you get away with not needing locks still.
It is only when the pool is attached to something that does span threads, such as a database connection pool, that you start needing locks, in which case use the mutex APIs to use the locks you need.
Again, APR gives you choice: do you use an in-process thread lock, or a machine wide process lock, which you choose is up to your application's needs.
Regards,
Graham
--
Re: apr_palloc is not thread safe
Posted by Thomas Eckert <th...@gmail.com>.
You do not need to expose pools to users through your API to make their
usage thread safe. Identify the spots which can trigger pool access and
wrap some thread safety mechanism around them, e.g. mutexes. APR does
supply you with good means to get your code thread safe - just use it ;-)
Look up
http://apr.apache.org/docs/apr/1.4/group__apr__proc__mutex.html
or
http://apr.apache.org/docs/apr/1.4/group___a_p_r___global_mutex.html
depending on what kind of thread safety you are interested in - within one
process or across process borders.
Cheers,
Thomas
On Fri, May 31, 2013 at 6:15 AM, TROY.LIU 劉春偉 <TR...@deltaww.com.cn>wrote:
> Thanks for reply, I got it. Now the problem is my APIs does not expose
> apr_pool_t to user like apr, so my APIs are not thread safe.
>
> Best Regards
> Chunwei Liu
>
> -----Original Message-----
> From: Philip Martin [mailto:codematters@ntlworld.com] On Behalf Of Philip
> Martin
> Sent: Thursday, May 30, 2013 17:22 troy
> To: TROY.LIU 劉春偉
> Cc: dev@apr.apache.org; modules-dev@httpd.apache.org
> Subject: Re: apr_palloc is not thread safe
>
> TROY.LIU 劉春偉 <TR...@DELTAWW.COM.CN> writes:
>
> > In our practice, we found two threads get same address returned by
> > apr_palloc. It will happen about one hour later after our server
> > starts. We are using apr 1.4.5, the issue still exists in the
> > latest subversion.
>
> From apr_pools.h:
>
> * Note that most operations on pools are not thread-safe: a single pool
> * should only be accessed by a single thread at any given time. The one
> * exception to this rule is creating a subpool of a given pool: one or
> more
> * threads can safely create subpools at the same time that another thread
> * accesses the parent pool.
>
> The pool system allows multiple threads to use multiple pools. There is
> no point trying to make apr_palloc "thread-safe" as the API is not designed
> to work that way. Even if apr_palloc was "thread-safe" how would
> apr_pool_clear work?
>
> A related discussion:
>
>
> http://mail-archives.apache.org/mod_mbox/apr-dev/201304.mbox/%3C792240597.462741.1366211671567.JavaMail.root@brainsware.org%3E
>
> --
> Philip
>
> *************************************************************************
> This email message, including any attachments, is for the sole
> use of the intended recipient(s) and may contain confidential and
> privileged information. Any unauthorized review, use, disclosure or
> distribution is prohibited. If you are not the intended recipient, please
> contact the sender by reply e-mail and destroy all copies of the original
> message. [Delta Electronics, INC. China]
> *************************************************************************
>
答复: apr_palloc is not thread safe
Posted by "TROY.LIU 劉春偉" <TR...@DELTAWW.COM.CN>.
No question now, I misunderstood the interface.
-----邮件原件-----
发件人: Branko Čibej [mailto:brane@apache.org]
发送时间: 2013年5月31日 17:37
收件人: dev@apr.apache.org
主题: Re: apr_palloc is not thread safe
On 31.05.2013 06:15, TROY.LIU 劉春偉 wrote:
> Thanks for reply, I got it. Now the problem is my APIs does not expose apr_pool_t to user like apr, so my APIs are not thread safe.
Excuse me; was that a question, an observation, or a complaint?
-- Brane
*************************************************************************
This email message, including any attachments, is for the sole
use of the intended recipient(s) and may contain confidential and
privileged information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient, please
contact the sender by reply e-mail and destroy all copies of the original
message. [Delta Electronics, INC. China]
*************************************************************************
Re: apr_palloc is not thread safe
Posted by Branko Čibej <br...@apache.org>.
On 31.05.2013 06:15, TROY.LIU 劉春偉 wrote:
> Thanks for reply, I got it. Now the problem is my APIs does not expose apr_pool_t to user like apr, so my APIs are not thread safe.
Excuse me; was that a question, an observation, or a complaint?
-- Brane
RE: apr_palloc is not thread safe
Posted by "TROY.LIU 劉春偉" <TR...@DELTAWW.COM.CN>.
Thanks for reply, I got it. Now the problem is my APIs does not expose apr_pool_t to user like apr, so my APIs are not thread safe.
Best Regards
Chunwei Liu
-----Original Message-----
From: Philip Martin [mailto:codematters@ntlworld.com] On Behalf Of Philip Martin
Sent: Thursday, May 30, 2013 17:22 troy
To: TROY.LIU 劉春偉
Cc: dev@apr.apache.org; modules-dev@httpd.apache.org
Subject: Re: apr_palloc is not thread safe
TROY.LIU 劉春偉 <TR...@DELTAWW.COM.CN> writes:
> In our practice, we found two threads get same address returned by
> apr_palloc. It will happen about one hour later after our server
> starts. We are using apr 1.4.5, the issue still exists in the
> latest subversion.
From apr_pools.h:
* Note that most operations on pools are not thread-safe: a single pool
* should only be accessed by a single thread at any given time. The one
* exception to this rule is creating a subpool of a given pool: one or more
* threads can safely create subpools at the same time that another thread
* accesses the parent pool.
The pool system allows multiple threads to use multiple pools. There is no point trying to make apr_palloc "thread-safe" as the API is not designed to work that way. Even if apr_palloc was "thread-safe" how would apr_pool_clear work?
A related discussion:
http://mail-archives.apache.org/mod_mbox/apr-dev/201304.mbox/%3C792240597.462741.1366211671567.JavaMail.root@brainsware.org%3E
--
Philip
*************************************************************************
This email message, including any attachments, is for the sole
use of the intended recipient(s) and may contain confidential and
privileged information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient, please
contact the sender by reply e-mail and destroy all copies of the original
message. [Delta Electronics, INC. China]
*************************************************************************
RE: apr_palloc is not thread safe
Posted by "TROY.LIU 劉春偉" <TR...@DELTAWW.COM.CN>.
Thanks for reply, I got it. Now the problem is my APIs does not expose apr_pool_t to user like apr, so my APIs are not thread safe.
Best Regards
Chunwei Liu
-----Original Message-----
From: Philip Martin [mailto:codematters@ntlworld.com] On Behalf Of Philip Martin
Sent: Thursday, May 30, 2013 17:22 troy
To: TROY.LIU 劉春偉
Cc: dev@apr.apache.org; modules-dev@httpd.apache.org
Subject: Re: apr_palloc is not thread safe
TROY.LIU 劉春偉 <TR...@DELTAWW.COM.CN> writes:
> In our practice, we found two threads get same address returned by
> apr_palloc. It will happen about one hour later after our server
> starts. We are using apr 1.4.5, the issue still exists in the
> latest subversion.
From apr_pools.h:
* Note that most operations on pools are not thread-safe: a single pool
* should only be accessed by a single thread at any given time. The one
* exception to this rule is creating a subpool of a given pool: one or more
* threads can safely create subpools at the same time that another thread
* accesses the parent pool.
The pool system allows multiple threads to use multiple pools. There is no point trying to make apr_palloc "thread-safe" as the API is not designed to work that way. Even if apr_palloc was "thread-safe" how would apr_pool_clear work?
A related discussion:
http://mail-archives.apache.org/mod_mbox/apr-dev/201304.mbox/%3C792240597.462741.1366211671567.JavaMail.root@brainsware.org%3E
--
Philip
*************************************************************************
This email message, including any attachments, is for the sole
use of the intended recipient(s) and may contain confidential and
privileged information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient, please
contact the sender by reply e-mail and destroy all copies of the original
message. [Delta Electronics, INC. China]
*************************************************************************
Re: apr_palloc is not thread safe
Posted by Philip Martin <ph...@codematters.co.uk>.
TROY.LIU 劉春偉 <TR...@DELTAWW.COM.CN> writes:
> In our practice, we found two threads get same address returned by
> apr_palloc. It will happen about one hour later after our server
> starts. We are using apr 1.4.5, the issue still exists in the
> latest subversion.
>From apr_pools.h:
* Note that most operations on pools are not thread-safe: a single pool
* should only be accessed by a single thread at any given time. The one
* exception to this rule is creating a subpool of a given pool: one or more
* threads can safely create subpools at the same time that another thread
* accesses the parent pool.
The pool system allows multiple threads to use multiple pools. There is
no point trying to make apr_palloc "thread-safe" as the API is not
designed to work that way. Even if apr_palloc was "thread-safe" how
would apr_pool_clear work?
A related discussion:
http://mail-archives.apache.org/mod_mbox/apr-dev/201304.mbox/%3C792240597.462741.1366211671567.JavaMail.root@brainsware.org%3E
--
Philip
Re: apr_palloc is not thread safe
Posted by Philip Martin <ph...@codematters.co.uk>.
TROY.LIU 劉春偉 <TR...@DELTAWW.COM.CN> writes:
> In our practice, we found two threads get same address returned by
> apr_palloc. It will happen about one hour later after our server
> starts. We are using apr 1.4.5, the issue still exists in the
> latest subversion.
>From apr_pools.h:
* Note that most operations on pools are not thread-safe: a single pool
* should only be accessed by a single thread at any given time. The one
* exception to this rule is creating a subpool of a given pool: one or more
* threads can safely create subpools at the same time that another thread
* accesses the parent pool.
The pool system allows multiple threads to use multiple pools. There is
no point trying to make apr_palloc "thread-safe" as the API is not
designed to work that way. Even if apr_palloc was "thread-safe" how
would apr_pool_clear work?
A related discussion:
http://mail-archives.apache.org/mod_mbox/apr-dev/201304.mbox/%3C792240597.462741.1366211671567.JavaMail.root@brainsware.org%3E
--
Philip