You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Jason Gionta <jj...@ncsu.edu> on 2011/11/11 20:09:09 UTC
Server Process Pool Behavior
Hi all,
I am developing a module in which I need to create a cache for the duration
of the server process. This cache will basically keep state information in
which requests will query and write too. As a test, I have implemented a
simple cache using a hash table in which I read the count of the hash table
and write that count into the hash table, there by incrementing the count
by one. I allocate the memory for the hash table using the server process
pool as this memory should have the life time of the process.
While I expect the count to increment by one after each request. It seems
like if there is a time gap between requests (over 10 seconds), the count
gets reset (apr_hash_count = 0). I have no idea how or why the hash table
is seemingly being emptied as I only increment and insert with a new key
and value. Can someone shed some light on this issue? Is there a process
pool timeout? I have compiled apache without threads to make things easier
(as with threads gave me even more unexpected results)
Here is the basic code for the cache.... Thanks for any help!
static void *http_create_server_config(apr_pool_t *p,server_rec *s)
{
http_config *pconfig=ap_get_module_config(s->module_config,
&http_module);
if(pconfig!=NULL)
return pconfig;
pconfig=apr_pcalloc(p,sizeof(http_config));
pconfig->enabled=0;
pconfig->server = s;
apr_status_t rv;
apr_pool_t * pchild = s->process->pool;
/* Derive our own pool from pchild */
rv = apr_pool_create(&pconfig->test_pool, pchild);
if (rv != APR_SUCCESS) {
ap_log_perror(APLOG_MARK, APLOG_CRIT, rv, pchild,
"Failed to create subpool for my_module");
}
/* Set up a thread mutex for when we need to manipulate the cache */
#if APR_HAS_THREADS
rv = apr_thread_mutex_create(&pconfig->test_mutex,
APR_THREAD_MUTEX_DEFAULT, pchild);
if (rv != APR_SUCCESS) {
ap_log_perror(APLOG_MARK, APLOG_CRIT, rv, pchild,
"Failed to create mutex for my_module");
}
#endif
/* Finally, create the cache itself (and prime it if applicable) */
pconfig->test_cache = apr_hash_make(pconfig->test_pool);
return pconfig;
}
static apr_status_t * test_cache(request_rec * r)
{
apr_status_t rv;
http_config *svr =
ap_get_module_config(r->server->module_config, &http_module);
const char *key;
char * tempkey=NULL;
int val;
#if APR_HAS_THREADS
/* If it isn't cached, we need to compute it and save it to
* the cache. That's a critical section, so we need the mutex.
*/
rv = apr_thread_mutex_lock(svr->test_mutex);
if (rv != APR_SUCCESS) {
ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r,
"Failed to acquire thread mutex");
return HTTP_SERVICE_UNAVAILABLE;
}
/* In case of a race condition between cache lookup and
* obtaining the lock, perform the lookup again.
* Not a performance problem unless this happens a lot.
*/
val = apr_hash_count(svr->test_cache);
key = apr_itoa(svr->test_pool, val);
tempkey = apr_hash_get(svr->test_cache, key, APR_HASH_KEY_STRING);
if (tempkey == NULL) {
/* OK, we really do need to compute it */
apr_hash_set(svr->test_cache, key, APR_HASH_KEY_STRING, key);
}
else
{
ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r,
"TEST_CACHE: FAILED! %s", tempkey);
return APR_ERELATIVE;
}
#else
/* No threads = no risk of a race condition. Just set it. */
val = apr_hash_count(svr->test_cache);
key = apr_itoa(svr->test_pool, val);
apr_hash_set(svr->test_cache, key, APR_HASH_KEY_STRING, key);
#endif
#if APR_HAS_THREADS
rv = apr_thread_mutex_unlock(svr->test_mutex);
if (rv != APR_SUCCESS) {
/* Something is seriously wrong. We need to log it,
* but it doesn't -– of itself -– invalidate this request.
*/
ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r,
"Failed to release thread mutex");
return rv;
}
#endif
ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r,
"TEST_CACHE: COUNT=%d", val);
return APR_SUCCESS;
}
static apr_status_t my_filter(ap_filter_t *f, apr_bucket_brigade *bb)
{
test_cache(f->r);
}
--
Jason Gionta
North Carolina State University
jjgionta@ncsu.edu
Re: Server Process Pool Behavior
Posted by Jason Gionta <jj...@ncsu.edu>.
I believe that is what is happening. The hash table still exists because
the new requests will count. But the hash table seems to be empty after
the keepalive timeout expires.
Do I have a misunderstanding of how the server process pool handles memory?
I will even being to bring to light this observation is for the most simple
test where the page being requested is static html that triggers no
referrer requests.
When I load a page that contains further resources (image, css, etc), the
counts seem to diverge within the single page request with simultaneous
requests for resources. (Multiple requests being handled at the same time)
Any help is much appreciated.
Thanks,
Jason
On Fri, Nov 11, 2011 at 2:16 PM, Jeff Trawick <tr...@gmail.com> wrote:
> On Fri, Nov 11, 2011 at 11:09 AM, Jason Gionta <jj...@ncsu.edu> wrote:
> > Hi all,
> > I am developing a module in which I need to create a cache for the
> duration
> > of the server process. This cache will basically keep state information
> in
> > which requests will query and write too. As a test, I have implemented a
> > simple cache using a hash table in which I read the count of the hash
> table
> > and write that count into the hash table, there by incrementing the
> count by
> > one. I allocate the memory for the hash table using the server process
> pool
> > as this memory should have the life time of the process.
> > While I expect the count to increment by one after each request. It
> seems
> > like if there is a time gap between requests (over 10 seconds), the count
> > gets reset (apr_hash_count = 0). I have no idea how or why the hash
> table
> > is seemingly being emptied as I only increment and insert with a new key
> and
> > value. Can someone shed some light on this issue? Is there a process
> pool
> > timeout? I have compiled apache without threads to make things easier
> (as
> > with threads gave me even more unexpected results)
>
> Once keepalive timeout expires the client reconnects and a different
> child process handles that connection, and its hash is still in the
> initial state?
>
> > Here is the basic code for the cache.... Thanks for any help!
> > static void *http_create_server_config(apr_pool_t *p,server_rec *s)
> > {
> > http_config *pconfig=ap_get_module_config(s->module_config,
> > &http_module);
> > if(pconfig!=NULL)
> > return pconfig;
> >
> > pconfig=apr_pcalloc(p,sizeof(http_config));
> > pconfig->enabled=0;
> > pconfig->server = s;
> > apr_status_t rv;
> > apr_pool_t * pchild = s->process->pool;
> > /* Derive our own pool from pchild */
> > rv = apr_pool_create(&pconfig->test_pool, pchild);
> > if (rv != APR_SUCCESS) {
> > ap_log_perror(APLOG_MARK, APLOG_CRIT, rv, pchild,
> > "Failed to create subpool for my_module");
> > }
> > /* Set up a thread mutex for when we need to manipulate the cache */
> > #if APR_HAS_THREADS
> > rv = apr_thread_mutex_create(&pconfig->test_mutex,
> > APR_THREAD_MUTEX_DEFAULT, pchild);
> > if (rv != APR_SUCCESS) {
> > ap_log_perror(APLOG_MARK, APLOG_CRIT, rv, pchild,
> > "Failed to create mutex for my_module");
> > }
> > #endif
> > /* Finally, create the cache itself (and prime it if applicable) */
> > pconfig->test_cache = apr_hash_make(pconfig->test_pool);
> >
> > return pconfig;
> > }
> >
> > static apr_status_t * test_cache(request_rec * r)
> > {
> > apr_status_t rv;
> > http_config *svr =
> > ap_get_module_config(r->server->module_config, &http_module);
> > const char *key;
> > char * tempkey=NULL;
> > int val;
> > #if APR_HAS_THREADS
> > /* If it isn't cached, we need to compute it and save it to
> > * the cache. That's a critical section, so we need the mutex.
> > */
> > rv = apr_thread_mutex_lock(svr->test_mutex);
> > if (rv != APR_SUCCESS) {
> > ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r,
> > "Failed to acquire thread mutex");
> > return HTTP_SERVICE_UNAVAILABLE;
> > }
> > /* In case of a race condition between cache lookup and
> > * obtaining the lock, perform the lookup again.
> > * Not a performance problem unless this happens a lot.
> > */
> > val = apr_hash_count(svr->test_cache);
> > key = apr_itoa(svr->test_pool, val);
> > tempkey = apr_hash_get(svr->test_cache, key, APR_HASH_KEY_STRING);
> > if (tempkey == NULL) {
> > /* OK, we really do need to compute it */
> > apr_hash_set(svr->test_cache, key, APR_HASH_KEY_STRING, key);
> > }
> > else
> > {
> > ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r,
> > "TEST_CACHE: FAILED! %s", tempkey);
> > return APR_ERELATIVE;
> > }
> > #else
> > /* No threads = no risk of a race condition. Just set it. */
> > val = apr_hash_count(svr->test_cache);
> > key = apr_itoa(svr->test_pool, val);
> > apr_hash_set(svr->test_cache, key, APR_HASH_KEY_STRING, key);
> > #endif
> > #if APR_HAS_THREADS
> > rv = apr_thread_mutex_unlock(svr->test_mutex);
> > if (rv != APR_SUCCESS) {
> > /* Something is seriously wrong. We need to log it,
> > * but it doesn't -– of itself -– invalidate this request.
> > */
> > ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r,
> > "Failed to release thread mutex");
> > return rv;
> > }
> > #endif
> > ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r,
> > "TEST_CACHE: COUNT=%d", val);
> > return APR_SUCCESS;
> > }
> > static apr_status_t my_filter(ap_filter_t *f, apr_bucket_brigade *bb)
> > {
> > test_cache(f->r);
> > }
> > --
> > Jason Gionta
> > North Carolina State University
> > jjgionta@ncsu.edu
> >
>
>
>
> --
> Born in Roswell... married an alien...
>
--
Jason Gionta
Cyber Defense Lab
North Carolina State University
jjgionta@ncsu.edu
Re: Server Process Pool Behavior
Posted by Jeff Trawick <tr...@gmail.com>.
On Fri, Nov 11, 2011 at 11:09 AM, Jason Gionta <jj...@ncsu.edu> wrote:
> Hi all,
> I am developing a module in which I need to create a cache for the duration
> of the server process. This cache will basically keep state information in
> which requests will query and write too. As a test, I have implemented a
> simple cache using a hash table in which I read the count of the hash table
> and write that count into the hash table, there by incrementing the count by
> one. I allocate the memory for the hash table using the server process pool
> as this memory should have the life time of the process.
> While I expect the count to increment by one after each request. It seems
> like if there is a time gap between requests (over 10 seconds), the count
> gets reset (apr_hash_count = 0). I have no idea how or why the hash table
> is seemingly being emptied as I only increment and insert with a new key and
> value. Can someone shed some light on this issue? Is there a process pool
> timeout? I have compiled apache without threads to make things easier (as
> with threads gave me even more unexpected results)
Once keepalive timeout expires the client reconnects and a different
child process handles that connection, and its hash is still in the
initial state?
> Here is the basic code for the cache.... Thanks for any help!
> static void *http_create_server_config(apr_pool_t *p,server_rec *s)
> {
> http_config *pconfig=ap_get_module_config(s->module_config,
> &http_module);
> if(pconfig!=NULL)
> return pconfig;
>
> pconfig=apr_pcalloc(p,sizeof(http_config));
> pconfig->enabled=0;
> pconfig->server = s;
> apr_status_t rv;
> apr_pool_t * pchild = s->process->pool;
> /* Derive our own pool from pchild */
> rv = apr_pool_create(&pconfig->test_pool, pchild);
> if (rv != APR_SUCCESS) {
> ap_log_perror(APLOG_MARK, APLOG_CRIT, rv, pchild,
> "Failed to create subpool for my_module");
> }
> /* Set up a thread mutex for when we need to manipulate the cache */
> #if APR_HAS_THREADS
> rv = apr_thread_mutex_create(&pconfig->test_mutex,
> APR_THREAD_MUTEX_DEFAULT, pchild);
> if (rv != APR_SUCCESS) {
> ap_log_perror(APLOG_MARK, APLOG_CRIT, rv, pchild,
> "Failed to create mutex for my_module");
> }
> #endif
> /* Finally, create the cache itself (and prime it if applicable) */
> pconfig->test_cache = apr_hash_make(pconfig->test_pool);
>
> return pconfig;
> }
>
> static apr_status_t * test_cache(request_rec * r)
> {
> apr_status_t rv;
> http_config *svr =
> ap_get_module_config(r->server->module_config, &http_module);
> const char *key;
> char * tempkey=NULL;
> int val;
> #if APR_HAS_THREADS
> /* If it isn't cached, we need to compute it and save it to
> * the cache. That's a critical section, so we need the mutex.
> */
> rv = apr_thread_mutex_lock(svr->test_mutex);
> if (rv != APR_SUCCESS) {
> ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r,
> "Failed to acquire thread mutex");
> return HTTP_SERVICE_UNAVAILABLE;
> }
> /* In case of a race condition between cache lookup and
> * obtaining the lock, perform the lookup again.
> * Not a performance problem unless this happens a lot.
> */
> val = apr_hash_count(svr->test_cache);
> key = apr_itoa(svr->test_pool, val);
> tempkey = apr_hash_get(svr->test_cache, key, APR_HASH_KEY_STRING);
> if (tempkey == NULL) {
> /* OK, we really do need to compute it */
> apr_hash_set(svr->test_cache, key, APR_HASH_KEY_STRING, key);
> }
> else
> {
> ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r,
> "TEST_CACHE: FAILED! %s", tempkey);
> return APR_ERELATIVE;
> }
> #else
> /* No threads = no risk of a race condition. Just set it. */
> val = apr_hash_count(svr->test_cache);
> key = apr_itoa(svr->test_pool, val);
> apr_hash_set(svr->test_cache, key, APR_HASH_KEY_STRING, key);
> #endif
> #if APR_HAS_THREADS
> rv = apr_thread_mutex_unlock(svr->test_mutex);
> if (rv != APR_SUCCESS) {
> /* Something is seriously wrong. We need to log it,
> * but it doesn't -– of itself -– invalidate this request.
> */
> ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r,
> "Failed to release thread mutex");
> return rv;
> }
> #endif
> ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r,
> "TEST_CACHE: COUNT=%d", val);
> return APR_SUCCESS;
> }
> static apr_status_t my_filter(ap_filter_t *f, apr_bucket_brigade *bb)
> {
> test_cache(f->r);
> }
> --
> Jason Gionta
> North Carolina State University
> jjgionta@ncsu.edu
>
--
Born in Roswell... married an alien...
Re: Server Process Pool Behavior
Posted by Sander Temme <sc...@apache.org>.
On Nov 11, 2011, at 1:21 PM, Jeff Trawick wrote:
> On Fri, Nov 11, 2011 at 1:02 PM, Jeff Trawick <tr...@gmail.com> wrote:
>> 2011/11/11 Torsten Förtsch <to...@gmx.net>:
>>> On Friday, 11 November 2011 20:09:09 Jason Gionta wrote:
>>>> While I expect the count to increment by one after each request. It
>>>> seems like if there is a time gap between requests (over 10 seconds),
>>>> the count gets reset (apr_hash_count = 0).
>>>
>>> May it be that your requests simply hit another worker process?
>>
>> That must be it.
>>
>> Jason, your code is going to run in different child processes as new
>> client connections are accepted by different child processes. The
>> children started off with the same empty hash inherited over fork(),
>> and in different child processes it will have different contents, at
>> least once a child handles its first connection.
>
> Also, note that in-memory caches served between child processes use
> shared memory.
Check out mod_example_ipc.c in trunk and the new 2.4 branch.
Note on that... should we port that to slotmem?
S.
> You can get a bit of mileage in the short term from configuring a
> single-child-process configuration (worker or event MPM,
> MaxClients==ThreadsPerChild) but that probably won't be viable
> (different reasons why you want more child processes, don't want to
> lose your cache if the child segfaults or you need to use
> MaxRequestsPerChild, etc.).
Re: Server Process Pool Behavior
Posted by Jeff Trawick <tr...@gmail.com>.
On Fri, Nov 11, 2011 at 1:02 PM, Jeff Trawick <tr...@gmail.com> wrote:
> 2011/11/11 Torsten Förtsch <to...@gmx.net>:
>> On Friday, 11 November 2011 20:09:09 Jason Gionta wrote:
>>> While I expect the count to increment by one after each request. It
>>> seems like if there is a time gap between requests (over 10 seconds),
>>> the count gets reset (apr_hash_count = 0).
>>
>> May it be that your requests simply hit another worker process?
>
> That must be it.
>
> Jason, your code is going to run in different child processes as new
> client connections are accepted by different child processes. The
> children started off with the same empty hash inherited over fork(),
> and in different child processes it will have different contents, at
> least once a child handles its first connection.
Also, note that in-memory caches served between child processes use
shared memory.
You can get a bit of mileage in the short term from configuring a
single-child-process configuration (worker or event MPM,
MaxClients==ThreadsPerChild) but that probably won't be viable
(different reasons why you want more child processes, don't want to
lose your cache if the child segfaults or you need to use
MaxRequestsPerChild, etc.).
Re: Server Process Pool Behavior
Posted by Jeff Trawick <tr...@gmail.com>.
2011/11/11 Torsten Förtsch <to...@gmx.net>:
> On Friday, 11 November 2011 20:09:09 Jason Gionta wrote:
>> While I expect the count to increment by one after each request. It
>> seems like if there is a time gap between requests (over 10 seconds),
>> the count gets reset (apr_hash_count = 0).
>
> May it be that your requests simply hit another worker process?
That must be it.
Jason, your code is going to run in different child processes as new
client connections are accepted by different child processes. The
children started off with the same empty hash inherited over fork(),
and in different child processes it will have different contents, at
least once a child handles its first connection.
Re: Server Process Pool Behavior
Posted by Torsten Förtsch <to...@gmx.net>.
On Friday, 11 November 2011 20:09:09 Jason Gionta wrote:
> While I expect the count to increment by one after each request. It
> seems like if there is a time gap between requests (over 10 seconds),
> the count gets reset (apr_hash_count = 0).
May it be that your requests simply hit another worker process?
Torsten Förtsch
--
Need professional modperl support? Hire me! (http://foertsch.name)
Like fantasy? http://kabatinte.net