You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Sander Striker <st...@apache.org> on 2002/06/28 13:28:33 UTC
[PATCH] MaxMemFree directive
Hi,
Since this patch introduces a new directive I'm presenting it here
first.
All MPMs that are creating their own allocator* (to get rid of
locking overhead), now have the MaxMemFree directive. MaxMemFree
allows you to set the maximum amount of memory (in kB), you want
the child allocator to hold on to. Anything over that threshold
is free()d back to the system. The default stays 'unlimited'.
Sander
/me just realized we have another magic value to get rid off and instead
do #define APR_ALLOCATOR_MAX_FREE_UNLIMITED 0, *sigh*
*) prefork, worker, beos, netware, threadpool and leader
Index: include/mpm_common.h
===================================================================
RCS file: /home/cvs/httpd-2.0/include/mpm_common.h,v
retrieving revision 1.37
diff -u -r1.37 mpm_common.h
--- include/mpm_common.h 23 May 2002 12:58:37 -0000 1.37
+++ include/mpm_common.h 28 Jun 2002 11:05:07 -0000
@@ -285,6 +285,13 @@
void ap_mpm_rewrite_args(process_rec *);
#endif
+#ifdef AP_MPM_WANT_SET_MAX_MEM_FREE
+extern apr_uint32_t ap_max_mem_free;
+const char *ap_mpm_set_max_mem_free(cmd_parms *cmd, void *dummy,
+ const char *arg);
+#endif
+
+
#ifdef __cplusplus
}
#endif
Index: server/core.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/core.c,v
retrieving revision 1.187
diff -u -r1.187 core.c
--- server/core.c 27 Jun 2002 05:18:19 -0000 1.187
+++ server/core.c 28 Jun 2002 11:05:19 -0000
@@ -3043,6 +3043,10 @@
AP_INIT_TAKE1("AcceptMutex", ap_mpm_set_accept_lock_mech, NULL, RSRC_CONF, \
ap_valid_accept_mutex_string),
#endif
+#ifdef AP_MPM_WANT_SET_MAX_MEM_FREE
+AP_INIT_TAKE1("MaxMemFree", ap_mpm_set_max_mem_free, NULL, RSRC_CONF,\
+ "Maximum number of 1k blocks a particular childs allocator may hold."),
+#endif
{ NULL }
};
Index: server/mpm_common.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm_common.c,v
retrieving revision 1.98
diff -u -r1.98 mpm_common.c
--- server/mpm_common.c 9 Jun 2002 03:44:03 -0000 1.98
+++ server/mpm_common.c 28 Jun 2002 11:05:21 -0000
@@ -880,3 +880,20 @@
#endif /* AP_MPM_WANT_SIGNAL_SERVER */
+#ifdef AP_MPM_WANT_SET_MAX_MEM_FREE
+apr_uint32_t ap_max_mem_free = 0;
+
+const char *ap_mpm_set_max_mem_free(cmd_parms *cmd, void *dummy,
+ const char *arg)
+{
+ const char *err = ap_check_cmd_context(cmd, GLOBAL_ONLY);
+ if (err != NULL) {
+ return err;
+ }
+
+ ap_max_mem_free = atoi(arg) * 1024;
+
+ return NULL;
+}
+
+#endif /* AP_MPM_WANT_SET_MAX_MEM_FREE */
Index: server/mpm/beos/beos.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/beos/beos.c,v
retrieving revision 1.96
diff -u -r1.96 beos.c
--- server/mpm/beos/beos.c 30 May 2002 00:21:27 -0000 1.96
+++ server/mpm/beos/beos.c 28 Jun 2002 11:05:24 -0000
@@ -383,6 +383,7 @@
sigprocmask(SIG_BLOCK, &sig_mask, NULL);
apr_allocator_create(&allocator);
+ apr_allocator_max_free_set(allocator, ap_max_mem_free);
apr_pool_create_ex(&ptrans, tpool, NULL, allocator);
apr_allocator_owner_set(allocator, ptrans);
Index: server/mpm/beos/mpm.h
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/beos/mpm.h,v
retrieving revision 1.13
diff -u -r1.13 mpm.h
--- server/mpm/beos/mpm.h 29 Mar 2002 16:15:33 -0000 1.13
+++ server/mpm/beos/mpm.h 28 Jun 2002 11:05:24 -0000
@@ -73,6 +73,7 @@
#define AP_MPM_WANT_SET_SCOREBOARD
#define AP_MPM_WANT_SET_MAX_REQUESTS
#define AP_MPM_WANT_SET_COREDUMPDIR
+#define AP_MPM_WANT_SET_MAX_MEM_FREE
extern int ap_max_child_assigned;
extern server_rec *ap_server_conf;
Index: server/mpm/experimental/leader/leader.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/experimental/leader/leader.c,v
retrieving revision 1.21
diff -u -r1.21 leader.c
--- server/mpm/experimental/leader/leader.c 7 Jun 2002 14:08:04 -0000 1.21
+++ server/mpm/experimental/leader/leader.c 28 Jun 2002 11:05:30 -0000
@@ -821,6 +821,7 @@
free(ti);
apr_allocator_create(&allocator);
+ apr_allocator_max_free_set(allocator, ap_max_mem_free);
apr_pool_create_ex(&ptrans, NULL, NULL, allocator);
apr_allocator_owner_set(allocator, ptrans);
bucket_alloc = apr_bucket_alloc_create(tpool);
Index: server/mpm/experimental/leader/mpm.h
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/experimental/leader/mpm.h,v
retrieving revision 1.4
diff -u -r1.4 mpm.h
--- server/mpm/experimental/leader/mpm.h 22 May 2002 01:04:59 -0000 1.4
+++ server/mpm/experimental/leader/mpm.h 28 Jun 2002 11:05:30 -0000
@@ -74,6 +74,7 @@
#define AP_MPM_WANT_SET_MAX_REQUESTS
#define AP_MPM_WANT_SET_COREDUMPDIR
#define AP_MPM_WANT_SET_ACCEPT_LOCK_MECH
+#define AP_MPM_WANT_SET_MAX_MEM_FREE
#define AP_MPM_DISABLE_NAGLE_ACCEPTED_SOCK
#define AP_MPM_USES_POD 1
Index: server/mpm/experimental/threadpool/mpm.h
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/experimental/threadpool/mpm.h,v
retrieving revision 1.2
diff -u -r1.2 mpm.h
--- server/mpm/experimental/threadpool/mpm.h 6 Jun 2002 23:58:14 -0000 1.2
+++ server/mpm/experimental/threadpool/mpm.h 28 Jun 2002 11:05:30 -0000
@@ -75,6 +75,7 @@
#define AP_MPM_WANT_SET_COREDUMPDIR
#define AP_MPM_WANT_SET_ACCEPT_LOCK_MECH
#define AP_MPM_WANT_SIGNAL_SERVER
+#define AP_MPM_WANT_SET_MAX_MEM_FREE
#define AP_MPM_DISABLE_NAGLE_ACCEPTED_SOCK
#define MPM_CHILD_PID(i) (ap_scoreboard_image->parent[i].pid)
Index: server/mpm/experimental/threadpool/threadpool.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/experimental/threadpool/threadpool.c,v
retrieving revision 1.11
diff -u -r1.11 threadpool.c
--- server/mpm/experimental/threadpool/threadpool.c 7 Jun 2002 14:08:04 -0000 1.11
+++ server/mpm/experimental/threadpool/threadpool.c 28 Jun 2002 11:05:37 -0000
@@ -1045,6 +1045,7 @@
ap_update_child_status_from_indexes(process_slot, thread_slot, SERVER_STARTING, NULL);
apr_allocator_create(&allocator);
+ apr_allocator_max_free_set(allocator, ap_max_mem_free);
apr_pool_create_ex(&ptrans, NULL, NULL, allocator);
apr_allocator_owner_set(allocator, ptrans);
Index: server/mpm/netware/mpm.h
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/netware/mpm.h,v
retrieving revision 1.5
diff -u -r1.5 mpm.h
--- server/mpm/netware/mpm.h 29 Mar 2002 16:15:33 -0000 1.5
+++ server/mpm/netware/mpm.h 28 Jun 2002 11:05:37 -0000
@@ -72,6 +72,7 @@
#define AP_MPM_WANT_SET_LOCKFILE
*/
#define AP_MPM_WANT_SET_MAX_REQUESTS
+#define AP_MPM_WANT_SET_MAX_MEM_FREE
#define AP_MPM_DISABLE_NAGLE_ACCEPTED_SOCK
/*#define AP_MPM_WANT_SET_COREDUMPDIR
#define AP_MPM_WANT_SET_ACCEPT_LOCK_MECH
Index: server/mpm/netware/mpm_netware.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/netware/mpm_netware.c,v
retrieving revision 1.55
diff -u -r1.55 mpm_netware.c
--- server/mpm/netware/mpm_netware.c 26 Jun 2002 21:53:25 -0000 1.55
+++ server/mpm/netware/mpm_netware.c 28 Jun 2002 11:05:40 -0000
@@ -373,6 +373,7 @@
tv.tv_usec = 0;
apr_allocator_create(&allocator);
+ apr_allocator_max_free_set(allocator, ap_max_mem_free);
apr_pool_create_ex(&ptrans, NULL, NULL, allocator);
apr_allocator_owner_set(allocator, ptrans);
apr_pool_tag(ptrans, "transaction");
Index: server/mpm/prefork/mpm.h
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/prefork/mpm.h,v
retrieving revision 1.20
diff -u -r1.20 mpm.h
--- server/mpm/prefork/mpm.h 23 May 2002 12:58:37 -0000 1.20
+++ server/mpm/prefork/mpm.h 28 Jun 2002 11:05:40 -0000
@@ -78,6 +78,7 @@
#define AP_MPM_WANT_SET_COREDUMPDIR
#define AP_MPM_WANT_SET_ACCEPT_LOCK_MECH
#define AP_MPM_WANT_SIGNAL_SERVER
+#define AP_MPM_WANT_SET_MAX_FREE
#define AP_MPM_DISABLE_NAGLE_ACCEPTED_SOCK
#define AP_MPM_USES_POD 1
Index: server/mpm/prefork/prefork.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/prefork/prefork.c,v
retrieving revision 1.267
diff -u -r1.267 prefork.c
--- server/mpm/prefork/prefork.c 14 Jun 2002 03:11:43 -0000 1.267
+++ server/mpm/prefork/prefork.c 28 Jun 2002 11:05:44 -0000
@@ -558,6 +558,7 @@
* we can have cleanups occur when the child exits.
*/
apr_allocator_create(&allocator);
+ apr_allocator_max_free_set(allocator, ap_max_mem_free);
apr_pool_create_ex(&pchild, pconf, NULL, allocator);
apr_allocator_owner_set(allocator, pchild);
Index: server/mpm/worker/mpm.h
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/worker/mpm.h,v
retrieving revision 1.13
diff -u -r1.13 mpm.h
--- server/mpm/worker/mpm.h 23 May 2002 12:58:37 -0000 1.13
+++ server/mpm/worker/mpm.h 28 Jun 2002 11:05:44 -0000
@@ -75,6 +75,7 @@
#define AP_MPM_WANT_SET_COREDUMPDIR
#define AP_MPM_WANT_SET_ACCEPT_LOCK_MECH
#define AP_MPM_WANT_SIGNAL_SERVER
+#define AP_MPM_WANT_SET_MAX_MEM_FREE
#define AP_MPM_DISABLE_NAGLE_ACCEPTED_SOCK
#define MPM_CHILD_PID(i) (ap_scoreboard_image->parent[i].pid)
Index: server/mpm/worker/worker.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/worker/worker.c,v
retrieving revision 1.128
diff -u -r1.128 worker.c
--- server/mpm/worker/worker.c 7 Jun 2002 14:08:05 -0000 1.128
+++ server/mpm/worker/worker.c 28 Jun 2002 11:05:50 -0000
@@ -796,6 +796,7 @@
apr_allocator_t *allocator;
apr_allocator_create(&allocator);
+ apr_allocator_max_free_set(allocator, ap_max_mem_free);
apr_pool_create_ex(&ptrans, NULL, NULL, allocator);
apr_allocator_owner_set(allocator, ptrans);
}
RE: [PATCH] MaxMemFree directive
Posted by Sander Striker <st...@apache.org>.
> From: Justin Erenkrantz [mailto:jerenkrantz@apache.org]
> Sent: 30 June 2002 09:51
> On Fri, Jun 28, 2002 at 01:28:33PM +0200, Sander Striker wrote:
>> All MPMs that are creating their own allocator* (to get rid of
>> locking overhead), now have the MaxMemFree directive. MaxMemFree
>> allows you to set the maximum amount of memory (in kB), you want
>> the child allocator to hold on to. Anything over that threshold
>> is free()d back to the system. The default stays 'unlimited'.
>
> +1 in concept.
>
>> /me just realized we have another magic value to get rid off and instead
>> do #define APR_ALLOCATOR_MAX_FREE_UNLIMITED 0, *sigh*
>
> That'd be nice to add when you cleanup the allocator stuff to
> make the types opaque. =)
*grunt* *grmbl* ;)
I'll do that before 2.0.40, but have something else on my plate right
now.
>> +#ifdef AP_MPM_WANT_SET_MAX_MEM_FREE
>> +apr_uint32_t ap_max_mem_free = 0;
>> +
>> +const char *ap_mpm_set_max_mem_free(cmd_parms *cmd, void *dummy,
>> + const char *arg)
>> +{
>> + const char *err = ap_check_cmd_context(cmd, GLOBAL_ONLY);
>> + if (err != NULL) {
>> + return err;
>> + }
>> +
>> + ap_max_mem_free = atoi(arg) * 1024;
>> +
>> + return NULL;
>> +}
>
> My one caveat would be to enforce that arg is positive.
> atoi() could return a negative, but it'll get casted to an
> unsigned 32-bit quantity.
Ack. Will fix.
> Or does the config core code reject negative numbers straight out?
> (Way too lazy to check myself.) I think you'd be safer if you used
> strtol (or better yet strtoul - strtoul is in C89, so that should be
> everywhere). -- justin
I went to try this and got a compiler error because
'strtoul_is_not_a_portable_function_use_strtol_instead' doesn't exist.
A quick grep on strtoul gave me:
/opt/src/build/httpd-2.0/CHANGES:8778: *) Remove strtoul() use from mod_proxy because it isn't available
/opt/src/build/httpd-2.0/include/httpd.h:1684:/** strtoul does not exist on sunos4. */
/opt/src/build/httpd-2.0/include/httpd.h:1685:#ifdef strtoul
/opt/src/build/httpd-2.0/include/httpd.h:1686:#undef strtoul
/opt/src/build/httpd-2.0/include/httpd.h:1688:#define strtoul strtoul_is_not_a_portable_function_use_strtol_instead
/opt/src/build/httpd-2.0/server/core.c:2471: /* WTF: If strtoul is not portable, then write a replacement.
/opt/src/build/httpd-2.0/server/.#core.c.1.187:2471: /* WTF: If strtoul is not portable, then write a replacement.
/opt/src/build/httpd-2.0/srclib/pcre/pcretest.c:464: ((size_offsets = (int)strtoul(argv[op+1], &endptr, 10)), *endptr == 0))
So, to summarize:
- PCRE uses it
- At other places we complain because we can't use strtoul because of the define
- There is no apr_strtoul, which would make sense if it isn't portable
- C89 is pretty ancient... does sunos still not implement it?
Sander
PS. Haven't heard any objections, so I've committed MemMaxFree.
Re: [PATCH] MaxMemFree directive
Posted by Justin Erenkrantz <je...@apache.org>.
On Fri, Jun 28, 2002 at 01:28:33PM +0200, Sander Striker wrote:
> All MPMs that are creating their own allocator* (to get rid of
> locking overhead), now have the MaxMemFree directive. MaxMemFree
> allows you to set the maximum amount of memory (in kB), you want
> the child allocator to hold on to. Anything over that threshold
> is free()d back to the system. The default stays 'unlimited'.
+1 in concept.
> /me just realized we have another magic value to get rid off and instead
> do #define APR_ALLOCATOR_MAX_FREE_UNLIMITED 0, *sigh*
That'd be nice to add when you cleanup the allocator stuff to
make the types opaque. =)
> +#ifdef AP_MPM_WANT_SET_MAX_MEM_FREE
> +apr_uint32_t ap_max_mem_free = 0;
> +
> +const char *ap_mpm_set_max_mem_free(cmd_parms *cmd, void *dummy,
> + const char *arg)
> +{
> + const char *err = ap_check_cmd_context(cmd, GLOBAL_ONLY);
> + if (err != NULL) {
> + return err;
> + }
> +
> + ap_max_mem_free = atoi(arg) * 1024;
> +
> + return NULL;
> +}
My one caveat would be to enforce that arg is positive.
atoi() could return a negative, but it'll get casted to an
unsigned 32-bit quantity.
Or does the config core code reject negative numbers straight out?
(Way too lazy to check myself.) I think you'd be safer if you used
strtol (or better yet strtoul - strtoul is in C89, so that should be
everywhere). -- justin