You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Aaron Bannert <aa...@clove.org> on 2002/02/07 20:57:09 UTC
[PATCH] new scoreboard creation logic, remove DEFAULT_SCOREBOARD from MPMs
This is the patch I promised a couple weeks ago. It changes the way
we create the scoreboard to be more friendly with 3rd party apps/processes
that want access to the scoreboard while maintaining the ability to
have an anonymous scoreboard. The logic was discussed on-list a week
or two ago, but I'll reproduce it here anyway:
if config specifies ScoreBoardFile
create name-based shared memory, errors are fatal
else /* we get to choose */
create anonymous shared memory
if ENOTIMPL
create name-based shared memory from DEFAULT_SCOREBOARD
else
errors are fatal
I had to modify the MPMs so they wouldn't try to set ap_scoreboard_fname
any more. This #define is now fully owned by the scoreboard.c file.
(Might we want to namespace-protect that #define? I don't know.)
I'm posting this here for feedback because it is a big change and could
use some testing on other platform/MPM combos, but I'd also like to
wait until the current release process finishes.
-aaron
p.s. this removes 99% of the #ifdef WIN32 stuff from scoreboard.c.
Index: server/scoreboard.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/scoreboard.c,v
retrieving revision 1.55
diff -u -u -r1.55 scoreboard.c
--- server/scoreboard.c 1 Feb 2002 18:07:08 -0000 1.55
+++ server/scoreboard.c 7 Feb 2002 19:46:52 -0000
@@ -161,6 +161,31 @@
ap_assert(more_storage == (char*)shared_score + scoreboard_size);
}
+/**
+ * Create a name-based scoreboard in the given pool using the
+ * given filename.
+ */
+static apr_status_t create_namebased_scoreboard(apr_pool_t *pool,
+ const char *fname)
+{
+#if APR_HAS_SHARED_MEMORY
+ apr_status_t rv;
+
+ /* The shared memory file must not exist before we create the
+ * segment. */
+ apr_file_remove(fname, pool); /* ignore errors */
+
+ rv = apr_shm_create(&ap_scoreboard_shm, scoreboard_size, fname, pool);
+ if (rv != APR_SUCCESS) {
+ ap_log_error(APLOG_MARK, APLOG_CRIT, rv, NULL,
+ "unable to create scoreboard "
+ "(name-based shared memory failure)");
+ return rv;
+ }
+#endif /* APR_HAS_SHARED_MEMORY */
+ return APR_SUCCESS;
+}
+
/* ToDo: This function should be made to handle setting up
* a scoreboard shared between processes using any IPC technique,
* not just a shared memory segment
@@ -183,35 +208,32 @@
return rv;
}
-#ifndef WIN32
- rv = apr_shm_create(&ap_scoreboard_shm, scoreboard_size, fname,
- global_pool);
- if ((rv != APR_SUCCESS) && (rv != APR_ENOTIMPL)) {
- ap_log_error(APLOG_MARK, APLOG_CRIT, rv, NULL,
- "Fatal error: could not create scoreboard "
- "(using anonymous shared memory)");
- return rv;
+ /* The config says to create a name-based shmem */
+ if (ap_scoreboard_fname) {
+ /* make sure it's an absolute pathname */
+ fname = ap_server_root_relative(pconf, ap_scoreboard_fname);
+
+ return create_namebased_scoreboard(global_pool, fname);
}
- if (rv == APR_ENOTIMPL) {
-#else
- {
-#endif
- if (ap_scoreboard_fname) {
- fname = ap_server_root_relative(global_pool, ap_scoreboard_fname);
- /* make sure the file doesn't exist before trying
- * to create the segment. */
- apr_file_remove(fname, global_pool);
- }
- rv = apr_shm_create(&ap_scoreboard_shm, scoreboard_size, fname,
- global_pool);
- if (rv != APR_SUCCESS) {
+ else { /* config didn't specify, we get to choose shmem type */
+ rv = apr_shm_create(&ap_scoreboard_shm, scoreboard_size, NULL,
+ global_pool); /* anonymous shared memory */
+ if ((rv != APR_SUCCESS) && (rv != APR_ENOTIMPL)) {
ap_log_error(APLOG_MARK, APLOG_CRIT, rv, NULL,
- "Fatal error: could not open(create) scoreboard");
+ "Unable to create scoreboard "
+ "(anonymous shared memory failure)");
return rv;
}
+ /* Make up a filename and do name-based shmem */
+ else if (rv == APR_ENOTIMPL) {
+ /* Make sure it's an absolute pathname */
+ ap_scoreboard_fname = DEFAULT_SCOREBOARD;
+ fname = ap_server_root_relative(pconf, ap_scoreboard_fname);
+
+ return create_namebased_scoreboard(global_pool, fname);
+ }
}
- /* everything will be cleared shortly */
-#endif
+#endif /* APR_HAS_SHARED_MEMORY */
return APR_SUCCESS;
}
Index: include/scoreboard.h
===================================================================
RCS file: /home/cvs/httpd-2.0/include/scoreboard.h,v
retrieving revision 1.39
diff -u -u -r1.39 scoreboard.h
--- include/scoreboard.h 30 Jan 2002 22:35:56 -0000 1.39
+++ include/scoreboard.h 7 Feb 2002 19:46:53 -0000
@@ -58,6 +58,7 @@
#ifndef APACHE_SCOREBOARD_H
#define APACHE_SCOREBOARD_H
+
#ifdef __cplusplus
extern "C" {
#endif
@@ -74,6 +75,11 @@
#include "apr_thread_proc.h"
#include "apr_portable.h"
#include "apr_shm.h"
+
+/* Scoreboard file, if there is one */
+#ifndef DEFAULT_SCOREBOARD
+#define DEFAULT_SCOREBOARD "logs/apache_runtime_status"
+#endif
/* Scoreboard info on a process is, for now, kept very brief ---
* just status value and pid (the latter so that the caretaker process
Index: server/mpm/beos/beos.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/beos/beos.c,v
retrieving revision 1.83
diff -u -u -r1.83 beos.c
--- server/mpm/beos/beos.c 1 Feb 2002 22:16:31 -0000 1.83
+++ server/mpm/beos/beos.c 7 Feb 2002 19:46:56 -0000
@@ -762,8 +762,6 @@
pconf = _pconf;
ap_server_conf = s;
- ap_scoreboard_fname = DEFAULT_SCOREBOARD;
-
/* Increase the available pool of fd's. This code from
* Joe Kloss <jo...@be.com>
*/
@@ -1032,7 +1030,6 @@
ap_thread_limit = HARD_THREAD_LIMIT;
ap_pid_fname = DEFAULT_PIDLOG;
ap_max_requests_per_thread = DEFAULT_MAX_REQUESTS_PER_THREAD;
- ap_scoreboard_fname = DEFAULT_SCOREBOARD;
apr_cpystrn(ap_coredump_dir, ap_server_root, sizeof(ap_coredump_dir));
Index: server/mpm/beos/mpm_default.h
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/beos/mpm_default.h,v
retrieving revision 1.6
diff -u -u -r1.6 mpm_default.h
--- server/mpm/beos/mpm_default.h 18 Dec 2001 13:48:53 -0000 1.6
+++ server/mpm/beos/mpm_default.h 7 Feb 2002 19:46:56 -0000
@@ -101,11 +101,6 @@
#define DEFAULT_PIDLOG "logs/httpd.pid"
#endif
-/* Scoreboard file, if there is one */
-#ifndef DEFAULT_SCOREBOARD
-#define DEFAULT_SCOREBOARD "logs/apache_runtime_status"
-#endif
-
/*
* Interval, in microseconds, between scoreboard maintenance.
*/
Index: server/mpm/netware/mpm_netware.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/netware/mpm_netware.c,v
retrieving revision 1.32
diff -u -u -r1.32 mpm_netware.c
--- server/mpm/netware/mpm_netware.c 6 Feb 2002 21:13:57 -0000 1.32
+++ server/mpm/netware/mpm_netware.c 7 Feb 2002 19:46:59 -0000
@@ -970,7 +970,6 @@
ap_threads_min_free = DEFAULT_MIN_FREE_THREADS;
ap_threads_max_free = DEFAULT_MAX_FREE_THREADS;
ap_threads_limit = HARD_THREAD_LIMIT;
- ap_scoreboard_fname = DEFAULT_SCOREBOARD;
ap_max_requests_per_child = DEFAULT_MAX_REQUESTS_PER_CHILD;
ap_extended_status = 0;
Index: server/mpm/netware/mpm_default.h
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/netware/mpm_default.h,v
retrieving revision 1.5
diff -u -u -r1.5 mpm_default.h
--- server/mpm/netware/mpm_default.h 19 Dec 2001 16:23:34 -0000 1.5
+++ server/mpm/netware/mpm_default.h 7 Feb 2002 19:46:59 -0000
@@ -122,11 +122,6 @@
#endif
*/
-/* Scoreboard file, if there is one */
-#ifndef DEFAULT_SCOREBOARD
-#define DEFAULT_SCOREBOARD "logs/apache_runtime_status"
-#endif
-
/* Where the main/parent process's pid is logged */
/*#ifndef DEFAULT_PIDLOG
#define DEFAULT_PIDLOG "logs/httpd.pid"
Index: server/mpm/perchild/perchild.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/perchild/perchild.c,v
retrieving revision 1.108
diff -u -u -r1.108 perchild.c
--- server/mpm/perchild/perchild.c 5 Feb 2002 22:18:49 -0000 1.108
+++ server/mpm/perchild/perchild.c 7 Feb 2002 19:47:04 -0000
@@ -1493,7 +1493,6 @@
max_spare_threads = DEFAULT_MAX_SPARE_THREAD;
max_threads = thread_limit;
ap_pid_fname = DEFAULT_PIDLOG;
- ap_scoreboard_fname = DEFAULT_SCOREBOARD;
ap_lock_fname = DEFAULT_LOCKFILE;
max_requests_per_child = DEFAULT_MAX_REQUESTS_PER_CHILD;
curr_child_num = 0;
Index: server/mpm/perchild/mpm_default.h
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/perchild/mpm_default.h,v
retrieving revision 1.8
diff -u -u -r1.8 mpm_default.h
--- server/mpm/perchild/mpm_default.h 18 Dec 2001 13:48:53 -0000 1.8
+++ server/mpm/perchild/mpm_default.h 7 Feb 2002 19:47:05 -0000
@@ -91,11 +91,6 @@
#define DEFAULT_LOCKFILE "logs/accept.lock"
#endif
-/* Scoreboard file, if there is one */
-#ifndef DEFAULT_SCOREBOARD
-#define DEFAULT_SCOREBOARD "logs/apache_runtime_status"
-#endif
-
/* Where the main/parent process's pid is logged */
#ifndef DEFAULT_PIDLOG
#define DEFAULT_PIDLOG "logs/httpd.pid"
Index: server/mpm/prefork/prefork.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/prefork/prefork.c,v
retrieving revision 1.239
diff -u -u -r1.239 prefork.c
--- server/mpm/prefork/prefork.c 4 Feb 2002 18:41:46 -0000 1.239
+++ server/mpm/prefork/prefork.c 7 Feb 2002 19:47:08 -0000
@@ -1247,7 +1247,6 @@
ap_daemons_max_free = DEFAULT_MAX_FREE_DAEMON;
ap_daemons_limit = server_limit;
ap_pid_fname = DEFAULT_PIDLOG;
- ap_scoreboard_fname = DEFAULT_SCOREBOARD;
ap_lock_fname = DEFAULT_LOCKFILE;
ap_max_requests_per_child = DEFAULT_MAX_REQUESTS_PER_CHILD;
ap_extended_status = 0;
Index: server/mpm/prefork/mpm_default.h
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/prefork/mpm_default.h,v
retrieving revision 1.7
diff -u -u -r1.7 mpm_default.h
--- server/mpm/prefork/mpm_default.h 18 Dec 2001 13:48:53 -0000 1.7
+++ server/mpm/prefork/mpm_default.h 7 Feb 2002 19:47:08 -0000
@@ -85,11 +85,6 @@
#define DEFAULT_LOCKFILE "logs/accept.lock"
#endif
-/* Scoreboard file, if there is one */
-#ifndef DEFAULT_SCOREBOARD
-#define DEFAULT_SCOREBOARD "logs/apache_runtime_status"
-#endif
-
/* Where the main/parent process's pid is logged */
#ifndef DEFAULT_PIDLOG
#define DEFAULT_PIDLOG "logs/httpd.pid"
Index: server/mpm/worker/worker.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/worker/worker.c,v
retrieving revision 1.72
diff -u -u -r1.72 worker.c
--- server/mpm/worker/worker.c 5 Feb 2002 23:17:22 -0000 1.72
+++ server/mpm/worker/worker.c 7 Feb 2002 19:47:13 -0000
@@ -1544,7 +1544,6 @@
ap_daemons_limit = server_limit;
ap_threads_per_child = DEFAULT_THREADS_PER_CHILD;
ap_pid_fname = DEFAULT_PIDLOG;
- ap_scoreboard_fname = DEFAULT_SCOREBOARD;
ap_lock_fname = DEFAULT_LOCKFILE;
ap_max_requests_per_child = DEFAULT_MAX_REQUESTS_PER_CHILD;
ap_extended_status = 0;
Index: server/mpm/worker/mpm_default.h
===================================================================
RCS file: /home/cvs/httpd-2.0/server/mpm/worker/mpm_default.h,v
retrieving revision 1.3
diff -u -u -r1.3 mpm_default.h
--- server/mpm/worker/mpm_default.h 18 Dec 2001 13:48:54 -0000 1.3
+++ server/mpm/worker/mpm_default.h 7 Feb 2002 19:47:13 -0000
@@ -89,11 +89,6 @@
#define DEFAULT_LOCKFILE "logs/accept.lock"
#endif
-/* Scoreboard file, if there is one */
-#ifndef DEFAULT_SCOREBOARD
-#define DEFAULT_SCOREBOARD "logs/apache_runtime_status"
-#endif
-
/* Where the main/parent process's pid is logged */
#ifndef DEFAULT_PIDLOG
#define DEFAULT_PIDLOG "logs/httpd.pid"
Re: [PATCH] new scoreboard creation logic, remove DEFAULT_SCOREBOARD from MPMs
Posted by Aaron Bannert <aa...@clove.org>.
On Fri, Feb 15, 2002 at 01:50:31AM -0800, Adam Sussman wrote:
> This works well with perfork and worker under Linux. I have a couple
> of comments though:
>
> 1) There are some not infrequent cases I have run into where apache
> needs to be killed (for unrelated reasons) and the shared memory
> segment does not get cleaned up. When this happens, you can't restart
> the server. You get a "file exists" error and apache refuses to start
> up. This is easy to fix with ipcrm but the error is confusing and
> does not make the solution obvious.
>
> It would be nice if apache would clean up the shared memory segement
> if it sees it, or have a more meaningfull error.
The problem here is that there's a way to automatically destroy a segment
when all the processes die that had it attached, but as soon as you mark
the segment for auto-death no new processes can attach to it.
This is obviously a side effect of a larger lingering process problem
in apache, but it is still a legitimate problem, IMHO. [To get rid of
at least one of those lingering process problems, grab HEAD from CVS.]
We could add a new routine to apr_shm.h in APR for preremoving a segment;
it could attach to a segment to see if it exists, set it for removal,
then detach. Then if the EXCL creation fails then there is another
process still running.
> 2) Unless I am missing something, there does not seem to be an easy way
> for an external application accessing the scoreboard to know how to
> navigate the data structure. You have to know the server limit and
> thread limit or else you run into problems. It would be nice to
> be able to derive those values from the scoreboard image instead of
> the httpd.conf file.
That may get us into a chicken-and-egg -- IIRC the scoreboard size is
proportional to the server limit and thread limit. :) I do agree that we
need a better way to access the structure of the scoreboard.
-aaron
RE: [PATCH] new scoreboard creation logic, remove DEFAULT_SCOREBOARD from MPMs
Posted by Ryan Bloom <rb...@covalent.net>.
> This works well with perfork and worker under Linux. I have a couple
> of comments though:
>
> 1) There are some not infrequent cases I have run into where apache
> needs to be killed (for unrelated reasons) and the shared memory
> segment does not get cleaned up. When this happens, you can't
restart
> the server. You get a "file exists" error and apache refuses to
start
> up. This is easy to fix with ipcrm but the error is confusing and
> does not make the solution obvious.
>
> It would be nice if apache would clean up the shared memory
segement
> if it sees it, or have a more meaningfull error.
The last time I looked at the scoreboard code, we tried very hard to
clean up after ourselves, by registering a cleanup to be called when the
process exited. If we aren't doing that, or the cleanup isn't being
called, that is a bug. If the parent is segfaulting, and thus not
calling the cleanup, there isn't much more we can do. :-(
> 2) Unless I am missing something, there does not seem to be an easy
way
> for an external application accessing the scoreboard to know how to
> navigate the data structure. You have to know the server limit and
> thread limit or else you run into problems. It would be nice to
> be able to derive those values from the scoreboard image instead of
> the httpd.conf file.
Those values should be found with the ap_mpm_query function. If you
can't get them out of that function, they should be added to it.
Ryan
Re: [PATCH] new scoreboard creation logic, remove DEFAULT_SCOREBOARD from MPMs
Posted by Adam Sussman <my...@vishnu.vidya.com>.
> I had to modify the MPMs so they wouldn't try to set ap_scoreboard_fname
> any more. This #define is now fully owned by the scoreboard.c file.
> (Might we want to namespace-protect that #define? I don't know.)
>
> I'm posting this here for feedback because it is a big change and could
> use some testing on other platform/MPM combos, but I'd also like to
> wait until the current release process finishes.
>
This works well with perfork and worker under Linux. I have a couple
of comments though:
1) There are some not infrequent cases I have run into where apache
needs to be killed (for unrelated reasons) and the shared memory
segment does not get cleaned up. When this happens, you can't restart
the server. You get a "file exists" error and apache refuses to start
up. This is easy to fix with ipcrm but the error is confusing and
does not make the solution obvious.
It would be nice if apache would clean up the shared memory segement
if it sees it, or have a more meaningfull error.
2) Unless I am missing something, there does not seem to be an easy way
for an external application accessing the scoreboard to know how to
navigate the data structure. You have to know the server limit and
thread limit or else you run into problems. It would be nice to
be able to derive those values from the scoreboard image instead of
the httpd.conf file.
-adam
Re: [PATCH] new scoreboard creation logic, remove DEFAULT_SCOREBOARD from MPMs
Posted by Justin Erenkrantz <je...@ebuilt.com>.
On Thu, Feb 07, 2002 at 11:57:09AM -0800, Aaron Bannert wrote:
> This is the patch I promised a couple weeks ago. It changes the way
> we create the scoreboard to be more friendly with 3rd party apps/processes
> that want access to the scoreboard while maintaining the ability to
> have an anonymous scoreboard. The logic was discussed on-list a week
> or two ago, but I'll reproduce it here anyway:
>
> if config specifies ScoreBoardFile
> create name-based shared memory, errors are fatal
> else /* we get to choose */
> create anonymous shared memory
> if ENOTIMPL
> create name-based shared memory from DEFAULT_SCOREBOARD
> else
> errors are fatal
+1. Works fine here on prefork/Linux. -- justin