You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Rodent of Unusual Size <Ke...@Golux.Com> on 2002/10/24 05:45:19 UTC
[STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
APACHE 2.0 STATUS: -*-text-*-
Last modified at [$Date: 2002/10/22 17:01:44 $]
Release:
2.0.44 : in development
2.0.43 : rolled October 2, 2002
2.0.42 : released September 24, 2002 as GA.
2.0.41 : rolled September 16, 2002. not released.
2.0.40 : released August 9, 2002 as GA.
2.0.39 : released June 17, 2002 as GA.
2.0.38 : rolled June 16, 2002. not released.
2.0.37 : rolled June 11, 2002. not released.
2.0.36 : released May 6, 2002 as GA.
2.0.35 : released April 5, 2002 as GA.
2.0.34 : tagged March 26, 2002.
2.0.33 : tagged March 6, 2002. not released.
2.0.32 : released Feburary 16, 2002 as beta.
2.0.31 : rolled Feburary 1, 2002. not released.
2.0.30 : tagged January 8, 2002. not rolled.
2.0.29 : tagged November 27, 2001. not rolled.
2.0.28 : released November 13, 2001 as beta.
2.0.27 : rolled November 6, 2001
2.0.26 : tagged October 16, 2001. not rolled.
2.0.25 : rolled August 29, 2001
2.0.24 : rolled August 18, 2001
2.0.23 : rolled August 9, 2001
2.0.22 : rolled July 29, 2001
2.0.21 : rolled July 20, 2001
2.0.20 : rolled July 8, 2001
2.0.19 : rolled June 27, 2001
2.0.18 : rolled May 18, 2001
2.0.17 : rolled April 17, 2001
2.0.16 : rolled April 4, 2001
2.0.15 : rolled March 21, 2001
2.0.14 : rolled March 7, 2001
2.0a9 : released December 12, 2000
2.0a8 : released November 20, 2000
2.0a7 : released October 8, 2000
2.0a6 : released August 18, 2000
2.0a5 : released August 4, 2000
2.0a4 : released June 7, 2000
2.0a3 : released April 28, 2000
2.0a2 : released March 31, 2000
2.0a1 : released March 10, 2000
Please consult the following STATUS files for information
on related projects:
* srclib/apr/STATUS
* srclib/apr-util/STATUS
* docs/STATUS
Contributors looking for a mission:
* just do an egrep on "TODO" and see what's there
CURRENT RELEASE NOTES:
RELEASE SHOWSTOPPERS:
CURRENT VOTES:
* httpd-std.conf and friends
a) httpd-std.conf should be tailored by install (from src or
binbuild) even if user has existing httpd.conf
+1: trawick, slive, gregames, ianh, Ken
b) tailored httpd-std.conf should be copied by install to
sysconfdir/examples
-0: striker
c) tailored httpd-std.conf should be installed to
sysconfdir/examples or manualdir/exampleconf/
+1: slive, trawick, Ken
d) Installing a set of default config files when upgrading a server
doesn't make ANY sense at all.
+1: striker
ianh - medium/big sites don't use 'standard config' anyway, as it
usually needs major customizations
-1: Ken
* If the parent process dies, should the remaining child processes
"gracefully" self-terminate. Or maybe we should make it a runtime
option, or have a concept of 2 parent processes (one being a
"hot spare").
See: Message-ID: <3C...@Golux.Com>
Self-destruct: Ken, Martin
Not self-destruct: BrianP, Ian, Cliff, BillS
Make it runtime configurable: Aaron, Jim, Justin
Have 2 parents: +1: Jim
-1: Justin, wrowe [for 2.0]
+0: Martin (while standing by, could it do
something useful?)
* Make the worker MPM the default MPM for threaded Unix boxes.
+1: Justin, Ian, Cliff, BillS, striker
+0: BrianP, Aaron (mutex contention is looking better with the
latest code, let's continue tuning and testing)
-0: Lars
RELEASE NON-SHOWSTOPPERS BUT WOULD BE REAL NICE TO WRAP THESE UP:
* There is a bug in how we sort some hooks, at least the pre-config
hook. The first time we call the hooks, they are in the correct
order, but the second time, we don't sort them correctly. Currently,
the modules/http/config.m4 file has been renamed to
modules/http/config2.m4 to work around this problem, it should moved
back when this is fixed.
OtherBill offers that this is a SERIOUS problem. We do not sort
correctly by the ordering arguments passed to the register hook
functions. This was proven when I reordered the open_logs hook
to attempt to open the error logs prior to the access logs. Possibly
the entire sorting code needs to be refactored.
* pipes deadlock on all platforms with limited pipe buffers (e.g. both
Linux and Win32, as opposed to only Win32 on 1.3). The right solution
is either GStein's proposal for a "CGI Brigade", or OtherBill's proposal
for "Poll Buckets" for "Polling Filter Chains".
* server pushed CGI's not working.
This might be an interaction with the above pipes deadlock issue.
PR: 8482
Message-ID: <3C...@apache.org>
* All handlers should always send content down even if r->header_only
is set. If not, it means that the HEAD requests don't generate the
same headers as a GET which is wrong.
* HP/UX 10.20: compile breakage in APR. Looks like it should be easy
to fix, probably just some extraneous #include's that are fouling
things up.
PR: 9457
Jeff: See my reply and patch in the PR (and previous commit to
stop using "pipe" as a field name). If patch is committed, we
should be okay. I'll wait to see if the user tests the patch.
Update by Jeff 20020722: I got an account on HP 10.20. It looks
like some of the APR thread detection is screwed up. If we find
pthread.h but we can't compile the pthread test program we still
think we can use threads. For that reason, the patch I posted
to the PR won't work as-is since a failed compile of the test
program means nothing.
* exec cmd and suexec arg-passing enhancements
Status: Patches proposed
Message-ID: <20...@prodigy.Redbrick.DCU.IE>
(see the "proc.patch" and "suexec-shell.patch" links in this message)
* The 2.0.36 worker MPM graceless shutdown changes work but are
a bit clunky on some platforms; eg, on Linux, the loop to
join each worker thread seems to hang, and the parent ends up
killing off the child with SIGKILL. But at least it shuts down.
* --enable-mods-shared="foo1 foo2" is busted on Darwin. Pier
posted a patch (Message-ID: <B8...@betaversion.org>).
* We do not properly substitute the prefix-variables in the configuration
scripts or generated-configs. (i.e. if sysconfdir is etc,
httpd-std.conf points to conf.)
* If any request gets through ap_process_request_internal() and is
scheduled to be served by the core handler, without a flag that this
r->filename was tested by dir/file_walk, we need to 500 at the very
end of the ap_process_request_internal() processing so sub_req-esters
know this request cannot be run. This provides authors of older
modules better compatibility, while still improving the security and
robustness of 2.0.
Status: still need to decide where this goes, OtherBill comments...
Message-ID: <06...@roweclan.net>
[Deleted comments regarding the ap_run_handler phase, as irrelevant
as BillS points out that "common case will be caught in
default_handler already (with the r->finfo.filetype == 0 check)"
and the issue is detecting this -before- we try to run the req.]
gregames says: can this happen somehow without a broken module
being involved? If not, why waste cycles trying to defend against
potential broken modules? It seems futile.
wrowe counters: no, it shouldn't happen unless the module is broken.
But the right answer is to fail the request up-front in dir/file
walk if the path was entirely invalid; and we can't do that either
UNTIL 2.1 or we break modules that haven't hooked map_to_storage.
* Rewrite core_output_filter. It is nearly impossible to support
it with predictable results as it is implemented now.
* With AP_MODE_EXHAUSTIVE in the core, it is finally clear to me
how the Perchild MPM should be re-written. It hasn't worked
correctly since filters were added because it wasn't possible to
get the content that had already been written and the socket at
the same time. This mode lets us do that, so the MPM can be
fixed.
* htpasswd blindly processes the file you give it, and does no
sanity checking before totally corrupting whatever file it was
you thought you had. It should check the input file and bail
if it finds non-comment lines that do not contain exactly 1
':' character.
Message-ID: <20...@clove.org>
* Can a static httpd be built reliably?
Message-ID: <20...@clove.org>
* [Ken] Test suite failures:
o worker is also failing some of the 'cgi' subtests
(see <URL:http://Source-Zone.Org/Apache/regression/>):
Justin says: "Worker should be fine and passes httpd-test here.
I think it's a perl or a httpd-test problem."
* Usage of APR_BRIGADE_NORMALIZE in core_input_filter should be
removed if possible.
Message-ID: <Pi...@deepthought.cs.virginia.edu>
Jeff wonders if we still care about this. It is no longer an
API issue but simply an extra trip through the brigade.
* The Add...Filter and Set...Filter directives do not allow the
administrator to order filters, beyond the order of filename (mime)
extensions. It isn't clear if Set...Filter(s) should be inserted
before or after the Add...Filter(s) which are ordered by sequence of
filename extensions. At minimum, some sort of +-[0-10] syntax seems
like a nice solution. See ROADMAP.
* Get perchild to work on platforms other than Linux. This
will require a portable mechanism to pass data and file/socket
descriptors between vhost child groups. An API was proposed
on dev@apr:
Message-ID: <20...@clove.org>
* Try to get libtool inter-library dependency code working on AIX.
Message-ID: <cm...@rdu163-40-092.nc.rr.com>
Justin says: If we get it working on AIX, we can enable this
on all platforms and clean up our build system
somewhat.
Jeff says: I thought I tested a patch for you sometime in
January that you were going to commit within a few
days.
* Handling of %2f in URIs. Currently both 1.3 and 2.0
completely disallow %2f in the request URI path (see
ap_unescape_url() in util.c). It's permitted and passed
through in the query string, however. Roy says the
original reason for disallowing it, from five years ago,
was to protect CGI scripts that applied PATH_INFO to
a filesystem location and which might be tricked by
..%2f..%2f(...). We *should* allow path-info of the
form 'http://foo.com/index.cgi/path/to/path%2finfo'.
Since we've revamped a lot of our processing of path
segments, it would be nice to allow this, or at least
allow it conditionally with a directive.
OtherBill adds that %2f as the SECOND character of a multibyte
sequence causes the request to fail! This happens notably in
the ja-jis encoding.
* FreeBSD, threads, and worker MPM. All seems to work fine
if you only have one worker process with many threads. Add
a second worker process and the accept lock seems to be
lost. This might be an APR issue with how it deals with
the child_init hook (i.e. the fcntl lock needs to be resynced).
More examination and analysis is required.
Status: This has also been reported on Cygwin.
Message-ID: <3C...@wapme-systems.de> (cygnus)
Aaron says: I spent some time disecting this and have come to
the conclusion that it is not a problem in the worker MPM
(or at least, it is not isolated to a problem in worker).
I'll list some of the problems I'm seeing in case someone
else wants to pick up where I've left off:
- Delivery of just about any signal to one of the child
processes will send it into an infinite loop as well.
- Even though the parent is spinning out of control,
at first the child or children will appear to work
properly. At times it is possible to get it into a state,
however, where a request will hang until another concurrent
request "kicks" the first, at which point the second will
hang. My theory is that this has to do with the
pthread_cond_*() implementation in FreeBSD, but it's still
possible that it is in APR.
Justin adds: Oh, FreeBSD threads are implemented entirely with
select()/poll()/longjmp(). Welcome to the nightmare.
So, that means a ktrace output also has the thread
scheduling internals in it (since it is all the same to
the kernel). Which makes it hard to distinguish between
our select() calls and their select() calls.
*bangs head on wall repeatedly* But, some of the libc_r
files have a DBG_MSG #define. This is moderately helpful
when used with -DNO_DETACH. The kernel scheduler isn't
waking up the threads on a select(). Yum. And, I bet
those decrementing select calls have to do with the
scheduler. Time to brush up on our OS fundamentals.
* There is increasing demand from module writers for an API
that will allow them to control the server � la apachectl.
Reasons include sole-function servers that need to die if
an external dependency (e.g., a database) fails, et cetera.
Perhaps something in the (ever more abused) scoreboard?
On the other hand, we already have a pipe that goes between parent
and child for graceful shutdown events, along with an API that
can be used to send a message down that pipe. In threaded MPMs,
it is easy enough to make that one pipe be used for graceful
and graceless events, and it is also easy to open that pipe
to both parent and child for writing. Then we just need to
figure out how to do graceless on non-threaded MPMs.
* Allow the DocumentRoot directive within <Location > scopes? This
allows the beloved (crusty) Alias /foo/ /somepath/foo/ followed
by a <Directory /somepath/foo> to become simply
<Location /foo/> DocumentRoot /somefile/foo (IMHO a bit more legible
and in-your-face.) DocumentRoot unset would be accepted [and would
not permit content to be served, only virtual resources such as
server-info or server-status.
This proposed change would _not_ depricate Alias.
striker: See the thread starting with Message-ID:
JLEGKKNELMHCJPNMOKHOGEEJFBAA.striker@apache.org.
* Win32: Rotatelogs sometimes is not terminated when Apache
goes down hard. FirstBill was looking at possibly tracking the
child's-child processes in the parent process.
stoddard: Shared scoreboard might offer a good way for the parent
to keep track of 'other child' processes and whack them if the child
goes down.
Other thoughts on walking the process chain using the NT kernel
have also been proposed on APR.
* Eliminate unnecessary creation of pipes in mod_cgid
* Combine log_child and piped_log_spawn. Clean up http_log.c.
Common logging API.
* Document mod_file_cache.
* Platforms that do not support fork (primarily Win32 and AS/400)
Architect start-up code that avoids initializing all the modules
in the parent process on platforms that do not support fork.
* Win32: Migrate the MPM over to use APR thread/process calls. This
would eliminate some code in the Win32 branch that essentially
duplicates what is in APR.
* There are still a number of places in the code where we are
losing error status (i.e. throwing away the error returned by a
system call and replacing it with a generic error code)
* Mass vhosting version of suEXEC.
* All DBMs suffer from confusion in support/dbmmanage (perl script) since
the dbmmanage employs the first-matched dbm format. This is not
necessarily the library that Apache was built with. Aught to
rewrite dbmmanage upon installation to bin/ with the proper library
for predictable mod_auth_dbm administration.
Questions; htdbm exists, time to kill dbmmanage, or does it remain
useful as a perl dbm management example? If we keep it,
do we address the issue above?
* Integrate mod_dav.
Some additional items remaining:
- case_preserved_filename stuff
(use the new canonical name stuff?)
- find a new home for ap_text(_header)
- is it possible to remove the DAV: namespace stuff from util_xml?
* ap_core_translate() and its use by mod_mmap_static and mod_file_cache
are a bit wonky. The function should probably be exposed as a utility
function (such as ap_translate_url2fs() or ap_validate_fs_url() or
something). Another approach would be a new hook phase after
"translate" which would allow the module to munge what the
translation has decided to do.
Status: Greg +1 (volunteers)
* Explore use of a post-config hook for the code in http_main.c which
calls ap_fixup_virutal_hosts(), ap_fini_vhost_config(), and
ap_sort_hooks() [to reduce the logic in main()]
* read the config tree just once, and process N times (as necessary)
* (possibly) use UUIDs in mod_unique_id and/or mod_usertrack
* (possibly) port the bug fix for PR 6942 (segv when LoadModule is put
into a VirtualHost container) to 2.0.
* shift stuff to mod_core.h
* callers of ap_run_create_request() should check the return value
for failure (Doug volunteers)
* Win32: Get Apache working on Windows 95/98. The following work
(at least) needs to be done:
- Document warning that OSR2 is required (for Crypt functions, in
rand.c, at least.) This could be resolved with an SSL library, or
randomization in APR itself.
- Bring the Win9xConHook.dll from 1.3 into 2.0 (no sense till it
actually works) and add in a splash of Win9x service code.
* Fix the worker MPM to use POD to kill child processes instead
of ap_os_killpg, regardless of how they should die.
* Scoreboard structures could be changed in the future such that
proper alignment is not maintained, leading to segfaults on
some systems. Cliff posted a patch to deal with this issue but
later recanted. See this message to dev@apr.apache.org:
Message-ID: <Pine.LNX.4.44.0203011354090.16457-200000@deepthought
.cs.virginia.edu>
TODO ISSUES REMAINING IN MOD_SSL:
* In order to use a DSO version of mod_ssl we have to link with
-lssl and -lcrypto. A workaround is in place right now where the
entire EXTRA_LIBS macro is being appended to the objects list, but
this is a hack. We should either revamp the APACHE_CHECK_SSL_TOOLKIT
autoconf function or come up with some other autoconf checks to
search for libssl and libcrypto and properly add them to mod_ssl's
link flags.
* SSL renegotiations in combination with POST request
* Port or dispose all code inside #if 0...#endif blocks that remain
from the porting effort.
* Do we need SSL_set_read_ahead()?
* the ssl_expr api is NOT THREAD SAFE. race conditions exist:
-in ssl_expr_comp() if SSLRequire is used in .htaccess
(ssl_expr_info is global)
-is ssl_expr_eval() if there is an error
(ssl_expr_error is global)
* SSLRequire directive (parsing of) leaks memory
* Diffie-Hellman-Parameters for temporary keys are hardcoded in
ssl_engine_dh.c, while the comment in ssl_engine_kernel.c says:
"it is suggested that keys be changed daily or every 500
transactions, and more often if possible."
* ssl_var_lookup could be rewritten to be MUCH faster
* CRL callback should be pluggable
* session cache store should be pluggable
* init functions should return status code rather than ssl_die()
* ssl_engine_pphrase.c needs to be reworked so it is generic enough
to also decrypt proxy keys
* the shmcb code should just align its memory segment rather than
jumping through all the "safe" memcpy and memset hoops
EXPERIMENTAL MODULES:
Experimental modules should eventually be be promoted to fully supported
status or removed from the repository entirely (ie, the
'experiment' failed). This section tracks what needs to happen to
get the modules promoted to fully supported status.
mod_cache/mod_mem_cache/mod_disk_cache:
* mod_cache: handle cache_control: no_cache "field_name" to enable
cacheing the response w/o header "field_name"
See RFC2616 section 14.9.1
* mod_cache: CacheEnable/CacheDisable should accept regular expressions.
* mod_cache: Fix dependency on ATOMIC operators. Need
APR_HAS_ATOMIC_* feature macros.
* mod_disk_cache: Implement garbage collection
* mod_mem_cache/mod_disk_cache: Need to be able to query cache
status (num of entries, cache object properties, etc.).
mod_status could be extended to query optional hooks defined
by modules for the purpose of reporting module status.
mod_cache (et. al.) could define optional hooks that are called
to collect status. Status should be queryable by
HTTP or SNMP?
* Enable mod_cache/mod_mem_cache/mod_disk_cache to handle
multiviews
* mod_mem_cache fd caching is broken on Unix. The cached fd cannot
be used across multiple threads except for use on sendfile().
mod_auth_ldap/util_ldap:
* implement cross-platform/cross-library TLS support
* General stabilization and testing
PRs that have been suspended forever waiting for someone to
put them into 'the next release':
* documentation and Q&A
PR#2221: Make online documentation search link back to my installation
Status:
PR#2906: Propose that Apache recommend $UNIQUE_ID for all "session id"
algorithms
Status:
PR#2793: When will Apache support P3P? Any Plans?
Status:
* build
PR#2113: HTTP Server Rebuild Line Needs Changing for the better
Status:
PR#2421: problem specifying ndbm library for build ?with autoconfigure
Status:
* config
PR#76: missing call to "setlocale();"
Status:
PR#628: Request of "Options SymLinksIfGroupMatch"
Status:
PR#793: RLimitCPU and RLimitMEM don't apply to all children like they should
Status:
PR#922: it is useful to allow specifiction that root-owned symlinks
should always be followed
Status:
PR#1028: DoS attacks involving memory consumption
Status:
PR#1191: setlogin() is not called, causing problems with e.g. identd
Status:
PR#1204: regerror() exists, use it
Status:
PR#2284: Can not POST to ErrorDocument - Apache/1.3b6
Status:
PR#2396: Proposal for TimeZone directive
Status:
PR#2446: AllowOverride FileInfo is too coarse
Status:
PR#2760: [PATCH] User/Group for <Directory> and <Location> i.e. not only
in global and <Virtual>.
Status:
PR#2907: suggestion: power up your Include directive :)
Status:
PR#3018: cannot limit some HTTP methods
Status:
PR#3677: New ErrorDocumentMatch directive
Status:
PR#4244: "Files" and "FilesMatch" regexp does not recognize bang as
negation operator
Status:
PR#5993: AllowOverride should have a 'CheckNone' and 'AllowNone' argument
instead of only 'None'
Status:
* mod_access
PR#537: mod_access syntax allows hosts that should be restricted
Status:
PR#1287: add allow,deny/deny,allow warning to mod_access
Status:
PR#2512: <IfDenied> directive wanted
Status:
* mod_auth-any
PR#557: ~UserHome directories are not honored in absolute pathname
requests (.htaccess)
Status:
PR#1117: Using NIS passwd.byname dbm files with AuthDBMUserFile
Status:
PR#1809: Suggestion for improving authentication modules and core source
code, problem with 401 and ErrorDocument
Status:
* mod_autoindex
PR#1263: Add frame-safe anchor attribute to mod_autoindex links
Status:
* mod_cgi (and suexec)
PR#921: suexec Uses cwd before filling it in, doesn't use syslog
Status:
PR#1176: Apache cannot handle continuation line in headers
Status:
PR#1120: suexec does not parse arguments to #exec cmd
Status:
PR#1268: CGI scripts running as Apache user: security (suexec etc.)
Status:
PR#1285: Error messages could be easier to spot in cgi.log file for suexec.c
Status:
PR#1905: suexec - Allow modules to set user:group for execution.
Status:
PR#2360: suexec for general access of user content?
Status:
PR#2460: TimeOut applies to output of CGI scripts
Status:
PR#2573: CGI's for general use still have to be run as another user
with suExec
Status:
PR#4241: Need to be able to override shebang line to make CGI scripts
more portable.
Status:
PR#4490: mod_cgi prevents handling of OPTIONS requests
Status:
* mod_env
PR#370: Modified PATH environemnt variable is not passed, instead
system's is used
Status:
* mod_headers
PR#1383: I make mod_headers to modify request headers as well as
response ones.
Status:
PR#1677: mod_headers should allow mod_log_config-style formats in
header values
Status:
* mod_imap
PR#759: imap should read <MAP><AREA>*</MAP> too!
Status:
* mod_include
PR#78: Additional status for XBitHack directive
Status:
PR#623: A smarter "Last Modified" value for SSI documents (see PR number 600)
Status:
PR#1145: mod_include
Allow for Last-Modified: without resorting to XBitHack
Status:
PR#1803: patches to mod_include to allow for file tests
Status:
PR#4459: Suggestion for better handling of Last-modified headers
Status:
* mod_info
PR#2415: /server-info doesn't check for the virtual host to list the info
Status:
* mod_log-any
PR#1050: Logging of virtual server to error_log as well
Status:
PR#1358: Selective url-encode of log fields (or maybe a pseudo
log_rewrite module?)
Status:
PR#2073: pipelined connections are not logged correctly
Status:
PR#4448: Please allow CGI env variables (QUERY_STRING, ...) to be logged
with %{}e
Status:
* mod_negotiation
PR#3191: no way to set global quality-of-source (qs) coneg values
with multiviews
Status:
* mod_proxy
PR#362: Mod_proxy doesn't allow change of error pages
Status:
PR#440: Proxy doesn't deliver documents if not connected
Status:
PR#534: proxy converts ~name to %7Ename when name starts with a dot (.)
Status:
PR#612: Proxy FTP Authentication Fails
Status:
PR#700: Proxy doesn't do links right for OpenVMS files through ftp:
Status:
PR#980: Controlling Access to Remote Proxies would be nice...
Status:
PR#994: Adding authentication "on the fly" through the proxy module
Status:
PR#1085: ProxyRemote make a dead cycle.
Status:
PR#1166: ``nph-'' not honored (no buffering) for ProxyRemote mapping
Status:
PR#1290: Need to know "hit-rate" on proxy cache
Status:
PR#1532: Proxy transfer logging
Status:
PR#1547: No HTTP_X_FORWARDED_FOR set...
Status:
PR#1567: ProxyRemote proxy requests fail authentication by firewall
Status:
PR#1702: mod_proxy to support persistent conns?
Status:
PR#1878: listing of proxy cache content
Status:
PR#2314: patterns in ProxyRemote
Status:
PR#2648: Cache file names in Proxy module
Status:
PR#3568: Accessing URL through proxy server corrupts data.
Status:
PR#3605: Some anonymous FTP URLs ask for authentication
Status:
* mod_rewrite
PR#1582: mod_rewrite forms REQUEST_URI different than mod_cgi does
Status:
PR#2074: mod_rewrite doesn't pass Proxy Throughput on internal subrequests
Status:
* mod_status
PR#2138: mod_status always displays 256 possible connection slots
Status:
PR#2343: Status module averages are for entire uptime
Status:
* apache-api
PR#1004: request_config field in request_rec is moderately bogus
Status:
PR#1158: improvements to child spawning API
Status:
PR#1233: there is no way to keep per-connection per-module state
Status:
PR#2024: adding auth_why to conn_rec
Status:
PR#2873: Feedback/Comment on APACI
Status:
PR#3143: No module specific data hook for per-connection data
Status:
* generally odds and ends
PR#2431: A small addition to rotatelogs.c to improve program functionality.
Status:
PR#2763: mailto tags and bundling bug report script
Status:
PR#2785: os-aix Support for System Resource Controller
Status:
PR#2889: Inclusion of RPM spec file in CVS/distributions
Status:
PR#5713: os-windows [PATCH] install as win32 service with domain account
Status: Cannot accept password-as-arg, we should prompt the
user when -k install/-k config with a user argument.
Other bugs that need fixing:
* ap_discard_request should be converted to use the bucket API
directly rather than waste cycles copying buffers with the old API.
* MaxRequestsPerChild measures connections, not requests.
Until someone has a better way, we'll probably just rename it
"MaxConnectionsPerChild".
* Regex containers don't work in an intutive way
Status: No one has come up with an efficient way to fix this
behavior. Dean has suggested getting rid of regex containers
completely.
OtherBill suggests: We at least seem to agree on eliminating
the <Container ~ foo> forms, and using only
<ContainerMatch foo> semantics.
* SIGSEGV on Linux (glibc 2.1.2) isn't caught properly by a
sigwaiting thread. We need to work around this, perhaps unless
there is hope soon for a fixed glibc.
* orig_ct in the byterange/multipart handling may not be
needed. Apache 1.3 just never stashed "multipart" into
r->content_type. We should probably follow suit since the
byterange stuff doesn't want the rest of the code to see the
multipart content-type; the other code should still think it is
dealing with the <orig_ct> stuff.
Status: Greg volunteers to investigate (esp. since he was most
likely the one to break it :-)
Binaries (2.0.43):
Platform Avail. Volunteer
------------------------------------------------------------------
AIX 4.3.3 no Bill Stoddard
Mandrake 8.1 no open
FreeBSD 4.1 no open
hppa2.0w-hp-hpux11.00 no Cliff Woolley
i386-pc-solaris2.8 no Aaron Bannert
i386-unknown-freebsd4.5 no
i386-unknown-freebsd4.6 no Cliff Woolley
i686-pc-linux-gnu-slackware81 no Cliff Woolley
i686-pc-linux-gnu-rh70 no Aaron Bannert
i686-pc-linux-gnu-rh73 no Cliff Woolley
ia64-hp-hpux11.20 no
powerpc-apple-darwin5.5 no Aaron Bannert
powerpc-unknown-linux-gnu no Graham Leggett
s390-ibm-linux no Greg Ames
sparc-sun-solaris2.8 no Jim Jagielski
NetWare no Brad Nicholes
OS/2 no Brian Havard
OS/390 no Greg Ames
Win32-x86 yes William Rowe
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Posted by Bojan Smojver <bo...@rexursive.com>.
> I recall you had tested a ton of 'little files' pipelined.
>
> What might be more interesting is a 100MB download (over a fast pipe)
> which is entirely 'sendfile'd out. Apache would consider itself done with
> the request long before it was finished with the connection.
I tested with an 80 MB file and got different numbers depending on when I
clicked stop in the browser. The testing was done on 127.0.0.1 (733 MHz Celeron
box), so guess it qualifies as a fast pipe. Is that what you meant? Or did you
mean pipelined large files?
However, I do have a question for you (or anyone else that understands Apache
2.0 internal dymamics better then me). Currently, my patch fetches the optional
function every time the connection is created (i.e. in core_pre_connection). I
reckon this should actually be done in core_post_config, only once. Do you think
that's safe (i.e. I'm guessing function pointers should be OK on graceful
restarts, forks etc.).?
Bojan
Re: mod_logio patch testing Re: [STATUS] (httpd-2.0) Wed Oct 23
23:45:19 EDT 2002
Posted by Bojan Smojver <bo...@rexursive.com>.
On Fri, 2002-10-25 at 15:31, Brian Pane wrote:
> The only test case in which mod_logio didn't report as small a
> byte count as expected was when I used telnet as the client and
> stopped it (with ctrl-C) while the server was still sending the
> response. A syscall trace on the server showed that, after I
> interrupted telnet, the httpd did the remainder of its sendfile
> calls (each one of which managed to send the next 48KB of the
> requestd file; that appears to be a kernel limit) very, very
> quickly. I think that's just because telnet did a half-close
> and continued reading and discarding data until the server
> closed its end of the connection before actually exiting.
I have observed the same. However, if I closed xterm in which telnet was
running, instead of pressing Ctrl-C, the behaviour would change and
sizes were different. I agree with you that telnet probably continued
reading, although not displaying it. There is no way an 80 MB file (in
my test) could fly by my eyes that quickly :-)
Bojan
mod_logio patch testing Re: [STATUS] (httpd-2.0) Wed Oct 23
23:45:19 EDT 2002
Posted by Brian Pane <br...@cnet.com>.
On Thu, 2002-10-24 at 19:45, David Burry wrote:
> At 08:45 PM 10/24/2002 -0500, William A. Rowe, Jr. wrote:
> >At 08:40 PM 10/24/2002, Bojan Smojver wrote:
> >>Quoting David Burry <db...@tagnet.org>:
> >>
> >>> Excellent! I will perform some tests with that when I get a chance! You
> >>> managed to get it working without breaking pipelining even? That's awesome!
> >>
> >>That's what I *think*, which has been known to deviate from the truth, from time
> >>to time. However, I appreciate all input, especially results of the actual tests.
> >
> > I recall you had tested a ton of 'little files' pipelined.
> >
> > What might be more interesting is a 100MB download (over a fast pipe)
> >which is entirely 'sendfile'd out. Apache would consider itself done with
> >the request long before it was finished with the connection.
>
>
> In case someone else wants to independently verify it...
>
> The exact test I was doing was with a 70+ meg .tar.gz file both over a 100mbit ethernet and a 1.5mbit DSL, starting and canceling it multiple times in Windoze Internet Explorer 5 or 6 (which appears to effectively use byte range requests for subsequent tries, by the way) and monitoring what was logged each time. This test isn't super precise on the byte count (I did not bother to go comb my IE cache) but it sure is obvious when it consistently logs the whole file size and I only received a small fraction according to the IE progress bar... Also looking at the byte range requests with %{Range}i makes it obvious how much IE received previously on each subsequent try (IE appears to only request the part of the file it hasn't cached yet).
>
> I was thinking of writing a script that did this in a more automated fashion... perhaps contributing a test to the apache test suite when I figure that thing out... ;o)
I just tried a similar test, using Bojan's latest patch. Using
IE 5.5 on Windows 2k, I repeatedly requested a 32MB file from Apache
w/mod_logio (on Linux with sendfile) and stopped the transfer before
the browser had received the whole response. Each time, the bytes-sent
value in the httpd access log was smaller than the file size.
I got similar results when using ab over the loopback as a test
client.
The only test case in which mod_logio didn't report as small a
byte count as expected was when I used telnet as the client and
stopped it (with ctrl-C) while the server was still sending the
response. A syscall trace on the server showed that, after I
interrupted telnet, the httpd did the remainder of its sendfile
calls (each one of which managed to send the next 48KB of the
requestd file; that appears to be a kernel limit) very, very
quickly. I think that's just because telnet did a half-close
and continued reading and discarding data until the server
closed its end of the connection before actually exiting.
Based on the test results, the patch does appear to be an
improvement over the current code. I'm planning to commit
it tonight.
Brian
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Posted by David Burry <db...@tagnet.org>.
At 08:45 PM 10/24/2002 -0500, William A. Rowe, Jr. wrote:
>At 08:40 PM 10/24/2002, Bojan Smojver wrote:
>>Quoting David Burry <db...@tagnet.org>:
>>
>>> Excellent! I will perform some tests with that when I get a chance! You
>>> managed to get it working without breaking pipelining even? That's awesome!
>>
>>That's what I *think*, which has been known to deviate from the truth, from time
>>to time. However, I appreciate all input, especially results of the actual tests.
>
> I recall you had tested a ton of 'little files' pipelined.
>
> What might be more interesting is a 100MB download (over a fast pipe)
>which is entirely 'sendfile'd out. Apache would consider itself done with
>the request long before it was finished with the connection.
In case someone else wants to independently verify it...
The exact test I was doing was with a 70+ meg .tar.gz file both over a 100mbit ethernet and a 1.5mbit DSL, starting and canceling it multiple times in Windoze Internet Explorer 5 or 6 (which appears to effectively use byte range requests for subsequent tries, by the way) and monitoring what was logged each time. This test isn't super precise on the byte count (I did not bother to go comb my IE cache) but it sure is obvious when it consistently logs the whole file size and I only received a small fraction according to the IE progress bar... Also looking at the byte range requests with %{Range}i makes it obvious how much IE received previously on each subsequent try (IE appears to only request the part of the file it hasn't cached yet).
I was thinking of writing a script that did this in a more automated fashion... perhaps contributing a test to the apache test suite when I figure that thing out... ;o)
Dave
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Posted by "William A. Rowe, Jr." <wr...@apache.org>.
At 08:40 PM 10/24/2002, Bojan Smojver wrote:
>Quoting David Burry <db...@tagnet.org>:
>
>> Excellent! I will perform some tests with that when I get a chance! You
>> managed to get it working without breaking pipelining even? That's awesome!
>
>That's what I *think*, which has been known to deviate from the truth, from time
>to time. However, I appreciate all input, especially results of the actual tests.
Bojon,
I recall you had tested a ton of 'little files' pipelined.
What might be more interesting is a 100MB download (over a fast pipe)
which is entirely 'sendfile'd out. Apache would consider itself done with
the request long before it was finished with the connection.
Just a thought.
Bill
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Posted by David Burry <db...@tagnet.org>.
At 09:38 PM 10/24/2002 -0400, Glenn wrote:
>Have you looked at the %...X directive in Apache2?
That's an interesting idea I hadn't thought of... it doesn't solve the chargeback issue but it's worth investigating for detecting successful downloads...
Dave
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Posted by Bojan Smojver <bo...@rexursive.com>.
Quoting David Burry <db...@tagnet.org>:
> Excellent! I will perform some tests with that when I get a chance! You
> managed to get it working without breaking pipelining even? That's awesome!
That's what I *think*, which has been known to deviate from the truth, from time
to time. However, I appreciate all input, especially results of the actual tests.
Bojan
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Posted by Glenn <gs...@gluelogic.com>.
On Thu, Oct 24, 2002 at 05:25:46PM -0700, David Burry wrote:
> Excellent! I will perform some tests with that when I get a chance! You managed to get it working without breaking pipelining even? That's awesome!
>
> Not meaning to belittle Bojan's hard work, but for my purposes mod_logio values are not as good as %b would be if %b worked properly... what I ideally need is the byte sent count without the headers... using Bojan's module I can get approximate results but they will be a hair off because they include headers... My main purpose is to detect if and when several meg files have been downloaded all the way vs. if they were cut off in the middle, including if a given user uses some byte-ranging download manager that lets you pause and restart... We also use it for chargebacks to the various departments for bandwidth usage (in this case mod_logio would of course be more accurate than %b though)... We actually had to fudge some of our statistics (duplicated nearby days' data with similar overall throughputs) due to us not catching the problem with Apache 2.0 soon enough...
Have you looked at the %...X directive in Apache2?
http://httpd.apache.org/docs-2.0/mod/mod_log_config.html
%...X: Connection status when response is completed.
'X' = connection aborted before the response completed.
'+' = connection may be kept alive after the response is sent.
'-' = connection will be closed after the response is sent.
(This directive was %...c in late versions of Apache 1.3, but
this conflicted with the historical ssl %...{var}c syntax.)
I don't know if this works in Apache2, but this is what the docs say.
Discussion of Apache2 design on this list would suggest that it might
suffer from the same limitation that %b currently exhibits.
(In the past, I used the %c in Apache 1.3 with success when tallying
whether or not a download of a .cab installer completed successfully.
And I didn't have to know the size of the file for comparison. Nice!)
-Glenn
> At 09:15 AM 10/25/2002 +1000, Bojan Smojver wrote:
> >On Fri, 2002-10-25 at 07:42, David Burry wrote:
> >
> >> I see.. ok, I'll keep waiting patiently...
> >
> >The patch for 2.0.43 is here:
> >
> >ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch
> >
> >You need to apply mod_logio patch for 2.0.43 first.
> >
> >Bojan
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Posted by David Burry <db...@tagnet.org>.
Excellent! I will perform some tests with that when I get a chance! You managed to get it working without breaking pipelining even? That's awesome!
Not meaning to belittle Bojan's hard work, but for my purposes mod_logio values are not as good as %b would be if %b worked properly... what I ideally need is the byte sent count without the headers... using Bojan's module I can get approximate results but they will be a hair off because they include headers... My main purpose is to detect if and when several meg files have been downloaded all the way vs. if they were cut off in the middle, including if a given user uses some byte-ranging download manager that lets you pause and restart... We also use it for chargebacks to the various departments for bandwidth usage (in this case mod_logio would of course be more accurate than %b though)... We actually had to fudge some of our statistics (duplicated nearby days' data with similar overall throughputs) due to us not catching the problem with Apache 2.0 soon enough...
Dave
At 09:15 AM 10/25/2002 +1000, Bojan Smojver wrote:
>On Fri, 2002-10-25 at 07:42, David Burry wrote:
>
>> I see.. ok, I'll keep waiting patiently...
>
>The patch for 2.0.43 is here:
>
>ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch
>
>You need to apply mod_logio patch for 2.0.43 first.
>
>Bojan
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Posted by Bojan Smojver <bo...@rexursive.com>.
Quoting Bojan Smojver <bo...@rexursive.com>:
Just realised that the URL is wrong :-( So, here is the correct one:
ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch.gz
Bojan
> On Fri, 2002-10-25 at 07:42, David Burry wrote:
>
> > I see.. ok, I'll keep waiting patiently...
>
> The patch for 2.0.43 is here:
>
> ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch
>
> You need to apply mod_logio patch for 2.0.43 first.
>
> Bojan
>
>
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Posted by Bojan Smojver <bo...@rexursive.com>.
On Fri, 2002-10-25 at 07:42, David Burry wrote:
> I see.. ok, I'll keep waiting patiently...
The patch for 2.0.43 is here:
ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch
You need to apply mod_logio patch for 2.0.43 first.
Bojan
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Posted by David Burry <db...@tagnet.org>.
----- Original Message -----
From: "Bojan Smojver" <bo...@rexursive.com>
> On Fri, 2002-10-25 at 03:31, David Burry wrote:
> > Is it possible to get some of the fixes to mod_logio committed?
Wouldn't
> > everyone agree that the current logging of the outgoing bytes is
incorrect
> > behavior? Currently it logs the full file size (plus headers) even if
it
> > gets cut off in the middle, instead of the actual number of bytes sent.
> > I've seen several patches to fix this but very little comment on it...
I've
> > seen lots of comments that it can't be done without major
rearchitecting,
> > but Bojan seems to have done it without that by breaking pipelining, am
I
> > correct?
>
> Actually, the last patch I sent contains one snag I'm still working on.
> It breaks the core's connection configuration structure, which gets
> attached to c->conn_config. However, I think I can get around that by
> using an optional function. As the matter of fact, I'm working on it
> right now.
I see.. ok, I'll keep waiting patiently...
> > Since I depend on correct outgoing byte count logging to see how many
people
> > sucessfully download files, I can live with broken pipelining for now in
> > 2.0, currently I've had to roll back to Apache 1.3 and put in 3 times as
> > many machines (12 instead of 4).... I'd really like to return those 8
> > borrowed machines someday and be able to upgrade to 2.0... but can't do
that
> > in the current broken log state.
>
> Glad to hear Apache 2.0 makes a huge performance difference. Not so glad
> to hear you had to resort to going back to 1.3. The only thing I can
> promise is a patch using optional function (this should guarantee
> compatibility of core between 43 and 44 and no MMN bump) during the day
> (Sydney time). It's up to the committers to review and, if they like,
> commit.
The memory savings is quite significant, and I'll admit that the 8 extra
machines are smaller than the original 4 so it's not exactly 3 times better
cpu-wise... the memory caching is where the largest savings comes with disk
IO, we have a very high traffic site--usually 3 terabytes transferred per
day, last few days have been more like 5 terabytes due to a new release.
> PS. By the number of messages on the list I'm guessing committers must
> be rather busy on their real jobs these days. Unfortunately there is no
> way of speeding things up, given this is volunteer effort. Unless, of
> course, you decide to bribe some of them ;-)
Not exactly the same thing as bribing a commit, but this could get similar
results: My manager's manager is actually not opposed to hiring a
contractor to fix this... anyone want a temporary job? I don't know if this
is the right place to say such things, tell me if there's a better place.
When you've got millions of dollars worth of sales depending on an open
source project, throwing a little at the open source project isn't such a
big deal... I'd gladly do it myself with my company's blessing (on the
clock, not volunteer) but I'm not a very experienced C programmer yet, more
of a Perl hacker and applications architect so far. This little paragraph
had better not get me too flooded with resumes or flames or I'm going to
feel dumb, whatever you do don't spam this list with personal replies! ;o)
Dave
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Posted by Bojan Smojver <bo...@rexursive.com>.
On Fri, 2002-10-25 at 03:31, David Burry wrote:
> Is it possible to get some of the fixes to mod_logio committed? Wouldn't
> everyone agree that the current logging of the outgoing bytes is incorrect
> behavior? Currently it logs the full file size (plus headers) even if it
> gets cut off in the middle, instead of the actual number of bytes sent.
> I've seen several patches to fix this but very little comment on it... I've
> seen lots of comments that it can't be done without major rearchitecting,
> but Bojan seems to have done it without that by breaking pipelining, am I
> correct?
Actually, the last patch I sent contains one snag I'm still working on.
It breaks the core's connection configuration structure, which gets
attached to c->conn_config. However, I think I can get around that by
using an optional function. As the matter of fact, I'm working on it
right now.
> I also wish that %b would be fixed in a similar manner but I haven't seen
> any patches for that (or comments about it). Wouldn't everyone agree that
> it too should log actual bytes sent not just the full file size every time?
> Apache 2.0 should do everything that 1.3 did, so this logging issue really
> should be considered a bug, right?
I agree with you on this one as well. But at this point I'm unsure how
to fix this one.
> Since I depend on correct outgoing byte count logging to see how many people
> sucessfully download files, I can live with broken pipelining for now in
> 2.0, currently I've had to roll back to Apache 1.3 and put in 3 times as
> many machines (12 instead of 4).... I'd really like to return those 8
> borrowed machines someday and be able to upgrade to 2.0... but can't do that
> in the current broken log state.
Glad to hear Apache 2.0 makes a huge performance difference. Not so glad
to hear you had to resort to going back to 1.3. The only thing I can
promise is a patch using optional function (this should guarantee
compatibility of core between 43 and 44 and no MMN bump) during the day
(Sydney time). It's up to the committers to review and, if they like,
commit.
Bojan
PS. By the number of messages on the list I'm guessing committers must
be rather busy on their real jobs these days. Unfortunately there is no
way of speeding things up, given this is volunteer effort. Unless, of
course, you decide to bribe some of them ;-)
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Posted by David Burry <db...@tagnet.org>.
Is it possible to get some of the fixes to mod_logio committed? Wouldn't
everyone agree that the current logging of the outgoing bytes is incorrect
behavior? Currently it logs the full file size (plus headers) even if it
gets cut off in the middle, instead of the actual number of bytes sent.
I've seen several patches to fix this but very little comment on it... I've
seen lots of comments that it can't be done without major rearchitecting,
but Bojan seems to have done it without that by breaking pipelining, am I
correct?
I also wish that %b would be fixed in a similar manner but I haven't seen
any patches for that (or comments about it). Wouldn't everyone agree that
it too should log actual bytes sent not just the full file size every time?
Apache 2.0 should do everything that 1.3 did, so this logging issue really
should be considered a bug, right?
Since I depend on correct outgoing byte count logging to see how many people
sucessfully download files, I can live with broken pipelining for now in
2.0, currently I've had to roll back to Apache 1.3 and put in 3 times as
many machines (12 instead of 4).... I'd really like to return those 8
borrowed machines someday and be able to upgrade to 2.0... but can't do that
in the current broken log state.
Dave