You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modules-dev@httpd.apache.org by oh...@cox.net on 2009/11/15 01:37:38 UTC
ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type always returns
'text/plain'?
Hi,
I'm not sure if this is the appropriate list, or if I should post on the regular httpd list...
I've been trying to work with an Apache module, mod_limitipconn:
http://dominia.org/djao/limitipconn2.html
This is with Apache 2.2.8/2.2.11.
Our use case is slightly different that the original one for this module. The original code is designed to limit the number of connections from any given IP address, whereas in my case, we want to limit the total number of connections to the entire Apache server instance.
To do that, I commented out one line (an 'if'), as shown below:
/* Count up the number of connections we are handling right now from
* this IP address */
/* This is modified to not do strcmp(address, ws_record->client check, in order
to count ALL connections to this server. */
for (i = 0; i < server_limit; ++i) {
for (j = 0; j < thread_limit; ++j) {
ws_record = ap_get_scoreboard_worker(i, j);
switch (ws_record->status) {
case SERVER_BUSY_READ:
case SERVER_BUSY_WRITE:
case SERVER_BUSY_KEEPALIVE:
case SERVER_BUSY_LOG:
case SERVER_BUSY_DNS:
case SERVER_CLOSING:
case SERVER_GRACEFUL:
/*
if (strcmp(address, ws_record->client) == 0)
*/
ip_count++;
break;
default:
break;
}
}
}
and, after I rebuilt the module, it seems to work ok.
Now, I'm trying to get the module's directive "NoIPLimit" to work.
According to the README, plus reading the code, the way that this is suppose to work is that if you specify a MIME type string (substring, actually), the module won't "count" requests that have any of the MIME types in the directive.
I have the Apache LogLevel set to debug, and it looks like the "content_type" that the module is retrieving from ALL requests is "text/plain", and I have confirmed that the actual MIME types are not "text/plain" (I see things like "image/gif", "text/css", etc.), so the "NoIPLimit" directive doesn't work.
>From reading the code, I think that the "content_type" is set in this snippet:
/* Only check the MIME-type if we have MIME-type stuff in our config.
The extra subreq can be quite expensive. */
if(cfg->no_limit->nelts > 0 || cfg->excl_limit->nelts > 0) {
/* Look up the Content-type of this request. We need a subrequest
* here since this module might be called before the URI has been
* translated into a MIME type. */
content_type = ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type;
/* If there's no Content-type, use the default. */
if (!content_type) {
content_type = ap_default_type(r);
}
ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
"mod_limitipconn: uri: %s Content-Type: %s",
r->uri, content_type);
/* Cycle through the exempt list; if our content_type is exempt,
* return OK */
for (i = 0; i < cfg->no_limit->nelts; i++) {
if ((ap_strcasecmp_match(content_type, nolim[i]) == 0)
|| (strncmp(nolim[i], content_type, strlen(nolim[i])) == 0))
{
ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
"mod_limitipconn: OK: %s exempt", content_type);
return DECLINED;
}
}
and, in particular:
content_type = ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type;
/* If there's no Content-type, use the default. */
if (!content_type) {
content_type = ap_default_type(r);
}
I'm not that familiar with Apache module development, but reading the docs, it *seems* like the above *should* work.
I'm assuming you all on this list *are* familiar with module development, so I was wondering if you might be able to tell me what, if anything, is wrong with that code (i.e., why is "content_type" not being set with the actual content type?
Thanks in advance!!
Jim
Re: ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type always
returns 'text/plain'?
Posted by Eric Covener <co...@gmail.com>.
On Sat, Nov 14, 2009 at 8:10 PM, <oh...@cox.net> wrote:
> "MaxClients" was one of the first approaches that I tried when i was asked about this.
>
> The problem with MaxClients that I found was that it appeared that when it was used, Apache would queue up the requests. The end-result from the user perspective was that either response time would look really slow, or their browser would eventually timeout, and they'd get (in IE) a "The page cannot be displayed".
Wanting your own errordoc makes sense, but the queuing is just:
http://httpd.apache.org/docs/2.2/mod/mpm_common.html#listenbacklog
--
Eric Covener
covener@gmail.com
Re: ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type always
returns 'text/plain'?
Posted by oh...@cox.net.
---- Eric Covener <co...@gmail.com> wrote:
> On Sat, Nov 14, 2009 at 7:37 PM, <oh...@cox.net> wrote:
>
> > Our use case is slightly different that the original one for this module. The original code is designed to limit the number of connections from any given IP address, whereas in my case, we want to limit the total number of connections to the entire Apache server instance.
>
> What does this get you over just setting MaxClients directly?
>
> --
> Eric Covener
> covener@gmail.com
Eric,
"MaxClients" was one of the first approaches that I tried when i was asked about this.
The problem with MaxClients that I found was that it appeared that when it was used, Apache would queue up the requests. The end-result from the user perspective was that either response time would look really slow, or their browser would eventually timeout, and they'd get (in IE) a "The page cannot be displayed".
For the situation that I've been asked to look into, they don't want either of the above (slow response or "The page cannot be displayed"), but they want the user to get some kind of message like "System too busy. Please try again later.".
With mod_limitipconn (vs. MaxClients), when mod_limitipconn is "triggered", Apache immediately (well, almost immediately) sends a 503 response, and with a custom ErrorDocument, we can customize the message.
This works pretty well, but the problem is that since most pages have embedded content (css, images, etc.), and, without the working NoIPLimit directive, mod_limitipconn will send 503 responses indiscriminately, i.e., it can send a 503 response to a GET for a CSS, which makes the pages that the user's see look broken.
I was hoping that I could get the NoIPLimit directive to work, then I could then configure mod_limitipconn to not count requests for things like CSS and GIFs, but per my original post, it looks like, for some reason the "content_type" that it's getting is always "text/plain". Actually, I think what's happening is when it calls that ap_...uri(), it's getting nothing, so it uses the default content type, which happens to be "text/plain".
Thanks,
Jim
Re: ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type always
returns 'text/plain'?
Posted by Eric Covener <co...@gmail.com>.
On Sat, Nov 14, 2009 at 7:37 PM, <oh...@cox.net> wrote:
> Our use case is slightly different that the original one for this module. The original code is designed to limit the number of connections from any given IP address, whereas in my case, we want to limit the total number of connections to the entire Apache server instance.
What does this get you over just setting MaxClients directly?
--
Eric Covener
covener@gmail.com
Re: May have found the culprit was Re: ap_sub_req_lookup_uri(r->uri,
r, NULL)->content_type always returns 'text/plain'?
Posted by Bob Ionescu <bo...@googlemail.com>.
2009/11/16 <oh...@cox.net>:
> Content-Type:
> - local ==> text/html
> - Via WebLogic Plugin ==> text/html; charset=UTF-8
>
> Could either of these (or both) differences be messing the mod_limitipconn module up, preventing it from getting the Content-Type properly?
Are you saying that "Via WebLogic Plugin ==> text/html; charset=UTF-8"
is being set before the content handler (proxy) runs or after?
The subrequest *lookup* won't invoke the content handler. You'll have
to call ap_run_sub_req() with the generated request_record (rr) of
your lookup to invoke the content handler.
Bob
Re: May have found the culprit was Re:
ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type always returns
'text/plain'?
Posted by oh...@cox.net.
---- ohaya@cox.net wrote:
> Hi,
>
> With further testing, I think that I've identified that mod_limitipconn is not able to get the Content-Type/content_type when the URL is being proxied to a WebLogic server via a plugin/module that comes with WebLogic (the "WebLogic Plugin for Apache").
>
> [NOTE: mod_limitipconn still DOES get the correct content_type for other URLs that are NOT being proxied by the WebLogic Plugin.]
>
> I've tried changing the LoadModule order, so that mod_limitipconn is both before and after the WebLogic plugin/module in httpd.conf, and the order doesn't seem to matter.
>
> I guess that this might take this thread slightly off-topic, except that I still don't understand why mod_limitipconn can't get the content_type, especially since the LoadModule order doesn't seem to affect things :(...
>
> Jim
Hi,
I've been continuing to try to figure this out, and quite frankly, I'm really stumped by this.
I don't know if this will help, but I enabled mod_dumpio, and recorded the responses both via the WebLogic plugin (when mod_limitipconn doesn't get the Content-Type) and for a GET for a resource (GIF) local to the Apache (when mod_limitipconn does get the correct Content-Type):
GOOD (GIF, local to Apache):
[Sat Nov 14 22:24:37 2009] [debug] mod_dumpio.c(74): mod_dumpio: dumpio_out (data-HEAP): HTTP/1.1 200 OK\r\nDate: Sun, 15 Nov 2009 03:24:37 GMT\r\nServer: Apache/2.2.14 (Unix)\r\nLast-Modified: Sat, 14 Nov 2009 23:41:34 GMT\r\nETag: "3220b-105-4785d510de780"\r\nAccept-Ranges: bytes\r\nContent-Length: 261\r\nKeep-Alive: timeout=5, max=99\r\nConnection: Keep-Alive\r\nContent-Type: image/gif\r\n\r\n
BAD (via the WebLogic Plugin/module):
[Sat Nov 14 22:17:24 2009] [debug] mod_dumpio.c(74): mod_dumpio: dumpio_out (data-HEAP): HTTP/1.1 200 OK\r\nDate: Sun, 15 Nov 2009 03:17:24 GMT\r\nServer: Apache/2.2.14 (Unix)\r\nX-Powered-By: Servlet/2.5 JSP/2.1\r\nKeep-Alive: timeout=5, max=100\r\nConnection: Keep-Alive\r\nTransfer-Encoding: chunked\r\nContent-Type: text/html; charset=UTF-8\r\n\r\n
Some of the things that I noticed that are different:
Transfer-Encoding:
- local ==> Response header Not present
- Via WebLogic Plugin ==> chunked
Content-Type:
- local ==> text/html
- Via WebLogic Plugin ==> text/html; charset=UTF-8
Could either of these (or both) differences be messing the mod_limitipconn module up, preventing it from getting the Content-Type properly?
Jim
May have found the culprit was Re: ap_sub_req_lookup_uri(r->uri,
r, NULL)->content_type always returns 'text/plain'?
Posted by oh...@cox.net.
Hi,
With further testing, I think that I've identified that mod_limitipconn is not able to get the Content-Type/content_type when the URL is being proxied to a WebLogic server via a plugin/module that comes with WebLogic (the "WebLogic Plugin for Apache").
[NOTE: mod_limitipconn still DOES get the correct content_type for other URLs that are NOT being proxied by the WebLogic Plugin.]
I've tried changing the LoadModule order, so that mod_limitipconn is both before and after the WebLogic plugin/module in httpd.conf, and the order doesn't seem to matter.
I guess that this might take this thread slightly off-topic, except that I still don't understand why mod_limitipconn can't get the content_type, especially since the LoadModule order doesn't seem to affect things :(...
Jim
Re: Static vs. Shared module precedence was Re:
ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type always returns
'text/plain'?
Posted by oh...@cox.net.
---- Eric Covener <co...@gmail.com> wrote:
> > Is the problem that the original Apache was on a 64-bit system?
> >
> > Could the Apache API on the 64-bit system be behaving differently than on a 32-bit system?
>
> Seems unlikely. I also checked and mod_limitipconn and mod_mime don't
> even operate in the same hook.
>
> Perhaps see the comments in mid_dir's usage of ap_sub_req_lookup_uri()?
>
> --
> Eric Covener
> covener@gmail.com
Eric,
I'm not clear what you meant by:
"Perhaps see the comments in mid_dir's usage of ap_sub_req_lookup_uri()?".
Did you mean "mod_dir"?
Jim
Re: Static vs. Shared module precedence was Re: ap_sub_req_lookup_uri(r->uri,
r, NULL)->content_type always returns 'text/plain'?
Posted by Eric Covener <co...@gmail.com>.
> Is the problem that the original Apache was on a 64-bit system?
>
> Could the Apache API on the 64-bit system be behaving differently than on a 32-bit system?
Seems unlikely. I also checked and mod_limitipconn and mod_mime don't
even operate in the same hook.
Perhaps see the comments in mid_dir's usage of ap_sub_req_lookup_uri()?
--
Eric Covener
covener@gmail.com
Re: Static vs. Shared module precedence was Re:
ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type always returns
'text/plain'?
Posted by oh...@cox.net.
---- ohaya@cox.net wrote:
>
> ---- ohaya@cox.net wrote:
> >
> > ---- ohaya@cox.net wrote:
> > >
> > > ---- ohaya@cox.net wrote:
> > > > Hi,
> > > >
> > > > I'm not sure if this is the appropriate list, or if I should post on the regular httpd list...
> > > >
> > > > I've been trying to work with an Apache module, mod_limitipconn:
> > > >
> > > > http://dominia.org/djao/limitipconn2.html
> > > >
> > > > This is with Apache 2.2.8/2.2.11.
> > > >
> > > > Our use case is slightly different that the original one for this module. The original code is designed to limit the number of connections from any given IP address, whereas in my case, we want to limit the total number of connections to the entire Apache server instance.
> > > >
> > > > To do that, I commented out one line (an 'if'), as shown below:
> > > >
> > > > /* Count up the number of connections we are handling right now from
> > > > * this IP address */
> > > > /* This is modified to not do strcmp(address, ws_record->client check, in order
> > > > to count ALL connections to this server. */
> > > > for (i = 0; i < server_limit; ++i) {
> > > > for (j = 0; j < thread_limit; ++j) {
> > > > ws_record = ap_get_scoreboard_worker(i, j);
> > > > switch (ws_record->status) {
> > > > case SERVER_BUSY_READ:
> > > > case SERVER_BUSY_WRITE:
> > > > case SERVER_BUSY_KEEPALIVE:
> > > > case SERVER_BUSY_LOG:
> > > > case SERVER_BUSY_DNS:
> > > > case SERVER_CLOSING:
> > > > case SERVER_GRACEFUL:
> > > > /*
> > > > if (strcmp(address, ws_record->client) == 0)
> > > > */
> > > > ip_count++;
> > > > break;
> > > > default:
> > > > break;
> > > > }
> > > > }
> > > > }
> > > >
> > > > and, after I rebuilt the module, it seems to work ok.
> > > >
> > > > Now, I'm trying to get the module's directive "NoIPLimit" to work.
> > > >
> > > > According to the README, plus reading the code, the way that this is suppose to work is that if you specify a MIME type string (substring, actually), the module won't "count" requests that have any of the MIME types in the directive.
> > > >
> > > > I have the Apache LogLevel set to debug, and it looks like the "content_type" that the module is retrieving from ALL requests is "text/plain", and I have confirmed that the actual MIME types are not "text/plain" (I see things like "image/gif", "text/css", etc.), so the "NoIPLimit" directive doesn't work.
> > > >
> > > > From reading the code, I think that the "content_type" is set in this snippet:
> > > >
> > > > /* Only check the MIME-type if we have MIME-type stuff in our config.
> > > > The extra subreq can be quite expensive. */
> > > > if(cfg->no_limit->nelts > 0 || cfg->excl_limit->nelts > 0) {
> > > > /* Look up the Content-type of this request. We need a subrequest
> > > > * here since this module might be called before the URI has been
> > > > * translated into a MIME type. */
> > > > content_type = ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type;
> > > >
> > > > /* If there's no Content-type, use the default. */
> > > > if (!content_type) {
> > > > content_type = ap_default_type(r);
> > > > }
> > > >
> > > > ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
> > > > "mod_limitipconn: uri: %s Content-Type: %s",
> > > > r->uri, content_type);
> > > >
> > > > /* Cycle through the exempt list; if our content_type is exempt,
> > > > * return OK */
> > > > for (i = 0; i < cfg->no_limit->nelts; i++) {
> > > > if ((ap_strcasecmp_match(content_type, nolim[i]) == 0)
> > > > || (strncmp(nolim[i], content_type, strlen(nolim[i])) == 0))
> > > > {
> > > > ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
> > > > "mod_limitipconn: OK: %s exempt", content_type);
> > > > return DECLINED;
> > > > }
> > > > }
> > > >
> > > > and, in particular:
> > > >
> > > > content_type = ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type;
> > > >
> > > > /* If there's no Content-type, use the default. */
> > > > if (!content_type) {
> > > > content_type = ap_default_type(r);
> > > > }
> > > >
> > > > I'm not that familiar with Apache module development, but reading the docs, it *seems* like the above *should* work.
> > > >
> > > > I'm assuming you all on this list *are* familiar with module development, so I was wondering if you might be able to tell me what, if anything, is wrong with that code (i.e., why is "content_type" not being set with the actual content type?
> > > >
> > > > Thanks in advance!!
> > > >
> > > > Jim
> > >
> > > Hi,
> > >
> > > BTW, here's an example of the Apache debug output for mod_limitipconn that illustrates what I'm seeing:
> > >
> > > [Sun Nov 15 02:21:09 2009] [debug] mod_limitipconn.c(143): [client 192.168.xx.yy] mod_limitipconn: uri: /anewwsclient/css/body.css Content-Type: text/plain, referer: http://foo.host.com/anewwsclient
> > >
> > > As you can see, the URI is for a .css, but the Content-Type comes back as "text/plain, referer: http://foo.host.com/anewwsclient", then, since the mod_limitipconn code checks for the MIME types in the NoIPLimit directive, (e.g., image/* text/css/*), it fails to detect this as a MIME type that it should ignore.
> > >
> > > Shouldn't the Content-Type be something like "text/css, referer: http://foo.host.com/anewwsclient"?
> > >
> > > Jim
> > >
> >
> >
> > Hi,
> >
> > I just noticed that when I run "apachectl -M", I get:
> >
> > bash-3.00# ./apachectl -M
> > Loaded Modules:
> > core_module (static)
> > authn_file_module (static)
> > authn_default_module (static)
> > authz_host_module (static)
> > authz_groupfile_module (static)
> > authz_user_module (static)
> > authz_default_module (static)
> > auth_basic_module (static)
> > include_module (static)
> > filter_module (static)
> > log_config_module (static)
> > env_module (static)
> > setenvif_module (static)
> > ssl_module (static)
> > mpm_prefork_module (static)
> > http_module (static)
> > mime_module (static)
> > status_module (static)
> > autoindex_module (static)
> > asis_module (static)
> > cgi_module (static)
> > negotiation_module (static)
> > dir_module (static)
> > actions_module (static)
> > userdir_module (static)
> > alias_module (static)
> > so_module (static)
> > limitipconn_module (shared)
> > Syntax OK
> >
> > Notice that the static modules apparently are listed before the shared module. I'm assuming that this list is in module, order, so it seems like the limitipconn module is "first".
> >
> > I'm wondering if *this* might be my problem, i.e., since the mod_limitipconn is first, and the mod_mime (which maps the suffix to mime type) is "after" the mod_limitipconn, if that might be why mod_limitipconn is apparently not able to determine the correct MIME type?
> >
> > Jim
> >
>
> Hi,
>
> To try to test the "precedence" theory, I built a new Apache 2.2.14, with all shared module. I then built a new copy of mod_limitipconn.so.
>
> Now, here's the weird thing :(...
>
> When I run this new Apache with the new mod_limitipconn.so, and check in the error_log file, it is now SHOWING the PROPER Content-Type!!
>
> Here's the 'apachectl -M' and '-V' from this new build:
>
> [root@fed9 htdocs]# ../bin/apachectl -M
> Loaded Modules:
> core_module (static)
> mpm_prefork_module (static)
> http_module (static)
> so_module (static)
> authn_file_module (shared)
> authn_dbm_module (shared)
> authn_anon_module (shared)
> authn_dbd_module (shared)
> authn_default_module (shared)
> authz_host_module (shared)
> authz_groupfile_module (shared)
> authz_user_module (shared)
> authz_dbm_module (shared)
> authz_owner_module (shared)
> authz_default_module (shared)
> auth_basic_module (shared)
> auth_digest_module (shared)
> dbd_module (shared)
> dumpio_module (shared)
> ext_filter_module (shared)
> include_module (shared)
> filter_module (shared)
> substitute_module (shared)
> deflate_module (shared)
> log_config_module (shared)
> log_forensic_module (shared)
> logio_module (shared)
> env_module (shared)
> mime_magic_module (shared)
> cern_meta_module (shared)
> expires_module (shared)
> headers_module (shared)
> ident_module (shared)
> usertrack_module (shared)
> unique_id_module (shared)
> setenvif_module (shared)
> version_module (shared)
> mime_module (shared)
> dav_module (shared)
> status_module (shared)
> autoindex_module (shared)
> asis_module (shared)
> info_module (shared)
> cgi_module (shared)
> dav_fs_module (shared)
> vhost_alias_module (shared)
> negotiation_module (shared)
> dir_module (shared)
> imagemap_module (shared)
> actions_module (shared)
> speling_module (shared)
> userdir_module (shared)
> alias_module (shared)
> rewrite_module (shared)
> limitipconn_module (shared)
> Syntax OK
> [root@fed9 htdocs]# ../bin/apachectl -V
> Server version: Apache/2.2.14 (Unix)
> Server built: Nov 14 2009 18:20:52
> Server's Module Magic Number: 20051115:23
> Server loaded: APR 1.3.9, APR-Util 1.3.9
> Compiled using: APR 1.3.9, APR-Util 1.3.9
> Architecture: 32-bit
> Server MPM: Prefork
> threaded: no
> forked: yes (variable process count)
> Server compiled with....
> -D APACHE_MPM_DIR="server/mpm/prefork"
> -D APR_HAS_SENDFILE
> -D APR_HAS_MMAP
> -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
> -D APR_USE_SYSVSEM_SERIALIZE
> -D APR_USE_PTHREAD_SERIALIZE
> -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
> -D APR_HAS_OTHER_CHILD
> -D AP_HAVE_RELIABLE_PIPED_LOGS
> -D DYNAMIC_MODULE_LIMIT=128
> -D HTTPD_ROOT="/apps/httpd-2.2.14"
> -D SUEXEC_BIN="/apps/httpd-2.2.14/bin/suexec"
> -D DEFAULT_PIDLOG="logs/httpd.pid"
> -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
> -D DEFAULT_LOCKFILE="logs/accept.lock"
> -D DEFAULT_ERRORLOG="logs/error_log"
> -D AP_TYPES_CONFIG_FILE="conf/mime.types"
> -D SERVER_CONFIG_FILE="conf/httpd.conf"
> [root@fed9 htdocs]#
>
> The above looks similar to the Apache that doesn't work. The main thing is the old Apache had all modules except for mod_limitipconnn.so as "static", whereas the new Apache has all modules as "shared", and the old Apache was on 64-bit Redhat, whereas the new Apache is on 32-bit Fedora 9.
>
> Anyone have any ideas about what might be causing the missing Content-Type on the old Apache?
>
> Thanks,
> Jim
>
>
Hi,
To try to eliminate the possibilities, I built another new Apache, but with all static modules. Again, this will on a 32-bit Fedora 9.
I then rebuilt mod_limitipconn.so.
When I tested this new Apache+mod_limitipconn.so, and check the Apache error_log, it is STILL displaying the Content-Type correctly!!
So, I am still puzzled at this point.
Is the problem that the original Apache was on a 64-bit system?
Could the Apache API on the 64-bit system be behaving differently than on a 32-bit system?
Anyone?
Thanks,
Jim
Re: Static vs. Shared module precedence was Re:
ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type always returns
'text/plain'?
Posted by oh...@cox.net.
---- ohaya@cox.net wrote:
>
> ---- ohaya@cox.net wrote:
> >
> > ---- ohaya@cox.net wrote:
> > > Hi,
> > >
> > > I'm not sure if this is the appropriate list, or if I should post on the regular httpd list...
> > >
> > > I've been trying to work with an Apache module, mod_limitipconn:
> > >
> > > http://dominia.org/djao/limitipconn2.html
> > >
> > > This is with Apache 2.2.8/2.2.11.
> > >
> > > Our use case is slightly different that the original one for this module. The original code is designed to limit the number of connections from any given IP address, whereas in my case, we want to limit the total number of connections to the entire Apache server instance.
> > >
> > > To do that, I commented out one line (an 'if'), as shown below:
> > >
> > > /* Count up the number of connections we are handling right now from
> > > * this IP address */
> > > /* This is modified to not do strcmp(address, ws_record->client check, in order
> > > to count ALL connections to this server. */
> > > for (i = 0; i < server_limit; ++i) {
> > > for (j = 0; j < thread_limit; ++j) {
> > > ws_record = ap_get_scoreboard_worker(i, j);
> > > switch (ws_record->status) {
> > > case SERVER_BUSY_READ:
> > > case SERVER_BUSY_WRITE:
> > > case SERVER_BUSY_KEEPALIVE:
> > > case SERVER_BUSY_LOG:
> > > case SERVER_BUSY_DNS:
> > > case SERVER_CLOSING:
> > > case SERVER_GRACEFUL:
> > > /*
> > > if (strcmp(address, ws_record->client) == 0)
> > > */
> > > ip_count++;
> > > break;
> > > default:
> > > break;
> > > }
> > > }
> > > }
> > >
> > > and, after I rebuilt the module, it seems to work ok.
> > >
> > > Now, I'm trying to get the module's directive "NoIPLimit" to work.
> > >
> > > According to the README, plus reading the code, the way that this is suppose to work is that if you specify a MIME type string (substring, actually), the module won't "count" requests that have any of the MIME types in the directive.
> > >
> > > I have the Apache LogLevel set to debug, and it looks like the "content_type" that the module is retrieving from ALL requests is "text/plain", and I have confirmed that the actual MIME types are not "text/plain" (I see things like "image/gif", "text/css", etc.), so the "NoIPLimit" directive doesn't work.
> > >
> > > From reading the code, I think that the "content_type" is set in this snippet:
> > >
> > > /* Only check the MIME-type if we have MIME-type stuff in our config.
> > > The extra subreq can be quite expensive. */
> > > if(cfg->no_limit->nelts > 0 || cfg->excl_limit->nelts > 0) {
> > > /* Look up the Content-type of this request. We need a subrequest
> > > * here since this module might be called before the URI has been
> > > * translated into a MIME type. */
> > > content_type = ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type;
> > >
> > > /* If there's no Content-type, use the default. */
> > > if (!content_type) {
> > > content_type = ap_default_type(r);
> > > }
> > >
> > > ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
> > > "mod_limitipconn: uri: %s Content-Type: %s",
> > > r->uri, content_type);
> > >
> > > /* Cycle through the exempt list; if our content_type is exempt,
> > > * return OK */
> > > for (i = 0; i < cfg->no_limit->nelts; i++) {
> > > if ((ap_strcasecmp_match(content_type, nolim[i]) == 0)
> > > || (strncmp(nolim[i], content_type, strlen(nolim[i])) == 0))
> > > {
> > > ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
> > > "mod_limitipconn: OK: %s exempt", content_type);
> > > return DECLINED;
> > > }
> > > }
> > >
> > > and, in particular:
> > >
> > > content_type = ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type;
> > >
> > > /* If there's no Content-type, use the default. */
> > > if (!content_type) {
> > > content_type = ap_default_type(r);
> > > }
> > >
> > > I'm not that familiar with Apache module development, but reading the docs, it *seems* like the above *should* work.
> > >
> > > I'm assuming you all on this list *are* familiar with module development, so I was wondering if you might be able to tell me what, if anything, is wrong with that code (i.e., why is "content_type" not being set with the actual content type?
> > >
> > > Thanks in advance!!
> > >
> > > Jim
> >
> > Hi,
> >
> > BTW, here's an example of the Apache debug output for mod_limitipconn that illustrates what I'm seeing:
> >
> > [Sun Nov 15 02:21:09 2009] [debug] mod_limitipconn.c(143): [client 192.168.xx.yy] mod_limitipconn: uri: /anewwsclient/css/body.css Content-Type: text/plain, referer: http://foo.host.com/anewwsclient
> >
> > As you can see, the URI is for a .css, but the Content-Type comes back as "text/plain, referer: http://foo.host.com/anewwsclient", then, since the mod_limitipconn code checks for the MIME types in the NoIPLimit directive, (e.g., image/* text/css/*), it fails to detect this as a MIME type that it should ignore.
> >
> > Shouldn't the Content-Type be something like "text/css, referer: http://foo.host.com/anewwsclient"?
> >
> > Jim
> >
>
>
> Hi,
>
> I just noticed that when I run "apachectl -M", I get:
>
> bash-3.00# ./apachectl -M
> Loaded Modules:
> core_module (static)
> authn_file_module (static)
> authn_default_module (static)
> authz_host_module (static)
> authz_groupfile_module (static)
> authz_user_module (static)
> authz_default_module (static)
> auth_basic_module (static)
> include_module (static)
> filter_module (static)
> log_config_module (static)
> env_module (static)
> setenvif_module (static)
> ssl_module (static)
> mpm_prefork_module (static)
> http_module (static)
> mime_module (static)
> status_module (static)
> autoindex_module (static)
> asis_module (static)
> cgi_module (static)
> negotiation_module (static)
> dir_module (static)
> actions_module (static)
> userdir_module (static)
> alias_module (static)
> so_module (static)
> limitipconn_module (shared)
> Syntax OK
>
> Notice that the static modules apparently are listed before the shared module. I'm assuming that this list is in module, order, so it seems like the limitipconn module is "first".
>
> I'm wondering if *this* might be my problem, i.e., since the mod_limitipconn is first, and the mod_mime (which maps the suffix to mime type) is "after" the mod_limitipconn, if that might be why mod_limitipconn is apparently not able to determine the correct MIME type?
>
> Jim
>
Hi,
To try to test the "precedence" theory, I built a new Apache 2.2.14, with all shared module. I then built a new copy of mod_limitipconn.so.
Now, here's the weird thing :(...
When I run this new Apache with the new mod_limitipconn.so, and check in the error_log file, it is now SHOWING the PROPER Content-Type!!
Here's the 'apachectl -M' and '-V' from this new build:
[root@fed9 htdocs]# ../bin/apachectl -M
Loaded Modules:
core_module (static)
mpm_prefork_module (static)
http_module (static)
so_module (static)
authn_file_module (shared)
authn_dbm_module (shared)
authn_anon_module (shared)
authn_dbd_module (shared)
authn_default_module (shared)
authz_host_module (shared)
authz_groupfile_module (shared)
authz_user_module (shared)
authz_dbm_module (shared)
authz_owner_module (shared)
authz_default_module (shared)
auth_basic_module (shared)
auth_digest_module (shared)
dbd_module (shared)
dumpio_module (shared)
ext_filter_module (shared)
include_module (shared)
filter_module (shared)
substitute_module (shared)
deflate_module (shared)
log_config_module (shared)
log_forensic_module (shared)
logio_module (shared)
env_module (shared)
mime_magic_module (shared)
cern_meta_module (shared)
expires_module (shared)
headers_module (shared)
ident_module (shared)
usertrack_module (shared)
unique_id_module (shared)
setenvif_module (shared)
version_module (shared)
mime_module (shared)
dav_module (shared)
status_module (shared)
autoindex_module (shared)
asis_module (shared)
info_module (shared)
cgi_module (shared)
dav_fs_module (shared)
vhost_alias_module (shared)
negotiation_module (shared)
dir_module (shared)
imagemap_module (shared)
actions_module (shared)
speling_module (shared)
userdir_module (shared)
alias_module (shared)
rewrite_module (shared)
limitipconn_module (shared)
Syntax OK
[root@fed9 htdocs]# ../bin/apachectl -V
Server version: Apache/2.2.14 (Unix)
Server built: Nov 14 2009 18:20:52
Server's Module Magic Number: 20051115:23
Server loaded: APR 1.3.9, APR-Util 1.3.9
Compiled using: APR 1.3.9, APR-Util 1.3.9
Architecture: 32-bit
Server MPM: Prefork
threaded: no
forked: yes (variable process count)
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/prefork"
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=128
-D HTTPD_ROOT="/apps/httpd-2.2.14"
-D SUEXEC_BIN="/apps/httpd-2.2.14/bin/suexec"
-D DEFAULT_PIDLOG="logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_LOCKFILE="logs/accept.lock"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"
[root@fed9 htdocs]#
The above looks similar to the Apache that doesn't work. The main thing is the old Apache had all modules except for mod_limitipconnn.so as "static", whereas the new Apache has all modules as "shared", and the old Apache was on 64-bit Redhat, whereas the new Apache is on 32-bit Fedora 9.
Anyone have any ideas about what might be causing the missing Content-Type on the old Apache?
Thanks,
Jim
Static vs. Shared module precedence was Re:
ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type always returns
'text/plain'?
Posted by oh...@cox.net.
---- ohaya@cox.net wrote:
>
> ---- ohaya@cox.net wrote:
> > Hi,
> >
> > I'm not sure if this is the appropriate list, or if I should post on the regular httpd list...
> >
> > I've been trying to work with an Apache module, mod_limitipconn:
> >
> > http://dominia.org/djao/limitipconn2.html
> >
> > This is with Apache 2.2.8/2.2.11.
> >
> > Our use case is slightly different that the original one for this module. The original code is designed to limit the number of connections from any given IP address, whereas in my case, we want to limit the total number of connections to the entire Apache server instance.
> >
> > To do that, I commented out one line (an 'if'), as shown below:
> >
> > /* Count up the number of connections we are handling right now from
> > * this IP address */
> > /* This is modified to not do strcmp(address, ws_record->client check, in order
> > to count ALL connections to this server. */
> > for (i = 0; i < server_limit; ++i) {
> > for (j = 0; j < thread_limit; ++j) {
> > ws_record = ap_get_scoreboard_worker(i, j);
> > switch (ws_record->status) {
> > case SERVER_BUSY_READ:
> > case SERVER_BUSY_WRITE:
> > case SERVER_BUSY_KEEPALIVE:
> > case SERVER_BUSY_LOG:
> > case SERVER_BUSY_DNS:
> > case SERVER_CLOSING:
> > case SERVER_GRACEFUL:
> > /*
> > if (strcmp(address, ws_record->client) == 0)
> > */
> > ip_count++;
> > break;
> > default:
> > break;
> > }
> > }
> > }
> >
> > and, after I rebuilt the module, it seems to work ok.
> >
> > Now, I'm trying to get the module's directive "NoIPLimit" to work.
> >
> > According to the README, plus reading the code, the way that this is suppose to work is that if you specify a MIME type string (substring, actually), the module won't "count" requests that have any of the MIME types in the directive.
> >
> > I have the Apache LogLevel set to debug, and it looks like the "content_type" that the module is retrieving from ALL requests is "text/plain", and I have confirmed that the actual MIME types are not "text/plain" (I see things like "image/gif", "text/css", etc.), so the "NoIPLimit" directive doesn't work.
> >
> > From reading the code, I think that the "content_type" is set in this snippet:
> >
> > /* Only check the MIME-type if we have MIME-type stuff in our config.
> > The extra subreq can be quite expensive. */
> > if(cfg->no_limit->nelts > 0 || cfg->excl_limit->nelts > 0) {
> > /* Look up the Content-type of this request. We need a subrequest
> > * here since this module might be called before the URI has been
> > * translated into a MIME type. */
> > content_type = ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type;
> >
> > /* If there's no Content-type, use the default. */
> > if (!content_type) {
> > content_type = ap_default_type(r);
> > }
> >
> > ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
> > "mod_limitipconn: uri: %s Content-Type: %s",
> > r->uri, content_type);
> >
> > /* Cycle through the exempt list; if our content_type is exempt,
> > * return OK */
> > for (i = 0; i < cfg->no_limit->nelts; i++) {
> > if ((ap_strcasecmp_match(content_type, nolim[i]) == 0)
> > || (strncmp(nolim[i], content_type, strlen(nolim[i])) == 0))
> > {
> > ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
> > "mod_limitipconn: OK: %s exempt", content_type);
> > return DECLINED;
> > }
> > }
> >
> > and, in particular:
> >
> > content_type = ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type;
> >
> > /* If there's no Content-type, use the default. */
> > if (!content_type) {
> > content_type = ap_default_type(r);
> > }
> >
> > I'm not that familiar with Apache module development, but reading the docs, it *seems* like the above *should* work.
> >
> > I'm assuming you all on this list *are* familiar with module development, so I was wondering if you might be able to tell me what, if anything, is wrong with that code (i.e., why is "content_type" not being set with the actual content type?
> >
> > Thanks in advance!!
> >
> > Jim
>
> Hi,
>
> BTW, here's an example of the Apache debug output for mod_limitipconn that illustrates what I'm seeing:
>
> [Sun Nov 15 02:21:09 2009] [debug] mod_limitipconn.c(143): [client 192.168.xx.yy] mod_limitipconn: uri: /anewwsclient/css/body.css Content-Type: text/plain, referer: http://foo.host.com/anewwsclient
>
> As you can see, the URI is for a .css, but the Content-Type comes back as "text/plain, referer: http://foo.host.com/anewwsclient", then, since the mod_limitipconn code checks for the MIME types in the NoIPLimit directive, (e.g., image/* text/css/*), it fails to detect this as a MIME type that it should ignore.
>
> Shouldn't the Content-Type be something like "text/css, referer: http://foo.host.com/anewwsclient"?
>
> Jim
>
Hi,
I just noticed that when I run "apachectl -M", I get:
bash-3.00# ./apachectl -M
Loaded Modules:
core_module (static)
authn_file_module (static)
authn_default_module (static)
authz_host_module (static)
authz_groupfile_module (static)
authz_user_module (static)
authz_default_module (static)
auth_basic_module (static)
include_module (static)
filter_module (static)
log_config_module (static)
env_module (static)
setenvif_module (static)
ssl_module (static)
mpm_prefork_module (static)
http_module (static)
mime_module (static)
status_module (static)
autoindex_module (static)
asis_module (static)
cgi_module (static)
negotiation_module (static)
dir_module (static)
actions_module (static)
userdir_module (static)
alias_module (static)
so_module (static)
limitipconn_module (shared)
Syntax OK
Notice that the static modules apparently are listed before the shared module. I'm assuming that this list is in module, order, so it seems like the limitipconn module is "first".
I'm wondering if *this* might be my problem, i.e., since the mod_limitipconn is first, and the mod_mime (which maps the suffix to mime type) is "after" the mod_limitipconn, if that might be why mod_limitipconn is apparently not able to determine the correct MIME type?
Jim
Re: ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type always
returns 'text/plain'?
Posted by oh...@cox.net.
---- ohaya@cox.net wrote:
> Hi,
>
> I'm not sure if this is the appropriate list, or if I should post on the regular httpd list...
>
> I've been trying to work with an Apache module, mod_limitipconn:
>
> http://dominia.org/djao/limitipconn2.html
>
> This is with Apache 2.2.8/2.2.11.
>
> Our use case is slightly different that the original one for this module. The original code is designed to limit the number of connections from any given IP address, whereas in my case, we want to limit the total number of connections to the entire Apache server instance.
>
> To do that, I commented out one line (an 'if'), as shown below:
>
> /* Count up the number of connections we are handling right now from
> * this IP address */
> /* This is modified to not do strcmp(address, ws_record->client check, in order
> to count ALL connections to this server. */
> for (i = 0; i < server_limit; ++i) {
> for (j = 0; j < thread_limit; ++j) {
> ws_record = ap_get_scoreboard_worker(i, j);
> switch (ws_record->status) {
> case SERVER_BUSY_READ:
> case SERVER_BUSY_WRITE:
> case SERVER_BUSY_KEEPALIVE:
> case SERVER_BUSY_LOG:
> case SERVER_BUSY_DNS:
> case SERVER_CLOSING:
> case SERVER_GRACEFUL:
> /*
> if (strcmp(address, ws_record->client) == 0)
> */
> ip_count++;
> break;
> default:
> break;
> }
> }
> }
>
> and, after I rebuilt the module, it seems to work ok.
>
> Now, I'm trying to get the module's directive "NoIPLimit" to work.
>
> According to the README, plus reading the code, the way that this is suppose to work is that if you specify a MIME type string (substring, actually), the module won't "count" requests that have any of the MIME types in the directive.
>
> I have the Apache LogLevel set to debug, and it looks like the "content_type" that the module is retrieving from ALL requests is "text/plain", and I have confirmed that the actual MIME types are not "text/plain" (I see things like "image/gif", "text/css", etc.), so the "NoIPLimit" directive doesn't work.
>
> From reading the code, I think that the "content_type" is set in this snippet:
>
> /* Only check the MIME-type if we have MIME-type stuff in our config.
> The extra subreq can be quite expensive. */
> if(cfg->no_limit->nelts > 0 || cfg->excl_limit->nelts > 0) {
> /* Look up the Content-type of this request. We need a subrequest
> * here since this module might be called before the URI has been
> * translated into a MIME type. */
> content_type = ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type;
>
> /* If there's no Content-type, use the default. */
> if (!content_type) {
> content_type = ap_default_type(r);
> }
>
> ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
> "mod_limitipconn: uri: %s Content-Type: %s",
> r->uri, content_type);
>
> /* Cycle through the exempt list; if our content_type is exempt,
> * return OK */
> for (i = 0; i < cfg->no_limit->nelts; i++) {
> if ((ap_strcasecmp_match(content_type, nolim[i]) == 0)
> || (strncmp(nolim[i], content_type, strlen(nolim[i])) == 0))
> {
> ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
> "mod_limitipconn: OK: %s exempt", content_type);
> return DECLINED;
> }
> }
>
> and, in particular:
>
> content_type = ap_sub_req_lookup_uri(r->uri, r, NULL)->content_type;
>
> /* If there's no Content-type, use the default. */
> if (!content_type) {
> content_type = ap_default_type(r);
> }
>
> I'm not that familiar with Apache module development, but reading the docs, it *seems* like the above *should* work.
>
> I'm assuming you all on this list *are* familiar with module development, so I was wondering if you might be able to tell me what, if anything, is wrong with that code (i.e., why is "content_type" not being set with the actual content type?
>
> Thanks in advance!!
>
> Jim
Hi,
BTW, here's an example of the Apache debug output for mod_limitipconn that illustrates what I'm seeing:
[Sun Nov 15 02:21:09 2009] [debug] mod_limitipconn.c(143): [client 192.168.xx.yy] mod_limitipconn: uri: /anewwsclient/css/body.css Content-Type: text/plain, referer: http://foo.host.com/anewwsclient
As you can see, the URI is for a .css, but the Content-Type comes back as "text/plain, referer: http://foo.host.com/anewwsclient", then, since the mod_limitipconn code checks for the MIME types in the NoIPLimit directive, (e.g., image/* text/css/*), it fails to detect this as a MIME type that it should ignore.
Shouldn't the Content-Type be something like "text/css, referer: http://foo.host.com/anewwsclient"?
Jim