problems with mod_proxy and mod_cache on 2.X

[Michal Mertl <mi...@traveller.cz>]

I don't know if you know about some persistent problems with $SUBJ. In
case you do, please ignore this posting.

I'm developping caching proxy server with antivirus. The best/cleanest
solution seems to me apache2 + mod_clamav. I had to fix mod_clamav
substantialy but now I stumbled across some bug in mod_proxy. I know
about several situations where the pages get loaded wrongly regardless
of the browser and or apache2 version (2.0.53 or 2.1.4). Fortunately the
situation is quite rare.

The best example is http://www.vtipky.cz/. The page loads well with
Apache1.3 base proxy.

Re: problems with mod_proxy and mod_cache on 2.X

[Michal Mertl <mi...@i.cz>]

Graham Leggett wrote:
> Michal Mertl wrote:
> 
> > I'd like to also have better error messages in case of problems with the
> > remote site - e.g. when I go to an address which doesn't have web server
> > running on it I get 404 instead of some message saying "connection
> > refused/timed out" or something. Do I have something configured wrong or
> > is it a problem with mod_proxy/mod_cache?
> 
> An error 404 indicates the resource doesn't exist. If a 404 was given 
> when the site was not available, the admin would be sent on a wild goose 
> chase trying to figure out why the file (or whatever) was missing, when 
> the real error was that the server wasn't running. The error codes have 
> very well defined meanings, and cannot be changed.

I don't understand what are you saying.

I'm complaining that the mod_proxy sends 404 page when I ask it to give
me page from something refusing connections on port 80.

When I ask mod_proxy to go no nonresolvable hostname it sends
(correctly) 502 Proxy Error and says - DNS lookup failure. When I ask it
to go to something refusing connections I'd expect to get 502 too and it
should say Connection refused (or timeout if that would be the case).

> 
> The human readable text that comes with the error message is definitely 
> improvable (if there is such a word), if you have specific examples of 
> error messages that are too short or too brief, post examples here.
> 
> Regards,
> Graham
> --

Re: problems with mod_proxy and mod_cache on 2.X

[Graham Leggett <mi...@sharp.fm>]

Michal Mertl wrote:

> I'd like to also have better error messages in case of problems with the
> remote site - e.g. when I go to an address which doesn't have web server
> running on it I get 404 instead of some message saying "connection
> refused/timed out" or something. Do I have something configured wrong or
> is it a problem with mod_proxy/mod_cache?

An error 404 indicates the resource doesn't exist. If a 404 was given 
when the site was not available, the admin would be sent on a wild goose 
chase trying to figure out why the file (or whatever) was missing, when 
the real error was that the server wasn't running. The error codes have 
very well defined meanings, and cannot be changed.

The human readable text that comes with the error message is definitely 
improvable (if there is such a word), if you have specific examples of 
error messages that are too short or too brief, post examples here.

Regards,
Graham
--

Re: problems with mod_proxy and mod_cache on 2.X

[Michal Mertl <mi...@traveller.cz>]

I wrote:
> I don't know if you know about some persistent problems with $SUBJ. In
> case you do, please ignore this posting.
> 
> I'm developping caching proxy server with antivirus. The best/cleanest
> solution seems to me apache2 + mod_clamav. I had to fix mod_clamav
> substantialy but now I stumbled across some bug in mod_proxy. I know
> about several situations where the pages get loaded wrongly regardless
> of the browser and or apache2 version (2.0.53 or 2.1.4). Fortunately the
> situation is quite rare.
> 
> The best example is http://www.vtipky.cz/. The page loads well with
> Apache1.3 base proxy.

I'm sorry to reply to my own post but I sent it by an accident.

I'd like to also have better error messages in case of problems with the
remote site - e.g. when I go to an address which doesn't have web server
running on it I get 404 instead of some message saying "connection
refused/timed out" or something. Do I have something configured wrong or
is it a problem with mod_proxy/mod_cache?

I can provide any needed info. If access to the machine I'm testing on
would be of any use I can provide it too.


Michal

Re: problems with mod_proxy and mod_cache on 2.X

[Michal Mertl <mi...@i.cz>]

Michal Mertl wrote:
> Graham Leggett wrote:
> > Michal Mertl wrote:
> > 
> > > I'm developping caching proxy server with antivirus. The best/cleanest
> > > solution seems to me apache2 + mod_clamav. I had to fix mod_clamav
> > > substantialy but now I stumbled across some bug in mod_proxy. I know
> > > about several situations where the pages get loaded wrongly regardless
> > > of the browser and or apache2 version (2.0.53 or 2.1.4). Fortunately the
> > > situation is quite rare.
> > > 
> > > The best example is http://www.vtipky.cz/. The page loads well with
> > > Apache1.3 base proxy.
> > 
> > Can you send more details on the problem that you found?
> 
> I really don't know exactly. I thought the problem is much more
> widespread but now it seems it's quite rare.
> 
> The problem seems to be in particular site (webserver) which responds
> from time to time to the same request differently. I've never seen error
> message in the browser when going direct of through Apache 1.3 based
> proxy. I suspect it was just luck I didn't notice it with 1.3 now. On
> the other hand I've seen it when using FreeBSD fetch utility (similar to
> wget) - it also fails the link from time to time with "unknown error"
> message.
> 
> Failing URL is for example this one (it's random):
> 
> http://adx1.adrenaline.cz/adrun.dll?action=1&id=4766&target=_top&border=1&bgcolor=ffffff&link=0000FF&ref=&rnd=662405
> 
> It's tempting to say it's the server's fault but because it doesn't
> appear in direct connection it's unfortunate. I managed to get tcpdump
> of a failed request with fetch. It really is server's fault - instead of
> replying with the data it just closes the connection. A second later the
> same request works. It seems the browser must react to the broken server
> by retrying while mod_proxy sends to the client the Bad Gateway error
> (which seems correct).
> 
> The bad thing is that the server in question is (I thing) pretty heavily
> used on some important pages on the Czech Internet. Unfortunately I
> don't browse much in there but the users will surely do so I can't be
> sure how often the problem appears. The server is an advertising server.
> 
> I'll check with squid - if it sends error to the client I would say it's
> a server's problem and give up on the issue. 

FYI: The page doesn't work with squid either. It only doesn't display as
visible an error but it operates the same.

In the squid log the problem can be seen like this:

1112876066.989     48 X.X.X.X TCP_MISS/000 1 GET
http://adx1.adrenaline.cz/adrun.dll? - DIRECT/194.213.53.212 -

Normally squid displays HTTP return code in place of '000'.

No big problem then.

Regards

Michal

Re: problems with mod_proxy and mod_cache on 2.X

[Michal Mertl <mi...@i.cz>]

Graham Leggett wrote:
> Michal Mertl wrote:
> 
> > I'm developping caching proxy server with antivirus. The best/cleanest
> > solution seems to me apache2 + mod_clamav. I had to fix mod_clamav
> > substantialy but now I stumbled across some bug in mod_proxy. I know
> > about several situations where the pages get loaded wrongly regardless
> > of the browser and or apache2 version (2.0.53 or 2.1.4). Fortunately the
> > situation is quite rare.
> > 
> > The best example is http://www.vtipky.cz/. The page loads well with
> > Apache1.3 base proxy.
> 
> Can you send more details on the problem that you found?

I really don't know exactly. I thought the problem is much more
widespread but now it seems it's quite rare.

The problem seems to be in particular site (webserver) which responds
from time to time to the same request differently. I've never seen error
message in the browser when going direct of through Apache 1.3 based
proxy. I suspect it was just luck I didn't notice it with 1.3 now. On
the other hand I've seen it when using FreeBSD fetch utility (similar to
wget) - it also fails the link from time to time with "unknown error"
message.

Failing URL is for example this one (it's random):

http://adx1.adrenaline.cz/adrun.dll?action=1&id=4766&target=_top&border=1&bgcolor=ffffff&link=0000FF&ref=&rnd=662405

It's tempting to say it's the server's fault but because it doesn't
appear in direct connection it's unfortunate. I managed to get tcpdump
of a failed request with fetch. It really is server's fault - instead of
replying with the data it just closes the connection. A second later the
same request works. It seems the browser must react to the broken server
by retrying while mod_proxy sends to the client the Bad Gateway error
(which seems correct).

The bad thing is that the server in question is (I thing) pretty heavily
used on some important pages on the Czech Internet. Unfortunately I
don't browse much in there but the users will surely do so I can't be
sure how often the problem appears. The server is an advertising server.

I'll check with squid - if it sends error to the client I would say it's
a server's problem and give up on the issue. 


The other issue happens probably even more seldomly. It may be even
browser bug but it doesn't happen when going direct or with Apache 1.3
based proxy either. The problem appears only with Mozilla with proxy
configured as HTTP 1.1 and pipelining enabled. Because it's be default
off I don't think it's as bad as the first one (would affect small
number of users).

The problem appears when I go to URL
http://tools.cisco.com/Support/Fusion/FusionHome.do

You need to be registered Cisco customer (partner?) to be able to access
it. It's normal page with several images and stuff and the problem is
that several of the images don't load. It happens always. When I sniffed
the traffic I didn't see the browser asking for the image even so it
looks like an issue in the browser. Still why doesn't it happen with
direct connection?

Thanks

Michal

Re: problems with mod_proxy and mod_cache on 2.X

[Graham Leggett <mi...@sharp.fm>]

Michal Mertl wrote:

> I'm developping caching proxy server with antivirus. The best/cleanest
> solution seems to me apache2 + mod_clamav. I had to fix mod_clamav
> substantialy but now I stumbled across some bug in mod_proxy. I know
> about several situations where the pages get loaded wrongly regardless
> of the browser and or apache2 version (2.0.53 or 2.1.4). Fortunately the
> situation is quite rare.
> 
> The best example is http://www.vtipky.cz/. The page loads well with
> Apache1.3 base proxy.

Can you send more details on the problem that you found?

Regards,
Graham
--