general/6180: ErrorDocument 404 /404.cgi not working with MSIE 5.0

[David Lau <da...@lau.com>]

>Number:         6180
>Category:       general
>Synopsis:       ErrorDocument 404 /404.cgi not working with MSIE 5.0
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    apache
>State:          open
>Class:          sw-bug
>Submitter-Id:   apache
>Arrival-Date:   Mon Jun 12 20:30:03 PDT 2000
>Closed-Date:
>Last-Modified:
>Originator:     david@lau.com
>Release:        Apache/1.3.9 (Unix) mod_ssl/2.4.6 OpenSSL/0.9.4 PHP/3.0.12
>Organization:
apache
>Environment:
Linux red 2.2.12-32 #1 Mon Oct 25 19:56:23 EDT 1999 i686 unknown
>Description:
I apologize for the brain dump:

I'm using the "ErrorDocument 404 /404.cgi" in the httpd.conf to cause a perl cgi to do a database lookup.

Using "http://www.gatepost.com/1.html" as an example: The 404.cgi script basically rips http://www.gatepost.com/1.html into http://www.gatepost.com/  and 1.html   It then does the db lookup on http://www.gatepost.com, finding the desired URL, then it pops 1.html onto the end and spits that out into a frameset.  

So what's the problem?

The problem is we have moved the above process from a sun/solaris/dbm box to an intel/linux/berkeley db box.  The port involved absolutely minimal code changes but the problem is much easier to isolate.

404.cgi went from doing it's job really well to doing it's job really well in Netscape and about 1% of the time in MSIE!!!  1%?  That's odd....  it should either work or not work.  Time to troubleshoot.  Combed through the code making minor changes, no effect, combed through some more, nothing, tweaked, nothing.... sometimes the page would get forwarded other times just a regular 404 error would appear.  Spoke to some people and they said that the newer releases of MSIE try to be "intelligent" and assume that if you are running a 404.cgi script and the output is less than 512bytes then it'll display it's own 404 text error.

I got pretty fed up with tweaking the 404.cgi code and getting nowhere so I went into full test mode.  How about I replace 404.cgi with a 404.cgi that just says "hello world" a few times.  Hmmmm, doesn't work.  That means that the 404.cgi isn't being called.  Called the same page http://www.gatepost.com/asdfasfdafd in netscape and up pops "hello world".  Ok, remembering the 512byte rule, I make it display a whack of environment variables and "hello world" a bunch of times.....  still doesn't work in MSIE 5.0... hang on, it works on a couple of tries, then not, then on about 5% of the times.  Ok, that's not good enough.  Called up my SA and he looks into it. He thinks that maybe my perl code is not quite up to snuff and writes his own 404.cgi

I run a test on a bunch of nonexistant pages and I get 100% success.  So let's look at his script... Darn it, he wrote just a bunch of "echo" commands and didn't do it in perl.  I port his 404.cgi into perl to see if I indeed messed up and boom.... doesn't work.  When I say "doesn't work" I mean that the script runs perfectly if you type in http://www.gatepost.com/404.cgi but it doesn't get executed when I type in http://www.gatepost.com/asfadsf

I would think that 404.cgi isn't being called at all *except* it works flawlessly in Netscape.
Ok, then it's not working for MSIE... not true as I can get it to work if I click "reload" often enough (cache set to 0).
Must be a caching problem.... can't be as I'm typing in different URL's all the time.
Must be an apache problem with calling 404.cgi......  can't be as when the 404.cgi is written in cshell (my guess here) that it runs perfectly.

So my question to you is: "What is going on here?"

I currently can not get 404.cgi's results to show in MSIE 5.0 no matter how large or small the resultant page is.  I'm not doing anything complex in my test 404.cgi, just displaying text.
>How-To-Repeat:
This is easily set up with a basic 404.cgi and setting "ErrorDocument 404 /404.cgi" in httpd.conf
>Fix:
Is MSIE looking at the headers and "intelligently" desiding on using it's own canned error messages?  Why can I get this to work on Solaris and not RedHat?
>Release-Note:
>Audit-Trail:
>Unformatted:
 [In order for any reply to be added to the PR database, you need]
 [to include <ap...@Apache.Org> in the Cc line and make sure the]
 [subject line starts with the report component and number, with ]
 [or without any 'Re:' prefixes (such as "general/1098:" or      ]
 ["Re: general/1098:").  If the subject doesn't match this       ]
 [pattern, your message will be misfiled and ignored.  The       ]
 ["apbugs" address is not added to the Cc line of messages from  ]
 [the database automatically because of the potential for mail   ]
 [loops.  If you do not include this Cc, your reply may be ig-   ]
 [nored unless you are responding to an explicit request from a  ]
 [developer.  Reply only with text; DO NOT SEND ATTACHMENTS!     ]