cgid on hp-ux - read hangs because shutdown doesn't close

[Gary Feldman <ga...@rtr.com>]

I'm running into a problem with some code based on the
mod_cgid module, on hp-ux.  I've also submitted
the underlying problem to comp.systems.hp.hpux.

The problem is that the underlying cgi hangs on a read
from stdin when there is actually no data left to be read.
I can reduce this to a simple discrepency between HP-UX
and other platforms, Linux being my example.  On Linux,
when the socket shutdown(sd, 1) is done by the sender, the 
read on the socket returns with eof.  On HP-UX, this
doesn't happen until the socket is actually closed.

My code happens to be based on 2.0.40, though I've looked
at the changes in 2.0.44 and they don't seem relevant 
(the changes are there to catch EINTR, but I don't see
any way to trigger that).

Any advice would be appreciated.

Gary

PS: I originally submitted this to the users list, where
someone suggested I resubmit here.

====================================================================
                  Ready-to-Run Software, Inc.  
              The Industry's Leading Provider of
              Cross-Platform Software Services
                 *****************************
email: gaf@rtr.com                 Gary Feldman 
fax  : 1-978-692-5401              Ready-to-Run Software, Inc.
voice: 1-978-251-5431              11 School Street
www  : http://www.rtr.com          North Chelmsford, MA 01863
                                                     USA
  

Re: cgid on hp-ux - read hangs because shutdown doesn't close

[Jeff Trawick <tr...@attglobal.net>]

Jeff Trawick wrote:

> If somebody doesn't care about local security, mod_cgid should be able
> to see something like
>
> scriptsock 127.0.0.1:9999
>
> in the config file and use a TCP socket instead of Unix socket for
> communication with the daemon.
>
> Alternatively the handler could store the request body in a temporary
> file and have the daemon set the child's stdin to that temporary file.


no, this is a bad idea...  it has to be something with flow control and 
limited buffering...  you can't have the handler sucking up 1GB of 
request body to stuff in a temp file before the CGI starts reading... 
and you also can't have a situation where a CGI reading faster than we 
read from the network thinks it hit the end of the data

> Alternatively some sort of named pipe could be used instead of a Unix
> socket.

Re: cgid on hp-ux - read hangs because shutdown doesn't close

[Jeff Trawick <tr...@attglobal.net>]

> >So use mod_cgi :)
> >
> >(If there isn't a compelling performance difference, that makes sense
> >from Apache's perspective because mod_cgid is unfortunately
> >complicated
> >and it isn't worth more tricks if there's no sense in using it in the
> >first place.)
>
>
> My understanding of the world is that mod_cgid is necessary
> for decent performance with worker threads.


That is what I've always been told.  I just did a few very unscientific 
tests using the default worker MPM config (25 threads per child I think) 
and found that mod_cgi was consistently around 12.5% slower.  Note that 
this was a test where all the server was doing was printenv.

Unfortunately I also found that mod_cgid was not at all reliable even 
for printenv, so potentially some of the performance boost was due to 
some cheap error paths.

Here's a sampling of stuff in the error log:

(9)Bad file number: write to cgi daemon process
Premature end of script headers: printenv
daemon couldn't find CGI process for connection 7
(232)Connection reset by peer: Error reading request on cgid socket
(9)Bad file number: ap_content_length_filter: apr_bucket_read() failed

The "daemon couldn't find CGI process for connection 7" error probably 
has my name on it.  I suspect some fatal error already occurred on the 
request and we neglected to avoid some later cleanup which, given the 
error, is not appropriate.

> Just as supporting mod_cgi.c and mod_cgid.c adds to Apache maintenance
> work, for us to have to support both models would be prohibitive.
> Hence my desire for a good workaround.

Apache has to support both mod_cgi.c and mod_cgid.c anyway, broken [on] 
HP-UX or not.  Your personal mileage may vary :)

If somebody doesn't care about local security, mod_cgid should be able 
to see something like

scriptsock 127.0.0.1:9999

in the config file and use a TCP socket instead of Unix socket for 
communication with the daemon.

Alternatively the handler could store the request body in a temporary 
file and have the daemon set the child's stdin to that temporary file.

Alternatively some sort of named pipe could be used instead of a Unix 
socket.

RE: cgid on hp-ux - read hangs because shutdown doesn't close

[Gary Feldman <ga...@rtr.com>]

> From: Jeff Trawick [mailto:trawick@attglobal.net] 
> Sent: Thursday, March 06, 2003 5:50 PM

Thanks for confirmation.  At least I know I'm not barking
up the wrong tree.

> We can't close the socket.  Otherwise, how would we read the 
> output of 
> the CGI back in the child process [thread] that sends the CGI output 
> back to the client?

I realize that (otherwise I'd do it myself).  I was hoping for some
other
workaround.

> So use mod_cgi :)
> 
> (If there isn't a compelling performance difference, that makes sense 
> from Apache's perspective because mod_cgid is unfortunately 
> complicated 
> and it isn't worth more tricks if there's no sense in using it in the 
> first place.)

My understanding of the world is that mod_cgid is necessary
for decent performance with worker threads.

Just as supporting mod_cgi.c and mod_cgid.c adds to Apache maintenance
work, for us to have to support both models would be prohibitive.
Hence my desire for a good workaround.

Gary


====================================================================
                  Ready-to-Run Software, Inc.  
              The Industry's Leading Provider of
              Cross-Platform Software Services
                 *****************************
email: gaf@rtr.com                 Gary Feldman 
fax  : 1-978-692-5401              Ready-to-Run Software, Inc.
voice: 1-978-251-5431              11 School Street
www  : http://www.rtr.com          North Chelmsford, MA 01863
                                                     USA

Re: cgid on hp-ux - read hangs because shutdown doesn't close

[Jeff Trawick <tr...@attglobal.net>]

Gary Feldman wrote:

> I'm running into a problem with some code based on the
> mod_cgid module, on hp-ux.  I've also submitted
> the underlying problem to comp.systems.hp.hpux.
>
> The problem is that the underlying cgi hangs on a read
> from stdin when there is actually no data left to be read.


uhhh, unfortunately I can recreate this behavior :(

 >On HP-UX, this
 >doesn't happen until the socket is actually closed.

We can't close the socket.  Otherwise, how would we read the output of 
the CGI back in the child process [thread] that sends the CGI output 
back to the client?

A non-Apache testcase with very simple use of the sockets seems to 
indicate that HP-UX doesn't support this idiom.  (Just what you said, I 
think :) ).

So use mod_cgi :)

(If there isn't a compelling performance difference, that makes sense 
from Apache's perspective because mod_cgid is unfortunately complicated 
and it isn't worth more tricks if there's no sense in using it in the 
first place.)