handling client abort instantly?

[Frank Meier <fr...@datacomm.ch>]

Hi
I'm working with a proprietary apache module which communicates (through
a socket) with another backend application (I have the C source code of
the module). I've now found out, when the client closes the http
connection during a request, the module does not "see" that the client
has disconnected. In some cases the request is just finished, but with
larger requests (larger amount of data from backend), the module keeps 
hanging in the ap_rwrite() or ap_rflush() function. This is only 
resolved if a timeout (default 300s, timeout directive in httpd.conf) 
occurs. I think the tx-buffer of the socket is filled and then the
write/flush function blocks.

I've read in "the Apache Modules Book" (p.138) that we can detect the
disconnection by checking r->connection->aborted. The problem is, this
is only set when the timeout occurs.

What is the nice/right way to go, if I like to abort
instantly my request when the client has closed the connection? I'm
reluctant to decrease the apache timeout directive to very low value,
because I don't know about side effects and It would be not a nice solution.

ApacheVersion: 2.2.9
OS: Solaris10

pseudocode:
while (notFinished)
{
        do
        {
            int received=readDataFromBackend(buf);

            result = ap_rwrite(buf, received, r);
            if (result <= 0 || r->connection->aborted)
            {
                ap_log_rerror(APLOG_MARK, APLOG_ERR|APLOG_NOERRNO, 0,  r,
                        "PID %d: Cannot write data to client, returning
%d (errno = %d), aborted:%d", getpid(), result, errno,
r->connection->aborted);
                return HTTP_INTERNAL_SERVER_ERROR;
            }

        } while (moreDataFromBackend);

        int result = ap_rflush(r);
        if (result < 0 || r->connection->aborted)
        {
            ap_log_rerror(APLOG_MARK, APLOG_ERR|APLOG_NOERRNO, 0,  r,
                    "PID %d: Response chunk flush failed, returning %d
(errno = %d), aborted:%d", getpid(), result, errno, r->connection->aborted);
            return HTTP_INTERNAL_SERVER_ERROR;
        }
    }


thanks for help, Frank

Re: handling client abort instantly?

[Chris Kukuchka <ch...@sequoiagroup.com>]

Frank Meier wrote:
> Hi
> I'm working with a proprietary apache module which communicates (through
> a socket) with another backend application (I have the C source code of
> the module). I've now found out, when the client closes the http
> connection during a request, the module does not "see" that the client
> has disconnected. In some cases the request is just finished, but with
> larger requests (larger amount of data from backend), the module keeps 
> hanging in the ap_rwrite() or ap_rflush() function. This is only 
> resolved if a timeout (default 300s, timeout directive in httpd.conf) 
> occurs. I think the tx-buffer of the socket is filled and then the
> write/flush function blocks.

Frank,

In Apache 2.2 your module should not need to communicate directly with 
the client.  Instead, pass your output into the bucket brigade.  
Something like this (Note: sock is a socket connection to an external host):

        bb = apr_brigade_create(r->pool, c->bucket_alloc);

        /* Put the body of the reply into the bucket brigade */
        b = apr_bucket_socket_create(sock, c->bucket_alloc);
        APR_BRIGADE_INSERT_TAIL(bb, b);
        b = apr_bucket_eos_create(c->bucket_alloc);
        APR_BRIGADE_INSERT_TAIL(bb, b);
        status = ap_pass_brigade(r->output_filters, bb);
        if (status != APR_SUCCESS) {
            ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r, "ap_pass_brigade 
failed");
            apr_socket_close(sock);
            return HTTP_INTERNAL_SERVER_ERROR;
        }

This code passes the socket into the bucket brigade which takes care of 
all the client communication.  In addition, it allows for other modules 
to still act upon the output data.

>
> I've read in "the Apache Modules Book" (p.138)

Try Nick's book ( http://books.google.com/books?id=HTo_AmTpQPMC ) as it 
is geared toward the newer Apache releases.  Chances are the module you 
inherited has not been properly updated to the new Apache standards.

Regards,

Chris Kukuchka
Sequoia Group, Inc.

Re: handling client abort instantly?

[Saju Pillai <sa...@gmail.com>]

Frank Meier wrote:

> 
> thanks for your replies, I tried the use of bucketbrigades as Chris 
> Kukuchka suggested. Unfortunately this lead  to the same behaviour.
> 
> I also tried the *hack* approach (accessing the socket directly), where 
> I had the problem of getting to the socket itself. I didn't understand 
> how I could use ap_filter_t->ctx since I don't know what is stored 

For the top level network filter 'ctx' is a core_net_rec (httpd.h) object.

My version of httpd.h has this definition.

typedef struct core_net_rec {
     /** Connection to the client */
     apr_socket_t *client_socket;

     /** connection record */
     conn_rec *c;

     core_output_filter_ctx_t *out_ctx;
     core_ctx_t *in_ctx;
} core_net_rec;




> there, but I used "ap_get_module_config" which should also get me to the 
> socket. right?
> 
> struct apr_socket_t *client_socket = 
> ap_get_module_config(r->connection->conn_config, &core_module);


Ok, this looks like a far saner way to get to the client socket than the 
hack I suggested.

> int rv = read(client_socket->socketdes, dummyBuf, len);
> 
> but rv is always 0 (if len is 0) / -1 (if len > 0) and errno is 2 (No 
> such file or directory)

Very interesting, don't have a clue why you are seeing this though. I 
don't think read() is even supposed to return ENOENT.

http://docs.sun.com/app/docs/doc/816-0212/6m6nd4nd4?a=view is a manpage 
for read() on solaris 9. No ENOENT here. What does your manpage say ?


> 
> I think my next step is to implement a test module myself to verify this 
> strange behaviour and also test it on another OS (linux) since Solaris 
> sometimes does strange things.
> 

That would be the way to go.

srp
-- 
http://saju.net.in

Re: handling client abort instantly?

[Frank Meier <fr...@datacomm.ch>]

Saju Pillai wrote:
> Frank Meier wrote:
>> Hi
>> I'm working with a proprietary apache module which communicates (through
>> a socket) with another backend application (I have the C source code of
>> the module). I've now found out, when the client closes the http
>> connection during a request, the module does not "see" that the client
>> has disconnected. In some cases the request is just finished, but with
>> larger requests (larger amount of data from backend), the module 
>> keeps hanging in the ap_rwrite() or ap_rflush() function. This is 
>> only resolved if a timeout (default 300s, timeout directive in 
>> httpd.conf) occurs. I think the tx-buffer of the socket is filled and 
>> then the
>> write/flush function blocks.
>
> I think if the client has gone away while httpd is trying to write() 
> to it, httpd will immediately error out - probably with a "Broken 
> Pipe" error.
>
> Even if your socket write buffer is filled, the actual attempt to 
> write() by your tcp stack must raise an error.
>
> Maybe I am not understanding the problem properly. If the tcp 
> connection is broken, attempts to read() or write() on that connection 
> should flag a detectable error. It maybe possible that httpd is 
> waiting too long to perform a write() and doesn't figure out fast 
> enough that the remote end has gone away.
>
> I can think of a *hack* to determine if the client has really gone away.
> In your module you can possibly do
>
> client = ap_filter_t->ctx->client_socket /* ap_filter_t is either the 
> input or output filter stack */
>
> to get the apr client socket. You can try do a 0-byte read() on this 
> socket to see if you get an error. An error means the remote side has 
> gone away. I have not tested this, I don't know if this will work for 
> you.
>
>
>
> srp

thanks for your replies, I tried the use of bucketbrigades as Chris 
Kukuchka suggested. Unfortunately this lead  to the same behaviour.

I also tried the *hack* approach (accessing the socket directly), where 
I had the problem of getting to the socket itself. I didn't understand 
how I could use ap_filter_t->ctx since I don't know what is stored 
there, but I used "ap_get_module_config" which should also get me to the 
socket. right?

struct apr_socket_t *client_socket = 
ap_get_module_config(r->connection->conn_config, &core_module);
int rv = read(client_socket->socketdes, dummyBuf, len);

but rv is always 0 (if len is 0) / -1 (if len > 0) and errno is 2 (No 
such file or directory)

 > I think if the client has gone away while httpd is trying to write() 
to it, httpd will immediately error out - probably with a "Broken Pipe" 
error.
that IS what I ought to think too :-)

I think my next step is to implement a test module myself to verify this 
strange behaviour and also test it on another OS (linux) since Solaris 
sometimes does strange things.

thanks anyway, Frank

Re: handling client abort instantly?

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

Saju Pillai wrote:
> 
> I can think of a *hack* to determine if the client has really gone away.
> In your module you can possibly do
> 
> client = ap_filter_t->ctx->client_socket /* ap_filter_t is either the 
> input or output filter stack */
> 
> to get the apr client socket. You can try do a 0-byte read() on this 
> socket to see if you get an error. An error means the remote side has 
> gone away. I have not tested this, I don't know if this will work for you.

You have to walk your filter back to the topmost (network) filter before
you'll find that ctx corresponds to the core's value (sock).  ctx is set
up per-filter.

Re: handling client abort instantly?

[Saju Pillai <sa...@gmail.com>]

Frank Meier wrote:
> Hi
> I'm working with a proprietary apache module which communicates (through
> a socket) with another backend application (I have the C source code of
> the module). I've now found out, when the client closes the http
> connection during a request, the module does not "see" that the client
> has disconnected. In some cases the request is just finished, but with
> larger requests (larger amount of data from backend), the module keeps 
> hanging in the ap_rwrite() or ap_rflush() function. This is only 
> resolved if a timeout (default 300s, timeout directive in httpd.conf) 
> occurs. I think the tx-buffer of the socket is filled and then the
> write/flush function blocks.

I think if the client has gone away while httpd is trying to write() to 
it, httpd will immediately error out - probably with a "Broken Pipe" error.

Even if your socket write buffer is filled, the actual attempt to 
write() by your tcp stack must raise an error.

Maybe I am not understanding the problem properly. If the tcp connection 
is broken, attempts to read() or write() on that connection should flag 
a detectable error. It maybe possible that httpd is waiting too long to 
perform a write() and doesn't figure out fast enough that the remote end 
has gone away.

I can think of a *hack* to determine if the client has really gone away.
In your module you can possibly do

client = ap_filter_t->ctx->client_socket /* ap_filter_t is either the 
input or output filter stack */

to get the apr client socket. You can try do a 0-byte read() on this 
socket to see if you get an error. An error means the remote side has 
gone away. I have not tested this, I don't know if this will work for you.



srp
-- 
http://saju.net.in