Patch pending on scanning for HTTP 1.1 - attached

[Eric Johnson <er...@tibco.com>]

I see that over the weekend lots happened on the mailing list.  I guess 
that one cannot safely go on vacation around here!

I've seen a few posts about problems with connections closing 
unexpectedly.  I had this problem, and traced it down to several issues. 
 The problem stems from the wrappers around the input streams that would 
occassionally close or not close the underlying socket, in ways that 
were unexpected.  There seems to be an odd expectation that the "close" 
function would never be called on the various InputStream variants, 
except by HttpClient and its associate classes.

I've attached the patch that I have, but I don't expect it to actually 
be applied yet - as I have mentioned before, I think it prudent to wait 
for the far more important HttpClient/HttpMultiClient merge.  So here it 
is - I'll resubmit after I see the big merge in CVS.

-Eric.

The patch consists of the following:

    * AutoCloseInputStream now does not call "super.read()" once close
      has been called on the connection itself.  The close() function is
      now implemented to close the underlying _connection_ rather than
      just the socket input stream.  Failing to close the connection
      would cause an unnecessary (and sometimes troublesome) recoverable
      exception with subsequent requests.
    * ChunkedInputStream does not close the underlying stream anymore.
       It does, however, "exhaust" the remaining chunked stream when the
      "close()" method is called, eliminating _almost_ all of the need
      for HttpClient to scan for HTTP/1.1 lines (which is error prone,
      due to the unpredictability of the content of the underlying
      responses).  The "read" methods check to see if the connection is
      closed and throw an exception.
    * ContentLengthInputStream - similar changes to that of
      ChunkedInputStream.
    * The HeadMethod now calls shouldCloseConnection and
      connection.close() as appropriate.
    * HttpClient.executeMethod() now calls InputStream.close() on the
      previous response's input stream (this is part that may be tricky
      to carry over to the MultiClient merge).  In this way, a client of
      a "Get" method, for example, need not actually read the entire
      response - calling close has the effect of doing so, or closing
      the underlying connection, as appropriate.
    * HttpConnection - now keeps track of the InputStream for the last
      executeMethod() request.
    * HttpMethodBase - Now keeps track of the number of "recoverable
      exceptions" - so that I could write test cases that monitor
      whether recoverable exceptions are being thrown when they
      shouldn't be!  Also calls "close" on the input stream when done
      reading it, and sets the connection's last response input stream.
    * TestGetMethodLocal - added an additional test where the "GET"
      method response body is never read, but is followed immediately by
      another GET request.
    * TestHttpClientLocalHost - now does "GET" requests from
      "/httpclienttest/body" (based on system props), rather than "/" as
      the root of the server is not controlled by the standard WAR file.
       I was getting 302 responses, and thus test failures, until I
      pointed to the new location.
    * TestMethodsLocalHost - same as TestHttpClientLocalHost
    * TestMethodsNoHost - added new tests of the HeadMethod.

Miscellaneous random notes:

    * I ran into my difficulties initially because I was using Tomcat
      4.1.10 with the old (now deprecated) HttpConnector (1.1
      compliant).  It turns out that it returns a "Connection: close"
      header on any request that fails to indicate content length either
      via a content-length header or a chunked encoding.  This sounds
      similar to someone else's posting about difficulties with the
      "Connection: close" header.
    * I had subsequent difficulties due to sending the results of a GET
      request to Xerces for XML parsing.  Xerces would eventually call
      "close" on the stream passed to it.  This would have the
      unfortunate effect, when working with ContentLengthInputStream for
      example, of closing the underlying socket - without notifying the
      connection.  Now, the close changes the state of the
      ContentLengthInputStream (by exhausting the input stream), but
      does not close the socket.  Of course, if the AutoCloseInputStream
      wrapper gets used, the connection will be closed....

Enjoy.  Please let me know of any feedback you might have, I will be 
happy to take another shot at revising.

-Eric.