[httpclient] Content-Length handling

[Ryan Hoegg <rh...@isisnetworks.net>]

Hello,

I am in the process of trying to build the commons HttpClient code into 
the apache XML-RPC library.  I have it working, however I have run into 
some interoperability issues.  Specifically, certain XML-RPC servers 
report a slightly smaller content-length than the actual response body. 
 java.net.URLConnection takes the entire body regardless, but I think 
HttpClient is terminating the body after Content-Length bytes have been 
received.

 From RFC2616 4.4.4 :

When a Content-Length is given in a message where a message-body is
allowed, its field value MUST exactly match the number of OCTETs in
the message-body. HTTP/1.1 user agents MUST notify the user when an
invalid length is received and detected.

Does this mean that HTTPClient should throw an exception when 
Content-Length does not match the length of the response body?  Or do we 
have some latitude here?

My immediate concern is not for strict RFC enforcement but for 
compliance with as many (possibly broken) HTTP server implementations as 
possible.  What are the priorities for the committers?

Ryan Hoegg
ISIS Networks


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: [httpclient] Content-Length handling

[Ryan Hoegg <rh...@isisnetworks.net>]

You are right.  If we wait, we are basically falling back to waiting for 
the server to close the connection in every case.  I believe that 
HttpClient behaves correctly.  I have already pursued this with the 
appropriate HTTP servers :)

--
Ryan Hoegg
ISIS Networks

Ortwin Glück wrote:

> My point of view:
>
> We should primarily stick to the RFCs. (Yes I know there are other 
> interest groups who do server testing). If there are servers that have 
> a faulty HTTP implementation that's their problem (which should be 
> corrected), not ours. Better write to them :-)
>
> But the actual problem here is more: How can we determine that the 
> content-length was reported too small?
>
> Imagine the following situation:
> Server reports content-lenth 1000 bytes. The actual content is 1015 
> bytes. After 10 bytes the server pauses (maybe because of network jam) 
> and after some seconds the remaining 15 bytes arrive. There is 
> absolutely no reason why the client should wait (for how long?) for 
> more data after it has received the reported number of bytes.
>
> So we can not detect this in general, right?





--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: [httpclient] Content-Length handling

[Ortwin Glück <or...@nose.ch>]

My point of view:

We should primarily stick to the RFCs. (Yes I know there are other 
interest groups who do server testing). If there are servers that have a 
faulty HTTP implementation that's their problem (which should be 
corrected), not ours. Better write to them :-)

But the actual problem here is more: How can we determine that the 
content-length was reported too small?

Imagine the following situation:
Server reports content-lenth 1000 bytes. The actual content is 1015 
bytes. After 10 bytes the server pauses (maybe because of network jam) 
and after some seconds the remaining 15 bytes arrive. There is 
absolutely no reason why the client should wait (for how long?) for more 
data after it has received the reported number of bytes.

So we can not detect this in general, right?

Odi

Ryan Hoegg wrote:
>  From RFC2616 4.4.4 :
> 
> When a Content-Length is given in a message where a message-body is
> allowed, its field value MUST exactly match the number of OCTETs in
> the message-body. HTTP/1.1 user agents MUST notify the user when an
> invalid length is received and detected.
> 
> Does this mean that HTTPClient should throw an exception when 
> Content-Length does not match the length of the response body?  Or do we 
> have some latitude here?
> 
> My immediate concern is not for strict RFC enforcement but for 
> compliance with as many (possibly broken) HTTP server implementations as 
> possible.  What are the priorities for the committers?
> 
> Ryan Hoegg
> ISIS Networks

-- 
_________________________________________________________________
  NOSE applied intelligence ag

  ortwin glück                      [www]      http://www.nose.ch
  hardturmstrasse 171               [email] ortwin.glueck@nose.ch
  8005 zurich                       [office]      +41-1-277 57 35
  switzerland                       [fax]         +41-1-277 57 12



--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>