You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Ben Cox <be...@hotmail.com> on 2010/02/02 22:07:45 UTC
HTTP Trailers in HttpClient 4.0.1
Hi all,
I'm testing out using chunked encoding with trailing headers at the moment, but seem to be having some problems getting the HttpClient to find the trailers. My code looks like this:
/***************************************************************/
HttpClient httpClient = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(url);
// Tell the server that we can deal with trailers
httpGet.addHeader("TE", "trailers");
// Get the response
HttpResponse response = httpClient.execute(httpGet);
// Print out all the headers we have
for (Header aHeader : response.getAllHeaders()) {
System.out.println(aHeader.getName() + ": " + aHeader.getValue());
}
/***************************************************************/
The resultant document is transferred in around 100 chunks.
I get all the other trailers I might expect to be printed, including the "Trailer: Mytrailer" one that describes what trailers I should I expect. However, I don't get "Mytrailer" at all, though I can see it on the wire if I use Wireshark.
I noticed in the 3.x documentation there seems to be a lot more info about chunked encoding and trailers/footers. However, there's not so much in the 4.0.1 documentation. Is this not supported any more, or just more integrated into normal headers and I've got something wrong?
Any help is much appreciated.
Thanks,
Ben
_________________________________________________________________
Tell us your greatest, weirdest and funniest Hotmail stories
http://clk.atdmt.com/UKM/go/195013117/direct/01/
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: HTTP Trailers in HttpClient 4.0.1
Posted by Ben Cox <be...@hotmail.com>.
Argh - I should say I get all the other HEADERS I might expect, not trailers! I DON'T get any trailers.
----------------------------------------
> From: ben_cox_21@hotmail.com
> To: httpclient-users@hc.apache.org
> Subject: HTTP Trailers in HttpClient 4.0.1
> Date: Tue, 2 Feb 2010 21:07:45 +0000
>
>
> Hi all,
>
> I'm testing out using chunked encoding with trailing headers at the moment, but seem to be having some problems getting the HttpClient to find the trailers. My code looks like this:
>
> /***************************************************************/
>
> HttpClient httpClient = new DefaultHttpClient();
> HttpGet httpGet = new HttpGet(url);
>
> // Tell the server that we can deal with trailers
> httpGet.addHeader("TE", "trailers");
>
> // Get the response
> HttpResponse response = httpClient.execute(httpGet);
>
> // Print out all the headers we have
> for (Header aHeader : response.getAllHeaders()) {
> System.out.println(aHeader.getName() + ": " + aHeader.getValue());
> }
>
> /***************************************************************/
>
> The resultant document is transferred in around 100 chunks.
>
> I get all the other trailers I might expect to be printed, including the "Trailer: Mytrailer" one that describes what trailers I should I expect. However, I don't get "Mytrailer" at all, though I can see it on the wire if I use Wireshark.
>
> I noticed in the 3.x documentation there seems to be a lot more info about chunked encoding and trailers/footers. However, there's not so much in the 4.0.1 documentation. Is this not supported any more, or just more integrated into normal headers and I've got something wrong?
>
> Any help is much appreciated.
>
> Thanks,
> Ben
>
> _________________________________________________________________
> Tell us your greatest, weirdest and funniest Hotmail stories
> http://clk.atdmt.com/UKM/go/195013117/direct/01/
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
_________________________________________________________________
Tell us your greatest, weirdest and funniest Hotmail stories
http://clk.atdmt.com/UKM/go/195013117/direct/01/
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: HTTP Trailers in HttpClient 4.0.1
Posted by Ben Cox <be...@hotmail.com>.
That did it! Thanks a lot, Oleg.
Ben
----------------------------------------
> Date: Thu, 4 Feb 2010 21:02:22 +0100
> From: olegk@apache.org
> To: httpclient-users@hc.apache.org
> Subject: Re: HTTP Trailers in HttpClient 4.0.1
>
> Ben Cox wrote:
>> Ah, getting there - almost! I was making the mistake of trying to cast the InputStream of a BufferedHttpEntity. If I instead try response.getEntity().getContent() (i.e. work with the BasicManagedEntity that it gives me) I get an EofSensorInputStream which, I can see in debug, wraps a ChunkedInputStream. However, the ChunkedInputStream is still invisible! Can I somehow get access to that ChunkedInputStream?
>>
>
> You can get access to the underlying input stream from a response
> interceptor:
>
>
> -----
> HttpHost targetHost = new HttpHost("www.yahoo.com", 80, "http");
>
> DefaultHttpClient httpclient = new DefaultHttpClient();
> httpclient.addResponseInterceptor(new HttpResponseInterceptor() {
>
> public void process(
> final HttpResponse response,
> final HttpContext context) throws HttpException, IOException {
> HttpEntity entity = response.getEntity();
> if (entity != null) {
> InputStream instream = entity.getContent();
> context.setAttribute("raw-insteam", instream);
> }
> }
> });
>
> BasicHttpContext localcontext = new BasicHttpContext();
>
> HttpGet httpget = new HttpGet("/");
> HttpResponse response = httpclient.execute(targetHost, httpget,
> localcontext);
>
> System.out.println(response.getStatusLine());
>
> HttpEntity entity = response.getEntity();
> if (entity != null) {
> entity.consumeContent();
> }
>
> InputStream instream = (InputStream)
> localcontext.getAttribute("raw-insteam");
> if (instream != null && instream instanceof ChunkedInputStream) {
> Header[] footers = ((ChunkedInputStream) instream).getFooters();
> System.out.println(footers.length);
> }
> ---
>
> Hope this helps
>
> Oleg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
_________________________________________________________________
Send us your Hotmail stories and be featured in our newsletter
http://clk.atdmt.com/UKM/go/195013117/direct/01/
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HTTP Trailers in HttpClient 4.0.1
Posted by Oleg Kalnichevski <ol...@apache.org>.
Ben Cox wrote:
> Ah, getting there - almost! I was making the mistake of trying to cast the InputStream of a BufferedHttpEntity. If I instead try response.getEntity().getContent() (i.e. work with the BasicManagedEntity that it gives me) I get an EofSensorInputStream which, I can see in debug, wraps a ChunkedInputStream. However, the ChunkedInputStream is still invisible! Can I somehow get access to that ChunkedInputStream?
>
You can get access to the underlying input stream from a response
interceptor:
-----
HttpHost targetHost = new HttpHost("www.yahoo.com", 80, "http");
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.addResponseInterceptor(new HttpResponseInterceptor() {
public void process(
final HttpResponse response,
final HttpContext context) throws HttpException, IOException {
HttpEntity entity = response.getEntity();
if (entity != null) {
InputStream instream = entity.getContent();
context.setAttribute("raw-insteam", instream);
}
}
});
BasicHttpContext localcontext = new BasicHttpContext();
HttpGet httpget = new HttpGet("/");
HttpResponse response = httpclient.execute(targetHost, httpget,
localcontext);
System.out.println(response.getStatusLine());
HttpEntity entity = response.getEntity();
if (entity != null) {
entity.consumeContent();
}
InputStream instream = (InputStream)
localcontext.getAttribute("raw-insteam");
if (instream != null && instream instanceof ChunkedInputStream) {
Header[] footers = ((ChunkedInputStream) instream).getFooters();
System.out.println(footers.length);
}
---
Hope this helps
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: HTTP Trailers in HttpClient 4.0.1
Posted by Ben Cox <be...@hotmail.com>.
Ah, getting there - almost! I was making the mistake of trying to cast the InputStream of a BufferedHttpEntity. If I instead try response.getEntity().getContent() (i.e. work with the BasicManagedEntity that it gives me) I get an EofSensorInputStream which, I can see in debug, wraps a ChunkedInputStream. However, the ChunkedInputStream is still invisible! Can I somehow get access to that ChunkedInputStream?
Agreed on instead adding any trailing information to the tail of the response body. However, I'm meant to be testing a client's full adherence to HTTP/1.1, so I have to check the trailers.
Thanks for your help,
Ben
----------------------------------------
> Date: Wed, 3 Feb 2010 23:38:36 +0100
> From: olegk@apache.org
> To: httpclient-users@hc.apache.org
> Subject: Re: HTTP Trailers in HttpClient 4.0.1
>
> Ben Cox wrote:
>> Oleg,
>>
>> Thanks for your response. That's interesting - as you say, using trailers is somewhat of an edge case, though there's something to be said for using them as part of mashup-type applications in which data may be being sent over an extended period, so some state may be unknown at the beginning of transmission.
>
> That begs a question: why not just send it at the end of the response body
>
>>
>> I'd spotted the ChunkedInputStream before, but was unable to use it. Which InputStream are you referring to? I tried casting entity.getContent(), but that's a ByteArrayInputStream, and I can't find an InputStream for the response itself.
>>
>
> I double-checked and am pretty sure HttpEntity#getContent() of the
> response entity should return ChunkedInputStream if the response body is
> chunk coded.
>
> Oleg
>
>> Thanks,
>> Ben
>>
>>
>> ----------------------------------------
>>> Subject: Re: HTTP Trailers in HttpClient 4.0.1
>>> From: olegk@apache.org
>>> To: httpclient-users@hc.apache.org
>>> Date: Wed, 3 Feb 2010 11:41:25 +0100
>>>
>>> On Tue, 2010-02-02 at 21:07 +0000, Ben Cox wrote:
>>>> Hi all,
>>>>
>>>> I'm testing out using chunked encoding with trailing headers at the moment, but seem to be having some problems getting the HttpClient to find the trailers. My code looks like this:
>>>>
>>>> /***************************************************************/
>>>>
>>>> HttpClient httpClient = new DefaultHttpClient();
>>>> HttpGet httpGet = new HttpGet(url);
>>>>
>>>> // Tell the server that we can deal with trailers
>>>> httpGet.addHeader("TE", "trailers");
>>>>
>>>> // Get the response
>>>> HttpResponse response = httpClient.execute(httpGet);
>>>>
>>>> // Print out all the headers we have
>>>> for (Header aHeader : response.getAllHeaders()) {
>>>> System.out.println(aHeader.getName() + ": " + aHeader.getValue());
>>>> }
>>>>
>>>> /***************************************************************/
>>>>
>>>> The resultant document is transferred in around 100 chunks.
>>>>
>>>> I get all the other trailers I might expect to be printed, including the "Trailer: Mytrailer" one that describes what trailers I should I expect. However, I don't get "Mytrailer" at all, though I can see it on the wire if I use Wireshark.
>>>>
>>>> I noticed in the 3.x documentation there seems to be a lot more info about chunked encoding and trailers/footers. However, there's not so much in the 4.0.1 documentation. Is this not supported any more, or just more integrated into normal headers and I've got something wrong?
>>>>
>>>> Any help is much appreciated.
>>>>
>>>> Thanks,
>>>> Ben
>>>>
>>> Hi Ben
>>>
>>> I personally have never seen trailers/footers used in any of the
>>> real-world application. This seems to be no convincing use case for
>>> them. This is the reason trailing headers in Httpclient 4.0 can only be
>>> accessed by casting the response InputStream to ChunkInputStream and
>>> calling ChunkInputStream#getFooters once the end of the stream has been
>>> reached.
>>>
>>> Hope this helps
>>>
>>> Oleg
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>
>>
>> _________________________________________________________________
>> Send us your Hotmail stories and be featured in our newsletter
>> http://clk.atdmt.com/UKM/go/195013117/direct/01/
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
_________________________________________________________________
Do you have a story that started on Hotmail? Tell us now
http://clk.atdmt.com/UKM/go/195013117/direct/01/
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HTTP Trailers in HttpClient 4.0.1
Posted by Oleg Kalnichevski <ol...@apache.org>.
Ben Cox wrote:
> Oleg,
>
> Thanks for your response. That's interesting - as you say, using trailers is somewhat of an edge case, though there's something to be said for using them as part of mashup-type applications in which data may be being sent over an extended period, so some state may be unknown at the beginning of transmission.
That begs a question: why not just send it at the end of the response body
>
> I'd spotted the ChunkedInputStream before, but was unable to use it. Which InputStream are you referring to? I tried casting entity.getContent(), but that's a ByteArrayInputStream, and I can't find an InputStream for the response itself.
>
I double-checked and am pretty sure HttpEntity#getContent() of the
response entity should return ChunkedInputStream if the response body is
chunk coded.
Oleg
> Thanks,
> Ben
>
>
> ----------------------------------------
>> Subject: Re: HTTP Trailers in HttpClient 4.0.1
>> From: olegk@apache.org
>> To: httpclient-users@hc.apache.org
>> Date: Wed, 3 Feb 2010 11:41:25 +0100
>>
>> On Tue, 2010-02-02 at 21:07 +0000, Ben Cox wrote:
>>> Hi all,
>>>
>>> I'm testing out using chunked encoding with trailing headers at the moment, but seem to be having some problems getting the HttpClient to find the trailers. My code looks like this:
>>>
>>> /***************************************************************/
>>>
>>> HttpClient httpClient = new DefaultHttpClient();
>>> HttpGet httpGet = new HttpGet(url);
>>>
>>> // Tell the server that we can deal with trailers
>>> httpGet.addHeader("TE", "trailers");
>>>
>>> // Get the response
>>> HttpResponse response = httpClient.execute(httpGet);
>>>
>>> // Print out all the headers we have
>>> for (Header aHeader : response.getAllHeaders()) {
>>> System.out.println(aHeader.getName() + ": " + aHeader.getValue());
>>> }
>>>
>>> /***************************************************************/
>>>
>>> The resultant document is transferred in around 100 chunks.
>>>
>>> I get all the other trailers I might expect to be printed, including the "Trailer: Mytrailer" one that describes what trailers I should I expect. However, I don't get "Mytrailer" at all, though I can see it on the wire if I use Wireshark.
>>>
>>> I noticed in the 3.x documentation there seems to be a lot more info about chunked encoding and trailers/footers. However, there's not so much in the 4.0.1 documentation. Is this not supported any more, or just more integrated into normal headers and I've got something wrong?
>>>
>>> Any help is much appreciated.
>>>
>>> Thanks,
>>> Ben
>>>
>> Hi Ben
>>
>> I personally have never seen trailers/footers used in any of the
>> real-world application. This seems to be no convincing use case for
>> them. This is the reason trailing headers in Httpclient 4.0 can only be
>> accessed by casting the response InputStream to ChunkInputStream and
>> calling ChunkInputStream#getFooters once the end of the stream has been
>> reached.
>>
>> Hope this helps
>>
>> Oleg
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>
>
> _________________________________________________________________
> Send us your Hotmail stories and be featured in our newsletter
> http://clk.atdmt.com/UKM/go/195013117/direct/01/
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: HTTP Trailers in HttpClient 4.0.1
Posted by Ben Cox <be...@hotmail.com>.
Oleg,
Thanks for your response. That's interesting - as you say, using trailers is somewhat of an edge case, though there's something to be said for using them as part of mashup-type applications in which data may be being sent over an extended period, so some state may be unknown at the beginning of transmission.
I'd spotted the ChunkedInputStream before, but was unable to use it. Which InputStream are you referring to? I tried casting entity.getContent(), but that's a ByteArrayInputStream, and I can't find an InputStream for the response itself.
Thanks,
Ben
----------------------------------------
> Subject: Re: HTTP Trailers in HttpClient 4.0.1
> From: olegk@apache.org
> To: httpclient-users@hc.apache.org
> Date: Wed, 3 Feb 2010 11:41:25 +0100
>
> On Tue, 2010-02-02 at 21:07 +0000, Ben Cox wrote:
>> Hi all,
>>
>> I'm testing out using chunked encoding with trailing headers at the moment, but seem to be having some problems getting the HttpClient to find the trailers. My code looks like this:
>>
>> /***************************************************************/
>>
>> HttpClient httpClient = new DefaultHttpClient();
>> HttpGet httpGet = new HttpGet(url);
>>
>> // Tell the server that we can deal with trailers
>> httpGet.addHeader("TE", "trailers");
>>
>> // Get the response
>> HttpResponse response = httpClient.execute(httpGet);
>>
>> // Print out all the headers we have
>> for (Header aHeader : response.getAllHeaders()) {
>> System.out.println(aHeader.getName() + ": " + aHeader.getValue());
>> }
>>
>> /***************************************************************/
>>
>> The resultant document is transferred in around 100 chunks.
>>
>> I get all the other trailers I might expect to be printed, including the "Trailer: Mytrailer" one that describes what trailers I should I expect. However, I don't get "Mytrailer" at all, though I can see it on the wire if I use Wireshark.
>>
>> I noticed in the 3.x documentation there seems to be a lot more info about chunked encoding and trailers/footers. However, there's not so much in the 4.0.1 documentation. Is this not supported any more, or just more integrated into normal headers and I've got something wrong?
>>
>> Any help is much appreciated.
>>
>> Thanks,
>> Ben
>>
>
> Hi Ben
>
> I personally have never seen trailers/footers used in any of the
> real-world application. This seems to be no convincing use case for
> them. This is the reason trailing headers in Httpclient 4.0 can only be
> accessed by casting the response InputStream to ChunkInputStream and
> calling ChunkInputStream#getFooters once the end of the stream has been
> reached.
>
> Hope this helps
>
> Oleg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
_________________________________________________________________
Send us your Hotmail stories and be featured in our newsletter
http://clk.atdmt.com/UKM/go/195013117/direct/01/
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HTTP Trailers in HttpClient 4.0.1
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2010-02-02 at 21:07 +0000, Ben Cox wrote:
> Hi all,
>
> I'm testing out using chunked encoding with trailing headers at the moment, but seem to be having some problems getting the HttpClient to find the trailers. My code looks like this:
>
> /***************************************************************/
>
> HttpClient httpClient = new DefaultHttpClient();
> HttpGet httpGet = new HttpGet(url);
>
> // Tell the server that we can deal with trailers
> httpGet.addHeader("TE", "trailers");
>
> // Get the response
> HttpResponse response = httpClient.execute(httpGet);
>
> // Print out all the headers we have
> for (Header aHeader : response.getAllHeaders()) {
> System.out.println(aHeader.getName() + ": " + aHeader.getValue());
> }
>
> /***************************************************************/
>
> The resultant document is transferred in around 100 chunks.
>
> I get all the other trailers I might expect to be printed, including the "Trailer: Mytrailer" one that describes what trailers I should I expect. However, I don't get "Mytrailer" at all, though I can see it on the wire if I use Wireshark.
>
> I noticed in the 3.x documentation there seems to be a lot more info about chunked encoding and trailers/footers. However, there's not so much in the 4.0.1 documentation. Is this not supported any more, or just more integrated into normal headers and I've got something wrong?
>
> Any help is much appreciated.
>
> Thanks,
> Ben
>
Hi Ben
I personally have never seen trailers/footers used in any of the
real-world application. This seems to be no convincing use case for
them. This is the reason trailing headers in Httpclient 4.0 can only be
accessed by casting the response InputStream to ChunkInputStream and
calling ChunkInputStream#getFooters once the end of the stream has been
reached.
Hope this helps
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org