streaming responses

["St Jacques, Robert" <RS...@crt.xerox.com>]

Howdy,

I've just started using HttpClient for testing.  The product that I am
developing includes a software download feature that downloads (sometimes
very large) files over the internet using HTTP.  The reason we use HTTP is
that many of our customers are unwilling to open their proxies or firewalls
to other protocols.

In order to accommodate these downloads, we must either be able to stream
the response to a GET, or we need to be able to insure that the response we
get from the server will be chunked in such a way that the chunks are small
enough to buffer.  I know that HttpClient supports chunking, but I am unable
to 'force' our web server to chunk up the data.  Furthermore, the HTTP 1.1
RFC indicates that the server does not need to guarantee that a response,
even if chunked, will be small enough to conveniently buffer on the client.

What is the best way to use HttpClient to retrieve large amounts of data (as
large as 200 MB or more)?

Thanks,
Bob St. Jacques
Member Research Technical Staff
Xerox Corp.
(585)231-8306

Re: streaming responses

[Oleg Kalnichevski <ol...@apache.org>]

Bob,

There's no special magic involved. Make sure you use
HttpMethod#getResponseBodyAsStream, which will return the raw input
stream, and not its buffering counterparts
HttpMethod#getResponseBodyAsString and HttpMethod#getResponseBody.

You should not worry about chunking. HttpClient will decode chunked
input on fly, if necessary. Just grab the raw input stream and do
whatever response retrieval suits your application best.

Hope this helps

Oleg


On Wed, 2004-09-29 at 20:43, St Jacques, Robert wrote:
> Howdy,
> 
> I've just started using HttpClient for testing.  The product that I am
> developing includes a software download feature that downloads (sometimes
> very large) files over the internet using HTTP.  The reason we use HTTP is
> that many of our customers are unwilling to open their proxies or firewalls
> to other protocols.
> 
> In order to accommodate these downloads, we must either be able to stream
> the response to a GET, or we need to be able to insure that the response we
> get from the server will be chunked in such a way that the chunks are small
> enough to buffer.  I know that HttpClient supports chunking, but I am unable
> to 'force' our web server to chunk up the data.  Furthermore, the HTTP 1.1
> RFC indicates that the server does not need to guarantee that a response,
> even if chunked, will be small enough to conveniently buffer on the client.
> 
> What is the best way to use HttpClient to retrieve large amounts of data (as
> large as 200 MB or more)?
> 
> Thanks,
> Bob St. Jacques
> Member Research Technical Staff
> Xerox Corp.
> (585)231-8306
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org