You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by B K <gr...@hotmail.com> on 2005/08/30 14:15:50 UTC
HttpClient bandwidth
Hi all,
I have developed an application using httpclient, and now that I have
started using it, it is using to much bandwidth I wonder if anybody has some
pointers on how to reduce the amount of data being transferred. My idea was
to use the response headers to only retrieve the response if the data had
changed, unfortunately I can't find any way of doing this, my idea was to
only retrieve the response when the web content changed. The web sites I am
dealing with are very dynamic and change every couple of minutes so I need
to check every minute for updates to the data.
The application is gathering data from 10 web sites, and I have developed it
so there are 10 instances of httpclient, one for each web site, the big
problem I see is everytime I send out the requests I have to download the
response even if there is no change in the data. Anybody got some bright
design ideas on how to cut down, I have researched and come up with nothing.
Thanks
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
Re: HttpClient bandwidth
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, Aug 30, 2005 at 10:15:50PM +1000, B K wrote:
> Hi all,
> I have developed an application using httpclient, and now that I
> have started using it, it is using to much bandwidth I wonder if anybody
> has some pointers on how to reduce the amount of data being transferred. My
> idea was to use the response headers to only retrieve the response if the
> data had changed, unfortunately I can't find any way of doing this, my idea
> was to only retrieve the response when the web content changed. The web
> sites I am dealing with are very dynamic and change every couple of minutes
> so I need to check every minute for updates to the data.
>
> The application is gathering data from 10 web sites, and I have developed
> it so there are 10 instances of httpclient, one for each web site, the big
> problem I see is everytime I send out the requests I have to download the
> response even if there is no change in the data. Anybody got some bright
> design ideas on how to cut down, I have researched and come up with nothing.
>
> Thanks
>
B K,
There's not much you can do unless the target servers play along. Please
refer to the HTTP spec [1] and take a look at the 304 Not Modified
mechanism [2] for details
Hope this helps
Oleg
[1] http://www.w3.org/Protocols/rfc2616/rfc2616.html
[2] http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org