[HTTPClient] warn long message about buffering

[Guy Katz <gk...@allot.com>]

hi;
i just upgraded to http client 2.0.2.
i noticed that i keep getting the following log message:

"2004-10-26 22:36:18 [http8080-Processor25] WARN
commons.httpclient.HttpMethodBa
se  - Going to buffer response body of large or unknown size. Using
getResponseA
sStream instead is recommended."

my usage of HTTP Client is for screen scrapping. i read the HTML of a page
from a URL as is and parse it to find the data i need.
i am using the getAsString method of the Http client API. this message
appears only for the new version.

i was wondering if someone could shed light about this.
should i really use the get as stream instead? is there a performance
benefit when using the getAsStream over the more convenient getAsString?
also, the message is pretty annoying cause i use thew screen scapping all
the time, i am not sure that it should be there. if its a matter of best
practice than maybe the mesage should not be logged at all.

thanks in advance.

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org

Re: [HTTPClient] warn long message about buffering

[Oleg Kalnichevski <ol...@apache.org>]

Guy,

Yes, you really should not be using buffering HttpMethod#getResponse*
methods unless you are really sure your application will be dealing with
relatively small response bodies. It may be a pretty bad idea to buffer
a 200MB response, hence this big fat warning. We do encourage HttpClient
users to use non-buffering getResponseAsStream and implement the content
retrieval and buffering logic that fits best their specific application
requirements. In most cases it is by far more efficient performance and
resource consumption wise to read directly from the InputStream

Hope this helps

Oleg


On Wed, 2004-10-27 at 11:57, Guy Katz wrote:
> hi;
> i just upgraded to http client 2.0.2.
> i noticed that i keep getting the following log message:
> 
> "2004-10-26 22:36:18 [http8080-Processor25] WARN
> commons.httpclient.HttpMethodBa
> se  - Going to buffer response body of large or unknown size. Using
> getResponseA
> sStream instead is recommended."
> 
> my usage of HTTP Client is for screen scrapping. i read the HTML of a page
> from a URL as is and parse it to find the data i need.
> i am using the getAsString method of the Http client API. this message
> appears only for the new version.
> 
> i was wondering if someone could shed light about this.
> should i really use the get as stream instead? is there a performance
> benefit when using the getAsStream over the more convenient getAsString?
> also, the message is pretty annoying cause i use thew screen scapping all
> the time, i am not sure that it should be there. if its a matter of best
> practice than maybe the mesage should not be logged at all.
> 
> thanks in advance.
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org