You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Eric Bloch <bl...@laszlosystems.com> on 2004/07/02 18:39:22 UTC
questionable default value for BufferedOutputStream size in HttpConnection
and memory usage?
Hi httpclient folks,
I've been looking at 2.0 source code and the default value for the
BufferedOutputStream that is used in an HttpConnectionn is coming from
socket.getSendBufferSize(). My hunch, is that, in general, this is
bigger than you'd want.
Most HTTP "sends" are less than 1KByte ('cept for big POSTs).
The default value I get for socket.getSendBufferSize for this is 8192.
I would think a better default for this buffer would be 1K, no?
Also, fyi, if someone happens to dork the system send buffer size hi
(say MB) and you are using the MultiThreadedConnectionManager in 2.0
(dunno about 3.0), you will use up a lot of memory for each connection
since the pool doesn't let idle connections (or their buffers) be gced.
I just got bit bad by that.
-Eric
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
Re: questionable default value for BufferedOutputStream
size in HttpConnection and memory usage?
Posted by Oleg Kalnichevski <ol...@apache.org>.
> Next time I have the cycles to sit down with JMX *and* port Laszlo's
> product from httpclient 2.0 to httpclient 3.0, I may be able to get to
> this :-). I'd be focused on settings for a client using the
> MultiThreadedConnectionManager.
That should be a very welcome contribution
>
> Btw, how do I go about getting the Laszlo Presentation Server listed on
> the applications page for httpclient? We've been a user since 1.0
> actually.
Currently one just needs to apply.
> We use httpclient as the core HTTP transport for a
> transcoding proxy inside the Laszlo Presentation Server. See
> http://www.laszlosystems.com/ for all sorts of details including free
> developer and non-commericial-use downloads and documentation. As to a
> short blurb for the 'applications page', something like:
>
I'll add an entry for Laszlo Presentation Server with the description
given below to the application page sometime later this week
> The Laszlo Presentation Server is an XML-native platform for the
> development and delivery of a new generation of Rich Internet
> Applications. For a quick introduction see:
> http://www.laszlosystems.com/lps/laszlo-in-ten-minutes/ .
>
> Btw, we also have an interest in using httpclient as the transport for a
> SOAP client if anyone else is working on that.
>
Apache Axis can use HttpClient as a transport agent of choice. Check it
out at
http://ws.apache.org/axis/
Cheers,
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
Re: questionable default value for BufferedOutputStream size in HttpConnection
and memory usage?
Posted by Eric Bloch <bl...@laszlosystems.com>.
>
> JMX has not been formally considered or discussed on this list. I
> personally do not think it would have made a good match
>
> HttpClient needs to allow for customization of lots of short-lived
> objects (HttpMethods, HostConfigurations, HttpConnections, etc), whereas
> JMS is better suited for relatively few long-lived objects that usually
> represent some type of services (at least IHMO). JMX would be simply too
> much, too heavy-weight. For HttpClient's needs light-weight linked
> hash-maps do just fine.
>
> This said, an JMX layer sitting on top of HttpClient that provide MBeans
> for HttpClient instances would be a great contribution
>
>
Hey Oleg,
Next time I have the cycles to sit down with JMX *and* port Laszlo's
product from httpclient 2.0 to httpclient 3.0, I may be able to get to
this :-). I'd be focused on settings for a client using the
MultiThreadedConnectionManager.
Btw, how do I go about getting the Laszlo Presentation Server listed on
the applications page for httpclient? We've been a user since 1.0
actually. We use httpclient as the core HTTP transport for a
transcoding proxy inside the Laszlo Presentation Server. See
http://www.laszlosystems.com/ for all sorts of details including free
developer and non-commericial-use downloads and documentation. As to a
short blurb for the 'applications page', something like:
The Laszlo Presentation Server is an XML-native platform for the
development and delivery of a new generation of Rich Internet
Applications. For a quick introduction see:
http://www.laszlosystems.com/lps/laszlo-in-ten-minutes/ .
Btw, we also have an interest in using httpclient as the transport for a
SOAP client if anyone else is working on that.
Best,
-Eric Bloch
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
Re: questionable default value for BufferedOutputStream size
in HttpConnection and memory usage?
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sat, 2004-07-03 at 02:26, Eric Bloch wrote:
> Thanks, I filed against 2.0 final. A question: did you guys consider
> jmx for your 'preferences architecture' ?
Eric,
JMX has not been formally considered or discussed on this list. I
personally do not think it would have made a good match
HttpClient needs to allow for customization of lots of short-lived
objects (HttpMethods, HostConfigurations, HttpConnections, etc), whereas
JMS is better suited for relatively few long-lived objects that usually
represent some type of services (at least IHMO). JMX would be simply too
much, too heavy-weight. For HttpClient's needs light-weight linked
hash-maps do just fine.
This said, an JMX layer sitting on top of HttpClient that provide MBeans
for HttpClient instances would be a great contribution
Oleg
>
> Thanks,
> Eric
>
>
> Oleg Kalnichevski wrote:
>
> > Hi Eric
> >
> > Thanks for bringing this up. HttpClient 3.0 allows for parameterization
> > of SO_SNDBUF and SO_RCVBUF settings. For HttpClient 2.0 (as well as for
> > 3.0 when falling back onto the system defaults), however, it would make
> > sense to set a cap on the size of the send and receive buffers.
> >
> > Feel free to open a ticket for this issue with Bugzilla
> >
> > Oleg
> >
> >
> > On Fri, 2004-07-02 at 18:39, Eric Bloch wrote:
> >
> >>Hi httpclient folks,
> >>
> >>I've been looking at 2.0 source code and the default value for the
> >>BufferedOutputStream that is used in an HttpConnectionn is coming from
> >>socket.getSendBufferSize(). My hunch, is that, in general, this is
> >>bigger than you'd want.
> >>
> >>Most HTTP "sends" are less than 1KByte ('cept for big POSTs).
> >>The default value I get for socket.getSendBufferSize for this is 8192.
> >>I would think a better default for this buffer would be 1K, no?
> >>
> >>Also, fyi, if someone happens to dork the system send buffer size hi
> >>(say MB) and you are using the MultiThreadedConnectionManager in 2.0
> >>(dunno about 3.0), you will use up a lot of memory for each connection
> >>since the pool doesn't let idle connections (or their buffers) be gced.
> >> I just got bit bad by that.
> >>
> >>-Eric
> >>
> >>
> >>---------------------------------------------------------------------
> >>To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
> >>For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
> >>
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
Re: questionable default value for BufferedOutputStream size in HttpConnection
and memory usage?
Posted by Eric Bloch <bl...@laszlosystems.com>.
Thanks, I filed against 2.0 final. A question: did you guys consider
jmx for your 'preferences architecture' ?
Thanks,
Eric
Oleg Kalnichevski wrote:
> Hi Eric
>
> Thanks for bringing this up. HttpClient 3.0 allows for parameterization
> of SO_SNDBUF and SO_RCVBUF settings. For HttpClient 2.0 (as well as for
> 3.0 when falling back onto the system defaults), however, it would make
> sense to set a cap on the size of the send and receive buffers.
>
> Feel free to open a ticket for this issue with Bugzilla
>
> Oleg
>
>
> On Fri, 2004-07-02 at 18:39, Eric Bloch wrote:
>
>>Hi httpclient folks,
>>
>>I've been looking at 2.0 source code and the default value for the
>>BufferedOutputStream that is used in an HttpConnectionn is coming from
>>socket.getSendBufferSize(). My hunch, is that, in general, this is
>>bigger than you'd want.
>>
>>Most HTTP "sends" are less than 1KByte ('cept for big POSTs).
>>The default value I get for socket.getSendBufferSize for this is 8192.
>>I would think a better default for this buffer would be 1K, no?
>>
>>Also, fyi, if someone happens to dork the system send buffer size hi
>>(say MB) and you are using the MultiThreadedConnectionManager in 2.0
>>(dunno about 3.0), you will use up a lot of memory for each connection
>>since the pool doesn't let idle connections (or their buffers) be gced.
>> I just got bit bad by that.
>>
>>-Eric
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
--
Laszlo Systems, Inc.
1040 Mariposa Street, SF, CA, USA 94107
voice: 415.241.2721
fax: 415.865.2914
im: eedeebee at yim; eedeebee3 at aim
email: bloch@laszlosystems.com
--
Laszlo allows Behr to deliver a breakthrough experience with
ColorSmart by BEHR application at http://www.behr.com
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
Re: questionable default value for BufferedOutputStream size in
HttpConnection and memory usage?
Posted by Oleg Kalnichevski <ol...@apache.org>.
Hi Eric
Thanks for bringing this up. HttpClient 3.0 allows for parameterization
of SO_SNDBUF and SO_RCVBUF settings. For HttpClient 2.0 (as well as for
3.0 when falling back onto the system defaults), however, it would make
sense to set a cap on the size of the send and receive buffers.
Feel free to open a ticket for this issue with Bugzilla
Oleg
On Fri, 2004-07-02 at 18:39, Eric Bloch wrote:
> Hi httpclient folks,
>
> I've been looking at 2.0 source code and the default value for the
> BufferedOutputStream that is used in an HttpConnectionn is coming from
> socket.getSendBufferSize(). My hunch, is that, in general, this is
> bigger than you'd want.
>
> Most HTTP "sends" are less than 1KByte ('cept for big POSTs).
> The default value I get for socket.getSendBufferSize for this is 8192.
> I would think a better default for this buffer would be 1K, no?
>
> Also, fyi, if someone happens to dork the system send buffer size hi
> (say MB) and you are using the MultiThreadedConnectionManager in 2.0
> (dunno about 3.0), you will use up a lot of memory for each connection
> since the pool doesn't let idle connections (or their buffers) be gced.
> I just got bit bad by that.
>
> -Eric
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org