You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Oleg Kalnichevski <ol...@apache.org> on 2008/10/25 14:35:55 UTC
Re: HttpConnection.flushRequestOutputStream() sometimes lasts too
long
On Thu, 2008-10-23 at 18:43 +0200, Furmaniak Christophe wrote:
> Hi,
>
> we've developped a web application performance tool. Our product intensivly uses httpclient (currently 3.1) to measure response time of these web applications (single url or sequence of url to copy a human being navigation).
>
> The main principle of our tool is to perform the same requests at a fixed frequency (5 minutes most of the times) to be able to compute Service Level Agreements.
>
> We've detected some "strange" behaviours on some monitors (=sequence of url request retreived with httpclient). Sometimes (1% of the executions), some requests take more time (like 3 sec instead of 100ms) to be performed.
>
> We cannot say for sure for the moment that the network elements of the architecture are not concerned but let's assume first that our problems are not network related (and I'm not saying that the guilty part is httpclient ;], if you can help me to find the proof that it's network related, let's go!)
>
> Logs have been added on the web server side (apache) using a %D in the LogFormat specification. The response times measured on apache side don't match the ones we get using httpclient, not even closer (we see 3 seconds on httpclient side, we see less that 1 ms on apache side).
>
> We've traced the httpclient call and we've been able to detect 2 different behaviours:
>
> the first one occurs during the connection opening, inside HttpConnection.open() call. I've added the "OPENED" trace that comes just after the isOpen=true.
>
> =============================
> isOpen = true;
> if (LOG.isDebugEnabled()) {
> LOG.debug("connection to " + host + ":" + port + " OPENED");
> }
> =============================
>
> our logs:
> ==============================
> [DEBUG] [22/10/08 12:38:26.250] [httpclient.HttpConnection]<7747> Open connection to 160.92.110.240:443 (HttpConnection.java:717)
> [DEBUG] [22/10/08 12:38:29.246] [httpclient.HttpConnection]<7747> connection to 160.92.110.240:443OPENED (HttpConnection.java:771)
> ==============================
>
> I'm aware that there can be a lot of reason why the HttpConnection.open() lasts so long, but does it exist explanation NOT related to network only issues?
>
>
>
> the second one occurs during the flush of the request OutputStream
>
> ==============================
> [DEBUG] [22/10/08 14:00:26.234] [httpclient.HttpConnection]<16536> enter HttpConnection.flushRequestOutputStream() (HttpConnection.java:851)
> [DEBUG] [22/10/08 14:00:29.240] [httpclient.HttpConnection]<16536> leaving HttpConnection.flushRequestOutputStream() (HttpConnection.java:854)
> ==============================
>
> my modifications are:
> ==============================
> public void flushRequestOutputStream() throws IOException {
> LOG.trace("enter HttpConnection.flushRequestOutputStream()");
> assertOpen();
> outputStream.flush();
> LOG.trace("leaving HttpConnection.flushRequestOutputStream()");
> }
> ==============================
>
> what can explain such a long time to flush the request OutputStream?
> Am I wrong or does this output stream only "contains" the request built by httpclient? That would mean that the web server is taking a (too) long time to accept the request?
>
>
> some more details: we have these behaviours using different hosts for our performance tool, so we can assume that these response times are not related to the load of the host that is running the httpclient.
>
Christophe,
HttpClient has no control over TCP/IP connections other than
java.net.Socket API provided by JRE. The problem can be related to the
particular JVM implementation, OS, network configuration, or all
combined.
Oleg
> Thanks for reading :)
>
> Christophe
>
> ________________________________
>
> Ce message et les pi?ces jointes sont confidentiels et r?serv?s ? l'usage exclusif de ses destinataires. Il peut ?galement ?tre prot?g? par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir imm?diatement l'exp?diteur et de le d?truire. L'int?grit? du message ne pouvant ?tre assur?e sur Internet, la responsabilit? du groupe Atos Origin ne pourra ?tre recherch?e quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'exp?diteur ne donne aucune garantie ? cet ?gard et sa responsabilit? ne saurait ?tre recherch?e pour tout dommage r?sultant d'un virus transmis.
>
> This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Multipart
Posted by Christine <ch...@christine.nl>.
On Sun, 2008-10-26 at 11:54 +0100, Oleg Kalnichevski wrote:
> I am glad you found HttpMime useful. The component has been officially
> released and can be found in the maven repository and in official
> release packages:
>
> http://people.apache.org/repo/m2-ibiblio-rsync-repository/org/apache/httpcomponents/httpmime/
> http://hc.apache.org/downloads.cgi
ok, thanks, hadn't seen that. Again, it works like a charm.
dagdag
Christine
> Oleg
>
>
> >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > > For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
--
dagdag is just a two character rotation of byebye
www.christine.nl
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Multipart
Posted by Christine <ch...@christine.nl>.
On Sun, 2008-10-26 at 11:54 +0100, Oleg Kalnichevski wrote:
> http://hc.apache.org/downloads.cgi
Right, now I see. I hadn't seen that the httpClient beta1 includes
httpmime, the version I was using doesn't have it, and now I'm working
whith the httpclient that's included in android, which does have it, but
it's not availaible to developers.
Christine
--
dagdag is just a two character rotation of byebye
www.christine.nl
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Multipart
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, 2008-10-26 at 01:14 +0200, Christine wrote:
> Oleg,
> I found your entity.mime package, for multipart messages in httpClient
> 4.0. It rocks! Works on android too. I found the sources in apache svn,
> when will there be an official package? Can I use it as is, for now?
>
> Thanks for making it anyway :-)
>
> Christine
>
Hi Christine
I am glad you found HttpMime useful. The component has been officially
released and can be found in the maven repository and in official
release packages:
http://people.apache.org/repo/m2-ibiblio-rsync-repository/org/apache/httpcomponents/httpmime/
http://hc.apache.org/downloads.cgi
Oleg
>
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Multipart
Posted by Christine <ch...@christine.nl>.
Oleg,
I found your entity.mime package, for multipart messages in httpClient
4.0. It rocks! Works on android too. I found the sources in apache svn,
when will there be an official package? Can I use it as is, for now?
Thanks for making it anyway :-)
Christine
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
--
dagdag is just a two character rotation of byebye
www.christine.nl
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: HttpConnection.flushRequestOutputStream() sometimes lasts too
long
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, 2008-10-27 at 09:24 +0100, Furmaniak Christophe wrote:
...
> Ok, thanks.
>
> Could you just confirm that the HttpConnection.flushRequestOutputStream() stuff happens when the request is sent to the remote server (and that it's not related to the response reading)?
>
I confirm that
Oleg
> christophe
>
>
> Ce message et les pièces jointes sont confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut également être protégé par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire. L'intégrité du message ne pouvant être assurée sur Internet, la responsabilité du groupe Atos Origin ne pourra être recherchée quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à cet égard et sa responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis.
>
> This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: HttpConnection.flushRequestOutputStream() sometimes lasts too
long
Posted by Furmaniak Christophe <Ch...@atosorigin.com>.
> -----Message d'origine-----
> De : Oleg Kalnichevski [mailto:olegk@apache.org]
> Envoyé : samedi 25 octobre 2008 14:36
> À : HttpClient User Discussion
> Objet : Re: HttpConnection.flushRequestOutputStream() sometimes lasts
> too long
>
> On Thu, 2008-10-23 at 18:43 +0200, Furmaniak Christophe wrote:
> > Hi,
> >
> > we've developped a web application performance tool. Our product
> intensivly uses httpclient (currently 3.1) to measure response time of
> these web applications (single url or sequence of url to copy a human
> being navigation).
> >
> > The main principle of our tool is to perform the same requests at a
> fixed frequency (5 minutes most of the times) to be able to compute
> Service Level Agreements.
> >
> > We've detected some "strange" behaviours on some monitors (=sequence
> of url request retreived with httpclient). Sometimes (1% of the
> executions), some requests take more time (like 3 sec instead of 100ms)
> to be performed.
> >
> > We cannot say for sure for the moment that the network elements of
> the architecture are not concerned but let's assume first that our
> problems are not network related (and I'm not saying that the guilty
> part is httpclient ;], if you can help me to find the proof that it's
> network related, let's go!)
> >
> > Logs have been added on the web server side (apache) using a %D in
> the LogFormat specification. The response times measured on apache side
> don't match the ones we get using httpclient, not even closer (we see 3
> seconds on httpclient side, we see less that 1 ms on apache side).
> >
> > We've traced the httpclient call and we've been able to detect 2
> different behaviours:
> >
> > the first one occurs during the connection opening, inside
> HttpConnection.open() call. I've added the "OPENED" trace that comes
> just after the isOpen=true.
> >
> > =============================
> > isOpen = true;
> > if (LOG.isDebugEnabled()) {
> > LOG.debug("connection to " + host + ":" +
> port + " OPENED");
> > }
> > =============================
> >
> > our logs:
> > ==============================
> > [DEBUG] [22/10/08 12:38:26.250] [httpclient.HttpConnection]<7747>
> Open connection to 160.92.110.240:443 (HttpConnection.java:717)
> > [DEBUG] [22/10/08 12:38:29.246] [httpclient.HttpConnection]<7747>
> connection to 160.92.110.240:443OPENED (HttpConnection.java:771)
> > ==============================
> >
> > I'm aware that there can be a lot of reason why the
> HttpConnection.open() lasts so long, but does it exist explanation NOT
> related to network only issues?
> >
> >
> >
> > the second one occurs during the flush of the request OutputStream
> >
> > ==============================
> > [DEBUG] [22/10/08 14:00:26.234] [httpclient.HttpConnection]<16536>
> enter HttpConnection.flushRequestOutputStream()
> (HttpConnection.java:851)
> > [DEBUG] [22/10/08 14:00:29.240] [httpclient.HttpConnection]<16536>
> leaving HttpConnection.flushRequestOutputStream()
> (HttpConnection.java:854)
> > ==============================
> >
> > my modifications are:
> > ==============================
> > public void flushRequestOutputStream() throws IOException {
> > LOG.trace("enter
> HttpConnection.flushRequestOutputStream()");
> > assertOpen();
> > outputStream.flush();
> > LOG.trace("leaving
> HttpConnection.flushRequestOutputStream()");
> > }
> > ==============================
> >
> > what can explain such a long time to flush the request OutputStream?
> > Am I wrong or does this output stream only "contains" the request
> built by httpclient? That would mean that the web server is taking a
> (too) long time to accept the request?
> >
> >
> > some more details: we have these behaviours using different hosts for
> our performance tool, so we can assume that these response times are
> not related to the load of the host that is running the httpclient.
> >
>
> Christophe,
>
> HttpClient has no control over TCP/IP connections other than
> java.net.Socket API provided by JRE. The problem can be related to the
> particular JVM implementation, OS, network configuration, or all
> combined.
>
> Oleg
[Furmaniak Christophe]
Ok, thanks.
Could you just confirm that the HttpConnection.flushRequestOutputStream() stuff happens when the request is sent to the remote server (and that it's not related to the response reading)?
christophe
Ce message et les pièces jointes sont confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut également être protégé par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire. L'intégrité du message ne pouvant être assurée sur Internet, la responsabilité du groupe Atos Origin ne pourra être recherchée quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à cet égard et sa responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis.
This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org