You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Na...@emc.com on 2009/04/27 17:49:43 UTC
RE: Connection metrics with Http Client/Core
We're using HttpClient version 3.0.1. Could anyone offer some pointers
as to how we might get this information in version 3.0.1?
Thanks
Valli
-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org]
Sent: Sunday, November 23, 2008 9:23 AM
To: HttpClient User Discussion
Subject: Re: Connection metrics with Http Client/Core
Chris Lowe wrote:
> Thanks Oleg. I presume you meant something like this for getting at
the
> connection metrics?
>
> new HttpResponseInterceptor() {
>
> public void process(final HttpResponse response,
> final HttpContext context) throws
HttpException,
> IOException {
>
> final AbstractClientConnAdapter connAdaptor =
> (AbstractClientConnAdapter)
>
context.getAttribute(ExecutionContext.HTTP_CONNECTION);
>
> HttpConnectionMetrics metrics =
connAdaptor.getMetrics();
>
> // ...
> }
> }
>
Yes, that is exactly what I meant.
>
> Looking at the GZip example, for other stats like TTFB etc I could
place an
> InputStream wrapper around the following line:
>
> InputStream wrappedin = wrappedEntity.getContent();
>
>
The best way to go about is to create a custom connection (class
possibly by extending DefaultClientConnection [1]) and collecting
additional metrics by overriding and decorating methods of the
AbstractHttpClientConnection class [2].
You will also have to create a custom connection manager in order to
provide a custom version of the ClientConnectionOperator interface to be
used for creating client connections.
> But wouldn't potentially skew my results since presumably the
InputStream
> reference that I get is already wrapped a few times and is buffered?
>
> Is there a way to get the genuine time to first byte from the
underlying
> socket?
>
If you really need to control I/O on the socket level, you should
decorate SessionInputBuffer and SessionOutputBuffer instances used by
the client connection class. See how DefaultClientConnection adds socket
level logging by decorating instances of those interfaces [3]
>
> BTW when I was looking at the GZip example, I added the code to my
> multithreaded implementation. It worked well. However, I tried
running the
> example with Charles HTTP proxy running so I could see the new headers
> coming through. Normally adding "-Djava.net.useSystemProxies=true" to
the
> JVM arguments routes all traffic through Charles but with HttpClient
it did
> not. Is there anything else that I need to do in order to use the
system
> proxy settings with HttpClient?
>
Use ProxySelectorRoutePlanner instead of the default one. By the way, a
simpler way to see what packets get sent across the wire is by
activating the wire log. Just configure whatever logging framework you
are using beneath Commons Logging to set 'org.apache.http' category to
DEBUG priority.
Hope this helps
Cheers
Oleg
[1]
http://hc.apache.org/httpcomponents-client/httpclient/xref/org/apache/ht
tp/impl/conn/DefaultClientConnection.html
[2]
http://hc.apache.org/httpcomponents-core/httpcore/xref/org/apache/http/i
mpl/AbstractHttpClientConnection.html
[3]
http://hc.apache.org/httpcomponents-client/httpclient/xref/org/apache/ht
tp/impl/conn/DefaultClientConnection.html#166
[4]
http://hc.apache.org/httpcomponents-client/httpclient/apidocs/org/apache
/http/impl/conn/ProxySelectorRoutePlanner.html
>
> Cheers,
>
> Chris.
>
>
> 2008/11/23 Oleg Kalnichevski <ol...@apache.org>
>
>> Chris Lowe wrote:
>>
>>> Hello,
>>>
>>> I'd like to use HttpClient 4.0b1 and HttpCore 4.0b3 to create a
stress
>>> test
>>> application from which I need to get the following stats:
>>>
>>> Connection hand shake time;
>>> Time to first byte sent/recevied;
>>> Time to last byte received;
>>> Overall connection metrics - total bytes sent and received from all
>>> connections.
>>>
>>> Regarding overall metrics, I've seen
>>> DefaultHttpClientConnection.getMetrics() but I cannot see how I can
access
>>> this information through something like ThreadSafeClientConnManager.
>>>
>>> Could anyone offer some pointers as to how I might derive these
stats?
>>>
>>> Cheers,
>>>
>>> Chris.
>>>
>>>
>> Hi Chris
>>
>> You should add a custom protocol interceptor, which would be able to
get
>> hold of the underlying connection from the execution context
>>
>> Here is an example how one can extend HttpClient capabilities by
adding
>> custom protocol interceptors:
>>
>>
>>
http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/module-c
lient/src/examples/org/apache/http/examples/client/ClientGZipContentComp
ression.java
>>
>> Hope this helps
>>
>> Oleg
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Connection metrics with Http Client/Core
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, Apr 27, 2009 at 11:49:43AM -0400, Natarajan_Valli@emc.com wrote:
> We're using HttpClient version 3.0.1. Could anyone offer some pointers
> as to how we might get this information in version 3.0.1?
>
> Thanks
> Valli
>
You just can't, without forking HttpClient, that is.
Oleg
> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Sunday, November 23, 2008 9:23 AM
> To: HttpClient User Discussion
> Subject: Re: Connection metrics with Http Client/Core
>
> Chris Lowe wrote:
> > Thanks Oleg. I presume you meant something like this for getting at
> the
> > connection metrics?
> >
> > new HttpResponseInterceptor() {
> >
> > public void process(final HttpResponse response,
> > final HttpContext context) throws
> HttpException,
> > IOException {
> >
> > final AbstractClientConnAdapter connAdaptor =
> > (AbstractClientConnAdapter)
> >
> context.getAttribute(ExecutionContext.HTTP_CONNECTION);
> >
> > HttpConnectionMetrics metrics =
> connAdaptor.getMetrics();
> >
> > // ...
> > }
> > }
> >
>
> Yes, that is exactly what I meant.
>
>
> >
> > Looking at the GZip example, for other stats like TTFB etc I could
> place an
> > InputStream wrapper around the following line:
> >
> > InputStream wrappedin = wrappedEntity.getContent();
> >
> >
>
> The best way to go about is to create a custom connection (class
> possibly by extending DefaultClientConnection [1]) and collecting
> additional metrics by overriding and decorating methods of the
> AbstractHttpClientConnection class [2].
>
> You will also have to create a custom connection manager in order to
> provide a custom version of the ClientConnectionOperator interface to be
>
> used for creating client connections.
>
> > But wouldn't potentially skew my results since presumably the
> InputStream
> > reference that I get is already wrapped a few times and is buffered?
> >
> > Is there a way to get the genuine time to first byte from the
> underlying
> > socket?
> >
>
> If you really need to control I/O on the socket level, you should
> decorate SessionInputBuffer and SessionOutputBuffer instances used by
> the client connection class. See how DefaultClientConnection adds socket
>
> level logging by decorating instances of those interfaces [3]
>
> >
> > BTW when I was looking at the GZip example, I added the code to my
> > multithreaded implementation. It worked well. However, I tried
> running the
> > example with Charles HTTP proxy running so I could see the new headers
> > coming through. Normally adding "-Djava.net.useSystemProxies=true" to
> the
> > JVM arguments routes all traffic through Charles but with HttpClient
> it did
> > not. Is there anything else that I need to do in order to use the
> system
> > proxy settings with HttpClient?
> >
>
> Use ProxySelectorRoutePlanner instead of the default one. By the way, a
> simpler way to see what packets get sent across the wire is by
> activating the wire log. Just configure whatever logging framework you
> are using beneath Commons Logging to set 'org.apache.http' category to
> DEBUG priority.
>
> Hope this helps
>
> Cheers
>
> Oleg
>
>
> [1]
> http://hc.apache.org/httpcomponents-client/httpclient/xref/org/apache/ht
> tp/impl/conn/DefaultClientConnection.html
> [2]
> http://hc.apache.org/httpcomponents-core/httpcore/xref/org/apache/http/i
> mpl/AbstractHttpClientConnection.html
> [3]
> http://hc.apache.org/httpcomponents-client/httpclient/xref/org/apache/ht
> tp/impl/conn/DefaultClientConnection.html#166
>
> [4]
> http://hc.apache.org/httpcomponents-client/httpclient/apidocs/org/apache
> /http/impl/conn/ProxySelectorRoutePlanner.html
> >
> > Cheers,
> >
> > Chris.
> >
> >
> > 2008/11/23 Oleg Kalnichevski <ol...@apache.org>
> >
> >> Chris Lowe wrote:
> >>
> >>> Hello,
> >>>
> >>> I'd like to use HttpClient 4.0b1 and HttpCore 4.0b3 to create a
> stress
> >>> test
> >>> application from which I need to get the following stats:
> >>>
> >>> Connection hand shake time;
> >>> Time to first byte sent/recevied;
> >>> Time to last byte received;
> >>> Overall connection metrics - total bytes sent and received from all
> >>> connections.
> >>>
> >>> Regarding overall metrics, I've seen
> >>> DefaultHttpClientConnection.getMetrics() but I cannot see how I can
> access
> >>> this information through something like ThreadSafeClientConnManager.
> >>>
> >>> Could anyone offer some pointers as to how I might derive these
> stats?
> >>>
> >>> Cheers,
> >>>
> >>> Chris.
> >>>
> >>>
> >> Hi Chris
> >>
> >> You should add a custom protocol interceptor, which would be able to
> get
> >> hold of the underlying connection from the execution context
> >>
> >> Here is an example how one can extend HttpClient capabilities by
> adding
> >> custom protocol interceptors:
> >>
> >>
> >>
> http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/module-c
> lient/src/examples/org/apache/http/examples/client/ClientGZipContentComp
> ression.java
> >>
> >> Hope this helps
> >>
> >> Oleg
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> >> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >>
> >>
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org