Usage of HttpConnection v. HttpClient

[sebb <se...@gmail.com>]

We are using HttpClient in Apache JMeter.
This uses multiple threads to simulate multiple users.

We don't use HttpClient and a Connection Manager, as we specifically
don't want to share connections between threads. Instead, we create
our own HttpConnection objects and pass these to

        method.execute(state.connection).

I'm now wondering whether this is a sensible approach?

Perhaps we are missing out by not using a method such as:

    httpClient.executeMethod(method)

I would have expected the two approaches to be much the same, but on
looking at the latter, this is handled via HttpMethodDirector, which
seems to do rather more than method.execute() does.

Should we be using a customised ConnectionManager instead?

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org

Re: Usage of HttpConnection v. HttpClient

[sebb <se...@gmail.com>]

On 13/01/06, Roland Weber <ht...@dubioso.net> wrote:
> Hi sebb,
>
> > JMeter can download embedded resources (images) etc referenced in a
> > page, and these may well be from a different host:port.
> >
> > If the next page is from the original host, would it not be better to
> > keep the original connection for further requests?
>
> That's what the connection manager is there for. A connection that is
> returned to the connection manager is "free", but not necessarily closed.
> The HostConfiguration passed to the CM when requesting a connection
> indicates the host:port to which the connection will be targetted, and
> the connection manager can then choose an open connection to that host.
> The SimpleConnectionManager has nothing to choose from, of course.
>
> HttpClient will detect whether the connection returned by the CM is an
> open connection to the target host:port and re-use it if so. Otherwise,
> the HttpConnection object will be pointed to the appropriate host:port
> and a new (socket) connection is opened.
>

Yes, but as you say, the Simple Manager has but one connection to play with.

The MultiThreaded Manager presumably has as many as are allocated to
it, but seemed like overkill for a single thread, and I did not want
JMeter to block if there were no connections free. Nor does JMeter
need connection idle checking.

I already had code to keep track of connections/host, so I just
adapted this to keep track of clients for each host.

Perhaps I could have used the MultiThreaded Manager instead.

S.

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org

Re: Usage of HttpConnection v. HttpClient

[Roland Weber <ht...@dubioso.net>]

Hi sebb,

> JMeter can download embedded resources (images) etc referenced in a
> page, and these may well be from a different host:port.
> 
> If the next page is from the original host, would it not be better to
> keep the original connection for further requests?

That's what the connection manager is there for. A connection that is
returned to the connection manager is "free", but not necessarily closed.
The HostConfiguration passed to the CM when requesting a connection
indicates the host:port to which the connection will be targetted, and
the connection manager can then choose an open connection to that host.
The SimpleConnectionManager has nothing to choose from, of course.

HttpClient will detect whether the connection returned by the CM is an
open connection to the target host:port and re-use it if so. Otherwise,
the HttpConnection object will be pointed to the appropriate host:port
and a new (socket) connection is opened.

cheers,
  Roland

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org

Re: Usage of HttpConnection v. HttpClient

[sebb <se...@gmail.com>]

On 13/01/06, Roland Weber <ht...@dubioso.net> wrote:
...

> >
> > I've now had a look at the code.
> >
> > SimpleConnectionManager only has a single connection, and does not
> > seem to block (unless closing the connection can do so).
> >
> > At present we use a single connection per host per thread - there may
> > be several different host:port combinations in a single thread.
> >
> > So I guess we could use a separate instance of HttpClient per "thread-host".
>
> I don't understand what you are trying to do. HttpClient is synchronous:
> a thread will block until the response for a request has been received.

Agreed.

> If your thread has to connect to different hosts, it should still be
> connected to a single host (or not at all) at any point in time. Hence,
> the single connection "managed" by the SimpleConnectionManager can be
> used for one host:port after another.

Agreed.

> What am I missing?

JMeter can download embedded resources (images) etc referenced in a
page, and these may well be from a different host:port.

If the next page is from the original host, would it not be better to
keep the original connection for further requests?

I imagine this is closer to how browsers work?

Sebastian

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org

Re: Usage of HttpConnection v. HttpClient

[Roland Weber <ht...@dubioso.net>]

Hi sebb,

>>I've looked at the SimpleConnectionManager docs, but I can't seem to
>>determine what happens if a single thread makes requests to different
>>host-port combinations without closing them in between.
>>
>>getConnection[WithTimeout] says that it blocks until a connection is
>>available, but there is no clue as to what "available" means.
>>
>>Also, there does not appear to be a shutdown() method. This suggests
>>that the thread will need to keep track of all its connections.
>>
> 
> 
> I've now had a look at the code.
> 
> SimpleConnectionManager only has a single connection, and does not
> seem to block (unless closing the connection can do so).
> 
> At present we use a single connection per host per thread - there may
> be several different host:port combinations in a single thread.
> 
> So I guess we could use a separate instance of HttpClient per "thread-host".

I don't understand what you are trying to do. HttpClient is synchronous:
a thread will block until the response for a request has been received.
If your thread has to connect to different hosts, it should still be
connected to a single host (or not at all) at any point in time. Hence,
the single connection "managed" by the SimpleConnectionManager can be
used for one host:port after another.

What am I missing?

cheers,
  Roland

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org

Re: Usage of HttpConnection v. HttpClient

[sebb <se...@gmail.com>]

On 12/01/06, sebb <se...@gmail.com> wrote:
> On 12/01/06, Roland Weber <ht...@dubioso.net> wrote:
> > sebb wrote:
> > > We are using HttpClient in Apache JMeter.
> > > This uses multiple threads to simulate multiple users.
> > >
> > > We don't use HttpClient and a Connection Manager, as we specifically
> > > don't want to share connections between threads. Instead, we create
> > > our own HttpConnection objects and pass these to
> > >
> > >         method.execute(state.connection).
> > >
> > > I'm now wondering whether this is a sensible approach?
> > >
> > > Perhaps we are missing out by not using a method such as:
> > >
> > >     httpClient.executeMethod(method)
> > >
> > > I would have expected the two approaches to be much the same, but on
> > > looking at the latter, this is handled via HttpMethodDirector, which
> > > seems to do rather more than method.execute() does.
> > >
> > > Should we be using a customised ConnectionManager instead?
> >
> > No need to use a customized ConnectionManager. Just create a
> > separate instance of HttpClient for each thread. By default,
> > HttpClient instances get a SimpleConnectionManager which uses
> > a single connection. This will provide the thread separation
> > you require, while allowing you to use httpClient.executeMethod.
> >
> > hope this helps,
>
> Thanks!
>
> I've looked at the SimpleConnectionManager docs, but I can't seem to
> determine what happens if a single thread makes requests to different
> host-port combinations without closing them in between.
>
> getConnection[WithTimeout] says that it blocks until a connection is
> available, but there is no clue as to what "available" means.
>
> Also, there does not appear to be a shutdown() method. This suggests
> that the thread will need to keep track of all its connections.
>

I've now had a look at the code.

SimpleConnectionManager only has a single connection, and does not
seem to block (unless closing the connection can do so).

At present we use a single connection per host per thread - there may
be several different host:port combinations in a single thread.

So I guess we could use a separate instance of HttpClient per "thread-host".

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org

Re: Usage of HttpConnection v. HttpClient

[sebb <se...@gmail.com>]

On 12/01/06, Roland Weber <ht...@dubioso.net> wrote:
> sebb wrote:
> > We are using HttpClient in Apache JMeter.
> > This uses multiple threads to simulate multiple users.
> >
> > We don't use HttpClient and a Connection Manager, as we specifically
> > don't want to share connections between threads. Instead, we create
> > our own HttpConnection objects and pass these to
> >
> >         method.execute(state.connection).
> >
> > I'm now wondering whether this is a sensible approach?
> >
> > Perhaps we are missing out by not using a method such as:
> >
> >     httpClient.executeMethod(method)
> >
> > I would have expected the two approaches to be much the same, but on
> > looking at the latter, this is handled via HttpMethodDirector, which
> > seems to do rather more than method.execute() does.
> >
> > Should we be using a customised ConnectionManager instead?
>
> No need to use a customized ConnectionManager. Just create a
> separate instance of HttpClient for each thread. By default,
> HttpClient instances get a SimpleConnectionManager which uses
> a single connection. This will provide the thread separation
> you require, while allowing you to use httpClient.executeMethod.
>
> hope this helps,

Thanks!

I've looked at the SimpleConnectionManager docs, but I can't seem to
determine what happens if a single thread makes requests to different
host-port combinations without closing them in between.

getConnection[WithTimeout] says that it blocks until a connection is
available, but there is no clue as to what "available" means.

Also, there does not appear to be a shutdown() method. This suggests
that the thread will need to keep track of all its connections.

S.

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org

Re: Usage of HttpConnection v. HttpClient

[Roland Weber <ht...@dubioso.net>]

sebb wrote:
> We are using HttpClient in Apache JMeter.
> This uses multiple threads to simulate multiple users.
> 
> We don't use HttpClient and a Connection Manager, as we specifically
> don't want to share connections between threads. Instead, we create
> our own HttpConnection objects and pass these to
> 
>         method.execute(state.connection).
> 
> I'm now wondering whether this is a sensible approach?
> 
> Perhaps we are missing out by not using a method such as:
> 
>     httpClient.executeMethod(method)
> 
> I would have expected the two approaches to be much the same, but on
> looking at the latter, this is handled via HttpMethodDirector, which
> seems to do rather more than method.execute() does.
> 
> Should we be using a customised ConnectionManager instead?

No need to use a customized ConnectionManager. Just create a
separate instance of HttpClient for each thread. By default,
HttpClient instances get a SimpleConnectionManager which uses
a single connection. This will provide the thread separation
you require, while allowing you to use httpClient.executeMethod.

hope this helps,
  Roland



---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org

Re: Usage of HttpConnection v. HttpClient

[sebb <se...@gmail.com>]

On 12/01/06, Oleg Kalnichevski <ol...@apache.org> wrote:
> sebb wrote:
> > We are using HttpClient in Apache JMeter.
> > This uses multiple threads to simulate multiple users.
> >
> > We don't use HttpClient and a Connection Manager, as we specifically
> > don't want to share connections between threads. Instead, we create
> > our own HttpConnection objects and pass these to
> >
> >         method.execute(state.connection).
> >
> > I'm now wondering whether this is a sensible approach?
> >
>
> Sebastian,
>
> This approach works more or less ok with HttpClient 2.0.x due to the
> fact that in HttpClient 2.0 almost all control logic is stuck in the
> HttpMethodBase class.  HttpClient 3.0 has a slightly saner design
> compared to that of HttpClient 2.0. The authentication, redirect
> handling, request retry handling logic has been moved from
> HttpMethodBase to HttpMethodDirector in HttpClient 3.0. Therefore, if

OK, I see - I'd not checked what 2.0 was doing.

BTW, the API changes document

http://svn.apache.org/viewcvs.cgi/jakarta/commons/proper/httpclient/trunk/API_CHANGES_3_0.txt?view=markup

does not mention this change.

> you want to continue using HttpConnection and HttpMethod classes
> directly, you face the prospect of having to re-implement all this logic
> in JMeter.
>

No thanks ... !

>
> > Perhaps we are missing out by not using a method such as:
> >
> >     httpClient.executeMethod(method)
> >
> > I would have expected the two approaches to be much the same, but on
> > looking at the latter, this is handled via HttpMethodDirector, which
> > seems to do rather more than method.execute() does.
> >
> > Should we be using a customised ConnectionManager instead?
> >
>
> I agree with Roland that in your case the simple HTTP connection manager
> should suffice. This said, there's no harm in writing your own
> connection manager if you want to exercise more control over the way
> connections are handled.
>

OK, let's see how that behaves.

Thanks.

S

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org

Re: Usage of HttpConnection v. HttpClient

[Oleg Kalnichevski <ol...@apache.org>]

sebb wrote:
> We are using HttpClient in Apache JMeter.
> This uses multiple threads to simulate multiple users.
> 
> We don't use HttpClient and a Connection Manager, as we specifically
> don't want to share connections between threads. Instead, we create
> our own HttpConnection objects and pass these to
> 
>         method.execute(state.connection).
> 
> I'm now wondering whether this is a sensible approach?
> 

Sebastian,

This approach works more or less ok with HttpClient 2.0.x due to the 
fact that in HttpClient 2.0 almost all control logic is stuck in the 
HttpMethodBase class.  HttpClient 3.0 has a slightly saner design 
compared to that of HttpClient 2.0. The authentication, redirect 
handling, request retry handling logic has been moved from 
HttpMethodBase to HttpMethodDirector in HttpClient 3.0. Therefore, if 
you want to continue using HttpConnection and HttpMethod classes 
directly, you face the prospect of having to re-implement all this logic 
in JMeter.


> Perhaps we are missing out by not using a method such as:
> 
>     httpClient.executeMethod(method)
> 
> I would have expected the two approaches to be much the same, but on
> looking at the latter, this is handled via HttpMethodDirector, which
> seems to do rather more than method.execute() does.
> 
> Should we be using a customised ConnectionManager instead?
> 

I agree with Roland that in your case the simple HTTP connection manager 
should suffice. This said, there's no harm in writing your own 
connection manager if you want to exercise more control over the way 
connections are handled.

Oleg


> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org