You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Saurabh Agrawal <sa...@sapient.com> on 2013/03/20 01:55:52 UTC
Tomcat Behavior on Multiple HTTP requests from same browser
Hi,
I have tried to find the simplest of answer from so many tomcat forums but have not got concrete answer to my query. The query I have is around management of multiple HTTP requests triggering from same browser one after other and being served by Tomcat container.
Let me take an example. Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL, tomcat will assign one of the worker threads (say Thread 1) from the pool to the HTTP request which will be processed and then response will be sent to the client. Now if I hit a link on homepage which is for login, a separate HTTP request will be initiated from the same browser.
What I want to understand is if the Tomcat will keep Thread 1 as persistent thread to server the second request (for login) from the same browser or will it assign a separate thread from the pool ?
If I look at my server logs for both the requests from client, the thread id remains the same which gives me a feeling the state of HTTP is persistent and till the time browser is not closed, the thread won't be returned to the pool.
Please can someone clarify the above behavior.
Note: I need to configure the maxThreads setting of Tomcat to support 20,000 concurrent users in the system. The above clarification will help me pick the appropriate setting for maxThreads.
Saurabh Agrawal
Manager Technology | Sapient
Aachvis Softech Private Limited SEZ,
"Oxygen", (Tower C), Ground - 3rd Floor,
Plot No. 7, Sector 144 Expressway,
Noida 201 301
Uttar Pradesh, India
Tel: +91 (120) 479 5000
Mobile: +91 981 866 4383
Fax: +91 (120) 479 5001
Email: sagrawal@sapient.com<ma...@sapient.com>
sapient.com
The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any (your) computer.
***Please consider the environment before printing this email.***
RE: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by Saurabh Agrawal <sa...@sapient.com>.
There will be a load of 20,000 users for a special festival for around 4-5 hours ONLY.
Regards,
Saurabh Agrawal
Manager Technology | Sapient
-----Original Message-----
From: Christopher Schultz [mailto:chris@christopherschultz.net]
Sent: Wednesday, March 20, 2013 1:48 PM
To: Tomcat Users List
Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Saurabh,
On 3/19/13 8:55 PM, Saurabh Agrawal wrote:
> What I want to understand is if the Tomcat will keep Thread 1 as
> persistent thread to server the second request (for login) from
> the same browser or will it assign a separate thread from the pool
> ?
Tomcat will assign an arbitrary (i.e. random) thread from the pool.
> If I look at my server logs for both the requests from client, the
> thread id remains the same which gives me a feeling the state of
> HTTP is persistent and till the time browser is not closed, the
> thread won't be returned to the pool.
>
> Please can someone clarify the above behavior.
What you are probably observing is a server under very low load (maybe
just your two example requests) that just happens to process the two
requests with the same thread.
Don't worry: the threads go back into the pool right away (after the
keepalive timeout, if you are using the BIO connector which is
currently the default).
> Note: I need to configure the maxThreads setting of Tomcat to
> support 20,000 concurrent users in the system. The above
> clarification will help me pick the appropriate setting for
> maxThreads.
20,000 concurrent users is no problem. How often do they all make
requests?
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEAREIAAYFAlFJvhAACgkQ9CaO5/Lv0PAkXwCfS5GE/d1yk+w1gN6xNvQ9LLfA
y7sAoJjfeun+2Njsu47uBMrWhDl1xzIv
=en0N
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Saurabh,
On 3/19/13 8:55 PM, Saurabh Agrawal wrote:
> What I want to understand is if the Tomcat will keep Thread 1 as
> persistent thread to server the second request (for login) from
> the same browser or will it assign a separate thread from the pool
> ?
Tomcat will assign an arbitrary (i.e. random) thread from the pool.
> If I look at my server logs for both the requests from client, the
> thread id remains the same which gives me a feeling the state of
> HTTP is persistent and till the time browser is not closed, the
> thread won't be returned to the pool.
>
> Please can someone clarify the above behavior.
What you are probably observing is a server under very low load (maybe
just your two example requests) that just happens to process the two
requests with the same thread.
Don't worry: the threads go back into the pool right away (after the
keepalive timeout, if you are using the BIO connector which is
currently the default).
> Note: I need to configure the maxThreads setting of Tomcat to
> support 20,000 concurrent users in the system. The above
> clarification will help me pick the appropriate setting for
> maxThreads.
20,000 concurrent users is no problem. How often do they all make
requests?
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEAREIAAYFAlFJvhAACgkQ9CaO5/Lv0PAkXwCfS5GE/d1yk+w1gN6xNvQ9LLfA
y7sAoJjfeun+2Njsu47uBMrWhDl1xzIv
=en0N
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
André,
On 3/20/13 6:52 AM, André Warnier wrote:
> If I may add something : at some point, on each Tomcat server,
> there is only 1 "listening socket" for one port (the point being :
> you could have several, but you won't have 8000). So even if your
> clients really send 8000 TCP requests to the server at the same
> moment, they will be serialized at some point, and from the
> server's point of view, they come in one by one.
While that is true, the threaded-dispatch of those requests does mean
that many requests can be processed simultaneously. Yes, they are
de-queued serially, but that happens very quickly compared to the
duration of the actual requests.
So, you can have 200 simultaneous requests /in-process/... not that
they all arrive at the exact same instant in time, but that they are
all being served (i.e. have threads assigned and are actually doing
work) simultaneously.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEAREIAAYFAlFJwWYACgkQ9CaO5/Lv0PAtSQCaA1QrSvIOd4GPEKsA2+RezsxQ
RW4AnRBtpcy4LBypi9ShRmproEcerrjd
=inUq
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by André Warnier <aw...@ice-sa.com>.
Mark Thomas wrote:
> Nick Williams <ni...@nicholaswilliams.net> wrote:
>
>> On Mar 19, 2013, at 8:49 PM, Saurabh Agrawal wrote:
>>
>>> Hi Nick,
>>>
>>> We currently have 8 tomcat nodes with each node configured to have
>> 1000 maxThreads. We did a round of performance test for 8000 concurrent
>> users and the observation was that the number of active executor
>> threads were far less.
>>> My understanding was if 8000 concurrent users hit the site at the
>> same time, 8000 executor threads are required to suffice the
>> requirement of all the requests. However, I see far less executor
>> threads in action when we put a load of 8000 concurrent users.
>> Yes, this is to be expected. Tomcat / the JVM will try to be as
>> efficient as possible. A thread will only be in use during an active
>> request.
>
> The correctness of that statement is highly dependent on the connector used and how it is configured.
>
>> Just because you have 8,000 simultaneous users does not mean
>> you have 8,000 simultaneous requests.
>
> +1
>
>>> If you need to support 20,000 simultaneous users, you are going to
>> need a farm of servers. Just one server will not be enough.
>
> How many servers you need will depend on the app and how those users use it. It is perfectly possible for one server to serve 20,000 simultaneous users. 20,000 simultaneous requests is more likely to require multiple servers but again - it depends.
>
If I may add something : at some point, on each Tomcat server, there is only 1 "listening
socket" for one port (the point being : you could have several, but you won't have 8000).
So even if your clients really send 8000 TCP requests to the server at the same moment,
they will be serialized at some point, and from the server's point of view, they come in
one by one. /Then/ the server (provided it's TCP stack and all the rest is fast enough)
can start distributing them over a pool of Threads.
I know that this is an absurd example, but just to provide one extreme point of view :
unless you have 8000 independent clients each firing one request at the sam time over 8000
cables connecting to 8000 ports, each one being served by one CPU core running its own TCP
stack and its own Tomcat, you will never really have 8000 requests being processed really
simultaneously.
So the "simultaneousness" is a question of degree, not an absolute value.
And the "degree of simultaneousness" depends on many factors, among which are (but none of
them to be considered alone) : the network, the front-end, the client keepalive setting,
the server keepalive setting, the number of connections that the client opens
simultaneously, the choice of Connector(s), the usage or not of an Executor, the degree to
which the requests (and responses) are really similar, the number of configured threads,
the time needed to process each request, the CPU speed, the amount of memory etc. etc.
And finally, the number of Threads that are started or are "running" simultaneously
depends on all these factors, and the only one who has access to all these factors is you.
If you fire 8000 "simultaneous" requests from clients, and you see only 200 Threads
running to process them, then obviously there is a bottleneck somewhere. But it is not
necessarily Tomcat which is not allocating all the Threads that it could be allocating.
It could just be that Tomcat does not get more requests to allocate a Thread for, because
they get slowed down (or discarded) somewhere else along the line.
Or it could be that Tomcat just is not getting enough CPU cycles to be able to allocate
more Threads and running them.
I some configurations, and with some kinds of client requests patterns, a longer keepalive
setting can make it so that one Tomcat thread stays allocated to one client connection,
waiting for futher requests on that connection (which may never come). In some kinds of
use cases, that is efficient, in others it is very inefficient. Like everything else, it
depends..
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by Mark Thomas <ma...@apache.org>.
Nick Williams <ni...@nicholaswilliams.net> wrote:
>On Mar 19, 2013, at 8:49 PM, Saurabh Agrawal wrote:
>
>> Hi Nick,
>>
>> We currently have 8 tomcat nodes with each node configured to have
>1000 maxThreads. We did a round of performance test for 8000 concurrent
>users and the observation was that the number of active executor
>threads were far less.
>>
>> My understanding was if 8000 concurrent users hit the site at the
>same time, 8000 executor threads are required to suffice the
>requirement of all the requests. However, I see far less executor
>threads in action when we put a load of 8000 concurrent users.
>>
>
>Yes, this is to be expected. Tomcat / the JVM will try to be as
>efficient as possible. A thread will only be in use during an active
>request.
The correctness of that statement is highly dependent on the connector used and how it is configured.
> Just because you have 8,000 simultaneous users does not mean
>you have 8,000 simultaneous requests.
+1
>> If you need to support 20,000 simultaneous users, you are going to
>need a farm of servers. Just one server will not be enough.
How many servers you need will depend on the app and how those users use it. It is perfectly possible for one server to serve 20,000 simultaneous users. 20,000 simultaneous requests is more likely to require multiple servers but again - it depends.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Nick,
On 3/19/13 10:31 PM, Nick Williams wrote:
> I'm glad to see that you have a working cluster of some sorts and
> that you are performing load tests to evaluate its performance.
> This means you are on the right track, making some good decisions,
> and have obviously done a little reading.
+1000
It's refreshing to see someone actually performing load-tests against
their configuration while tweaking their settings rather than just
saying "I need to support a million users", setting
maxThreads="1000000", setting PermGen to 3GiB, changing the thread
stack size, and micro-managing the garbage collector.
Well done.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEAREIAAYFAlFJvvcACgkQ9CaO5/Lv0PDthgCaA+XQogABsObUGwSUYCeOXadA
BAEAnA7+BHlzmZkgKgAKwXZ0v+9pGrga
=yQdQ
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Saurabh,
On 3/20/13 3:27 AM, Saurabh Agrawal wrote:
> The point I was trying to make when I mentioned that all requests
> from the same browser results in same thread is not a mere
> coincidence. The reason why it may not be sheer luck is because we
> have seen this on production which is accessed by more than 1000
> simultaneous users on the site. Currently we trace the user
> journey by the thread id in the logs and my observation is all 1000
> users (across the globe) gets the same thread id from the pool when
> they access subsequent URLS from the browser.
Is the reverse true? Do you ever see thread A used by more than one
client? Or is it a one-to-one mapping?
> It makes me feel as if the thread is not returned to the pool till
> the session times out or the user has closed the browser. Although
> the behavior is weird because tomcat won't should not have a
> persistent connection as it would mean improper use of the
> resources (thread).
Tomcat is doing its job just fine, otherwise half the Java servers in
the world would lock-up and stop working.
What may be happening is that your browser is maintaining a keep-alive
connection long enough (maybe 10 seconds?) that the "next" request can
be sent over the same connection, and will get the same thread. So, if
you have fairly quick-responding clients, you may see that each one is
kind of monopolizing a thread for a while. Honestly, if that's the
case, it's a *good* case as long as you aren't running out of threads.
It would be interesting to see how you have your <Connector>
configured, and how you have your load-balancer configured, too.
> Also I understand there may be difference in milli seconds or
> seconds between 10000 concurrent user requests and it may not be
> exactly 10000 users in system at one go but I can also simulate
> 10,000 concurrent request at same time even in milli seconds.
> Google analytics does show 10,000 concurrent users. In that case I
> assume it will leverage 10,000 concurrent threads.
Users != requests. 10,000 concurrent users might generate 1000
simultaneous requests (that is, requests in-progress at a particular
instant), unless they are really pounding on your cluster. What do
your thread-pool usage metrics look like? Are you collecting that data
(hint: do it)? Are you graphing them (hint: do that, too)?
Doing time-based graphing can help you visualize your user load
(measured in requests-per-second versus total average concurrent
users, which is a nearly useless metric unless you have 100MiB
sessions or something and you have to use that as part of your
capacity plan) and you can even see things like "hey, every day at
11:00 PDT we hit our maxThreads for 5 minutes". That lets you know
that you need to give yourself a little more breathing room and then
watch for another couple of days.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEAREIAAYFAlFJwI8ACgkQ9CaO5/Lv0PD8QACdHdYnkzzZbcbmq4dmayaguTp/
eDcAn2dGIn63MJRtbllnJcaicIClmJDp
=AfQh
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by Saurabh Agrawal <sa...@sapient.com>.
Hi Nick,
Thanks for your valuable feedback.
The point I was trying to make when I mentioned that all requests from the same browser results in same thread is not a mere coincidence. The reason why it may not be sheer luck is because we have seen this on production which is accessed by more than 1000 simultaneous users on the site. Currently we trace the user journey by the thread id in the logs and my observation is all 1000 users (across the globe) gets the same thread id from the pool when they access subsequent URLS from the browser.
It makes me feel as if the thread is not returned to the pool till the session times out or the user has closed the browser. Although the behavior is weird because tomcat won't should not have a persistent connection as it would mean improper use of the resources (thread).
I read the connector documentation and my gathering is threads are allocated per request and they are returned to the pool once the HTTP response in sent back to the client. However, if that's the case my thread ids should be different when the user accesses multiple pages on the site from same browser. Also printing the thread id may not be the right way to trace user journey then.
Also I understand there may be difference in milli seconds or seconds between 10000 concurrent user requests and it may not be exactly 10000 users in system at one go but I can also simulate 10,000 concurrent request at same time even in milli seconds. Google analytics does show 10,000 concurrent users. In that case I assume it will leverage 10,000 concurrent threads.
Let me know your thoughts around it.
Regards,
Saurabh Agrawal
Manager Technology | Sapient
-----Original Message-----
From: Nick Williams [mailto:nicholas@nicholaswilliams.net]
Sent: Wednesday, March 20, 2013 2:31 AM
To: Tomcat Users List
Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser
On Mar 19, 2013, at 8:49 PM, Saurabh Agrawal wrote:
> Hi Nick,
>
> We currently have 8 tomcat nodes with each node configured to have 1000 maxThreads. We did a round of performance test for 8000 concurrent users and the observation was that the number of active executor threads were far less.
>
> My understanding was if 8000 concurrent users hit the site at the same time, 8000 executor threads are required to suffice the requirement of all the requests. However, I see far less executor threads in action when we put a load of 8000 concurrent users.
>
Yes, this is to be expected. Tomcat / the JVM will try to be as efficient as possible. A thread will only be in use during an active request. Just because you have 8,000 simultaneous users does not mean you have 8,000 simultaneous requests. Remember that "real" users will have some time between each request, ranging from a few seconds to several minutes depending on the page and the user. A good load testing tool like JMeter, The Grinder, or NeoLoad (just examples) will mimic this behavior by putting varying pauses between requests. It sounds like whatever tool you are using is doing this--that's good! So, 8,000 simultaneous users might result in only 400-500 simultaneous requests (just a guess; depends on your application and its user base). In this case, you'd only see 400-500 threads used across the cluster.
In your small-scale test, where each request from your browser got the same thread, that could just be pure luck. If you're the only person using the server, it's likely that the Thread pool is extremely small, thus the odds are high that the same thread would get picked over and over again. However, in a high-load environment, this will not be the case, and the number of threads will nearly always be less than the number of simultaneous users.
I'm glad to see that you have a working cluster of some sorts and that you are performing load tests to evaluate its performance. This means you are on the right track, making some good decisions, and have obviously done a little reading. I suspect a little more reading of the Connector documentation will help you understand exactly how all of this works.
Nick
P.S. Please remember to bottom post. Don't top post.
> -----Original Message-----
> From: Nick Williams [mailto:nicholas@nicholaswilliams.net]
> Sent: Wednesday, March 20, 2013 1:44 AM
> To: Tomcat Users List
> Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser
>
>
> On Mar 19, 2013, at 8:37 PM, Caldarale, Charles R wrote:
>
>>> From: Saurabh Agrawal [mailto:sagrawal@sapient.com]
>>> Subject: Tomcat Behavior on Multiple HTTP requests from same browser
>>
>>> Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL,
>>> tomcat will assign one of the worker threads (say Thread 1) from the pool
>>> to the HTTP request which will be processed and then response will be sent
>>> to the client. Now if I hit a link on homepage which is for login, a separate
>>> HTTP request will be initiated from the same browser.
>>
>>> What I want to understand is if the Tomcat will keep Thread 1 as persistent
>>> thread to server the second request (for login) from the same browser or will
>>> it assign a separate thread from the pool ?
>>
>> Normally, the first available thread from the pool is used for each request. However, if you are using the BIO <Connector> with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active.
>>
>> See the HTTP <Connector> doc, especially the Connector Comparison at the end.
>>
>> http://tomcat.apache.org/tomcat-7.0-doc/config/http.html
>>
>> - Chuck
>
> I think the most important thing to say here is that there is NO guarantee that the browser will always keep the connection alive, therefore there is NO guarantee that every request will get the same thread. You should never rely on having access to the same thread from one request to the next. (That is what HttpSessions are for.)
>
> If you need to support 20,000 simultaneous users, you are going to need a farm of servers. Just one server will not be enough. One simultaneous user does not equal one thread needed: when a user is between requests, a thread can be servicing some other request. You should read the Tomcat documentation thoroughly, especial the sections on connection management, session management and clustering.
>
> Nick
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by Nick Williams <ni...@nicholaswilliams.net>.
On Mar 19, 2013, at 8:49 PM, Saurabh Agrawal wrote:
> Hi Nick,
>
> We currently have 8 tomcat nodes with each node configured to have 1000 maxThreads. We did a round of performance test for 8000 concurrent users and the observation was that the number of active executor threads were far less.
>
> My understanding was if 8000 concurrent users hit the site at the same time, 8000 executor threads are required to suffice the requirement of all the requests. However, I see far less executor threads in action when we put a load of 8000 concurrent users.
>
Yes, this is to be expected. Tomcat / the JVM will try to be as efficient as possible. A thread will only be in use during an active request. Just because you have 8,000 simultaneous users does not mean you have 8,000 simultaneous requests. Remember that "real" users will have some time between each request, ranging from a few seconds to several minutes depending on the page and the user. A good load testing tool like JMeter, The Grinder, or NeoLoad (just examples) will mimic this behavior by putting varying pauses between requests. It sounds like whatever tool you are using is doing this--that's good! So, 8,000 simultaneous users might result in only 400-500 simultaneous requests (just a guess; depends on your application and its user base). In this case, you'd only see 400-500 threads used across the cluster.
In your small-scale test, where each request from your browser got the same thread, that could just be pure luck. If you're the only person using the server, it's likely that the Thread pool is extremely small, thus the odds are high that the same thread would get picked over and over again. However, in a high-load environment, this will not be the case, and the number of threads will nearly always be less than the number of simultaneous users.
I'm glad to see that you have a working cluster of some sorts and that you are performing load tests to evaluate its performance. This means you are on the right track, making some good decisions, and have obviously done a little reading. I suspect a little more reading of the Connector documentation will help you understand exactly how all of this works.
Nick
P.S. Please remember to bottom post. Don't top post.
> -----Original Message-----
> From: Nick Williams [mailto:nicholas@nicholaswilliams.net]
> Sent: Wednesday, March 20, 2013 1:44 AM
> To: Tomcat Users List
> Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser
>
>
> On Mar 19, 2013, at 8:37 PM, Caldarale, Charles R wrote:
>
>>> From: Saurabh Agrawal [mailto:sagrawal@sapient.com]
>>> Subject: Tomcat Behavior on Multiple HTTP requests from same browser
>>
>>> Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL,
>>> tomcat will assign one of the worker threads (say Thread 1) from the pool
>>> to the HTTP request which will be processed and then response will be sent
>>> to the client. Now if I hit a link on homepage which is for login, a separate
>>> HTTP request will be initiated from the same browser.
>>
>>> What I want to understand is if the Tomcat will keep Thread 1 as persistent
>>> thread to server the second request (for login) from the same browser or will
>>> it assign a separate thread from the pool ?
>>
>> Normally, the first available thread from the pool is used for each request. However, if you are using the BIO <Connector> with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active.
>>
>> See the HTTP <Connector> doc, especially the Connector Comparison at the end.
>>
>> http://tomcat.apache.org/tomcat-7.0-doc/config/http.html
>>
>> - Chuck
>
> I think the most important thing to say here is that there is NO guarantee that the browser will always keep the connection alive, therefore there is NO guarantee that every request will get the same thread. You should never rely on having access to the same thread from one request to the next. (That is what HttpSessions are for.)
>
> If you need to support 20,000 simultaneous users, you are going to need a farm of servers. Just one server will not be enough. One simultaneous user does not equal one thread needed: when a user is between requests, a thread can be servicing some other request. You should read the Tomcat documentation thoroughly, especial the sections on connection management, session management and clustering.
>
> Nick
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by Saurabh Agrawal <sa...@sapient.com>.
Hi Nick,
We currently have 8 tomcat nodes with each node configured to have 1000 maxThreads. We did a round of performance test for 8000 concurrent users and the observation was that the number of active executor threads were far less.
My understanding was if 8000 concurrent users hit the site at the same time, 8000 executor threads are required to suffice the requirement of all the requests. However, I see far less executor threads in action when we put a load of 8000 concurrent users.
Regards,
Saurabh Agrawal
Manager Technology | Sapient
-----Original Message-----
From: Nick Williams [mailto:nicholas@nicholaswilliams.net]
Sent: Wednesday, March 20, 2013 1:44 AM
To: Tomcat Users List
Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser
On Mar 19, 2013, at 8:37 PM, Caldarale, Charles R wrote:
>> From: Saurabh Agrawal [mailto:sagrawal@sapient.com]
>> Subject: Tomcat Behavior on Multiple HTTP requests from same browser
>
>> Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL,
>> tomcat will assign one of the worker threads (say Thread 1) from the pool
>> to the HTTP request which will be processed and then response will be sent
>> to the client. Now if I hit a link on homepage which is for login, a separate
>> HTTP request will be initiated from the same browser.
>
>> What I want to understand is if the Tomcat will keep Thread 1 as persistent
>> thread to server the second request (for login) from the same browser or will
>> it assign a separate thread from the pool ?
>
> Normally, the first available thread from the pool is used for each request. However, if you are using the BIO <Connector> with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active.
>
> See the HTTP <Connector> doc, especially the Connector Comparison at the end.
>
> http://tomcat.apache.org/tomcat-7.0-doc/config/http.html
>
> - Chuck
I think the most important thing to say here is that there is NO guarantee that the browser will always keep the connection alive, therefore there is NO guarantee that every request will get the same thread. You should never rely on having access to the same thread from one request to the next. (That is what HttpSessions are for.)
If you need to support 20,000 simultaneous users, you are going to need a farm of servers. Just one server will not be enough. One simultaneous user does not equal one thread needed: when a user is between requests, a thread can be servicing some other request. You should read the Tomcat documentation thoroughly, especial the sections on connection management, session management and clustering.
Nick
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by Nick Williams <ni...@nicholaswilliams.net>.
On Mar 19, 2013, at 8:37 PM, Caldarale, Charles R wrote:
>> From: Saurabh Agrawal [mailto:sagrawal@sapient.com]
>> Subject: Tomcat Behavior on Multiple HTTP requests from same browser
>
>> Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL,
>> tomcat will assign one of the worker threads (say Thread 1) from the pool
>> to the HTTP request which will be processed and then response will be sent
>> to the client. Now if I hit a link on homepage which is for login, a separate
>> HTTP request will be initiated from the same browser.
>
>> What I want to understand is if the Tomcat will keep Thread 1 as persistent
>> thread to server the second request (for login) from the same browser or will
>> it assign a separate thread from the pool ?
>
> Normally, the first available thread from the pool is used for each request. However, if you are using the BIO <Connector> with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active.
>
> See the HTTP <Connector> doc, especially the Connector Comparison at the end.
>
> http://tomcat.apache.org/tomcat-7.0-doc/config/http.html
>
> - Chuck
I think the most important thing to say here is that there is NO guarantee that the browser will always keep the connection alive, therefore there is NO guarantee that every request will get the same thread. You should never rely on having access to the same thread from one request to the next. (That is what HttpSessions are for.)
If you need to support 20,000 simultaneous users, you are going to need a farm of servers. Just one server will not be enough. One simultaneous user does not equal one thread needed: when a user is between requests, a thread can be servicing some other request. You should read the Tomcat documentation thoroughly, especial the sections on connection management, session management and clustering.
Nick
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP
requests from same browser
Posted by "Howard W. Smith, Jr." <sm...@gmail.com>.
On Fri, Mar 22, 2013 at 10:04 AM, Jeffrey Janner <
Jeffrey.Janner@polydyne.com> wrote:
> > -----Original Message-----
> > From: André Warnier [mailto:aw@ice-sa.com]
> > Sent: Thursday, March 21, 2013 8:51 AM
> > To: Tomcat Users List
> > Subject: Re: [a bit, but not totally OT] Tomcat Behavior on Multiple
> > HTTP requests from same browser
> >
> > Christopher Schultz wrote:
> > > HTTP connections for long periods of time, but that's really abuse of
> > > the protocol IMO. You can send bowling balls via carrier pigeon, but
> > > there are better ways to send bowling balls.
> >
> > You would need a fairly large, and well-disciplined team of pigeons to
> > do that though. I don't think that this was a good metaphor, You should
> > have chosen a bigger bird and/or a smaller load. Eagles and tennis
> > balls maybe ?
> >
>
> Or swallows and coconuts.
>
> Jeff
> (sorry, couldn't resist.)
>
>
Wow, you all are funny! LOL
Re: [totally OT] Tomcat Behavior on Multiple HTTP requests from same
browser
Posted by Mark Eggers <it...@yahoo.com>.
On 3/22/2013 9:35 AM, André Warnier wrote:
> Caldarale, Charles R wrote:
>>> From: Jeffrey Janner [mailto:Jeffrey.Janner@PolyDyne.com] Subject:
>>> RE: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP
>>> requests from same browser
>>
>>>> You would need a fairly large, and well-disciplined team of pigeons to
>>>> do that though. I don't think that this was a good metaphor, You should
>>>> have chosen a bigger bird and/or a smaller load. Eagles and tennis
>>>> balls maybe ?
>>
>>> Or swallows and coconuts.
>>
>> Someone had to bring that up. African or European?
>>
>> I think we can remove the "not" from the subject line now...
>>
> Done.
> This all reminds me of this (locally) well-known Belgian bird : the
> oye-oye-oye bird.
> For those who don't know the species :
> It is a very strong, sturdy bird. Rather bad-tempered too, you shouldn't
> mess with it.
> It is a bit the bird-equivalent of the Belgian horse really.
> It has a big round head, with a strong beak, say oh a good 5 inches wide
> and 7 inches long. A really strong beak, he can crush nuts or bones
> with it.
> Then it has very strong wings too, strong enough to lift a small sheep
> for instance (or a bowling ball for that matter).
> And tough feathers, you can make brooms with them.
> And also rather short, but very strong sturdy legs; like 2 inches thick
> and 2 inches long.
> And then its balls, no kidding, they are the size of coconuts.
> And each time it lands, it goes "oye oye oye".
Sounds very much like the African oomy-goomie bird.
There's a rugby song about that - something about going off to see a
wild west show . . . elephants and kangaroos.
Looks like at least one subspecies migrated north.
/mde/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [totally OT] Tomcat Behavior on Multiple HTTP requests from same
browser
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
André,
On 3/22/13 12:35 PM, André Warnier wrote:
> Caldarale, Charles R wrote:
>>> From: Jeffrey Janner [mailto:Jeffrey.Janner@PolyDyne.com]
>>> Subject: RE: [a bit, but not totally OT] Tomcat Behavior on
>>> Multiple HTTP requests from same browser
>>
>>>> You would need a fairly large, and well-disciplined team of
>>>> pigeons to do that though. I don't think that this was a good
>>>> metaphor, You should have chosen a bigger bird and/or a
>>>> smaller load. Eagles and tennis balls maybe ?
>>
>>> Or swallows and coconuts.
>>
>> Someone had to bring that up. African or European?
>>
>> I think we can remove the "not" from the subject line now...
>>
> Done. This all reminds me of this (locally) well-known Belgian bird
> : the oye-oye-oye bird. For those who don't know the species : It
> is a very strong, sturdy bird. Rather bad-tempered too, you
> shouldn't mess with it. It is a bit the bird-equivalent of the
> Belgian horse really. It has a big round head, with a strong beak,
> say oh a good 5 inches wide and 7 inches long. A really strong
> beak, he can crush nuts or bones with it. Then it has very strong
> wings too, strong enough to lift a small sheep for instance (or a
> bowling ball for that matter). And tough feathers, you can make
> brooms with them. And also rather short, but very strong sturdy
> legs; like 2 inches thick and 2 inches long. And then its balls, no
> kidding, they are the size of coconuts. And each time it lands, it
> goes "oye oye oye".
That sounds absolutely terrifying. A 7-inch-long beak that's 5-inches
wide? That's like an oil tunnel for a car.
I don't even want to hear about the Belgian Horse...
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRTNzSAAoJEBzwKT+lPKRY79EP/0xwu/G4cTHvKoJldQRdFYUX
cT9vBgr/xrdkNuV8gO/+cwBhiAGVAw2LOtkT6aF85zdv0Da6PALk6DOrvx4PLcnz
KVAGG2mp6KHngISB09ti1I/QycNfKFQUkA3ogHFB8N3LDxQoTc1ZfbcWm1+UcRYu
BMvXrzAcAapx0ZKYP4ZId7Z9vtIyB5mpGmCXu88x0bJs+D/shzLTPGiE7tYzZfpi
R/VJhE4TezajaRFvnCNLLooeHAGEx3qJ/FNLaynr1QB8X7QRhOomulr5tR4n5+Za
ELoGLkeIfcYCrXiSDANJAEGrLUf5/4ub/yZqPhXuL2dgSI5FvI4+Np7PfscfJ0xi
BtTZ8JYB6I0FQfJqRHnB0krpvnBhncmnyqdk+OKuxoSqYeHabaX8PlqGMQu6mlpR
ay71I8qrHYsBvOjTF+rexstcNEkzWeEON06Q+AzlWElp/NAyI+OT6lpHXAew7A5e
wX8RcVU7/5vC14RnpUVwaA4OSvqQpMH29hk9n0s5rEjR8zro+fm5ZFUCHxsg6j51
Xmgq0AwENY2hm6EVducDs2aFUK1n1xoz0z09Vc9Jb9dIiXuz0DgPW/Ls7YcoiRce
kh04Jn+M8oQl1nZsteb3egXasExm/S8EoCg/HrFpenLje30z+riW7mBPRpwx4v5p
WenLW/6m51hgZNpuccxx
=QmXY
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [totally OT] Tomcat Behavior on Multiple HTTP requests from same
browser
Posted by André Warnier <aw...@ice-sa.com>.
Caldarale, Charles R wrote:
>> From: Jeffrey Janner [mailto:Jeffrey.Janner@PolyDyne.com]
>> Subject: RE: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser
>
>>> You would need a fairly large, and well-disciplined team of pigeons to
>>> do that though. I don't think that this was a good metaphor, You should
>>> have chosen a bigger bird and/or a smaller load. Eagles and tennis
>>> balls maybe ?
>
>> Or swallows and coconuts.
>
> Someone had to bring that up. African or European?
>
> I think we can remove the "not" from the subject line now...
>
Done.
This all reminds me of this (locally) well-known Belgian bird : the oye-oye-oye bird.
For those who don't know the species :
It is a very strong, sturdy bird. Rather bad-tempered too, you shouldn't mess with it.
It is a bit the bird-equivalent of the Belgian horse really.
It has a big round head, with a strong beak, say oh a good 5 inches wide and 7 inches
long. A really strong beak, he can crush nuts or bones with it.
Then it has very strong wings too, strong enough to lift a small sheep for instance (or a
bowling ball for that matter).
And tough feathers, you can make brooms with them.
And also rather short, but very strong sturdy legs; like 2 inches thick and 2 inches long.
And then its balls, no kidding, they are the size of coconuts.
And each time it lands, it goes "oye oye oye".
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: [totally OT] Tomcat Behavior on Multiple HTTP requests from
same browser
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Christopher Schultz [mailto:chris@christopherschultz.net]
> Subject: Re: [totally OT] Tomcat Behavior on Multiple HTTP requests from same browser
> > > Or swallows and coconuts.
> > Someone had to bring that up. African or European?
> > I think we can remove the "not" from the subject line now...
> I think someone gets thrown off a bridge into an abyss if we keep
> going. The Spring Generation probably doesn't have any idea what we're
> tottering on about.
The rules say the Spring Generation goes over if they can't answer the questions...
Auuuuuuuugh.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [totally OT] Tomcat Behavior on Multiple HTTP requests from same
browser
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Chuck,
On 3/22/13 10:25 AM, Caldarale, Charles R wrote:
>> From: Jeffrey Janner [mailto:Jeffrey.Janner@PolyDyne.com]
>> Subject: RE: [a bit, but not totally OT] Tomcat Behavior on
>> Multiple HTTP requests from same browser
>
>>> You would need a fairly large, and well-disciplined team of
>>> pigeons to do that though. I don't think that this was a good
>>> metaphor, You should have chosen a bigger bird and/or a smaller
>>> load. Eagles and tennis balls maybe ?
>
>> Or swallows and coconuts.
>
> Someone had to bring that up. African or European?
>
> I think we can remove the "not" from the subject line now...
I think someone gets thrown off a bridge into an abyss if we keep
going. The Spring Generation probably doesn't have any idea what we're
tottering on about.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRTNxAAAoJEBzwKT+lPKRY3koQAIkmmGnn4SA1u1VeLkU8ZAre
8YWDktOOvf6J0xJQl8iRajoV2ZZT0EjoHSO9ODopS7BiYRcbuCfLROXaIYLSD/PR
nLiporwrtehxW7yTHxtpM3P599E/Gr2vvLhw3/9Wl7BcM1exnQrsDkmTa3bLEIxT
XvUaisgwhDquCyfFyPxV5fy46Fn128fyHa6E4ZUaeRGYYH4hcwkAJRc7lM1TaX+y
BOmNr0jR60Z2KfSMEWeXR2J4FpYoFUrkS3KDRWi7RruZp28BR7MPLX/49inioy3K
a+K7lgus9s6p0QySBREDEpi2oCXEO31m2gRYQ4O7iXOoKaKcOkMS7Vx2hKkbO2eq
unJ/pcjkTWYXVx5oahbi9VEOSf3PiXGV3xYA/JFFbG/xYkKeVReofTmCoeHIfVTh
H9GKbzGR0jQAWUkZpnuMumg+Dbra6/J3+Nau/ItDy5kQ1e1xIbLyYswEbzTjxnHa
0wNkQkfID/IfOpz9mnM5wkowE8cDrqVFO3YkWDzYo5GsIliGT8o6NapTf7y8ll1D
rVyVhPfMUJWMjq6QlJRcPJE3EuluOtRP+73UejZmqssL/V2RJCH+5f2sbBipB1gW
X+VwBRVuD0qCGgniDIrXYjLZqTKsp+Y/9d38H1SgZgdTdIOPtqMVJ8IwU1uYopTa
L2rZsjHHYmB7+eYyzvMf
=Ttie
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP
requests from same browser
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Jeffrey Janner [mailto:Jeffrey.Janner@PolyDyne.com]
> Subject: RE: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP requests from same browser
> > You would need a fairly large, and well-disciplined team of pigeons to
> > do that though. I don't think that this was a good metaphor, You should
> > have chosen a bigger bird and/or a smaller load. Eagles and tennis
> > balls maybe ?
> Or swallows and coconuts.
Someone had to bring that up. African or European?
I think we can remove the "not" from the subject line now...
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP
requests from same browser
Posted by Jeffrey Janner <Je...@PolyDyne.com>.
> -----Original Message-----
> From: André Warnier [mailto:aw@ice-sa.com]
> Sent: Thursday, March 21, 2013 8:51 AM
> To: Tomcat Users List
> Subject: Re: [a bit, but not totally OT] Tomcat Behavior on Multiple
> HTTP requests from same browser
>
> Christopher Schultz wrote:
> > HTTP connections for long periods of time, but that's really abuse of
> > the protocol IMO. You can send bowling balls via carrier pigeon, but
> > there are better ways to send bowling balls.
>
> You would need a fairly large, and well-disciplined team of pigeons to
> do that though. I don't think that this was a good metaphor, You should
> have chosen a bigger bird and/or a smaller load. Eagles and tennis
> balls maybe ?
>
Or swallows and coconuts.
Jeff
(sorry, couldn't resist.)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP
requests from same browser
Posted by André Warnier <aw...@ice-sa.com>.
Christopher Schultz wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> André,
>
> On 3/21/13 7:15 AM, André Warnier wrote:
>> Christopher Schultz wrote:
>>> I think there might be a problem with the instrumentation, or
>>> just coincidences at a fairly implausible level. The trust of the
>>> matter is that Tomcat does not allocate a thread permanently to a
>>> remote client until ... whenever the client "disconnects"
>>> (whatever that means, as HTTP is a connection-less protocol).
>> (See the nitpick (*) below)
>>
>> Possible, but see above again with the httpd/tomcat connections
>> managed by the mod_jk module. It does have and manage its own pool
>> of connections, with each connection potentially "staying alive"
>> for a time much longer than any individual client request. I do not
>> deny that.
>
> Right, but the AJP connections are managed in a connection pool. I
> haven't really checked-into this, but I suspect that two requests
> coming from the same keepalive connection have no guarantee of being
> sent across the same AJP connection to Tomcat, and thus no guarantee
> that they will be served by the same JVM thread.
>
>> mod_jk is aware that the client/httpd connection is keepalive, and
>> it does not have any way to know that this client is not going to
>> send another request to Tomcat. So what does mod_jk really do ?
>> Does it relinquish the one connection that he had to Tomcat back to
>> the pool immediately after the first response has been served ? or
>> does it keep its handle on that pool connection until the
>> client/httpd timeout has expired ?
>
> It would be a mistake for mod_jk to retain control of the AJP
> connection for that keepalive request -- there's no guarantee that the
> /next/ request across that connection would even be routed through
> mod_jk: it might be for some other resource that another module handles.
On the other hand, if there were 10 successive requests for Tomcat from the same client on
the same connection, then it might be argued that it would be counterproductive to return
the connection to the pool each time, just to go obtain another one right after, and this
10 times in a row.
May be there should be an "adaptative" or "predictive" algorithm here : if this client in
the recent past has always sent several requests in short succession, then maybe I'll keep
this connection for now, just in case he does it again.
I can already hear Rainer saying "patches are always welcome".
;-)
But the real point is : does mod_jk keep the connection, or does it return it to the pool
at the end of each response ? Barring Rainer reading this, I guess that only looking at
the code would tell.
Note that Apache httpd already maintains the client/httpd connection, and keeps a count of
how many requests have been received over this connection. It has to, for
MaxKeepAliveRequests. So it would not be too much of a complication for mod_jk to keep
its own count, of how many requests forwarded to Tomcat have been received over this same
connection. That would already be a good predictor of whether the same is likely in the
future.
a = time for which this client connection has been alive
b = number of requests forwarded to tomcat over this connection
c = a / b = average time between 2 requests forwarded to tomcat
if c is lower than the overhead for obtaining and returning a connection from the pool,
then keep the connection.
It would be self-adaptative, because if the client slows down its request rate, then c
would become larger, and the connection would be returned to the pool; and vice-versa.
>
>> There is also kind of a weird question here : what is really the
>> purpose of the keepAliveTimeout attribute on the Tomcat AJP
>> Connector ?
>
> +1
>
>> (*) nitpick about HTTP being connection-less : that may be true in
>> the sense that each request+response is supposedly independent from
>> any other request+response. But HTTP 1.1 explicitly introduces
>> "persistent" TCP connections.
>
> Yes, and HTTP sessions are standard fare these days, too. But the fact
> is that HTTP itself is connection-less. We as engineers can make it
> feel like it's not and do stupid things like put JDBC connections into
> HttpSession objects and try to tie everything together to make the
> user feel like they have a permanent connection. We can even hold-open
> HTTP connections for long periods of time, but that's really abuse of
> the protocol IMO. You can send bowling balls via carrier pigeon, but
> there are better ways to send bowling balls.
You would need a fairly large, and well-disciplined team of pigeons to do that though. I
don't think that this was a good metaphor, You should have chosen a bigger bird and/or a
smaller load. Eagles and tennis balls maybe ?
I should also probably remind you of RFC 1149 :
http://en.wikipedia.org/wiki/IP_over_Avian_Carriers
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP
requests from same browser
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
André,
On 3/21/13 7:15 AM, André Warnier wrote:
> Christopher Schultz wrote:
>>
>> I think there might be a problem with the instrumentation, or
>> just coincidences at a fairly implausible level. The trust of the
>> matter is that Tomcat does not allocate a thread permanently to a
>> remote client until ... whenever the client "disconnects"
>> (whatever that means, as HTTP is a connection-less protocol).
>
> (See the nitpick (*) below)
>
> Possible, but see above again with the httpd/tomcat connections
> managed by the mod_jk module. It does have and manage its own pool
> of connections, with each connection potentially "staying alive"
> for a time much longer than any individual client request. I do not
> deny that.
Right, but the AJP connections are managed in a connection pool. I
haven't really checked-into this, but I suspect that two requests
coming from the same keepalive connection have no guarantee of being
sent across the same AJP connection to Tomcat, and thus no guarantee
that they will be served by the same JVM thread.
> mod_jk is aware that the client/httpd connection is keepalive, and
> it does not have any way to know that this client is not going to
> send another request to Tomcat. So what does mod_jk really do ?
> Does it relinquish the one connection that he had to Tomcat back to
> the pool immediately after the first response has been served ? or
> does it keep its handle on that pool connection until the
> client/httpd timeout has expired ?
It would be a mistake for mod_jk to retain control of the AJP
connection for that keepalive request -- there's no guarantee that the
/next/ request across that connection would even be routed through
mod_jk: it might be for some other resource that another module handles.
> There is also kind of a weird question here : what is really the
> purpose of the keepAliveTimeout attribute on the Tomcat AJP
> Connector ?
+1
> (*) nitpick about HTTP being connection-less : that may be true in
> the sense that each request+response is supposedly independent from
> any other request+response. But HTTP 1.1 explicitly introduces
> "persistent" TCP connections.
Yes, and HTTP sessions are standard fare these days, too. But the fact
is that HTTP itself is connection-less. We as engineers can make it
feel like it's not and do stupid things like put JDBC connections into
HttpSession objects and try to tie everything together to make the
user feel like they have a permanent connection. We can even hold-open
HTTP connections for long periods of time, but that's really abuse of
the protocol IMO. You can send bowling balls via carrier pigeon, but
there are better ways to send bowling balls.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRSwfNAAoJEBzwKT+lPKRYAMgQAKyONnliLXad6HDwO9raxN+d
evcEl8zAeYr6vbevZJ/KJK/FALoFVOmHdMDj+twGEWM+zrIOkHel2y9LHKzxR+St
6Fz1466yDeHM45D/PBcyMow2WaOSR9a6Ewj8uDprJLuVjT20qRlaiw0pQjvfB5M2
rPfX0rEP6NPMQNaTTdaTiq56LP4j/kNMiIhNZyZrq3Gu+9hP/LGEZgKW4j9PDPRF
wvNUnWrHwhl4cUJwtRF1CuyynmJmnsrglQWpLgj1cYvBnzHp/19I3CKUevut5JUj
NtcCmZ+u/is9zsJbWn6g1yRpyVFNForyV2XF2UFMDm/4UO+Ofyq1lVsGtvkMu3b2
2PQ7vMPqMM34JBySI3ZWVMFxD3GZUMm+Bqc4H5iKIzcGxRg0MgQn5z6DHniuIOmw
lUxsjiwHiJ8xF/W3cdN1cxzPPzG92MOp4FWpsnMF+XcAN8ctGr5MuRJzDJKfct1o
VcQojNqhmDyBHlJd3188aAz94KUXIGyXkmsLNdvnYhSZLZWxjFwBNxYm/4RzmHeA
Dm/Heoe+qpxsk868YGKvJcIAk/1Rdxje8ZEJv44YRNXyCqfDkq0t9HUCduzyNBJF
4H/RVCSSS6OEXvdXUMywS2JJElcss560fSZ1ZF45YSW6NiLMIyl5PjFR1mb0Kfsf
YYwN2L9egDE8ZDibeON2
=Bxsy
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [a bit, but not totally OT] Tomcat Behavior on Multiple HTTP
requests from same browser
Posted by André Warnier <aw...@ice-sa.com>.
Christopher,
Christopher Schultz wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> André,
>
> On 3/20/13 2:25 PM, André Warnier wrote:
>> Saurabh Agrawal wrote:
>>> All our assets are served from L3 CDN. So the asset requests
>>> never come to the application server.
>> That, I do not understand. I do not understand what you mean by
>> "assets" here, and I do not understand "L3 CDN". So I cannot tell
>> of this is relevant or not to the problem.
>
> CDN = Content Delivery Network. I'm not sure what "L3" (probably
> "Level 3", a data center operations company) is, but a CDN is
> basically a whole bunch of copies of your files geographically
> distributed such that requesting a file always gets the bits that are
> closest to you. Kind of a cool thing. ;)
>
Thank your for the above Rosetta Stone. This computer business os so full of acronyms of
all kinds - some of them with multiple interpretations - that it is sometimes difficult to
grasp the meaning. And I really don't feel like having to use Wikipedia every 3 words of
a post on the list. Not when "static content is delivered by the Apache front-end" would
have done it.
> The bottom line is that Saurabh expects only dynamic requests to come
> to Tomcat, so keepalives should be much less useful than if Tomcat
> were to be serving everything. Imagine httpd out front serving all
> static content and forwarding dynamic stuff to Tomcat via AJP --
> that's almost exactly what's going on here, except that the static
> stuff is being served very efficiently from a network-topology
> perspective.
>
> Since AJP is in use, keepalive is almost entirely a red herring as
> typical AJP connections are permanently-connected to the web server.
>
Well, I would say so indeed forthe case of a html page wit embedded images e.g.
Butit may not be so in the "benchmark" case that Saurabh explained, with each of the 10000
clients making multiple requests to non-static content, presumably served by Tomcat.
A human user may have delays, that his testcase might not have.
>> So, by default, the keepAliveTimeout [for AJP] is set to
>> "infinite".
>>
>> [snip]
>>
>> And if the client keeps the connection open, but does not send any
>> additional request on that connection, the Thread will wait
>> theoretically forever (because that is what the documentation says
>> about the default value of these parameters).
>
> No, the defaults are different for non-AJP connections. Tomcat's
> default default is 60 seconds but the stock server.xml configures it
> to 20 seconds.
Right. But my explanation was meant as an example only, to point out the interlinked
effects of the various attributes and timeouts.
And 20 seconds is still an incredibly long time, in the context of 10000 "simultaneous"
clients sending multiple requests each.
>
>> Now your case is a bit different, because - you are not using the
>> HTTP BIO connector (you use AJP)
>
> I think you've gotten yourself confused, here, unfortunately. You can
> use AJP with BIO, NIO, or APR (maybe you mixed-up AJP and APR between
> your eyes and your brain... the two are honestly too close to each
> other and it is very easy to do that).
Yes, I was confused and thank you for making me see the light. I though that the AJP
Connector was a beast all of it's own, and did not "use" these different underlying
layers. It is clear fom the documentation that it does though, I don't know how I could
have overlooked that for so long.
>
> He is in fact using the BIO connector because he has specified
> protocol="org.apache.coyote.ajp.AjpProtocol".
>
>> - in front of your Tomcat, is an Apache httpd server. This server
>> has its own keep-alive settings which apply to the connection of
>> the client with Apache httpd. And these keep-alive settings are a
>> bit different from the Tomcat ones (for example, there is a
>> keep-alive timeout, but also a MaxKeepAliveRequests)
>
> +1
>
>> - between Apache httpd and Tomcat, there is the mod_jk module in
>> Apache, and that module uses its own timeouts (as set in
>> workers.properties), and in addition it uses itself a pool of
>> connections to Tomcat, and this pool of connections has its own
>> rules for keeping alive a connection between Apache and Tomcat.
>>
>> But the basic principles above apply, and may explain why you are
>> seeing what appears to be one Thread dedicated to one client,
>> forever.
>
> I think there might be a problem with the instrumentation, or just
> coincidences at a fairly implausible level. The trust of the matter is
> that Tomcat does not allocate a thread permanently to a remote client
> until ... whenever the client "disconnects" (whatever that means, as
> HTTP is a connection-less protocol).
(See the nitpick (*) below)
Possible, but see above again with the httpd/tomcat connections managed by the mod_jk
module. It does have and manage its own pool of connections, with each connection
potentially "staying alive" for a time much longer than any individual client request.
I do not deny that.
But what I am not so sure of (and maybe Rainer could comment here) is this scenario :
- a client, via httpd+mod_jk, sends a single request to tomcat on a keep-alive connection,
and receives a response. Now the client does not send another request anymore on the same
connection, but keeps it open "just in case".
Now say that at the client/httpd level, the keep-alive timeout is set to 30 seconds; and
say that on the Tomcat AJP connector, the keep-alive timeout is set for 20 seconds.
What really happens ?
After this first request, Apache httpd would have to keep the client/httpd connection
alive for 30 seconds, right ? At the Apache-httpd level, that means that this "Apache
child" process keeps its connection to the client open, and sits there waiting for another
request. Mod_jk is not a separate animal; it is an Apache module, so it is code
"embedded" in this one Apache-httpd child process.
So what does mod_jk really do ? he needed one connection to Tomcat, to send the first
request and get the response, so he got it from the pool of connections.
mod_jk is aware that the client/httpd connection is keepalive, and it does not have any
way to know that this client is not going to send another request to Tomcat. So what does
mod_jk really do ?
Does it relinquish the one connection that he had to Tomcat back to the pool immediately
after the first response has been served ? or does it keep its handle on that pool
connection until the client/httpd timeout has expired ?
And assuming for a moment that it keeps this handle to himself for a while, what does that
mean at the Tomcat level ? is a Tomcat Thread also waiting on that same httpd/tomcat
connection (at least for 20 seconds) ? or do Tomcat Threads *always* go back to the pool
after serving one request ? or does it depend, and on what ?
There is also kind of a weird question here : what is really the purpose of the
keepAliveTimeout attribute on the Tomcat AJP Connector ? I mean, connections to that
Connector always come from some front-end module (be it mod_jk, mod_proxy_ajp or the
isapi_redirector). (It is very unlikely that they would come from some independent
AJP-capable client).
Why would the AJP Connector need its own keepAliveTimeout attribute, if these front-end
modules are themselves managing their connections to Tomcat, in the way that they deem
appropriate ?
These front-end modules could pass to Tomcat the timeout that /they/ want, when they open
their individual pool connections to Tomcat, right?
Isn't that Connector keepAliveTimeout then more confusing than really useful ?
(And I guess that the same could be said for the connectionTimeout).
Come to think of it, that is probably why, in the case of the AJP Connector, the defaults
are "infinite". And maybe the attributes are left accessible for the rare edge cases where
they could prove useful.
If so, then I guess that a small note in the documentation would be useful.
(*) nitpick about HTTP being connection-less : that may be true in the sense that each
request+response is supposedly independent from any other request+response. But HTTP 1.1
explicitly introduces "persistent" TCP connections. And Microsoft has its own take on
this, as for instance for Windows Integrated Authentication purposes, it is the
*connection* which is authenticated, and each time the connection changes, the client has
to re-authenticate. So the practice is a bit less connection-less than the theory.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
André,
On 3/20/13 2:25 PM, André Warnier wrote:
> Saurabh Agrawal wrote:
>> All our assets are served from L3 CDN. So the asset requests
>> never come to the application server.
>
> That, I do not understand. I do not understand what you mean by
> "assets" here, and I do not understand "L3 CDN". So I cannot tell
> of this is relevant or not to the problem.
CDN = Content Delivery Network. I'm not sure what "L3" (probably
"Level 3", a data center operations company) is, but a CDN is
basically a whole bunch of copies of your files geographically
distributed such that requesting a file always gets the bits that are
closest to you. Kind of a cool thing. ;)
The bottom line is that Saurabh expects only dynamic requests to come
to Tomcat, so keepalives should be much less useful than if Tomcat
were to be serving everything. Imagine httpd out front serving all
static content and forwarding dynamic stuff to Tomcat via AJP --
that's almost exactly what's going on here, except that the static
stuff is being served very efficiently from a network-topology
perspective.
Since AJP is in use, keepalive is almost entirely a red herring as
typical AJP connections are permanently-connected to the web server.
> So, by default, the keepAliveTimeout [for AJP] is set to
> "infinite".
>
> [snip]
>
> And if the client keeps the connection open, but does not send any
> additional request on that connection, the Thread will wait
> theoretically forever (because that is what the documentation says
> about the default value of these parameters).
No, the defaults are different for non-AJP connections. Tomcat's
default default is 60 seconds but the stock server.xml configures it
to 20 seconds.
> Now your case is a bit different, because - you are not using the
> HTTP BIO connector (you use AJP)
I think you've gotten yourself confused, here, unfortunately. You can
use AJP with BIO, NIO, or APR (maybe you mixed-up AJP and APR between
your eyes and your brain... the two are honestly too close to each
other and it is very easy to do that).
He is in fact using the BIO connector because he has specified
protocol="org.apache.coyote.ajp.AjpProtocol".
> - in front of your Tomcat, is an Apache httpd server. This server
> has its own keep-alive settings which apply to the connection of
> the client with Apache httpd. And these keep-alive settings are a
> bit different from the Tomcat ones (for example, there is a
> keep-alive timeout, but also a MaxKeepAliveRequests)
+1
> - between Apache httpd and Tomcat, there is the mod_jk module in
> Apache, and that module uses its own timeouts (as set in
> workers.properties), and in addition it uses itself a pool of
> connections to Tomcat, and this pool of connections has its own
> rules for keeping alive a connection between Apache and Tomcat.
>
> But the basic principles above apply, and may explain why you are
> seeing what appears to be one Thread dedicated to one client,
> forever.
I think there might be a problem with the instrumentation, or just
coincidences at a fairly implausible level. The trust of the matter is
that Tomcat does not allocate a thread permanently to a remote client
until ... whenever the client "disconnects" (whatever that means, as
HTTP is a connection-less protocol).
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRSlo8AAoJEBzwKT+lPKRYodAP/Rz73chvzhyYPNWpePlvPaBm
x7bFxFCNg7hbHV7X1skgfG4CYbR9AsjFqcgkXovVSBcIZjKoNT/yCqwL20SMdNIo
glkx1Gz6gBfM8Ad68VL4xr7nGtwC1WraN2tCszc8dNrBJOjmQwp5tX09OWtpsxqe
DNQVfhGk8wtHmPWjXgK8Kr0lZDjfz540bp4OSAwohjixsCTr3C1PqVEiiZyYWazC
B/5tEkeJ7YtPQJqmJhgx1uR+CmU3ty5iQpLmr4K9uKTLSTjs3HGFV0nkCKQIXmQs
vzjt0eWr8VzbbZnY4QBfBIntgjQiDIoK9Mi+Q3uilYbjCJuFw4jJGICAjaZjlYxW
eW29Ag/WGthB6shOwStiLS+/KUenzXY9aNzHQ1sCeB9Pdsy2eI+Yg15TomwcfxcV
2LjtHZO13QQeGcHpr5vBId8x/B9x5W1bzs8gzUALPQhdsYqGixohu4Ad4nqmeBI0
VZ02jiEDQqr4zT6oJwDVPpm8FSS2wBZKURIp80iI4B2+bS1VPcKzY/J/aKCQRQCw
KP4VDtNiAOXD4a+8sHuZrOh8Q/splNnbhf2G687PryjfxQuk+HQPY/XfFvY1CXzD
ZZD1yVjkSaRfXDmDCfFxfD+a3lOmGhAa8NNR4dtJJo1w1lhrYXvx1ujdS5V3FgO5
xdKlzkg4u30lRrAQq6IO
=yLs6
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by André Warnier <aw...@ice-sa.com>.
Saurabh Agrawal wrote:
> -----Original Message-----
> From: André Warnier [mailto:aw@ice-sa.com]
> Sent: Wednesday, March 20, 2013 3:27 PM
> To: Tomcat Users List
> Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser
>
> Christopher Schultz wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> Saurabh,
>>
>> On 3/19/13 9:46 PM, Saurabh Agrawal wrote:
>>> <Executor name="hybrisExecutor" namePrefix="hybrisHTTP"
>>> maxThreads="${tomcat.maxthreads}"
>>> minSpareThreads="${tomcat.minsparethreads}"
>>> maxIdleTime="${tomcat.maxidletime}"/>
<Connector
>>> port="${tomcat.ajp.port}" maxHttpHeaderSize="8192" maxThreads="200"
>>> protocol="org.apache.coyote.ajp.AjpProtocol"
>>> executor="hybrisExecutor" enableLookups="false" acceptCount="100"
>>> connectionTimeout="20000" URIEncoding="UTF-8"
>>> disableUploadTimeout="true" />
>>>
>>> <Connector port="${tomcat.http.port}" maxHttpHeaderSize="8192"
>>> maxThreads="${tomcat.maxthreads}"
>>> protocol="org.apache.coyote.http11.Http11Protocol"
>>> executor="hybrisExecutor" enableLookups="false" acceptCount="100"
>>> connectionTimeout="20000" URIEncoding="UTF-8"
>>> disableUploadTimeout="true" />
>> Note that your <Executor> has maxThreads="200" and your <Connector>
>> uses that <Executor>: your ${tomcat.maxthreads} is being ignored.
>>
> That, and the default keepalive setting, are probably the keys here.
> And the observation of Chuck about the HTTP and AJP connectors. Over which Connector do
> the test requests actually come in ?
>
> Saurabh - The actual front end requests come on AJP port. We are using AJP protocol for communication between Apache and Tomcat.
Right. So then I suppose that Christopher's note is not applicable. Probablya he misread,
because the way in which you pasted the configuration in the email makes it difficult to
read, after a couple of cut-and-paste.
As far as I can tell, the AJP connector refers to the Executor, and the Executor specifies
maxThreads="${tomcat.maxthreads}".
The main point of Christopher was that you specify a "maxThreads" parameter in both of
your Connectors, but because they both use the Executor, this parameter is being ignored
in the <Connector>, and it is only the maxThreads in the Executor that counts.
It helps in load balancing across the application servers in cluster. There is a
separate internal application (not exposed on internet) used by CMS team which is using
HTTP connector. I hope that clarifies.
>
Yes.
> And a question : is the "simulation" with the 10000 clients really comparable to what you
> expect in the reality ? For example, if the simulation requests one page per client, and
> then does nothing else with that page; but the real clients would get a page, and then
> immediately request the 50 thumbnail images referenced by that page, conditions would be
> really different, and keepalive would have a very different effect.
>
> Saurabh - The way we have configured our user journeys are as follows:
>
> User 1: Hits homepage, clicks football link on home page, makes a selection, adds to cart and checkout. So this is one user journey which triggers multiple requests.
Hi. Your usage of "user journey" is a bit obscure (to me at least) in the context of
analysing a matter of tomcat request/response performance.
I kind of understand what you mean, but it does not really provide the answer to the
questions :
- is this what you are using in your tests ?
- are you doing this same series of requests for each of your 10000 "test clients" ?
- does this represent (more or less) what you are expecting later "in production" ?
The point here was to avoid a case where you would be "optimising" the parameters in
function of a benchmark test, and then find out later that your production case is totally
different, and your optimal benchmark settings are totally inappropriate for the
production case.
> All our assets are served from L3 CDN. So the asset requests never come to the
application server.
That, I do not understand. I do not understand what you mean by "assets" here, and I do
not understand "L3 CDN". So I cannot tell of this is relevant or not to the problem.
Have pity for the people trying to help you here, who only know Tomcat and HTTP/AJP.
Try to use vocabulary that we understand, and you may get better help.
> We have not set keep alive explicitly anywhere in tomcat.
>
What Chuck was telling you in an earlier message, is that even if you do not set it
explicitly, it is set to some default non-zero value by Tomcat.
Look at this page in the on-line documentation :
http://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html#Standard_Implementations
So, by default (unless you explicitly define it),
keepAliveTimeout :
The number of milliseconds this Connector will wait for another AJP request before closing
the connection. The default value is to use the value that has been set for the
connectionTimeout attribute.
and
connectionTimeout
The number of milliseconds this Connector will wait, after accepting a connection, for the
request URI line to be presented. The default value for AJP protocol connectors is -1
(i.e. infinite).
So, by default, the keepAliveTimeout is set to "infinite".
So now, read very carefully, because this describes a very specific set of circumstances,
which are not really applicable to you, but it is just for basic understanding.
/If/ your clients were connecting directly to a Tomcat basic HTTP Connector (named BIO),
and you did not set any explicit keepAliveTimeout or connectionTimeout
and /If/ you were not using an Executor
/Then/ what would happen is this :
1) the client opens a connection to the server's HTTP connector, requesting a "keepalive"
connection (that is what browsers do by default in HTTP 1.1)
2) the server, by default, considers this connection as "persistent"
3) the server (in this case, the Connector) allocates a Thread to this connection, and
this Thread now listens to what the client is going to say
4) the client sends one request on this connection
5) the allocated Thread receives the request, processes the request, and sends the
response. Then it waits, to see if another request comes on the same connection.
And if the client keeps the connection open, but does not send any additional request on
that connection, the Thread will wait theoretically forever (because that is what the
documentation says about the default value of these parameters).
So /in this particular case/, you would have one Tomcat Thread that is now dedicated to
this connection, doing nothing, but being unavailable to process any other request from
any other client.
In other words, /in the particular set of conditions above/, if you wanted to avoid this
situation, then you should set the keepAliveTimeout *explicitly*, to some reasonable value.
What is a reasonable value ?
That depends very much on your application.
Let's imagine a counter-example, and suppose that the connection was not a keepalive
connection.
In that case, for each new request of the client, the client would have to create a *new*
TCP connection to the server, to send one request and receive one response. Then the
server would close the connection, and for the next request the client would again need to
open a new connection.
That would be fine if a client was only sending a single request from time to time,
because then you would not be tying up resources on the server, that can be better used
for something else.
But it would be inefficient if the client always retrieves a first html page, and then in
that page there are 10 embedded images that the client also needs to retrieve. The client
would then have to open and close 10 consecutive times a new connection, each time to
retrieve one single image.
So if you know that your application is so that a client will always need to retrieve
several "objects" one after the other in a short interval, it is better to use one single
connection to do this, and thus to ask for a "persistent" or "keep-alive" connection.
You should set the keepAliveTimeout to a time sufficient so that the client has the time
to request all the additional objects that it wants, before the server closes the connection.
But it should not be too long, because otherwise the Thread is going to wait a long time
after the client has sent his last request, before the Thread decides that it has waited
long enough, closes the connection, and returns to the pool of available Threads, to do
something else.
So in the practice today, with normal Internet connection speeds, and normal clients and
servers, a keepAliveTimeout of maximum 5 seconds is probably more than enough.
And for that, you should set it /explicitly/ to 5000 (milliseconds).
Now your case is a bit different, because
- you are not using the HTTP BIO connector (you use AJP)
- in front of your Tomcat, is an Apache httpd server. This server has its own keep-alive
settings which apply to the connection of the client with Apache httpd. And these
keep-alive settings are a bit different from the Tomcat ones (for example, there is a
keep-alive timeout, but also a MaxKeepAliveRequests)
- between Apache httpd and Tomcat, there is the mod_jk module in Apache, and that module
uses its own timeouts (as set in workers.properties), and in addition it uses itself a
pool of connections to Tomcat, and this pool of connections has its own rules for keeping
alive a connection between Apache and Tomcat.
But the basic principles above apply, and may explain why you are seeing what appears to
be one Thread dedicated to one client, forever.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by Saurabh Agrawal <sa...@sapient.com>.
-----Original Message-----
From: André Warnier [mailto:aw@ice-sa.com]
Sent: Wednesday, March 20, 2013 3:27 PM
To: Tomcat Users List
Subject: Re: Tomcat Behavior on Multiple HTTP requests from same browser
Christopher Schultz wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Saurabh,
>
> On 3/19/13 9:46 PM, Saurabh Agrawal wrote:
>> <Executor name="hybrisExecutor" namePrefix="hybrisHTTP"
>> maxThreads="${tomcat.maxthreads}"
>> minSpareThreads="${tomcat.minsparethreads}"
>> maxIdleTime="${tomcat.maxidletime}"/> <Connector
>> port="${tomcat.ajp.port}" maxHttpHeaderSize="8192" maxThreads="200"
>> protocol="org.apache.coyote.ajp.AjpProtocol"
>> executor="hybrisExecutor" enableLookups="false" acceptCount="100"
>> connectionTimeout="20000" URIEncoding="UTF-8"
>> disableUploadTimeout="true" />
>>
>> <Connector port="${tomcat.http.port}" maxHttpHeaderSize="8192"
>> maxThreads="${tomcat.maxthreads}"
>> protocol="org.apache.coyote.http11.Http11Protocol"
>> executor="hybrisExecutor" enableLookups="false" acceptCount="100"
>> connectionTimeout="20000" URIEncoding="UTF-8"
>> disableUploadTimeout="true" />
>
> Note that your <Executor> has maxThreads="200" and your <Connector>
> uses that <Executor>: your ${tomcat.maxthreads} is being ignored.
>
That, and the default keepalive setting, are probably the keys here.
And the observation of Chuck about the HTTP and AJP connectors. Over which Connector do
the test requests actually come in ?
Saurabh - The actual front end requests come on AJP port. We are using AJP protocol for communication between Apache and Tomcat. It helps in load balancing across the application servers in cluster. There is a separate internal application (not exposed on internet) used by CMS team which is using HTTP connector. I hope that clarifies.
And a question : is the "simulation" with the 10000 clients really comparable to what you
expect in the reality ? For example, if the simulation requests one page per client, and
then does nothing else with that page; but the real clients would get a page, and then
immediately request the 50 thumbnail images referenced by that page, conditions would be
really different, and keepalive would have a very different effect.
Saurabh - The way we have configured our user journeys are as follows:
User 1: Hits homepage, clicks football link on home page, makes a selection, adds to cart and checkout. So this is one user journey which triggers multiple requests. All our assets are served from L3 CDN. So the asset requests never come to the application server. We have not set keep alive explicitly anywhere in tomcat.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by André Warnier <aw...@ice-sa.com>.
Christopher Schultz wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Saurabh,
>
> On 3/19/13 9:46 PM, Saurabh Agrawal wrote:
>> <Executor name="hybrisExecutor" namePrefix="hybrisHTTP"
>> maxThreads="${tomcat.maxthreads}"
>> minSpareThreads="${tomcat.minsparethreads}"
>> maxIdleTime="${tomcat.maxidletime}"/> <Connector
>> port="${tomcat.ajp.port}" maxHttpHeaderSize="8192" maxThreads="200"
>> protocol="org.apache.coyote.ajp.AjpProtocol"
>> executor="hybrisExecutor" enableLookups="false" acceptCount="100"
>> connectionTimeout="20000" URIEncoding="UTF-8"
>> disableUploadTimeout="true" />
>>
>> <Connector port="${tomcat.http.port}" maxHttpHeaderSize="8192"
>> maxThreads="${tomcat.maxthreads}"
>> protocol="org.apache.coyote.http11.Http11Protocol"
>> executor="hybrisExecutor" enableLookups="false" acceptCount="100"
>> connectionTimeout="20000" URIEncoding="UTF-8"
>> disableUploadTimeout="true" />
>
> Note that your <Executor> has maxThreads="200" and your <Connector>
> uses that <Executor>: your ${tomcat.maxthreads} is being ignored.
>
That, and the default keepalive setting, are probably the keys here.
And the observation of Chuck about the HTTP and AJP connectors. Over which Connector do
the test requests actually come in ?
And a question : is the "simulation" with the 10000 clients really comparable to what you
expect in the reality ? For example, if the simulation requests one page per client, and
then does nothing else with that page; but the real clients would get a page, and then
immediately request the 50 thumbnail images referenced by that page, conditions would be
really different, and keepalive would have a very different effect.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Saurabh,
On 3/19/13 9:46 PM, Saurabh Agrawal wrote:
> <Executor name="hybrisExecutor" namePrefix="hybrisHTTP"
> maxThreads="${tomcat.maxthreads}"
> minSpareThreads="${tomcat.minsparethreads}"
> maxIdleTime="${tomcat.maxidletime}"/> <Connector
> port="${tomcat.ajp.port}" maxHttpHeaderSize="8192" maxThreads="200"
> protocol="org.apache.coyote.ajp.AjpProtocol"
> executor="hybrisExecutor" enableLookups="false" acceptCount="100"
> connectionTimeout="20000" URIEncoding="UTF-8"
> disableUploadTimeout="true" />
>
> <Connector port="${tomcat.http.port}" maxHttpHeaderSize="8192"
> maxThreads="${tomcat.maxthreads}"
> protocol="org.apache.coyote.http11.Http11Protocol"
> executor="hybrisExecutor" enableLookups="false" acceptCount="100"
> connectionTimeout="20000" URIEncoding="UTF-8"
> disableUploadTimeout="true" />
Note that your <Executor> has maxThreads="200" and your <Connector>
uses that <Executor>: your ${tomcat.maxthreads} is being ignored.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEAREIAAYFAlFJwdIACgkQ9CaO5/Lv0PDOMwCfUxQ+hIaIIWlVfOok6b2+tawK
eSoAn2Ivy26EMiLbhnaPs6VH35ZECbgB
=6T32
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Saurabh Agrawal [mailto:sagrawal@sapient.com]
> Subject: RE: Tomcat Behavior on Multiple HTTP requests from same browser
> We have not set the "keep alive" explicitly in tomcat's server.xml.
It's on by default.
> <Connector port="${tomcat.ajp.port}"
> <Connector port="${tomcat.http.port}"
You have both HTTP and AJP <Connector>s defined; are the requests coming in over both or just one of them? The discussion so far has been primarily related to HTTP, not AJP.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by Saurabh Agrawal <sa...@sapient.com>.
Hi chuck,
We have not set the "keep alive" explicitly in tomcat's server.xml. Howeverm when I print the thread id for all requests from same browser, it prints the same thread it. It gives me a feeling that all HTTP requests are holding worker thread of tomcat if they originate from same browser.
My tomcat configuration is as follows:
<Server port="${tomcat.internalserver.port}" shutdown="SHUTDOWN">
<Listener className="org.apache.catalina.core.JasperListener" />
<Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" />
<Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
<Service name="Catalina" >
<Executor name="hybrisExecutor"
namePrefix="hybrisHTTP"
maxThreads="${tomcat.maxthreads}"
minSpareThreads="${tomcat.minsparethreads}"
maxIdleTime="${tomcat.maxidletime}"/>
<Connector port="${tomcat.ajp.port}"
maxHttpHeaderSize="8192"
maxThreads="200"
protocol="org.apache.coyote.ajp.AjpProtocol"
executor="hybrisExecutor"
enableLookups="false"
acceptCount="100"
connectionTimeout="20000"
URIEncoding="UTF-8"
disableUploadTimeout="true" />
<Connector port="${tomcat.http.port}"
maxHttpHeaderSize="8192"
maxThreads="${tomcat.maxthreads}"
protocol="org.apache.coyote.http11.Http11Protocol"
executor="hybrisExecutor"
enableLookups="false"
acceptCount="100"
connectionTimeout="20000"
URIEncoding="UTF-8"
disableUploadTimeout="true" />
<Engine name="Catalina" defaultHost="localhost" jvmRoute="${tomcat.ajp.worker}">
<Valve className="org.apache.catalina.valves.FastCommonAccessLogValve"
directory="${HYBRIS_LOG_DIR}/tomcat"
prefix="access."
suffix=".log"
pattern="%{X-Forwarded-For}i %l %u %t %T %r %s %b %{User-Agent}i %{Referer}i %{JSESSIONID}c"
/>
<Host name="localhost"
appBase="webapps"
unpackWARs="true"
autoDeploy="false"
xmlValidation="false"
xmlNamespaceAware="false">
${tomcat.webapps.60}
</Host>
</Engine>
</Service>
</Server>
Regards,
Saurabh Agrawal
Manager Technology | Sapient
-----Original Message-----
From: Caldarale, Charles R [mailto:Chuck.Caldarale@unisys.com]
Sent: Wednesday, March 20, 2013 1:37 AM
To: Tomcat Users List
Subject: RE: Tomcat Behavior on Multiple HTTP requests from same browser
> From: Saurabh Agrawal [mailto:sagrawal@sapient.com]
> Subject: Tomcat Behavior on Multiple HTTP requests from same browser
> Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL,
> tomcat will assign one of the worker threads (say Thread 1) from the pool
> to the HTTP request which will be processed and then response will be sent
> to the client. Now if I hit a link on homepage which is for login, a separate
> HTTP request will be initiated from the same browser.
> What I want to understand is if the Tomcat will keep Thread 1 as persistent
> thread to server the second request (for login) from the same browser or will
> it assign a separate thread from the pool ?
Normally, the first available thread from the pool is used for each request. However, if you are using the BIO <Connector> with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active.
See the HTTP <Connector> doc, especially the Connector Comparison at the end.
http://tomcat.apache.org/tomcat-7.0-doc/config/http.html
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat Behavior on Multiple HTTP requests from same browser
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Saurabh Agrawal [mailto:sagrawal@sapient.com]
> Subject: Tomcat Behavior on Multiple HTTP requests from same browser
> Let's say I hit http://localhost:9001/homepage.html. Upon hitting the URL,
> tomcat will assign one of the worker threads (say Thread 1) from the pool
> to the HTTP request which will be processed and then response will be sent
> to the client. Now if I hit a link on homepage which is for login, a separate
> HTTP request will be initiated from the same browser.
> What I want to understand is if the Tomcat will keep Thread 1 as persistent
> thread to server the second request (for login) from the same browser or will
> it assign a separate thread from the pool ?
Normally, the first available thread from the pool is used for each request. However, if you are using the BIO <Connector> with keep-alives enable *and* the browser is using keep-alives, the same thread as long as the HTTP connection is active.
See the HTTP <Connector> doc, especially the Connector Comparison at the end.
http://tomcat.apache.org/tomcat-7.0-doc/config/http.html
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org