Re: Configuration question for 2500 simultaneous users.

[Tomcat Random <to...@gmail.com>]

Thanks Mark, I've been getting up to speed on JMeter. I've used selenium
before. "The particulars depend on your particular application" - agreed.
That part I can work out myself by looking at bottlenecks, generally timing
areas of the application with more and more granularity, in places where
the app behaves slowly.

Best,
Alec


On Tue, Jul 30, 2013 at 6:17 PM, Mark Eggers <it...@yahoo.com> wrote:

> On 7/30/2013 1:17 PM, Tomcat Random wrote:
>
>> Thanks Mark, I will give it a close read.
>>
>> As far as profiling, are you using any tools that are worth mentioning?
>>
>>
> Nothing outstanding, since currently all of our applications are pretty
> lightweight. That may change if we redo the architecture.
>
> JMeter / Selenium in combination can generate a lot of traffic. Generate a
> selenium test script, export to JUnit, couple with HTMLUnit, and hammer
> away.
>
> There are several ways to watch what goes on with your application:
>
> JConsole
> VisualVM
>
> The Tomcat Wiki page has more:
>
> http://wiki.apache.org/tomcat/**FAQ/Monitoring<http://wiki.apache.org/tomcat/FAQ/Monitoring>
>
> For lighter weight profiling (usually to figure out where the application
> bottlenecks are), I run the project under NetBeans and instrument the
> project.
>
> Access logs are usually a good first source for generating JMeter tests.
>
> In general, people can only give you guidelines concerning sizing,
> profiling, and benchmarking. The particulars depend on your particular
> application.
>
>
> . . . . just my two cents.
> /mde/
>
> PS - Please don't top post.
>
>  Best,
>> A
>>
>>
>>
>> On Tue, Jul 30, 2013 at 4:02 PM, Mark Eggers <it...@yahoo.com>
>> wrote:
>>
>>  On 7/30/2013 12:42 PM, Tomcat Random wrote:
>>>
>>>  The project I'm working on has 5000 simultaneous users average. I have
>>>> two
>>>> physical servers both running an instance of Tomcat 7.0. They're behind
>>>> a
>>>> physical load balancer with sticky, least connections balancing. Nothing
>>>> in
>>>> front of the Tomcats. Port 80 to is routed to them by iptables.
>>>>
>>>> Anyone out there willing to offer some tips (or point me to them) on
>>>> configuration for this amount of traffic?
>>>>
>>>> Environment is:
>>>> DELL PowerEdge R720 - 32 GB DELL RAM, GB Memory: 32
>>>> Single Socket Six Core Intel Xeon E5-2640 2.5GHz, #Processors: 1, #Cores
>>>> per Proc: 6
>>>> RHEL 6
>>>>
>>>> TIA,
>>>> Alec
>>>>
>>>>
>>>>  A great overview, and a solid outline of the process you should follow:
>>>
>>> http://people.apache.org/~****markt/presentations/2009-04-**<http://people.apache.org/~**markt/presentations/2009-04-**>
>>> 01-TomcatTuning.pdf<http://**people.apache.org/~markt/**
>>> presentations/2009-04-01-**TomcatTuning.pdf<http://people.apache.org/~markt/presentations/2009-04-01-TomcatTuning.pdf>
>>> >
>>>
>>>
>>> That, plus profiling your application with real-world traffic to
>>> understand bottlenecks and use cases . . .
>>>
>>> . . . just my two cents.
>>> /mde/
>>>
>>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.**apache.org<us...@tomcat.apache.org>
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

Re: Configuration question for 2500 simultaneous users.

[Christopher Schultz <ch...@christopherschultz.net>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Alec,

On 8/1/13 6:08 PM, Tomcat Random wrote:
> Thanks Mark, I've been getting up to speed on JMeter. I've used
> selenium before. "The particulars depend on your particular
> application" - agreed. That part I can work out myself by looking
> at bottlenecks, generally timing areas of the application with more
> and more granularity, in places where the app behaves slowly.

Just remember that you can't really benchmark your application with a
single client. There are behaviors that only emerge under some kind of
load. For instance, uncontended object monitors (i.e. synchronized
methods, blocks, etc.) in the JVM can be obtained very quickly. So, if
you have a resource that requires serialized access (e.g. db
connection pool), it may perform very well under an isolated test: you
find that your most time-consuming request/response pair takes, say,
850ms.

That's great: you do the math and say that you can handle 1.18 req/sec
on average and with users expected to make 1 request every 10 seconds,
that means that you can handle ~12 simultaneous users with no apparent
slow-down.

Then you get contended locks, which are slower. Instead of roughly
zero time to obtain an object lock, let's say it takes 50ms (that's
grossly overstating the amount of time required to obtain an object
lock, but somewhat instructive). So now you're up to 900ms for your
transaction and you can handle 1.11 req/sec with no slowdown and you
can only handle 11 simultaneous users.

Multiply those effects by a lot (lots of transactions, lots of users,
lots of multi-threaded conflicts) and you can find that you really can
only handle 91% of the traffic you thought you could.

If you use JMeter, your software will get better: you will be able to
identify those transactions that don't perform well and focus your
optimization efforts in those places, instead of just reading code,
changing StringBuffer to StringBuilder and new Integer() to
Integer.valueOf() and convincing yourself that you now have an
optimized piece of software.

With a load-testing suite, you can prove to yourself (and, perhaps
more importantly, your boss) that your software can handle the load
you expect. It also helps with resource planning: if you know that a
single instance of Tomcat on X hardware can handle Y load and you
expect Z actual load, then you know you need Z / Y pieces of X
hardware to run properly. Then multiply by some fudge factor (say 20%)
and take the ceiling of that (so you always have an extra server...
just in case). So maybe it's more like [ Z * 1.2 / Y ]. Oh, and
remember that Z needs to represent *peak load* and not average load.
If you have an online flower shop, you need to plan for the week of
Valentine's Day during business hours, not the following Saturday in
the middle of the night.

Gather a ton of data and look at it. Graph it. You will learn a lot.

Oh, and if you're using httpd (you didn't say), you might want to look
into the "event" MPM: it won't waste connections waiting around for
KeepAlive requests that never come from clients. The default MPM for
httpd 2.4 on *NIX is "event" when supported by the OS and prefork for
httpd 2.2. I believe "event" is stable on 2.2 -- just not the default.
Benchmark with and without it and see if it makes a difference. With
JMeter, your tests might be distorted because of how the tests run
(e.g. constantly). You might have to fiddle with the JMeter
configuration to get it to act more like a "real" browser by leaving
connections open for a few seconds after making all the requests for a
page. Then you may notice a difference.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJR/HgrAAoJEBzwKT+lPKRYkR8P/R5W8g/vnfEH/QxFfsQMCJnT
TDv4oZecfNZ2PDZjGj+KpAJv6OoDe9wQLlAR4VWZMmnI3qJPqVwWg04ZSnEszf22
XkiuvnbPQXwt95bTXk/zPp5lcb4kuLAwvokAyMQQo7P941SYOmvA+xEADmlE8Vfc
AgxnQY7GnifBU1wLZtzXrDnUWOzYwEL2vsec1bmlasbBg5NpxsQ9IyAq3DcPmDtO
hQUbET12JTjka51Ub6TG5gP2O+ZzYz50XyVtUWcn86u4acZYmvfCtFq/XilZXv4G
962bNxXinf9GmoeCokCOejJ/Ob8XC7UpXtTXTJlzx64Gn8EDNKYyZxg6wt6MLktq
xppytGQ+FfsxEFRtn9a0UM5Am0+7L4OpQWXesm5IcdypGNWn9Ng4bg2YI3c+P//O
Q0eyS7WN19oFMvq60tbKHfhCv7QFYQatKnxiAChDsFoA53u6xZIlzmz5h/0G2kCH
h7u57zUh2sSiBkSEs7QOxYNHS2tfTAAO2d5CMTuiAnIYk0joo0gOZ7u50DpAXaoR
lnxcimpO+wy5Qc3AEfZtYlmco+EhjJ4hHZyCvcE3MRkGdrWODHZkV1+B9TB0TH3M
sp1MEr3dlKeMDTj9jwB7mny8BY8lmIsPjWLl1s0wW+IST4Z4ax2t5eDvFDwtgtHy
t9b2VNjEizaTBrp88r7T
=iVJy
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org