You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Roy Wilson <de...@bellatlantic.net> on 2000/11/01 02:24:21 UTC
Tomcat 4.0 Milestone 4 and performance issues
Hi,
Has anyone used ab to stress test any versions of Tomcat when used as an
in-process servlet container (Costin, Craig, Pier)? (I am a newbie at
both Apache and Tomcat: I assume that using Tomcat and Apache together
implies that Tomcat is used in an in-process mode.) If so, do you
anticipate any procedural changes needed to do the same with Tomcat-4.0?
I ask because I'd like to do some measurements of thruput and resource
demand once the connector is available and am wondering whether I'll need
to do more than install Tomcat, get it running, and fire up ab. I know
the definitive answer that won't be known until the connector is up and
working.
Per an early suggestion from Costin or Craig, I've been doing some
simplistic static HTML page ApacheBench based measurements (of Apache
only) in localhost mode (yeah, I know ...) representing an Intranet usage
scenario. In short, I use ab to drive Apache while running sar and sa in
the background. From sar and sa, I derive per request CPU and disk
service time estimates for ab and httpd. These values are used as inputs
to a (too simple) queueing model. The model generates average queue
lengths, resource utilizations and average system response time for large
populations of users. For the intranet case, it also allows me to look at
the impact of a non-zero think time: ab assumes (in effect) a population
of C continuously active users. The model also makes possible a SWAG
(scientific wild-ass-guess) about performance and capacity across
processors (using CPUmark99 or some such numbers) and processor
configurations. This is rude and crude, I admit, but sometimes good for
comparing systems. And the model runs in a matter of seconds, so
sensitivity analysis is almost free.
If there's any interest, I can post a link to a PowerPoint presentation
that summarizes what I've done so far (and let the barbecue begin :-)).
--
Roy Wilson
E-mail: designrw@bellatlantic.net
>>>>>>>>>>>>>>>>>> Original Message <<<<<<<<<<<<<<<<<<
On 10/31/00, 7:09:41 PM, "Craig R. McClanahan"
<Cr...@eng.sun.com> wrote regarding Tomcat 4.0 Milestone 4:
> Hey folks,
> I'm planning on cutting a fourth milestone of Tomcat 4.0 tomorrow
> (Wednesday) evening. This milestone will reflect all of the changes
> that occurred in the servlet 2.3 and JSP 1.2 specifications between
> public draft and proposed final draft (and there were a *bunch* of
> them), completion of the remaining new 2.3/1.2 functionality, and
> several bug fixes.
> This will be the last big push of spec-related functionality additions
> for Tomcat 4.0. Now, we can turn our attention more towards bug fixes
> and performance tuning. You can help in that process by downloading and
> playing with the Tomcat 4.0 milestone. I'd like to see us shake it out
> enough to be production quality by Christmas time.
> Besides bug fixing and tuning, I know of several pieces of functionality
> yet to be added that are being worked on, including:
> * Web connectors (using a new connector protocol
> called mod_warp that is aware of webapp configuration
> settings, so you will not have to configure things twice).
> * JNDI context support (like that used in J2EE servers)
> for the <env-entry> and <resource-ref> configuration
> parameters in the deployment descriptor.
> If you are interested in contributing to Tomcat 4.0, there is a "wish
> list" document in file "catalina/STATUS.html" in the jakarta-tomcat-4.0
> source tree. Feel free to propose new ideas, or to volunteer to work on
> one of these.
> Craig McClanahan
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
TC4.0 Linux/Unix measurement scripts
Posted by Roy Wilson <de...@bellatlantic.net>.
Hi,
Per Craig's request, here are the script files I used to develop the
Apache-based presentation mentioned in an earlier post.
Assuming I understand the ab source code, the scripts represent a setting
involving single-request clients (who make a request and wait for a reply
before making another) generating a homogeneous workload (ie, HelloWorld)
on the server: I'm working on the single-client heterogeneous case now.
If you have heartburn about these scripts or want to propose something
better, please do.
Attached are two simplistic ascii bash shell scripts that work with
Apache on Redhat6.1. Since I haven't finished installing TC4.0, so I
don't know first-hand what's required to use ab with servlets: I'm told
that having an http-address of the following form is sufficient: "
https://localhost:8080/servlet/HTTPGetServlet"
Also attached is a sample summary file. I'm writing a shell/awk script to
process the summary files that are posted.
1) tcdriver
which which invokes tcscript with two parameters: the number of
concurrent connections (1, 20, 40, 60, 80, and 100) and one (!) servlet
of your choice (make it HelloWorld the first time you use the script -
and then send the summary before using it again with your second most
favorite servlet).
2) tcscript
which invokes ab using the two parameters from tcdriver, dumps filtered
data to a summary file, and cleans up the intermediate files. You will
need to modify the line that invokes ab to give your localhost pathname.
By running ab in localhost mode, network variability is eliminated (even
though TCPIP loopback processing isn't).
The data collected includes:
(from ab) info on per request connection times, total test, time and
throughput;
(from sar) cpu utilization (%) and disk activity;
(from sa) cpu minutes per process.
I ran each iteration of the script after a cold boot, but I think running
the whole thing immediately after a cold boot is sufficient.
Assuming Linux/Unix people decide to use these scripts, please attach the
summary file in ascii format to a post that describes your system
configuration (CPU, disk, memory, OS, etc.) I will process that files
post the results every TBD days.
Sorry about the length. If I knew how to make it shorter, I would. :-)
Roy
--
Roy Wilson
E-mail: designrw@bellatlantic.net
>>>>>>>>>>>>>>>>> Original Message <<<<<<<<<<<<<<<<<<
On 11/2/00, 1:32:20 PM, "Craig R. McClanahan"
<Cr...@eng.sun.com> wrote regarding Re: Tomcat 4.0 Milestone 4
performance measurement:
> Roy, would it be possible for you to write us a simple "cookbook" to
follow
> in executing tests, so that we can each follow the same script and then
> report the results consistently?
> Craig McClanahan
> Roy Wilson wrote:
> > Hi,
> >
> > (1) System throughput and response time data collection
> >
> > I plan to do measurements myself using Henri Gomez' rpmized version of
> > TC-4.0-m4, but since machines and environments differ, it might be
> > interesting to know what kind of total time, throughput, avg and max
> > connection and processing time per request people see using the
> > HelloWorld servlet driving their system with ab in localhost mode
> > (immediately after a cold boot). These are the kinds of data presented by
> > Costin at ApacheCon Europe. I'd like to compile such numbers if people
> > are willing to post them along with info on CPU (e.g. Celeron 400Mhz),
> > memory (eg. 256M SDRAM100), disk (eg. WD Caviar 10.2MB, 5400 rpms), OS
> > (Redhat6.1), JVM (Classic), etc.
> >
> > (2) System resource usage data collection
> >
> > CPU and disk demand per request on each system might be interesting. I
> > have done some very simple resource measurement and performance modeling
> > of Apache that is described in a Office2000 PowerPoint (but created in
> > StarImpress) presentation at
> >
> > http://members.bellatlantic.net/~designrw/BENCH1.zip
> >
> > My current plan is to apply the same approach to TC-4.0 and then to
> > extend it by considering a mix of servlets, not just the HelloWorld
> > servlet (again in keeping with Costin's pitch at ApacheCon).
> >
> > Roy Wilson
> > --
> > Roy Wilson
> > E-mail: designrw@bellatlantic.net
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Performance measurement scripting
Posted by Roy Wilson <de...@bellatlantic.net>.
Craig,
I have a bash shell script that (currently) drives Apache, sar, and sa on
a RedHat6.1 system. I will doctor it a bit and post it tonight to see if
there are suggestions about what to include, modify, leave out, so that
it will span most Unix/Linux systems.
Roy
--
Roy Wilson
E-mail: designrw@bellatlantic.net
>>>>>>>>>>>>>>>>>> Original Message <<<<<<<<<<<<<<<<<<
On 11/2/00, 1:32:20 PM, "Craig R. McClanahan"
<Cr...@eng.sun.com> wrote regarding Re: Tomcat 4.0 Milestone 4
performance measurement:
> I agree that it's definitely time to start doing this. However, to have
> useful results, it is important that we execute the same test procedures
--
> otherwise, you cannot even reproduce the same results reliably on the
same
> platform.
> Roy, would it be possible for you to write us a simple "cookbook" to
follow
> in executing tests, so that we can each follow the same script and then
> report the results consistently?
> Craig McClanahan
> Roy Wilson wrote:
> > Hi,
> >
> > (1) System throughput and response time data collection
> >
> > I plan to do measurements myself using Henri Gomez' rpmized version of
> > TC-4.0-m4, but since machines and environments differ, it might be
> > interesting to know what kind of total time, throughput, avg and max
> > connection and processing time per request people see using the
> > HelloWorld servlet driving their system with ab in localhost mode
> > (immediately after a cold boot). These are the kinds of data presented by
> > Costin at ApacheCon Europe. I'd like to compile such numbers if people
> > are willing to post them along with info on CPU (e.g. Celeron 400Mhz),
> > memory (eg. 256M SDRAM100), disk (eg. WD Caviar 10.2MB, 5400 rpms), OS
> > (Redhat6.1), JVM (Classic), etc.
> >
> > (2) System resource usage data collection
> >
> > CPU and disk demand per request on each system might be interesting. I
> > have done some very simple resource measurement and performance modeling
> > of Apache that is described in a Office2000 PowerPoint (but created in
> > StarImpress) presentation at
> >
> > http://members.bellatlantic.net/~designrw/BENCH1.zip
> >
> > My current plan is to apply the same approach to TC-4.0 and then to
> > extend it by considering a mix of servlets, not just the HelloWorld
> > servlet (again in keeping with Costin's pitch at ApacheCon).
> >
> > Roy Wilson
> > --
> > Roy Wilson
> > E-mail: designrw@bellatlantic.net
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: Tomcat 4.0 Milestone 4 performance measurement
Posted by "Craig R. McClanahan" <Cr...@eng.sun.com>.
I agree that it's definitely time to start doing this. However, to have
useful results, it is important that we execute the same test procedures --
otherwise, you cannot even reproduce the same results reliably on the same
platform.
Roy, would it be possible for you to write us a simple "cookbook" to follow
in executing tests, so that we can each follow the same script and then
report the results consistently?
Craig McClanahan
Roy Wilson wrote:
> Hi,
>
> (1) System throughput and response time data collection
>
> I plan to do measurements myself using Henri Gomez' rpmized version of
> TC-4.0-m4, but since machines and environments differ, it might be
> interesting to know what kind of total time, throughput, avg and max
> connection and processing time per request people see using the
> HelloWorld servlet driving their system with ab in localhost mode
> (immediately after a cold boot). These are the kinds of data presented by
> Costin at ApacheCon Europe. I'd like to compile such numbers if people
> are willing to post them along with info on CPU (e.g. Celeron 400Mhz),
> memory (eg. 256M SDRAM100), disk (eg. WD Caviar 10.2MB, 5400 rpms), OS
> (Redhat6.1), JVM (Classic), etc.
>
> (2) System resource usage data collection
>
> CPU and disk demand per request on each system might be interesting. I
> have done some very simple resource measurement and performance modeling
> of Apache that is described in a Office2000 PowerPoint (but created in
> StarImpress) presentation at
>
> http://members.bellatlantic.net/~designrw/BENCH1.zip
>
> My current plan is to apply the same approach to TC-4.0 and then to
> extend it by considering a mix of servlets, not just the HelloWorld
> servlet (again in keeping with Costin's pitch at ApacheCon).
>
> Roy Wilson
> --
> Roy Wilson
> E-mail: designrw@bellatlantic.net
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: Tomcat 4.0 Milestone 4 performance measurement
Posted by Roy Wilson <de...@bellatlantic.net>.
Hi,
(1) System throughput and response time data collection
I plan to do measurements myself using Henri Gomez' rpmized version of
TC-4.0-m4, but since machines and environments differ, it might be
interesting to know what kind of total time, throughput, avg and max
connection and processing time per request people see using the
HelloWorld servlet driving their system with ab in localhost mode
(immediately after a cold boot). These are the kinds of data presented by
Costin at ApacheCon Europe. I'd like to compile such numbers if people
are willing to post them along with info on CPU (e.g. Celeron 400Mhz),
memory (eg. 256M SDRAM100), disk (eg. WD Caviar 10.2MB, 5400 rpms), OS
(Redhat6.1), JVM (Classic), etc.
(2) System resource usage data collection
CPU and disk demand per request on each system might be interesting. I
have done some very simple resource measurement and performance modeling
of Apache that is described in a Office2000 PowerPoint (but created in
StarImpress) presentation at
http://members.bellatlantic.net/~designrw/BENCH1.zip
My current plan is to apply the same approach to TC-4.0 and then to
extend it by considering a mix of servlets, not just the HelloWorld
servlet (again in keeping with Costin's pitch at ApacheCon).
Roy Wilson
--
Roy Wilson
E-mail: designrw@bellatlantic.net