You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by simon colston <si...@lexues.co.jp> on 2001/09/18 02:45:14 UTC
threads on Linux
Set up
------
Linux Redhat 6.2 (with glibc 2.1.3), IBM JVM V1.3.0, Tomcat 3.2.3
Problem
-------
The number of threads shown by ps gradually increases over time.
Research (Mailing lists, documentation, etc.)
---------------------------------------------
1. The java entries on ps are threads not processes.
However, I'd rather they didn't continually increase because I'm sure that resources will eventually run out, even for threads.
2. I have set max_threads=500, max_spare_threads=30, min_spare_threads=10.
After restarting Tomcat I get about 37 threads (= max_spare_threads + tomcat threads not connected to pooling, I guess). However, over time this residual number gradually increases. (It does occasionally decrease but never by as much as it increases)
3. I have tried several alterations to my servlet to try and fix this (I no longer explicitly create threads of my own, been completely anal about closing any stream that's open, etc) but no change.
4. Below is the most informative information that I have found on this subject but as you can see there was no final answer to the last question.
Desparate plea
--------------
Can anyone shed any light on this?
--- taken from tomcat-user mailing list ---
From: myatt83@potsdam.edu
Subject: Re: ** JVM and Processes
Date: Fri, 01 Jun 2001 16:36:19 -0400
Using the Pool connector and the min_spare_threads,max_spare_threads,
and max_threads, I set max_threads to 30 just to test it.
Once I restart the server.xml file, if I wait a little while (after some people have
visited the site and used some of the servlets) more than 30 threads appear
(listed as previously mentioned: /usr/java/jdk1.3/bin/i386/native_threads/java ).
Sometimes as many as 80 or 90 appear after a while. It appears the
max_threads variable has no affect and is useless. Can you shed
light on the issue?
- Thanks,
Adam
At 11:57 AM 6/1/2001 -0700, you wrote:
>Hi Adam,
>
>No, the garbage collector runs as a low priority background process and, on
>a lightly loaded server, may never get called because the server's not using
>enough resources to warrant it. I really wouldn't worry about it too much
>and I would definitely avoid killing threads individually, especially since
>you're now utilizing a Pool connector. (you don't want to kill threads that
>are marked as available in the pool...) The min_spare_threads and
>max_spare_threads settings are supposed to take care of cleaning up any
>"extra" unused threads that are laying around.
>
>I think the best benefit you could do yourself would be to upgrade your 3.1
>version of Tomcat to the newly released 3.2.2 final to take advantage of
>upgrades and bug fixes.
>
>Thanks,
>--jeff
>
>----- Original Message -----
>From: <my...@potsdam.edu>
>To: <to...@jakarta.apache.org>
>Sent: Friday, June 01, 2001 11:03 AM
>Subject: Re: ** JVM and Processes
>
>
>> Jeff,
>>
>> Thanks a bunch. Your answer appears to be the best so far. I have
>> implemented the PoolTCPConnector in the server xml file and it appears to
>> be limiting the number of threads as it should. However, something that has
>> been happening (even before switching to PoolTCPConnector) is that when
>> running multiple java servlets the threads stay alive long after they
>> should have died or been garbage collected. Even after a long wait, the
>> only way (apparently) to get rid of them is to go through and kill them one
>> at a time. Is there a setting somewhere that is telling the java threads to
>> stay alive indefinitely?
>>
>> Thanks for your help,
>> - Adam
>>
>>
>> At 10:34 AM 6/1/2001 -0700, you wrote:
>> >When Java first came to the Linux platform (via the Blackdown port),
>> >green-threads were the only option. Native threads took a little longer to
>> >implement, but are a much better option for the reasons listed in the
>> >previous message. So, I would recommend avoiding green-threads unless you
>> >have a specific reason for using them.
>> >
>> >A lot of people freak out when they see the number of "processes" being
>> >reported by ps or top, without realizing that these are merely threads and
>> >not full-blown processes. If you have a lightly loaded Tomcat, you can tune
>> >down the number of threads being spawned by using the max_threads,
>> >max_spare_threads, and min_spare_threads parameters of the PoolTCPConnector
>> >in your server.xml file. For an example of this, take a look at the tomcat
>> >user's guide:
>> >
>> >http://jakarta.apache.org/tomcat/tomcat-3.2-doc/index.html
>> >
>> >Do a "find" in your web browser for "max_threads". I use this to limit the
>> >number of ajp12 threads and maximize ajp13 threads -- because I'm using
>> >ajp13 for my servlets and ajp12 only for startup/shutdown of Tomcat.
>> >
>> >Conversely, if you have a heavily loaded Tomcat, you should use these
>> >parameters to increase performance.
>> >
>> >Thanks,
>> >--jeff
>> >
>> >----- Original Message -----
>> >From: "Michael Jennings" <so...@home.com>
>> >To: <to...@jakarta.apache.org>
>> >Sent: Friday, June 01, 2001 9:32 AM
>> >Subject: Re: ** JVM and Processes
>> >
>> >
>> >RE: ** JVM and ProcessesMy understanding of green vs. native threads is as
>> >follows:
>> >With native threads, an actual system thread is created when a Java thread
>> >is created.
>> >On linux a system thread takes the form of another process, but one that
>> >shares memory
>> >etc. with another process. This is why if you create a program that
>> >allocates 100 megs of memory,
>> >then spins off 10 threads, it looks like 10 processes each taking up 100
>> >megs of memory, when in
>> >fact the amount of memory is 100 megs + 10*overhead for each thread (not
>> >much more than 100 megs).
>> >
>> >On WIN32 systems, threads do not show up as separate processes, they are
>> >separate threads of execution
>> >inside the same process (essentially the same as the Linux implementation
>> >with differences too subtle to care about)
>> >
>> >Green threads on the other hand use timers, signals, setjmp etc. voodoo to
>> >"simulate" threads within one process.
>> >Essentially taking over the scheduling from the kernel.
>> >
>> >I believe the command-line option for green threads is simply "-green" as in
>> >java -green MyThreaddedApp
>> >
>> >If you have a multi-cpu system, green threads will only take advantage of
>> >one cpu, whereas native threads
>> >will use all the cpus on your system (that's the theory anyway)
>> >
>> >I've heard of problems with blocking I/O with green threads, but have no
>> >first hand knowledge.
>> >
>> >Hope this helps.
>> >-Mike Jennings
>> >
>> > ----- Original Message -----
>> > From: BARRAUD Val駻ie
>> > To: 'tomcat-user@jakarta.apache.org'
>> > Sent: Friday, June 01, 2001 9:01 AM
>> > Subject: RE: ** JVM and Processes
>> >
>> >
>> >
>> >
>> > http://java.sun.com/products/jdk/1.1/packs/native-
threads/README
>> >
>> > -----Message d'origine-----
>> > De: myatt83@potsdam.edu
[SMTP:myatt83@potsdam.edu]
>> > Date: vendredi 1 juin 2001 17:46
>> > @: tomcat-user@jakarta.apache.org
>> > Objet: RE: ** JVM and Processes
>> >
>> > Randy,
>> >
>> > Thanks for the advice. Could you be a little more
specific, though,
>> >about
>> > how to use green threads instead of native threads
and possibly
>> >differences
>> > between the two? Thanks.
>> >
>> > - Adam
>> >
>> >
>> >
>> > At 10:59 AM 6/1/2001 -0400, you wrote:
>> > >
>> > > Don't use ps - these are actually
threads. ps is showing
>them
>> >as
>> > >processes because that is what it does. If you
use green thread (as
>> >opposed
>> > >to the native threads you are using now), the
display will go away,
>but
>> >you
>> > >will experience a slowdown (how much depends on
your operating
>system
>> >and
>> > >other activity on the system).
>> > >
>> > > Randy
>> > >
>> > >
>> > >> -----Original Message-----
>> > >> From: myatt83@potsdam.edu
[mailto:myatt83@potsdam.edu]
>> > >> Sent: Friday, June 01, 2001 10:37 AM
>> > >> To: tomcat-user@jakarta.apache.org
>> > >> Subject: ** JVM and Processes
>> > >>
>> > >>
>> > >> Hi,
>> > >>
>> > >> For a particular web server we are running
with Tomcat 3.1,
>> > >> we are having
>> > >> an issue with the java servlets that are
running. What appears to
>be
>> > >> happening is that each time a servlet is
called from the web
>> > >> site, a new
>> > >> process is created to run the java
program. When I view
>> > >> processes with "ps
>> > >> ax", I see dozens of instances of:
>> > >>
/usr/java/jdk1.3/bin/i386/native_threads/java
>> > >>
>> > >> It was briefly stated in Java Servlet
Programming by Hunter &
>> > >> Crawford, (c)
>> > >> Oreilly that 'most servlet containers
execute all servlets in
>> > >> a single JVM
>> > >> ... the exception being high-end
containers that support
>> > >> execution across
>> > >> multiple backend servers...'
>> > >>
>> > >> We are only using 1 web server with an
average weekly load of
>> > >> a couple of
>> > >> hundred visitors.
>> > >>
>> > >> Any ideas as to why we would be seeing so
many identical
>> > >> processes and if
>> > >> so, how to modify that?
>> > >>
>> > >> Thanks in advance.
>> > >>
>> > >> -Adam
>> > >>
>> > >>
>> > >
>> > >
>> >
>> >
>> >
>> >
>> >
>>
>
>
>
--
simon colston
simon@lexues.co.jp
Re: how to test the website scalability
Posted by Charles Webber <ch...@earthlink.net>.
Mercury Interactive LoadRunner is probably the most comprehensive tool, but
there are others. Radview (www.radview.com) has a tool called webload, that
does a fairly good job of this as well.
Depends on what kind of load test you require as well. If you just want to
simulate a couple hudred virtual clients, you don't need much hardware on
the load generation side of the network. On the other hand, if you want
many thousand users, you might want to hire the test performed by one of the
companies listed, as they should have hardware capable of generating load.
You'll also need to figure out exactly what you want to test. Do you want
to test the performance of the web server or the entire application and
database layers as well? What kind of response time threshholds are
acceptable? What kind of results do you need to get? Do you need averages?
Do you need min/max response times? Do you need server resource utilization
statistics, such as memory, CPU, system calls, other resources?
There's a ton of things you need to consider when doing one of these tests,
and just a little of the benefit of my somewhat limited experience.
----- Original Message -----
From: "Bryan Lipscy" <x4...@monroe.net>
To: <to...@jakarta.apache.org>
Sent: Monday, September 17, 2001 9:10 PM
Subject: RE: how to test the website scalability
> Mercury Interactive's LoadRunner is probably one of the best on the
market.
> Just a bit on the pricey side.
>
> -----Original Message-----
> From: Huaxin [mailto:hxzhang@cs.ualberta.ca]
> Sent: Sunday, September 17, 2000 7:09 PM
> To: tomcat-user@jakarta.apache.org
> Subject: how to test the website scalability
>
>
> This is quite general to all website development.
>
> When the website is completed, how can I test its
> scalability by faking a lot of URL request for it
> from various IPs concurrently? is there kind of such
> tools for that?
>
> thanks a lot!
>
>
RE: how to test the website scalability
Posted by Bryan Lipscy <x4...@monroe.net>.
Mercury Interactive's LoadRunner is probably one of the best on the market.
Just a bit on the pricey side.
-----Original Message-----
From: Huaxin [mailto:hxzhang@cs.ualberta.ca]
Sent: Sunday, September 17, 2000 7:09 PM
To: tomcat-user@jakarta.apache.org
Subject: how to test the website scalability
This is quite general to all website development.
When the website is completed, how can I test its
scalability by faking a lot of URL request for it
from various IPs concurrently? is there kind of such
tools for that?
thanks a lot!
how to test the website scalability
Posted by Huaxin <hx...@cs.ualberta.ca>.
This is quite general to all website development.
When the website is completed, how can I test its
scalability by faking a lot of URL request for it
from various IPs concurrently? is there kind of such
tools for that?
thanks a lot!