You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by A C <at...@gmail.com> on 2007/10/10 20:55:31 UTC
placing jars in /server/lib - can cause oome?
We've been experiencing horrendous OOMEs recently after deploying 40 clones
of a vendor's .war file which encompasses their application/portal. The
context is about 100MB
Our customized piece of the puzzle is quite small. We place the jTDS jar
for a quick database connection tha'ts made to their database, to
Tomcat/server/lib.
Then, we have 40 contexts for 40 customers, each one has its own WEB-INF/lib
folder with an identical set of class folders and jars etc etc.
When we deploy just ONE of these contexts onto our tomcat server,
eveything's "fine".
When we have 30, we need a daily restart, even though we never get more than
25 sessions to the database in that time …
My question is, does placing the jtds.jar in /server/lib cause an issue? I
know that it's bad practice to do this, and we're planning on refitting the
deployment and placing this in common/lib …
We're running a verbose:gc as well and have optimized to the best of our
ability xmx xms settings and evaluated the one JSP we have created as part
of the war package for leaks/finalizing objects properly. I've read to
exhaustion all the OOME threads/memory leaks/ "tomcat's fine, it's your BAD
CODE" threads, but now question some of our deployment mechanics.
Thanks in advance !!
Re: placing jars in /server/lib - can cause oome?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
A C,
A C wrote:
> Application: Reporting application we'll call "reporting.war" , which
> unpacks to a directory size of ~ 100MB of jars/classes/static content/etc.
Aah, that's a different story. Certainly, not all 100MB is being loaded
into memory and staying there. You probably have images and stuff that
are never loaded.
> # webapp contexts (is that what these are called?): 40
The servlet spec and J2EE interchangeably call them "contexts" and "web
applications". I suppose a context only makes sense when the web
application is actually running. You could call a WAR file a "web
application", but you wouldn't call it a context. On the other hand, a
deployed web application is running in a context which .. well, IS the
web application.
Semantics.
> -Xms: 512MB
> -Xmx: 1024M
Just for kicks, set these to the same value.
Then, attach something to the Tomcat process that can read the heap info
(free, user, min/max) as well as request a full GC from the JVM. You can
use a profiler or something less bulky such as LambdaProbe.
Deploy a single web application (and leave the other 39 of them off).
Use the web app a bit to exercise it a little bit. Look at the memory
usage over time. You should get a mem usage curve that looks like this:
\/\/\/\/\/\/\/\/\/\
Nice ASCII art, eh?
Seriously, though, your memory should be allocated, then freed, pretty
regularly. You should be able to look at the peaks and valleys to
determine how much memory a single copy of your webapp requires while
running on Tomcat (yes, I know Tomcat is there too, but bear with me).
Next, deploy two instances of the webapp and repeat the process (make
sure to exercise both copies and not just one). You should be able to
see how much memory is used by two instances running on Tomcat. By
subtracting the amount you got from test 2 from test 1, you should be
able to guess at how much memory a single instance takes.
Try your guesses out by repeating the process with 3 or more instances
to see if the memory usage increases according to how much you think
each instance "takes".
It's possible that each instance needs maybe 50 or 60MB of heap space to
run comfortably. If you've capped the heap at 1GB, that means that you
can only have 1024 / 50 ~= 20 instances running comfortably in that
amount of memory (and that doesn't even cover the overhead of running
Tomcat itself). You either need to increase the available memory (you
can probably squeeze up to 1.6GB out of a 32-bit JVM or get yourself a
64-bit OS and JVM) or run some of these instances on another server.
> The only "thing" running on this machine is a Windows apache tomcat
> service. this application "hooks" into a sister app server that performs
> the "meat and potatos" activities of the application which we've never
> needed to restart but have done so to see if it was somehow related to
> web-layer OOMEs (hey, my father used to bang the t.v. set to make the
> picture more clear and it worked for HIM, so ...)
Do all 40 instances use that particular resource? What is it, may I ask?
> webapp/reporting/web-inf/lib = 93 jar files = 22 Mb
Note that memory footprint rarely has anything to do with the size of
the WAR file that was used to deploy it.
> Connection con = ds.getConnection;
>
> we do some SQL and then close/null any variables using or related to the
> connection/resultset/etc
Do you have this wrapped-up in a try/catch/finally block? Is it possible
that you have JDBC resource leaks?
You should set:
removeAbandoned="true"
removeAbandonedTimeout="30"
logAbandoned="true"
in your <Resource>. If your SQL queries take a long time to run, 30
seconds might not be enough "timeout" time, so feel free to increase that.
This configuration will log a stack trace for any connection that does
not get returned to the pool within (approximately)
"removeAbandonedTimeout" seconds. You might find that you are leaking
connections. Then again, you might not be. It's always good to check.
> Here's the permgen after 1.5 hours of my simulation (35 deployed webapps of
> this application)
Let's see how your heap looks after a while...
> 1247.073: [GC 1247.073: [DefNew: 51055K->276K(52480K), 0.0630231 secs]
> 123930K->75721K(518528K), 0.0630960 secs]
You are reclaiming a bunch of memory in each GC.
Let's keep going:
> 1538.764: [Full GC 1538.764: [Tenured: 86527K->59881K(466048K),
> 5.3928429secs] 133369K->59881K(518528K), [Perm :
> 32768K->32610K(32768K)], 5.3929301 secs]
Looks like you're busting PermGen. You nearly busted it at the last Full
GC (had 32768K in it, and freed only about 150K).
> 2357.129: [Full GC 2357.129: [Tenured: 89508K->71494K(466048K),
> 7.1956506secs] 105965K->71494K(518528K), [Perm :
> 49151K->48973K(49152K)], 7.2276270 secs]
PermGen size has increased, but you're still very close to filling it up.
Note that the full heap is still only 1/2 GB (your initial heap size). 7
seconds is a loooong GC.
> 2485.690: [Full GC 2485.690: [Tenured: 90945K->91468K(466048K),
> 3.2931426secs] 138286K->91468K(518528K), [Perm :
> 53247K->53247K(53248K)],
PermGen keeps getting bigger. Over what time period was this?
> 2969.556: [Full GC 2969.556: [Tenured: 97335K->97396K(466048K),
> 0.4763756secs] 97664K->97396K(518528K), [Perm :
> 65535K->65535K(65536K)],
> 0.4764480 secs]
Does it bomb after this? You are always tight on PermGen space. I'd
guess this was the problem.
> when i remove some 30-odd *.xml deployment files from /conf.../localhost and
> have it load just TWO webapps, it lasts much longer
Well, whatever the problem is, having 40 instances is certainly
compounding it. Does it eventually die when running only two webapps?
> what should i do? is this a PermGen problem?
Almost certainly. Take a look at the rest of your heap: you are never
really exceeding your initial heap size, so it's not regular heap space
you need... it's PermGen.
> And if it is, why does my
> error message keep saying "ServeletException" and not the OutOfMemory
> PermGen message?
It doesn't always say "PermGen ran out of space".
> thanks - and if anyone needs graphs/files/blood i'll be happy to drop them
> somewhere
Didn't there used to be some kind of UNIX administrators guide to
sacrifices and stuff? I remember reading about what you were supposed to
do with chicken blood when you were having some specific issue waaaay
back in the day. Who knows; maybe it'll help.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFHD9oC9CaO5/Lv0PARAq8HAJ4+EjFXkVKgkdFp4s8unGlIsj/nWwCfWvW6
+ZGCvQhfywUKuFIJOjf4kWc=
=FJyG
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: placing jars in /server/lib - can cause oome?
Posted by A C <at...@gmail.com>.
har har. glad i could give everyone on this thread a good chuckle.
anyway... I didn't want to rattle off more crud, so instead I collected as
best I could, information that might elucidate my issue more:
and everyone was right, my jar file did NOT live in \server.
Application: Reporting application we'll call "reporting.war" , which
unpacks to a directory size of ~ 100MB of jars/classes/static content/etc.
# webapp contexts (is that what these are called?): 40
Server: Tomcat 5.0.27
JDK: 1.4.2_13
Tomcat runs as a windows service: Tomcat\bin\tomcat5.exe
//RS//reportingTomcat
-Xms: 512MB
-Xmx: 1024M
-Xrs
________________________________________
The only "thing" running on this machine is a Windows apache tomcat
service. this application "hooks" into a sister app server that performs
the "meat and potatos" activities of the application which we've never
needed to restart but have done so to see if it was somehow related to
web-layer OOMEs (hey, my father used to bang the t.v. set to make the
picture more clear and it worked for HIM, so ...)
webapp/reporting/web-inf/lib = 93 jar files = 22 Mb
webapp/reporting01
webapp/reporting02
.
.
webapp/reporting30
each of these contexts has a corresponding config file in :
\conf\Catalina\localhost\reporting01.xml
\conf\Catalina\localhost\reporting02.xml
.
.
\conf\Catalina\localhost\reporting30.xml
In this XMl file, I have:
<Context docBase="E:\tools\portal\reporting.war" path="/reporting01"
crossContext="false" debug="0" reloadable="false" trusted="false">
<Logger className="org.apache.catalina.logger.FileLogger"
prefix="localhost_reporting01_log." suffix=".txt" timestamp="true"/>
<Resource name="jdbc/myapp" auth="Container" type="javax.sql.DataSource" />
<ResourceParams name="jdbc/myapp">
<parameter>
<name>factory</name>
<value>org.apache.commons.dbcp.BasicDataSourceFactory</value>
</parameter>
<parameter>
<name>driverClassName</name>
<value>net.sourceforge.jtds.jdbc.Driver</value>
</parameter>
<parameter>
<name>url</name>
<value>jdbc:jtds:sqlserver://sqlserver:1433/reporting01;User=xxx;Password=xxx</value>
<!-- to add DB instance, use "...:port/databaseName;instance=<name>;User=xx"
notation -->
</parameter>
<parameter>
<name>validationQuery</name>
<value>select 1</value>
</parameter>
<parameter>
<name>maxActive</name>
<value>30</value>
</parameter>
</ResourceParams>
</Context>
________________________________________________
The *only* customization we've done is to make a brief call to a database,
grab a valid userlogin from our application and perform a handshake to the
reporting application.
_________ custom code lives in \webapps\reportingXX\customcode\ ________
and all apps use " jtds-1.2.jar " which lives in \Tomcat\common\lib
our customization is something like:
file > db.include, which has:
InitialContext initCtx = new InitialContext();
Context envCtx = (Context) initCtx.lookup("java:comp/env");
DataSource ds = (DataSource) envCtx.lookup("jdbc/myapp");
...and we create the connection in:
file > login.jsp which has:
includes db.include
Connection con = ds.getConnection;
we do some SQL and then close/null any variables using or related to the
connection/resultset/etc
After about a day, we start seeing these:
__________________________________________________
2007-10-12 12:24:20 ApplicationDispatcher[/reportingXX] Servlet.service()
for servlet jsp threw exception
java.lang.OutOfMemoryError
2007-10-12 12:24:21 ApplicationDispatcher[[/reportingXX] ] Servlet.service()
for servlet jsp threw exception
java.lang.OutOfMemoryError
2007-10-12 12:24:21 StandardWrapperValve[action]: Servlet.service() for
servlet action threw exception
java.lang.OutOfMemoryError
__________________________________________________
Here's the permgen after 1.5 hours of my simulation (35 deployed webapps of
this application)
Start:
9.643: [GC 9.643: [DefNew: 46591K->3981K(52416K), 0.0273929 secs]
46591K->3981K(518464K), 0.0276010 secs]
13.569: [GC 13.569: [DefNew: 50573K->3714K(52416K), 0.0351330 secs]
50573K->7132K(518464K), 0.0352592 secs]
19.409: [GC 19.409: [DefNew: 50306K->4897K(52416K), 0.0262670 secs]
53724K->9800K(518464K), 0.0263980 secs]
22.330: [GC 22.330: [DefNew: 51489K->1120K(52416K), 0.0177011 secs]
56392K->10751K(518464K), 0.0178319 secs]
22.978: [GC 22.978: [DefNew: 47712K->1128K(52416K), 0.0031998 secs]
57343K->10760K(518464K), 0.0033108 secs]
24.976: [GC 24.976: [DefNew: 47720K->3289K(52416K), 0.0163705 secs]
57352K->12920K(518464K), 0.0164999 secs]
38.553: [GC 38.553: [DefNew: 49881K->5047K(52416K), 0.0225587 secs]
59512K->15380K(518464K), 0.0227062 secs]
71.451: [GC 71.451: [DefNew: 51635K->249K(52416K), 0.0224671 secs]
61967K->13712K(518464K), 0.0225193 secs]
97.070: [GC 97.070: [DefNew: 46841K->1806K(52416K), 0.0066059 secs]
60304K->15268K(518464K), 0.0066603 secs]
107.194: [GC 107.194: [DefNew: 48398K->4604K(52416K), 0.0193955 secs]
61860K->18066K(518464K), 0.0194505 secs]
136.409: [GC 136.409: [DefNew: 51195K->251K(52416K), 0.0164256 secs]
64658K->16415K(518464K), 0.0164884 secs]
169.849: [GC 169.849: [DefNew: 46843K->1755K(52416K), 0.0072353 secs]
63007K->17919K(518464K), 0.0072906 secs]
.
.
.
.
.
.
1247.073: [GC 1247.073: [DefNew: 51055K->276K(52480K), 0.0630231 secs]
123930K->75721K(518528K), 0.0630960 secs]
1279.579: [GC 1279.579: [DefNew: 46932K->1585K(52480K), 0.0256284 secs]
122377K->77031K(518528K), 0.0256938 secs]
1291.668: [GC 1291.668: [DefNew: 48240K->4514K(52480K), 0.0205993 secs]
123686K->79959K(518528K), 0.0206722 secs]
1313.524: [GC 1313.524: [DefNew: 51169K->254K(52480K), 0.0317320 secs]
126615K->78344K(518528K), 0.0317929 secs]
1344.525: [GC 1344.525: [DefNew: 46910K->1729K(52480K), 0.0149128 secs]
125000K->79820K(518528K), 0.0149729 secs]
1351.479: [GC 1351.479: [DefNew: 48381K->4582K(52480K), 0.0218740 secs]
126471K->82673K(518528K), 0.0219304 secs]
1356.378: [Full GC 1356.379: [Tenured: 78090K->68675K(466048K),
6.8007345secs] 100262K->68675K(518528K), [Perm :
24575K->24575K(24576K)],
6.8080206 secs]
1368.802: [GC 1368.802: [DefNew: 46653K->4549K(52480K), 0.0286182 secs]
115328K->73225K(518528K), 0.0286805 secs]
1374.837: [GC 1374.837: [DefNew: 51205K->5688K(52480K), 0.0866753 secs]
119881K->76341K(518528K), 0.0867317 secs]
1380.416: [GC 1380.416: [DefNew: 52344K->5824K(52480K), 0.1281856 secs]
122997K->78947K(518528K), 0.1282481 secs]
1387.302: [GC 1387.302: [DefNew: 52479K->5824K(52480K), 0.1806688 secs]
125603K->81752K(518528K), 0.1807336 secs]
1393.171: [GC 1393.171: [DefNew: 52479K->5823K(52480K), 0.0392198 secs]
128408K->84507K(518528K), 0.0392715 secs]
1400.469: [GC 1400.469: [DefNew: 52479K->4437K(52480K), 0.0609195 secs]
131163K->85732K(518528K), 0.0609793 secs]
1403.957: [Full GC 1403.958: [Tenured: 81295K->76875K(466048K),
0.3270033secs] 106066K->76875K(518528K), [Perm :
28671K->28671K(28672K)],
0.3270935 secs]
1410.753: [GC 1410.753: [DefNew: 46656K->4605K(52480K), 0.0423694 secs]
123531K->81480K(518528K), 0.0424263 secs]
1416.358: [GC 1416.358: [DefNew: 51257K->5458K(52480K), 0.0375570 secs]
128133K->86701K(518528K), 0.0376129 secs]
1421.792: [GC 1421.792: [DefNew: 52114K->5339K(52480K), 0.1025094 secs]
133357K->91866K(518528K), 0.1025714 secs]
1538.764: [Full GC 1538.764: [Tenured: 86527K->59881K(466048K),
5.3928429secs] 133369K->59881K(518528K), [Perm :
32768K->32610K(32768K)],
5.3929301 secs]
1551.420: [GC 1551.420: [DefNew: 46655K->1848K(52480K), 0.0192879 secs]
106537K->61729K(518528K), 0.0193421 secs]
1557.712: [Full GC 1557.712: [Tenured: 59881K->64638K(466048K),
0.3245996secs] 99584K->64638K(518528K), [Perm :
36863K->36863K(36864K)],
0.3246770 secs]
1599.653: [GC 1599.653: [DefNew: 46655K->2500K(52480K), 0.0269780 secs]
111294K->67139K(518528K), 0.0270333 secs]
1603.423: [Full GC 1603.423: [Tenured: 64638K->67911K(466048K),
0.3214652secs] 77743K->67911K(518528K), [Perm :
40959K->40959K(40960K)],
0.3215498 secs]
1627.139: [GC 1627.139: [DefNew: 46655K->1289K(52480K), 0.0142188 secs]
114567K->69201K(518528K), 0.0142742 secs]
1670.137: [Full GC 1670.137: [Tenured: 67911K->69866K(466048K),
0.3690838secs] 90651K->69866K(518528K), [Perm :
45054K->45054K(45056K)],
0.3691751 secs]
1866.589: [GC 1866.589: [DefNew: 46418K->1563K(52480K), 0.0130402 secs]
116284K->71430K(518528K), 0.0131005 secs]
2324.705: [GC 2324.705: [DefNew: 48219K->4327K(52480K), 0.1842139 secs]
118086K->74194K(518528K), 0.1842687 secs]
2329.115: [GC 2329.115: [DefNew: 50983K->3916K(52480K), 0.2362021 secs]
120850K->75941K(518528K), 0.2362599 secs]
2332.533: [GC 2332.533: [DefNew: 50572K->1846K(52480K), 2.1846159 secs]
122597K->77692K(518528K), 2.1846729 secs]
2338.547: [GC 2338.547: [DefNew: 48502K->5824K(52480K), 0.4543022 secs]
124348K->84948K(518528K), 0.4543563 secs]
2341.175: [GC 2341.175: [DefNew: 52480K->1973K(52480K), 0.0189354 secs]
131604K->83257K(518528K), 0.0189910 secs]
.
.
.
.
.
2346.742: [GC 2346.742: [DefNew: 52479K->463K(52480K), 0.0085631 secs]
135685K->85193K(518528K), 0.0086173 secs]
2348.384: [GC 2348.384: [DefNew: 47119K->929K(52480K), 0.0053222 secs]
131849K->85658K(518528K), 0.0053758 secs]
2350.048: [GC 2350.048: [DefNew: 47585K->2118K(52480K), 0.0071565 secs]
132314K->86847K(518528K), 0.0072113 secs]
2351.522: [GC 2351.522: [DefNew: 48774K->1679K(52480K), 0.0072613 secs]
133503K->86409K(518528K), 0.0073149 secs]
2353.277: [GC 2353.277: [DefNew: 48335K->5823K(52480K), 0.1749974 secs]
133065K->94648K(518528K), 0.1750549 secs]
2355.637: [GC 2355.637: [DefNew: 52479K->3109K(52480K), 0.0142968 secs]
141304K->92617K(518528K), 0.0143513 secs]
2357.129: [Full GC 2357.129: [Tenured: 89508K->71494K(466048K),
7.1956506secs] 105965K->71494K(518528K), [Perm :
49151K->48973K(49152K)],
7.2276270 secs]
2369.631: [GC 2369.631: [DefNew: 46655K->1199K(52480K), 0.0261916 secs]
118150K->72693K(518528K), 0.0262450 secs]
2452.783: [GC 2452.783: [DefNew: 47855K->2579K(52480K), 0.0135995 secs]
119349K->74073K(518528K), 0.0136626 secs]
2457.584: [GC 2457.584: [DefNew: 49235K->5823K(52480K), 0.0369991 secs]
120729K->78211K(518528K), 0.0370544 secs]
2461.478: [GC 2461.478: [DefNew: 52479K->1589K(52480K), 0.0268260 secs]
124867K->79759K(518528K), 0.0268830 secs]
2464.523: [GC 2464.523: [DefNew: 48245K->1914K(52480K), 0.0117291 secs]
126415K->80083K(518528K), 0.0117836 secs]
2470.175: [GC 2470.175: [DefNew: 48570K->3163K(52480K), 0.0165518 secs]
126739K->81333K(518528K), 0.0166066 secs]
2473.213: [GC 2473.213: [DefNew: 49819K->2853K(52480K), 0.0148212 secs]
127989K->82160K(518528K), 0.0148874 secs]
2474.596: [GC 2474.596: [DefNew: 49509K->3104K(52480K), 0.0218307 secs]
128816K->82411K(518528K), 0.0218863 secs]
2478.481: [GC 2478.481: [DefNew: 49760K->3208K(52480K), 0.0126334 secs]
129067K->82833K(518528K), 0.0126879 secs]
2479.982: [GC 2479.983: [DefNew: 49864K->5824K(52480K), 0.1148272 secs]
129489K->92140K(518528K), 0.1148850 secs]
2481.667: [GC 2481.667: [DefNew: 52479K->3775K(52480K), 0.0376813 secs]
138796K->91694K(518528K), 0.0377389 secs]
2483.403: [GC 2483.403: [DefNew: 50431K->5823K(52480K), 0.1317053 secs]
138350K->96769K(518528K), 0.1317564 secs]
2485.690: [Full GC 2485.690: [Tenured: 90945K->91468K(466048K),
3.2931426secs] 138286K->91468K(518528K), [Perm :
53247K->53247K(53248K)],
3.2932258 secs]
2490.649: [GC 2490.649: [DefNew: 46655K->5824K(52480K), 0.0365206 secs]
138124K->99492K(518528K), 0.0365753 secs].
.
.
.
.
.
2888.080: [GC 2888.080: [DefNew: 47683K->2102K(52480K), 0.0122331 secs]
168795K->123214K(518528K), 0.0122898 secs]
2889.591: [GC 2889.591: [DefNew: 48758K->2383K(52480K), 0.0198578 secs]
169870K->123496K(518528K), 0.0199112 secs]
2891.947: [GC 2891.947: [DefNew: 49039K->3107K(52480K), 0.0112011 secs]
170152K->124219K(518528K), 0.0112553 secs]
2893.455: [GC 2893.455: [DefNew: 49763K->5718K(52480K), 0.0198187 secs]
170875K->127604K(518528K), 0.0198690 secs]
2895.015: [GC 2895.015: [DefNew: 52374K->5824K(52480K), 0.0642596 secs]
174260K->132189K(518528K), 0.0643112 secs]
2896.954: [GC 2896.954: [DefNew: 52480K->3493K(52480K), 0.0117051 secs]
178845K->130019K(518528K), 0.0117596 secs]
2899.660: [GC 2899.660: [DefNew: 50149K->1101K(52480K), 0.0075071 secs]
176675K->127790K(518528K), 0.0075585 secs]
2901.454: [GC 2901.454: [DefNew: 47756K->587K(52480K), 0.0068498 secs]
174445K->127276K(518528K), 0.0069012 secs]
2905.571: [Full GC 2905.571: [Tenured: 126688K->100740K(466048K),
10.5908514secs] 157839K->100740K(518528K), [Perm :
65535K->65063K(65536K)],
10.6232907 secs]
2919.226: [Full GC 2919.226: [Tenured: 100740K->101030K(466048K),
0.5165606secs] 138033K->101030K(518528K), [Perm :
65535K->65535K(65536K)],
0.5166402 secs]
2919.748: [Full GC 2919.748: [Tenured: 101030K->101030K(466048K),
0.5793060secs] 101030K->101030K(518528K), [Perm :
65535K->65535K(65536K)],
0.5794041 secs]
2920.335: [Full GC 2920.335: [Tenured: 101030K->101030K(466048K),
0.4588885secs] 101030K->101030K(518528K), [Perm :
65535K->65535K(65536K)],
0.4589650 secs]
2920.802: [Full GC 2920.802: [Tenured: 101030K->97189K(466048K),
0.6423210secs] 101030K->97189K(518528K), [Perm :
65535K->65497K(65536K)],
0.6424079 secs]
2968.834: [Full GC 2968.834: [Tenured: 97189K->97335K(466048K),
0.6619274secs] 99370K->97335K(518528K), [Perm :
65535K->65535K(65536K)],
0.6620179 secs]
2969.556: [Full GC 2969.556: [Tenured: 97335K->97396K(466048K),
0.4763756secs] 97664K->97396K(518528K), [Perm :
65535K->65535K(65536K)],
0.4764480 secs]
when i remove some 30-odd *.xml deployment files from /conf.../localhost and
have it load just TWO webapps, it lasts much longer
what should i do? is this a PermGen problemo? And if it is, why does my
error message keep saying "ServeletException" and not the OutOfMemory
PermGen message?
thanks - and if anyone needs graphs/files/blood i'll be happy to drop them
somewhere
AC
On 10/10/07, Pid <p...@pidster.com> wrote:
>
> Christopher Schultz wrote:
> > A C,
> >
> > A C wrote:
> >> We've been experiencing horrendous OOMEs recently after deploying 40
> clones
> >> of a vendor's .war file which encompasses their
> application/portal. The
> >> context is about 100MB
> >
> > Lesee... 100 MB times 40 instances .. mm... carry the 7 ... hey, that's
> > 4GB! Where have I heard that number before... hmm.. something about ...
> > right! The maximum process memory on a 32-bit OS!
>
> lol
>
>
>
> A C, I have to say this: the clue is in the question...
>
> I'll summarise; it works OK when when you deploy one copy of a 100Mb
> app, but when you deploy FORTY it gets a bit messy and you run out of
> memory.
>
> How is it that you're not thinking about more servers at this point?
>
> (I'd be waving a nice diagram at the accountants and saying 'shiny new
> boxes please')
>
>
> p
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: placing jars in /server/lib - can cause oome?
Posted by Pid <p...@pidster.com>.
Christopher Schultz wrote:
> A C,
>
> A C wrote:
>> We've been experiencing horrendous OOMEs recently after deploying 40 clones
>> of a vendor's .war file which encompasses their application/portal. The
>> context is about 100MB
>
> Lesee... 100 MB times 40 instances .. mm... carry the 7 ... hey, that's
> 4GB! Where have I heard that number before... hmm.. something about ...
> right! The maximum process memory on a 32-bit OS!
lol
A C, I have to say this: the clue is in the question...
I'll summarise; it works OK when when you deploy one copy of a 100Mb
app, but when you deploy FORTY it gets a bit messy and you run out of
memory.
How is it that you're not thinking about more servers at this point?
(I'd be waving a nice diagram at the accountants and saying 'shiny new
boxes please')
p
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: placing jars in /server/lib - can cause oome?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
A C,
A C wrote:
> We've been experiencing horrendous OOMEs recently after deploying 40 clones
> of a vendor's .war file which encompasses their application/portal. The
> context is about 100MB
Lesee... 100 MB times 40 instances .. mm... carry the 7 ... hey, that's
4GB! Where have I heard that number before... hmm.. something about ...
right! The maximum process memory on a 32-bit OS!
Nevermind the fact that a 32-bit JVM can't allocate that much memory in
one huge chunk.
It looks like your requirements are outstripping your capabilities. If
you want to run all these instances on the same machine, you're going to
need a 64-bit OS.
> When we deploy just ONE of these contexts onto our tomcat server,
> eveything's "fine".
Probably because it uses 1/40 of the memory that 40 of them require.
> When we have 30, we need a daily restart, even though we never get more than
> 25 sessions to the database in that time …
You should look at what is taking up so much memory.
> My question is, does placing the jtds.jar in /server/lib cause an issue?
Almost certainly not. It's probably not doing you any good, either,
since your application can't use it (only the server can, but maybe you
have Tomcat managing your connection pool or something).
> We're running a verbose:gc as well and have optimized to the best of our
> ability xmx xms settings
Like what? An example would be enlightening.
> [I] now question some of our deployment mechanics.
I'm pretty sure that you just need more memory than you have. If you're
right, and each instance of this app takes ~100MB, then you simply
cannot deploy 40 of them on a single 32-bit machine.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFHDSSV9CaO5/Lv0PARAtljAKCX1LhV8nl+cxnc5pSfRh6NAqNDLQCgkxDU
xbMHY6ha/JLZcPRCDB7K5RE=
=GBVj
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: placing jars in /server/lib - can cause oome?
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: A C [mailto:atulsmailbox@gmail.com]
> Subject: placing jars in /server/lib - can cause oome?
>
> We've been experiencing horrendous OOMEs recently after
> deploying 40 clones of a vendor's .war file which encompasses their
> application/portal.
You're a tad short on specifics: Tomcat version, JRE/JDK version, OS
platform, actual exception thrown, etc.
> Then, we have 40 contexts for 40 customers, each one has its
> own WEB-INF/lib folder with an identical set of class folders
> and jars etc etc.
Each copy of the classes has to be loaded separately, since Tomcat uses
one classloader per webapp. You are likely running out of PermGen
space, but can't tell for sure without more real information.
> When we have 30, we need a daily restart, even though we
> never get more than 25 sessions to the database in that time ...
Which indicates you may well have a memory leak in your webapp, and
having multiple copies of it just makes things worse. You need a
profiler.
> My question is, does placing the jtds.jar in /server/lib
> cause an issue?
Unless you've done something funny with conf/catalina.properties, that
may be completely irrelevant, since none of the webapps can even see
classes from that location. Are you sure that jar is even being used?
> we're planning on refitting the deployment and
> placing this in common/lib ...
Which would make it visible to the webapps, whereas now it's only
visible to Tomcat code. Moving it to common/lib should not require any
changes.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is thus for use only by the intended recipient. If you
received this in error, please contact the sender and delete the e-mail
and its attachments from all computers.
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org