You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Mark Eggers <it...@yahoo.com> on 2013/07/16 23:43:45 UTC
[OT] Cannot cleanly undeploy a web application
Folks,
Off topic, so I'm going to be burning a little karma (OK, a lot).
Environment:
Window 7 64 bit
JRE 1.7.0_25
Tomcat 7.0.42 (with tcnative) run from startup.bat
I have a web application that scans for a resource path using:
this.getClass().getClassLoader().getResources(resourcePath)
It then wanders through the resources, checking the protocol for each
URL in the returned enumeration.
Based on the protocol, the URL is scanned for desired information and
the names of those resources are collected.
For a JAR file, I use the following code (don't shoot me - I'm an admin
/ architect, not a developer).
public ArrayList<String> getResources(String resourcePath,
String ending) {
JarURLConnection urlCon = null;
Pattern p =
Pattern.compile("^" + resourcePath + "/.+(" + ending + ")$");
try {
urlCon = (JarURLConnection) url.openConnection();
urlCon.setUseCaches(false);
JarFile jar = urlCon.getJarFile();
Enumeration<JarEntry> je = jar.entries();
while (je.hasMoreElements()) {
String entry = je.nextElement().getName();
if (log.isDebugEnabled()) {
log.debug("JAR entry: " + entry);
}
Matcher m = p.matcher(entry);
if (m.matches()) {
resources.add(entry);
}
}
} catch (IOException ex) {
resources.add("Could not open resource");
if (log.isErrorEnabled()) {
log.error("Could not open resource",ex);
}
} finally {
if (null != urlCon) {
urlCon = null;
}
}
return this.resources;
}
Finally,
this.getClass().getClassLoader().getResourceAsStream()
is used to obtain the resources, process them, and stuff them into a
HashMap managed by a singleton.
This is all done via a servlet context listener, and the HashMap is
emptied in the listener's contextDestroyed method.
On Linux, this all works well. I can deploy and undeploy with abandon.
The Tomcat manager reports no leaks.
On Windows, I cannot cleanly undeploy. The JAR files containing the
resourcePath (and hence the resources) remain in
%CATALINA_HOME%\webapps\appname.
However, once even a minor garbage collection occurs I can then cleanly
undeploy the web application.
I have several solutions:
1. context.xml antiResourceLocking="true"
This works, but there are a lot of downsides ( noted in the
documentation). This also indicates that an underlying InputStream is
being held open when parsing the JAR file. The solution is also Tomcat -
specific.
2. Never package the desired resources in a JAR
This works, but developers may balk. Also, the path to the desired
resources must be unique, otherwise you could still trigger loading from
a JAR (although I could just not handle JAR scanning).
3. Copy the JAR file to java.io.tmpdir, extract, and read
This is ugly, but might work. It would impact the application start up
time even more, but the JAR scanning issue would no longer exist. I
think Glassfish does this.
4. Use a properties file
Read in a properties file with a list of desired resources, and use that
list to populate the singleton. This is actually what I had in place
before, but this then requires developers to manage both the resources
and the list.
This solution also makes developing a modular, platform-independent
build system more difficult (it can be done, just creates an extra step
using Maven). Right now the build is a convoluted Ant script that
requires a specific environment. I'm trying to get away from that.
5. Do this right
How, I'm not exactly sure. Hence, the question.
/mde/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Cannot cleanly undeploy a web application
Posted by Mark Thomas <ma...@apache.org>.
On 16/07/2013 22:43, Mark Eggers wrote:
> 5. Do this right
>
> How, I'm not exactly sure. Hence, the question.
At a guess, call close on the JarFile.
Looks like you might need to refactor the code to be able to do that.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Cannot cleanly undeploy a web application
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Mark,
On 7/16/13 7:47 PM, Mark Eggers wrote:
>> For a JAR file, I use the following code (don't shoot me - I'm
>> an admin / architect, not a developer).
This may be part of the problem. Don't feel bad: you just need a
developer in this case to bail you out.
> Mark, thanks for pointing out (gently) to RTFJD. While
> JarURLConnection does not have a close method, JarFile certainly
> does. As well as I should be checking the return value from
> JarURLConnection.getJarFile . . .
>
> Taking those two thoughts in mind, a more correct (still ugly, but
> it works) method might be:
>
> public ArrayList<String> getResources(String resourcePath, String
> ending) { Pattern p = Pattern.compile("^" + resourcePath + "/.+(" +
> ending + ")$"); try { JarURLConnection urlCon = (JarURLConnection)
> url.openConnection(); urlCon.setUseCaches(false); JarFile jar =
> urlCon.getJarFile(); if (null != jar) { Enumeration<JarEntry> je =
> jar.entries(); while (je.hasMoreElements()) { String entry =
> je.nextElement().getName(); if (log.isDebugEnabled()) {
> log.debug("JAR entry: " + entry); } Matcher m = p.matcher(entry);
> if (m.matches()) { resources.add(entry); } } jar.close();
You definitely want to clean-up your resources in your "finally"
block. That requires you to declare them before the "try" and then
test for null in the finally. But it's much safer: if a
RuntimeException gets thrown in your try block, you will not close the
JAR file unless you move the "close" to the finally block.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJR5rGOAAoJEBzwKT+lPKRYzUUP/1ixwHilpyqKFNHFp9NQVWI2
AakLMsOixPrhQO9MO4AZ+lhXLuDPE8PkbmcGnSercmfHH9Fi2OnhTjnNNzAteeNQ
8y/hTRNRh4FPz3rrW6KoF28JEzGZ+xr+5JhXm+Td0pkqNA/rl5byoyN94NoEfOWb
P1Un2B4j44qI+6L4s7WeFvmj+FoRPZQHBevkbyZpB0UigT8Wp2W2bD7TIX0ZGeTc
4WCt8BEKjq+7uL8MP2IoWqS7JdsDIEa6DldJSTQPVhkEYAQkgZL86i6cEkyzJSEo
ZWvOiYw97UVzuGrm/Styn9XZ31OMDNWpDAs9/L01VQj6nOHnjsFamGGykGIqeFAp
1iPXeIGWH9qmfwDKjDifdzXMI0+qfthmkmLnQBpq3ICJzplUTjTmv4UBwQYcLBdO
xCyRuYfHqvaLN8azpf7+At/7zWV66UCQogN74abz4g1Uwv+aWV8YrvU1Dj8JBs6b
KfIxgaCa4SzuiPLvLmE0Yld4NKTZZAu/SOB1EJisR/2RKJKtROsSA307ocrttBBQ
vPV2hjn1kYjGueLxCXLG+N807pHbFdoCuJ72r28NrK+pFNnD3gVxjbINvlvCn1Zz
JxebAffObdXD0UdnyHUSjCXek6SXwsS/j8auzfMEZkRYEZlRTxnlH+RGt7hZ1SP6
e1uuQ05Iw5zsCerxcYro
=FzeN
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Cannot cleanly undeploy a web application
Posted by Mark Eggers <it...@yahoo.com>.
On 7/16/2013 4:07 PM, Caldarale, Charles R wrote:
>> From: André Warnier [mailto:aw@ice-sa.com] Subject: Re: [OT] Cannot
>> cleanly undeploy a web application
>
>> What probably happens here, is that one level below the InputStream
>> which holds the filehandle in Java, is some native file object
>> which has the file open. The InputStream object is discarded at tbe
>> Java level, but is only really destroyed at the next GC.
>
> That sounds quite feasible. The real error is that whoever owned the
> InputStream failed to close it, and simply removed all references to
> it, making it unreachable. The next GC did the close() as part of
> its cleanup work. The fix would be to find the InputStream and
> explicitly close it.
>
> - Chuck
>
Folks,
It's unfortunate that I can't include multiple parts into the reply.
Ultimately, it's a programmer error. Reading javadoc is good for what
ails you.
Andre, exactly right. When I caught this on visualvm, there was an
InputStream with reference to native code but nothing else.
Yep Chuck, I failed to close the InputStream - mainly because . . .
Mark, thanks for pointing out (gently) to RTFJD. While JarURLConnection
does not have a close method, JarFile certainly does. As well as I
should be checking the return value from JarURLConnection.getJarFile . . .
Taking those two thoughts in mind, a more correct (still ugly, but it
works) method might be:
public ArrayList<String> getResources(String resourcePath,
String ending) {
Pattern p =
Pattern.compile("^" + resourcePath + "/.+(" + ending + ")$");
try {
JarURLConnection urlCon =
(JarURLConnection) url.openConnection();
urlCon.setUseCaches(false);
JarFile jar = urlCon.getJarFile();
if (null != jar) {
Enumeration<JarEntry> je = jar.entries();
while (je.hasMoreElements()) {
String entry = je.nextElement().getName();
if (log.isDebugEnabled()) {
log.debug("JAR entry: " + entry);
}
Matcher m = p.matcher(entry);
if (m.matches()) {
resources.add(entry);
}
}
jar.close();
} else {
if (log.isErrorEnabled()) {
log.error("Failed to get JarFile entry");
}
}
} catch (IOException ex) {
if (log.isErrorEnabled()) {
log.error("Could not open resource", ex);
}
}
return this.resources;
}
Thanks, I was looking at the wrong javadoc.
. . . . sticking to admin stuff, it's less dangerous
/mde/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: [OT] Cannot cleanly undeploy a web application
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: André Warnier [mailto:aw@ice-sa.com]
> Subject: Re: [OT] Cannot cleanly undeploy a web application
> What probably happens here, is that one level below the InputStream which
> holds the filehandle in Java, is some native file object which has the file
> open. The InputStream object is discarded at tbe Java level, but is only
> really destroyed at the next GC.
That sounds quite feasible. The real error is that whoever owned the InputStream failed to close it, and simply removed all references to it, making it unreachable. The next GC did the close() as part of its cleanup work. The fix would be to find the InputStream and explicitly close it.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Cannot cleanly undeploy a web application
Posted by André Warnier <aw...@ice-sa.com>.
Mark Eggers wrote:
> Folks,
>
> Off topic, so I'm going to be burning a little karma (OK, a lot).
>
> Environment:
>
> Window 7 64 bit
> JRE 1.7.0_25
> Tomcat 7.0.42 (with tcnative) run from startup.bat
>
> I have a web application that scans for a resource path using:
>
> this.getClass().getClassLoader().getResources(resourcePath)
>
> It then wanders through the resources, checking the protocol for each
> URL in the returned enumeration.
>
> Based on the protocol, the URL is scanned for desired information and
> the names of those resources are collected.
>
> For a JAR file, I use the following code (don't shoot me - I'm an admin
> / architect, not a developer).
>
> public ArrayList<String> getResources(String resourcePath,
> String ending) {
> JarURLConnection urlCon = null;
> Pattern p =
> Pattern.compile("^" + resourcePath + "/.+(" + ending + ")$");
> try {
> urlCon = (JarURLConnection) url.openConnection();
> urlCon.setUseCaches(false);
> JarFile jar = urlCon.getJarFile();
> Enumeration<JarEntry> je = jar.entries();
> while (je.hasMoreElements()) {
> String entry = je.nextElement().getName();
> if (log.isDebugEnabled()) {
> log.debug("JAR entry: " + entry);
> }
> Matcher m = p.matcher(entry);
> if (m.matches()) {
> resources.add(entry);
> }
> }
> } catch (IOException ex) {
> resources.add("Could not open resource");
> if (log.isErrorEnabled()) {
> log.error("Could not open resource",ex);
> }
> } finally {
> if (null != urlCon) {
> urlCon = null;
> }
> }
> return this.resources;
> }
>
> Finally,
>
> this.getClass().getClassLoader().getResourceAsStream()
>
> is used to obtain the resources, process them, and stuff them into a
> HashMap managed by a singleton.
>
> This is all done via a servlet context listener, and the HashMap is
> emptied in the listener's contextDestroyed method.
>
> On Linux, this all works well. I can deploy and undeploy with abandon.
> The Tomcat manager reports no leaks.
>
> On Windows, I cannot cleanly undeploy. The JAR files containing the
> resourcePath (and hence the resources) remain in
> %CATALINA_HOME%\webapps\appname.
>
> However, once even a minor garbage collection occurs I can then cleanly
> undeploy the web application.
>
> I have several solutions:
>
> 1. context.xml antiResourceLocking="true"
>
> This works, but there are a lot of downsides ( noted in the
> documentation). This also indicates that an underlying InputStream is
> being held open when parsing the JAR file. The solution is also Tomcat -
> specific.
>
> 2. Never package the desired resources in a JAR
>
> This works, but developers may balk. Also, the path to the desired
> resources must be unique, otherwise you could still trigger loading from
> a JAR (although I could just not handle JAR scanning).
>
> 3. Copy the JAR file to java.io.tmpdir, extract, and read
>
> This is ugly, but might work. It would impact the application start up
> time even more, but the JAR scanning issue would no longer exist. I
> think Glassfish does this.
>
> 4. Use a properties file
>
> Read in a properties file with a list of desired resources, and use that
> list to populate the singleton. This is actually what I had in place
> before, but this then requires developers to manage both the resources
> and the list.
>
> This solution also makes developing a modular, platform-independent
> build system more difficult (it can be done, just creates an extra step
> using Maven). Right now the build is a convoluted Ant script that
> requires a specific environment. I'm trying to get away from that.
>
> 5. Do this right
>
> How, I'm not exactly sure. Hence, the question.
>
I'll go a bit on a limb here, so someone may shoot me too and you won't be alone. (*)
It's also [OT][OT], just to provide some background information.
I believe that the underlying issue here is the difference in behaviour between Windows
and Unix/Linux, with regard to a file opened by one process, when another process tries to
delete the file.
Unix/Linux allow this. The file entry in the directory is deleted, the process deleting it
gets no error. The file itself is not deleted, and the process which has it open still has
a handle to the file and can continue to access it.
Under Windows however, the process trying to delete the file gets an error, at the very
moment of the "delete".
What probably happens here, is that one level below the InputStream which holds the
filehandle in Java, is some native file object which has the file open. The InputStream
object is discarded at tbe Java level, but is only really destroyed at the next GC. In the
meantime, the underlying native object still holds the file. When the GC happens, the
underlying filehandle is closed, and everyone is happy.
(*) I don't see why they would though. I'm pretty impressed by your admin/architect code
above, and wish I was as fluent in Java as you seem to be.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: [OT] Cannot cleanly undeploy a web application
Posted by Konstantin Preißer <ve...@t-online.de>.
Hi,
> From: André Warnier [mailto:aw@ice-sa.com]
> Subject: Re: [OT] Cannot cleanly undeploy a web application
>
> I believe that the underlying issue here is the difference in behaviour between Windows
> and Unix/Linux, with regard to a file opened by one process, when another process tries to
> delete the file.
> Unix/Linux allow this. The file entry in the directory is deleted, the process deleting it
> gets no error. The file itself is not deleted, and the process which has it open still has
> a handle to the file and can continue to access it.
> Under Windows however, the process trying to delete the file gets an error, at the very
> moment of the "delete".
I just wanted to add something:
The Unix behavior you describe is also possible under Windows. It depends on the "file share" flag that is used to open a file.
I don't know how this works in the native WinAPI, but if you try the following C# code for example (.Net) to read from a (big) file:
using (Stream s = new FileStream("C:\\MyBigFile.bin", FileMode.Open, FileAccess.Read, FileShare.Delete | FileShare.Read)) {
byte[] buf = new byte[128];
int read;
while ((read = s.Read(buf, 0, buf.Length)) > 0) {
Console.WriteLine("Read: " + read + " bytes.");
Thread.Sleep(2000); // wait 2 seconds
}
}
Then you can delete the file while it is still opened and the program reads from it. If you look at the free space of the drive, you will see that the space that the file uses is not yet available. As soon as you close the process / cancel the reading, the space will become available.
It seems that most programs don't open a file that way - that is why an error occurs when trying to delete such a file. However, I don't know why most programs (or a Java InputStream etc.) open a file for reading without specifying such a file share flag (E.g., in the above code example, if you use a FileStream constructor that doesn't specify a FileShare flag, the file cannot be deleted from other processes while it is opened). Maybe it is a design principle in Windows to not be able to delete a opened file by default, so that a user doesn't accidentally delete a file on which he is still working... but I don't know.
Regards,
Konstantin Preißer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org