ClassLoader File Locking Issues Fixed

[Andy <an...@orprovision.com>]

Fixed (Possibly a Windows only issue) ClassLoader jar file locking issues which were caused by several factors.

1. ClassLoaderUtil was using wrong key to remove and close cached jar files, so added a first (most common) and 
fall-back strategy to create a valid removal key. Also added private method to close jar file loaders.

2. DeploymentLoader was not actually using the temp files it created to create the lookup ClassLoader.

3. DeploymentLoader was trying to delete temp jar files before jar file references had been removed.

The order of action is:

Create temporary ClassLoader.
Destroy temporary ClassLoader.
System.gc() - required to ensure release.
Delete temporary files.
Create application ClassLoader.
Deploy application.
Undeploy application.
Destroy application ClassLoader.
System.gc() - required to ensure release.

All files are now unlocked (and can be deleted) as expected. Previously it was necessary to restart OpenEJB to redeploy 
an [application].jar with the same name.

Andy.

PS. Revision: 1041646 breaks due to missing library?

Re: ClassLoader File Locking Issues Fixed

[AndyG <an...@orprovision.com>]

The close is safe as long as the key entry is removed at the same time - and
that the removal is synchronized on the factory.
I have seen several implementations that close the jar but then leave the
closed jar in the map, and that is very 'game over'.
I have looked at the Java source to try and ensure that what I am doing is
safe, and have tested a cycling redeployment over several hours.

Based on some of the original code comments and existing code I believe that
all I have done is to 'fix' and enhance the originally intended
functionality that was broken.
In particular in ClassLoaderUtil.clearSunJarFileFactoryCacheImpl:

JarFile jarFile = (JarFile) fileCache.remove(url);

The old line can return null because the key is not always a URL (At least
since JDK 1.6.0), it is now a String that starts with 'file:///', but I
think there was a period where either a URL or a String was being used as a
key (pre generics), and this is what led to intermittent behaviour.

On Windows URI or URL will return (toString, toExternalForm etc.)
'file://C:/blah.jar' and it needs to be 'file:///C:/blah.jar' (note the
extra slash).
The JarFileFactory fudges the key together internally, and I have changed
the code to do the same. The first fall back option is to use the toString.

To that end, the JarFile was not always being removed and closed, and was
'sometimes' locked on Windows systems, dependent on the JDK version.
On Unix like systems it has nearly always been possible to delete or replace
these 'used' jar files without issue because a file lock is not obtained
until a physical read operation occurs.

The 'temp file' scenario (Revision: 637617) seems to have only been
developed due to these unresolved file locking issues in the past (most
likely caused by this very issue), and I feel that after my tests is
actually no longer required. I left it in because I did not want to make too
many changes to this very 'touchy' area, and it is also very safe to do it
this way.

My previous modification in this class was to address the
ConcurrentModificationException, which was simply caused by an
unsynchronized iteration over the map. What I had not noticed was that the
JarFile was null, but I hadn't actually been looking for it until now.

Good that you forced me to rethink over this, as I have also now just added
the pre JDK 1.6 fallback, which is to still use the URL... Doh!

Also good to know that JDK 7 has actually got a ClassLoader.close method,
let's just hope that they get it right!

Andy.
-- 
View this message in context: http://openejb.979440.n4.nabble.com/ClassLoader-File-Locking-Issues-Fixed-tp3071131p3077795.html
Sent from the OpenEJB Dev mailing list archive at Nabble.com.

Re: ClassLoader File Locking Issues Fixed

[Mark Struberg <st...@yahoo.de>]

andy, if you use tomcat and define your own <Context> sections in server.xml, then make sure that you use the 

<Context antiJARLocking="true"

option.

Otherwise you will might get jar locks if you use manual resource loading from jars in your code. I had this while serving JSF facelets from a shared jar...

LieGrue,
strub

--- On Fri, 12/3/10, David Blevins <da...@visi.com> wrote:

> From: David Blevins <da...@visi.com>
> Subject: Re: ClassLoader File Locking Issues Fixed
> To: dev@openejb.apache.org, andy.gumbrecht@orprovision.com
> Date: Friday, December 3, 2010, 5:13 PM
> On Fri, Dec 3, 2010 at 6:38 AM, Andy
> <an...@orprovision.com>
> wrote:
> > Fixed (Possibly a Windows only issue) ClassLoader jar
> file locking issues
> > which were caused by several factors.
> >
> > 1. ClassLoaderUtil was using wrong key to remove and
> close cached jar files,
> > so added a first (most common) and fall-back strategy
> to create a valid
> > removal key. Also added private method to close jar
> file loaders.
> 
> Careful with the jar file close technique.  I did some
> experimentation
> with that a few years ago and I seem to recall that closing
> it means
> it can never again be used.  The form of the "never
> again" I can't
> remember.  We might have issues though in embedded
> environments where
> we don't really control the classloaders.
> 
> Anyway, something to check into.
> 
> I know we had this working at one point.  Grab the 3.0
> final binary
> and try a redeploy on your windows machine and see if that
> works.
> Might be we have added something in between then and now
> that is
> getting in the way of windows redeploy working.
> 
> 
> -David
> 


      

Re: ClassLoader File Locking Issues Fixed

[David Blevins <da...@visi.com>]

On Fri, Dec 3, 2010 at 6:38 AM, Andy <an...@orprovision.com> wrote:
> Fixed (Possibly a Windows only issue) ClassLoader jar file locking issues
> which were caused by several factors.
>
> 1. ClassLoaderUtil was using wrong key to remove and close cached jar files,
> so added a first (most common) and fall-back strategy to create a valid
> removal key. Also added private method to close jar file loaders.

Careful with the jar file close technique.  I did some experimentation
with that a few years ago and I seem to recall that closing it means
it can never again be used.  The form of the "never again" I can't
remember.  We might have issues though in embedded environments where
we don't really control the classloaders.

Anyway, something to check into.

I know we had this working at one point.  Grab the 3.0 final binary
and try a redeploy on your windows machine and see if that works.
Might be we have added something in between then and now that is
getting in the way of windows redeploy working.


-David