You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by "Dickson, Matt MR" <ma...@defence.gov.au> on 2019/09/16 03:02:13 UTC
Virtual Classpath Issue [SEC=UNOFFICIAL]
UNOFFICIAL
I have a question about Accumulo and classloading. We have an iterator that does not run very often, sometimes 1 week might go by without it running. When it does run after a period of inactivity we get an exception:
Java.lang.ClassNotFoundException: ...BlahIterator
at org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:184)
...
Caused by: org.apache.commons.vfs2.provider.DefaultFileContent.getAttributes(DefaultFileContent.java:257)
...
Caused by: java.lang.NullPointerException
at org.apache.commons.vfs2.provider.jar.JarFileSystem.getAttributes(JarFileSystem:91)
...
It looks as though the class is aged out of a cache, but then fails to load it back into the cache for some reason. The classes are all located in HDFS and we did not get this problem before moving to a virtual classpath (VFS). The classes used to be local on each tablet server.
Is there some way to stop it unloading the class from the cache? The only way we have to fix this right now is to re-start Accumulo across the whole cluster.
Re: Virtual Classpath Issue [SEC=UNOFFICIAL]
Posted by Dave Marion <dl...@comcast.net>.
I agree. My guess is that it's a combination of:
1. JVM Class Unloading is enabled
2. Default general.vfs.cache.dir property value
3. tmpwatch, tmpfiles-clean (CentOS 7) or some other process cleaning up /tmp.
For (3) you could configure it to skip the vfs cache directory or disable., For (2) you could specify a different directory.
> On September 16, 2019 at 1:02 AM James Srinivasan <ja...@gmail.com> wrote:
>
> I can't tell without the full stack trace, but we had an issue when tmpwatch cleaned up /tmp which was where the VFS classloader cached jars by default. Might that be your issue?
>
> On Mon, 16 Sep 2019, 04:02 Dickson, Matt MR, < matt.dickson@defence.gov.au mailto:matt.dickson@defence.gov.au > wrote:
>
> > >
> > UNOFFICIAL
> >
> >
> > I have a question about Accumulo and classloading. We have an iterator that does not run very often, sometimes 1 week might go by without it running. When it does run after a period of inactivity we get an exception:
> >
> >
> >
> > Java.lang.ClassNotFoundException: …BlahIterator
> >
> > at org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:184)
> >
> > …
> >
> > Caused by: org.apache.commons.vfs2.provider.DefaultFileContent.getAttributes(DefaultFileContent.java:257)
> >
> > …
> >
> > Caused by: java.lang.NullPointerException
> >
> > at org.apache.commons.vfs2.provider.jar.JarFileSystem.getAttributes(JarFileSystem:91)
> >
> > …
> >
> >
> >
> > It looks as though the class is aged out of a cache, but then fails to load it back into the cache for some reason. The classes are all located in HDFS and we did not get this problem before moving to a virtual classpath (VFS). The classes used to be local on each tablet server.
> >
> >
> >
> > Is there some way to stop it unloading the class from the cache? The only way we have to fix this right now is to re-start Accumulo across the whole cluster.
> >
> >
> >
> >
> >
> > >
Re: Virtual Classpath Issue [SEC=UNOFFICIAL]
Posted by James Srinivasan <ja...@gmail.com>.
I can't tell without the full stack trace, but we had an issue when
tmpwatch cleaned up /tmp which was where the VFS classloader cached jars by
default. Might that be your issue?
On Mon, 16 Sep 2019, 04:02 Dickson, Matt MR, <ma...@defence.gov.au>
wrote:
>
>
> *UNOFFICIAL *
>
> I have a question about Accumulo and classloading. We have an iterator
> that does not run very often, sometimes 1 week might go by without it
> running. When it does run after a period of inactivity we get an exception:
>
>
>
> Java.lang.ClassNotFoundException: …BlahIterator
>
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:184)
>
> …
>
> Caused by:
> org.apache.commons.vfs2.provider.DefaultFileContent.getAttributes(DefaultFileContent.java:257)
>
> …
>
> Caused by: java.lang.NullPointerException
>
> at
> org.apache.commons.vfs2.provider.jar.JarFileSystem.getAttributes(JarFileSystem:91)
>
> …
>
>
>
> It looks as though the class is aged out of a cache, but then fails to
> load it back into the cache for some reason. The classes are all located
> in HDFS and we did not get this problem before moving to a virtual
> classpath (VFS). The classes used to be local on each tablet server.
>
>
>
> Is there some way to stop it unloading the class from the cache? The only
> way we have to fix this right now is to re-start Accumulo across the whole
> cluster.
>
>
>
>
>