You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by Grant Ingersoll <gs...@apache.org> on 2008/11/14 23:24:25 UTC

Classloader and SolrResourceLoader fun

I'm working on SOLR-282.  I'm building it out as a contrib.  The  
contrib has some source code implementing an ExtractingReqHandler  
(ERH) and the Tika library.  For testing, I copied the JAR file I  
create for the contrib and the Tika library into solr/lib and fire up  
the example, which leads to a NoClassDefFound exception (see [1])  
caused by the classloader that is loading the ERH failing to find the  
Tika dependencies, even though the Tika library is in the lib directory.

So, in debugging this, I noticed that we construct two different  
SolrResourceLoaders, one right after the other in the CoreContainer at  
lines 117 and 118.  Well, we create a SRL at 117, then the SolrConfig  
creates one in it's constructor, which gets called at 118.

I'm not sure if this is the problem, but it leads to [2] in my log  
files, which seems really weird.

Anyone with some classloader magic knowledge have any insight?

I also tried merging my contrib jar and the tika jar together, but  
that didn't resolve anything.

Thanks,
Grant

[1]
Nov 14, 2008 5:10:29 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.NoClassDefFoundError: org/apache/tika/exception/ 
TikaException
         at java.lang.Class.forName0(Native Method)
         at java.lang.Class.forName(Class.java:247)
         at  
org 
.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java: 
260)
         at  
org 
.apache 
.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:281)
         at  
org 
.apache 
.solr 
.util.plugin.AbstractPluginLoader.create(AbstractPluginLoader.java:84)
         at org.apache.solr.core.RequestHandlers 
$1.create(RequestHandlers.java:153)
         at org.apache.solr.core.RequestHandlers 
$1.create(RequestHandlers.java:138)
         at  
org 
.apache 
.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java: 
141)
         at  
org 
.apache 
.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java: 
170)
         at org.apache.solr.core.SolrCore.<init>(SolrCore.java:527)
         at org.apache.solr.core.CoreContainer 
$Initializer.initialize(CoreContainer.java:120)
         at  
org 
.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:69)
         at  
org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99)
....
Caused by: java.lang.ClassNotFoundException:  
org.apache.tika.exception.TikaException
         at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:316)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
         at  
org 
.mortbay 
.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:375)
         at  
org 
.mortbay 
.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:337)
         at java.lang.ClassLoader.loadClassInternal(ClassLoader.java: 
374)


[2]  Note, in this case I merged the Tika and the Solr contrib jars  
together:

INFO: Solr home set to 'solr/'
Nov 14, 2008 5:07:29 PM org.apache.solr.core.SolrResourceLoader  
createClassLoader
INFO: Adding 'file:.../solr-clean/example/solr/lib/apache-solr- 
extraction-1.4-dev.jar' to Solr classloader
Nov 14, 2008 5:08:07 PM org.apache.solr.core.SolrResourceLoader  
locateInstanceDir
INFO: JNDI not configured for solr (NoInitialContextEx)
Nov 14, 2008 5:08:07 PM org.apache.solr.core.SolrResourceLoader  
locateInstanceDir
INFO: solr home defaulted to 'solr/' (could not find system property  
or JNDI)
Nov 14, 2008 5:08:07 PM org.apache.solr.core.SolrResourceLoader <init>
INFO: Solr home set to 'solr/'
Nov 14, 2008 5:08:07 PM org.apache.solr.core.SolrResourceLoader  
createClassLoader
INFO: Adding 'file:.../solr-clean/example/solr/lib/apache-solr- 
extraction-1.4-dev.jar' to Solr classloader
Nov 14, 2008 5:10:15 PM org.apache.solr.core.SolrConfig <init>
INFO: Loaded SolrConfig: solrconfig.xml

Re: Classloader and SolrResourceLoader fun

Posted by Chris Hostetter <ho...@fucit.org>.
: Although, it still does seem weird that two SolrResourceLoader classes get
: created.

I think that's because the CoreContainer always has one SolrResourceLoader 
for the main instanceDir (ie: where it finds solr.xml, which might have a 
sharedLib) and then each SolrCore has one.

But that's just a guess ... CoreContainer constructor semantics are kind 
of a mess, it's hard to tell what's going on in there (three public 
constructors, none of which delegate to each other or a common 
initialization method)


-Hoss

Re: Classloader and SolrResourceLoader fun

Posted by Grant Ingersoll <gs...@apache.org>.
Never mind.  Found the problem.  There was a name collision due to  
overlap between using Tika's standalone JAR (which packages all of the  
dependencies into a single JAR) and some Solr deps, such that the  
wrong classloader was being invoked.

Although, it still does seem weird that two SolrResourceLoader classes  
get created.

-Grant

On Nov 14, 2008, at 5:24 PM, Grant Ingersoll wrote:

> I'm working on SOLR-282.  I'm building it out as a contrib.  The  
> contrib has some source code implementing an ExtractingReqHandler  
> (ERH) and the Tika library.  For testing, I copied the JAR file I  
> create for the contrib and the Tika library into solr/lib and fire  
> up the example, which leads to a NoClassDefFound exception (see [1])  
> caused by the classloader that is loading the ERH failing to find  
> the Tika dependencies, even though the Tika library is in the lib  
> directory.
>
> So, in debugging this, I noticed that we construct two different  
> SolrResourceLoaders, one right after the other in the CoreContainer  
> at lines 117 and 118.  Well, we create a SRL at 117, then the  
> SolrConfig creates one in it's constructor, which gets called at 118.
>
> I'm not sure if this is the problem, but it leads to [2] in my log  
> files, which seems really weird.
>
> Anyone with some classloader magic knowledge have any insight?
>
> I also tried merging my contrib jar and the tika jar together, but  
> that didn't resolve anything.
>
> Thanks,
> Grant
>
> [1]
> Nov 14, 2008 5:10:29 PM org.apache.solr.common.SolrException log
> SEVERE: java.lang.NoClassDefFoundError: org/apache/tika/exception/ 
> TikaException
>        at java.lang.Class.forName0(Native Method)
>        at java.lang.Class.forName(Class.java:247)
>        at  
> org 
> .apache 
> .solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:260)
>        at  
> org 
> .apache 
> .solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:281)
>        at  
> org 
> .apache 
> .solr 
> .util.plugin.AbstractPluginLoader.create(AbstractPluginLoader.java:84)
>        at org.apache.solr.core.RequestHandlers 
> $1.create(RequestHandlers.java:153)
>        at org.apache.solr.core.RequestHandlers 
> $1.create(RequestHandlers.java:138)
>        at  
> org 
> .apache 
> .solr 
> .util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:141)
>        at  
> org 
> .apache 
> .solr 
> .core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:170)
>        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:527)
>        at org.apache.solr.core.CoreContainer 
> $Initializer.initialize(CoreContainer.java:120)
>        at  
> org 
> .apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java: 
> 69)
>        at  
> org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99)
> ....
> Caused by: java.lang.ClassNotFoundException:  
> org.apache.tika.exception.TikaException
>        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:316)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>        at  
> org 
> .mortbay 
> .jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:375)
>        at  
> org 
> .mortbay 
> .jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:337)
>        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java: 
> 374)
>
>
> [2]  Note, in this case I merged the Tika and the Solr contrib jars  
> together:
>
> INFO: Solr home set to 'solr/'
> Nov 14, 2008 5:07:29 PM org.apache.solr.core.SolrResourceLoader  
> createClassLoader
> INFO: Adding 'file:.../solr-clean/example/solr/lib/apache-solr- 
> extraction-1.4-dev.jar' to Solr classloader
> Nov 14, 2008 5:08:07 PM org.apache.solr.core.SolrResourceLoader  
> locateInstanceDir
> INFO: JNDI not configured for solr (NoInitialContextEx)
> Nov 14, 2008 5:08:07 PM org.apache.solr.core.SolrResourceLoader  
> locateInstanceDir
> INFO: solr home defaulted to 'solr/' (could not find system property  
> or JNDI)
> Nov 14, 2008 5:08:07 PM org.apache.solr.core.SolrResourceLoader <init>
> INFO: Solr home set to 'solr/'
> Nov 14, 2008 5:08:07 PM org.apache.solr.core.SolrResourceLoader  
> createClassLoader
> INFO: Adding 'file:.../solr-clean/example/solr/lib/apache-solr- 
> extraction-1.4-dev.jar' to Solr classloader
> Nov 14, 2008 5:10:15 PM org.apache.solr.core.SolrConfig <init>
> INFO: Loaded SolrConfig: solrconfig.xml