Deployment with LUCENE-2899 patch can't load solr.OpenNLPTokenizerFactory

[Sean Glover <wl...@gmail.com>]

Hi,

Platform: ubuntu 12.04
Package: apache-solr-4.0-2012-07-07_11-55-05-src.tgz
Web: Apache Tomcat/7.0.26

I'm trying to use the LUCENE-2899 patch
(https://issues.apache.org/jira/browse/LUCENE-2899).  As an end-user I
believe this is the correct list to post to.

I'm new to Solr, so I started by successfully deploying the
solr/example project to tomcat7.

To deploy a 2nd instance of solr that uses the OpenNLP configuration I
performed the following steps as per the OpenNLP wiki page
(http://wiki.apache.org/solr/OpenNLP).

ant compile
cd solr/contrib/opennlp/src/test-files/training
run 'bin/trainall.sh'
run ant-testcontrib

All these tasks executed successfully.  Then I attempted to actually
deploy my Solr w/ OpenNLP instance with the following steps.

Downloaded real models from http://opennlp.sourceforge.net/models-1.5/
(except for the content of coref, do I need to get this?)
copied my solr-example deployment to create a solr-nlp deployment
copied opennlp config to my deployment config
 source: solr/contrib/opennlp/src/test-files/opennlp/solr/collection1/conf
 dest: /var/tomcat/solr/nlp/solr/collection1/conf
copied opennlp libs to my deployment libs
- source: solr/contrib/opennlp/lib
- dest: /var/tomcat/solr/nlp/solr/collection1/lib
updated my deployed solrconfig.xml
- set dataDir: <dataDir>${solr.data.dir:/var/tomcat/solr/nlp/solr/collection1/data}</dataDir>
 - an absolute path was recommended by the Tomcat7 Solr deployment guide
added a lib: <lib dir="/var/tomcat/solr/nlp/solr/collection1/lib"
regex=".*\.jar"/>
- again, i specified an absolute path so tomcat would know exactly
where to load the opennlp libs from

When I attempt to hit my NLP instance of solr I get the following
error (http://localhost:8080/solr-nlp/admin)

"This interface requires that you activate the admin request handlers,
add the following configuration to your solrconfig.xml:"

However, I have the admin requestHandler defined exactly as requested.
 Is this a catch-all error of some sort?

When I dig a little deeper and look at the Catalina logs I found an
error and stack trace.

SEVERE: null:org.apache.solr.common.SolrException: Plugin init failure
for [schema.xml] fieldType "text_opennlp": Plugin init failure for
[schema.xml] analyzer/tokenizer: Error loading class
'solr.OpenNLPTokenizerFactory'
        at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:168)
        at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:364)
        at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:111)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:816)
        at org.apache.solr.core.CoreContainer.load(CoreContainer.java:514)
        at org.apache.solr.core.CoreContainer.load(CoreContainer.java:335)
        at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:284)
        at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:106)
        at org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:277)
        at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:258)
        at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:382)
        at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:103)
        at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4638)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5294)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:895)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:871)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615)
        at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649)
        at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1581)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.solr.common.SolrException: Plugin init failure
for [schema.xml] analyzer/tokenizer: Error loading class
'solr.OpenNLPTokenizerFactory'
        at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:168)
        at org.apache.solr.schema.FieldTypePluginLoader.readAnalyzer(FieldTypePluginLoader.java:322)
        at org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:95)
        at org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:43)
        at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:142)
        ... 25 more
Caused by: org.apache.solr.common.SolrException: Error loading class
'solr.OpenNLPTokenizerFactory'
        at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:415)
        at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:430)
        at org.apache.solr.util.plugin.AbstractPluginLoader.create(AbstractPluginLoader.java:86)
        at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:142)
        ... 29 more
Caused by: java.lang.ClassNotFoundException: solr.OpenNLPTokenizerFactory
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
        at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:787)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:399)
        ... 32 more

It looks as though it can't find the solr.OpenNLPTokenizerFactory
referenced by my schema.xml.  But in the Catalina log I can see that
it's picking up the OpenNLP lib jar's successfully.

Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/opennlp-tools-1.5.2-incubating.jar'
to classloader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/opennlp-tools-1.5.2-incubating.jar.sha1'
to classloader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/opennlp-maxent-3.0.2-incubating.jar'
to classloader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/jwnl-1.4_rc3-LICENSE-BSD_LIKE.txt'
to classloader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/opennlp-maxent-3.0.2-incubating-LICENSE-ASL.txt'
to classloader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/jwnl-1.4_rc3-NOTICE.txt'
to classloader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/opennlp-tools-1.5.2-incubating-LICENSE-ASL.txt'
to classloader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/opennlp-tools-1.5.2-incubating-NOTICE.txt'
to classloader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/opennlp-maxent-3.0.2-incubating.jar.sha1'
to classloader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/jwnl-1.4_rc3.jar.sha1'
to classloader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/jwnl-1.4_rc3.jar'
to classloader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/opennlp-maxent-3.0.2-incubating-NOTICE.txt'
to classloader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrConfig initLibs
INFO: Adding specified lib dirs to ClassLoader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/opennlp-tools-1.5.2-incubating.jar'
to classloader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/opennlp-maxent-3.0.2-incubating.jar'
to classloader
Jul 08, 2012 4:53:22 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding 'file:/var/tomcat/solr/nlp/solr/collection1/lib/jwnl-1.4_rc3.jar'
to classloader

Please advise on how I can troubleshoot this further.

Regards,

Sean