Beginner problems with solr.ICUCollationField

[Björn Keil <gr...@yahoo.de>]

Hello,

I am trying to understand solr 5.1 (trying to overcome some problems I
have with solr 3.6) by experimenting with the distributed package, but I
am having problems using a "solr.ICUCollationField" field.

Trying to create the collection using
bin/create_collection aborts, because it says it does the classloader
failed to find "solr.ICUCollationField".

The documentations says:
###### QUOTE #######
solr.ICUCollationField is included in the Solr analysis-extras contrib -
see solr/contrib/analysis-extras/README.txt for instructions on which
jars you need to add to your SOLR_HOME/lib in order to use it."
####################

The mentioned README.txt file says:
###### QUOTE #######
ICU relies upon lucene-libs/lucene-analyzers-icu-X.Y.jar
and lib/icu4j-X.Y.jar
####################

Well, that's a bid odd in so far as there is no SOLR_HOME/lib directory.
When I start solr with the "verbose" flag it says:

SOLR_HOME = (...)/solr-5.1.0/server/solr

However, what I did is first symlink and then copy the respective
libraries to solr-5.1.0/server/lib/ext. That leads to the server not
starting at all. I also tried
solr-5.1.0/server/solr-webapp/webapp/WEB-INF/lib, but that does not make
any difference at all.

Then I had a look at the example configurations how it's done there and
notices the <lib> tags in the respective solrconfig.xml files. But the
process still fails, even though the logs indicate that the .jar files
*did* load.

There is a message:
201946 [qtp1055930828-14] INFO  org.apache.solr.core.SolrConfig
[booklooker shard2  booklooker_shard2_replica1] – Adding specified lib
dirs to ClassLoader
201947 [qtp1055930828-14] INFO  org.apache.solr.core.SolrResourceLoader
 [booklooker shard2  booklooker_shard2_replica1] – Adding
'file:/home/bjoern/solr-5.1.0/contrib/analysis-extras/lib/icu4j-54.1.jar' t
o classloader
201948 [qtp1055930828-14] INFO  org.apache.solr.core.SolrResourceLoader
 [booklooker shard2  booklooker_shard2_replica1] – Adding
'file:/home/bjoern/solr-5.1.0/contrib/analysis-extras/lucene-libs/lucene-an
alyzers-icu-5.1.0.jar' to classloader

and later:
202810 [qtp1055930828-14] ERROR org.apache.solr.core.CoreContainer
[booklooker shard2  booklooker_shard2_replica1] – Error creating core
[booklooker_shard2_replica1]: (...) Caused by:
java.lang.ClassNotFoundException: solr.ICUCollationField

So the respective jar files have been added to the class loader, but it
does not find the field? Well, the class solr.ICUCollationField itself
should be found somewhere in the org.apache.solr tree. Not in the
org.apache.lucene tree and certainly not under com.ibm.icu4j. But I
can't find the proper jar file for it...

It would nice if someone could tell me what's wrong here.

Re: Beginner problems with solr.ICUCollationField

[Shawn Heisey <ap...@elyograg.org>]

On 5/13/2015 4:16 AM, Björn Keil wrote:
> Thanks you for your help. That was only part of the problem, though.You also need ${solr.install.dir}/dist/solr-analysis-extras-X.jar
> where "X" is the version.
> 
> The other two libraries are dependencies, but the do not contain the actual ICUCollationField class. It might be helpful if that was mentioned in the respective spots in the documentation and README.txt file.

I have not used that particular class.  I have used the ICU tokenizers
and filters, which are in the lucene jar.

The docs you quoted say this:

-----------
solr.ICUCollationField is included in the Solr analysis-extras contrib
see solr/contrib/analysis-extras/README.txt for instructions on which
jars you need to add to your SOLR_HOME/lib in order to use it."
-----------

That sounds to me like an indication that you need the solr analysis
extras jar, which has the lucene-analyzers and icu4j jars as additional
dependencies.  The referenced README probably should mention that the
required jar can be found in the dist/ folder of the binary download.

Thanks,
Shawn

Re: Beginner problems with solr.ICUCollationField

[Björn Keil <gr...@yahoo.de>]

Thanks you for your help. That was only part of the problem, though.You also need ${solr.install.dir}/dist/solr-analysis-extras-X.jar
where "X" is the version.

The other two libraries are dependencies, but the do not contain the actual ICUCollationField class. It might be helpful if that was mentioned in the respective spots in the documentation and README.txt file.

Re: Beginner problems with solr.ICUCollationField

[Shawn Heisey <ap...@elyograg.org>]

On 5/12/2015 7:03 AM, Björn Keil wrote:
> Well, that's a bid odd in so far as there is no SOLR_HOME/lib
> directory. When I start solr with the "verbose" flag it says:
>
> SOLR_HOME = (...)/solr-5.1.0/server/solr
>
> However, what I did is first symlink and then copy the respective
> libraries to solr-5.1.0/server/lib/ext. That leads to the server not
> starting at all. I also tried
> solr-5.1.0/server/solr-webapp/webapp/WEB-INF/lib, but that does not
> make any difference at all.

The ${solr.solr.home}/lib directory does not exist by default in the
Solr example, but if you create it and use it for all your contrib/user
jars that your Solr config needs, it will work.  You should completely
remove all <lib> config elements from solrconfig.xml at the same time,
and make sure that any jar you need is in that lib directory.  All the
jars will be loaded once and available to all cores.

There seems to be some kind of problem with the classloader when certain
jars (the ICU jars being the one example I'm sure about) are loaded more
than once by the same classloader.

https://issues.apache.org/jira/browse/SOLR-4852

Thanks,
Shawn