You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Max Bridgewater <ma...@gmail.com> on 2016/03/29 23:14:37 UTC

Load Resource from within Solr Plugin

HI,

I am facing the exact issue described here:
http://stackoverflow.com/questions/25623797/solr-plugin-classloader.

Basically I'm writing a solr plugin by extending SearchComponent class. My
new class is part of a.jar archive. Also my class depends on a jar b.jar. I
placed both jars in my own folder and declared in it solrconfig.xml with:

<lib dir="/home/bridge/Workspaces/search-webapp/target/WEB-INF/lib/"
regex=".*\.jar" />

I also declared my new component in solrconfig.xml. The component is
invoked correctly up to a point where a class ClassFromB from b.jar
attempts to load a classpath resource personal-words.txt from classpath.

The piece of code in class ClassFromB looks like this:

Thread.currentThread().getContextClassLoader().getResources("personal-words.txt")


Unfortunately, this returns an empty list. Any recommendation?


Thanks,

Max.

Re: Load Resource from within Solr Plugin

Posted by Max Bridgewater <ma...@gmail.com>.
Hi Folks,

Thanks for all the great suggestions. i will try and see which one works
best.
@Hoss: The WEB-INF folder is just in my dev environment. I have a localo
Solr instance and I points it to the target/WEB-INF. Simple convenient
setup for development purposes.

Much appreciated.

Max.

On Wed, Mar 30, 2016 at 4:24 PM, Rajesh Hazari <ra...@gmail.com>
wrote:

> Max,
> Have you looked in External file field which is reload on every hard
> commit,
> only disadvantage of this is the file (personal-words.txt) has to be placed
> in all data folders in each solr core,
> for which we have a bash script to do this job.
>
>
> https://cwiki.apache.org/confluence/display/solr/Working+with+External+Files+and+Processes
>
> Ignore this if this does not meets your requirement.
>
> *Rajesh**.*
>
> On Wed, Mar 30, 2016 at 1:21 PM, Chris Hostetter <hossman_lucene@fucit.org
> >
> wrote:
>
> > :
> > : <lib dir="/home/bridge/Workspaces/search-webapp/target/WEB-INF/lib/"
> > : regex=".*\.jar" />
> >
> > 1) as a general rule, if you have a <lib/> delcaration which includes
> > "WEB-INF" you are probably doing something wrong.
> >
> > Maybe not in this case -- maybe "search-webapp/target" is a completley
> > distinct java application and you are just re-using it's jars.  But 9
> > times out of 10, when people have a  WEB-INF path they are trying to load
> > jars from, it's because they *first* added their jars to Solr's WEB_INF
> > directory, and then when that didn't work they added the path to the
> > WEB-INF dir as a <lib/> ... but now you've got those classes being loaded
> > twice, and you've multiplied all of your problems.
> >
> > 2) let's ignore the fact that your path has WEB-INF in it, and just
> > assume it's some path to somewhere where on disk that has nothing to
> > do with solr, and you want to load those jars.
> >
> > great -- solr will do that for you, and all of those classes will be
> > available to plugins.
> >
> > Now if you wnat to explicitly do something classloader related, you do
> > *not* want to be using Thread.currentThread().getContextClassLoader() ...
> > because the threads that execute everything in Solr are a pool of worker
> > threads that is created before solr ever has a chance to parse your <lib
> > /> directive.
> >
> > You want to ensure anything you do related to a Classloader uses the
> > ClassLoader Solr sets up for plugins -- that's available from the
> > SolrResourceLoader.
> >
> > You can always get the SolrResourceLoader via
> > SolrCore.getSolrResourceLoader().  from there you can getClassLoader() if
> > you really need some hairy custom stuff -- or if you are just trying to
> > load a simple resource file as an InputStream, use openResource(String
> > name) ... that will start by checking for it in the conf dir, and will
> > fallback to your jar -- so you can have a default resource file shipped
> > with your plugin, but allow users to override it in their collection
> > configs.
> >
> >
> > -Hoss
> > http://www.lucidworks.com/
> >
>

Re: Load Resource from within Solr Plugin

Posted by Rajesh Hazari <ra...@gmail.com>.
Max,
Have you looked in External file field which is reload on every hard commit,
only disadvantage of this is the file (personal-words.txt) has to be placed
in all data folders in each solr core,
for which we have a bash script to do this job.

https://cwiki.apache.org/confluence/display/solr/Working+with+External+Files+and+Processes

Ignore this if this does not meets your requirement.

*Rajesh**.*

On Wed, Mar 30, 2016 at 1:21 PM, Chris Hostetter <ho...@fucit.org>
wrote:

> :
> : <lib dir="/home/bridge/Workspaces/search-webapp/target/WEB-INF/lib/"
> : regex=".*\.jar" />
>
> 1) as a general rule, if you have a <lib/> delcaration which includes
> "WEB-INF" you are probably doing something wrong.
>
> Maybe not in this case -- maybe "search-webapp/target" is a completley
> distinct java application and you are just re-using it's jars.  But 9
> times out of 10, when people have a  WEB-INF path they are trying to load
> jars from, it's because they *first* added their jars to Solr's WEB_INF
> directory, and then when that didn't work they added the path to the
> WEB-INF dir as a <lib/> ... but now you've got those classes being loaded
> twice, and you've multiplied all of your problems.
>
> 2) let's ignore the fact that your path has WEB-INF in it, and just
> assume it's some path to somewhere where on disk that has nothing to
> do with solr, and you want to load those jars.
>
> great -- solr will do that for you, and all of those classes will be
> available to plugins.
>
> Now if you wnat to explicitly do something classloader related, you do
> *not* want to be using Thread.currentThread().getContextClassLoader() ...
> because the threads that execute everything in Solr are a pool of worker
> threads that is created before solr ever has a chance to parse your <lib
> /> directive.
>
> You want to ensure anything you do related to a Classloader uses the
> ClassLoader Solr sets up for plugins -- that's available from the
> SolrResourceLoader.
>
> You can always get the SolrResourceLoader via
> SolrCore.getSolrResourceLoader().  from there you can getClassLoader() if
> you really need some hairy custom stuff -- or if you are just trying to
> load a simple resource file as an InputStream, use openResource(String
> name) ... that will start by checking for it in the conf dir, and will
> fallback to your jar -- so you can have a default resource file shipped
> with your plugin, but allow users to override it in their collection
> configs.
>
>
> -Hoss
> http://www.lucidworks.com/
>

Re: Load Resource from within Solr Plugin

Posted by Chris Hostetter <ho...@fucit.org>.
: 
: <lib dir="/home/bridge/Workspaces/search-webapp/target/WEB-INF/lib/"
: regex=".*\.jar" />

1) as a general rule, if you have a <lib/> delcaration which includes 
"WEB-INF" you are probably doing something wrong.

Maybe not in this case -- maybe "search-webapp/target" is a completley 
distinct java application and you are just re-using it's jars.  But 9 
times out of 10, when people have a  WEB-INF path they are trying to load 
jars from, it's because they *first* added their jars to Solr's WEB_INF 
directory, and then when that didn't work they added the path to the 
WEB-INF dir as a <lib/> ... but now you've got those classes being loaded 
twice, and you've multiplied all of your problems.

2) let's ignore the fact that your path has WEB-INF in it, and just 
assume it's some path to somewhere where on disk that has nothing to 
do with solr, and you want to load those jars.

great -- solr will do that for you, and all of those classes will be 
available to plugins.

Now if you wnat to explicitly do something classloader related, you do 
*not* want to be using Thread.currentThread().getContextClassLoader() ... 
because the threads that execute everything in Solr are a pool of worker 
threads that is created before solr ever has a chance to parse your <lib 
/> directive.

You want to ensure anything you do related to a Classloader uses the 
ClassLoader Solr sets up for plugins -- that's available from the 
SolrResourceLoader.

You can always get the SolrResourceLoader via 
SolrCore.getSolrResourceLoader().  from there you can getClassLoader() if 
you really need some hairy custom stuff -- or if you are just trying to 
load a simple resource file as an InputStream, use openResource(String 
name) ... that will start by checking for it in the conf dir, and will 
fallback to your jar -- so you can have a default resource file shipped 
with your plugin, but allow users to override it in their collection 
configs.


-Hoss
http://www.lucidworks.com/

Re: Load Resource from within Solr Plugin

Posted by Ahmet Arslan <io...@yahoo.com.INVALID>.

Hi Max,

Why not implement org.apache.lucene.analysis.util.ResourceLoaderAware?
Existing implementation all load/read text files.

Ahmet

On Wednesday, March 30, 2016 12:14 AM, Max Bridgewater <ma...@gmail.com> wrote:



HI,

I am facing the exact issue described here:
http://stackoverflow.com/questions/25623797/solr-plugin-classloader.

Basically I'm writing a solr plugin by extending SearchComponent class. My
new class is part of a.jar archive. Also my class depends on a jar b.jar. I
placed both jars in my own folder and declared in it solrconfig.xml with:

<lib dir="/home/bridge/Workspaces/search-webapp/target/WEB-INF/lib/"
regex=".*\.jar" />

I also declared my new component in solrconfig.xml. The component is
invoked correctly up to a point where a class ClassFromB from b.jar
attempts to load a classpath resource personal-words.txt from classpath.

The piece of code in class ClassFromB looks like this:

Thread.currentThread().getContextClassLoader().getResources("personal-words.txt")


Unfortunately, this returns an empty list. Any recommendation?


Thanks,

Max.