Creating new Solr cores using relative paths

[Jay Hill <ja...@gmail.com>]

I'm having trouble getting the core CREATE command to work with relative
paths in the solr.xml configuration.

I'm working with a layout like this:
/opt/solr [this is solr.solr.home: $SOLR_HOME]
/opt/solr/solr.xml
/opt/solr/core0/ [this is the "template" core]
/opt/solr/core0/conf/schema.xml [etc.]

/opt/tomcat/bin [where tomcat is started from: $TOMCAT_HOME/bin]

My very basic solr.xml:
<solr persistent="true">
<cores adminPath="/admin/cores">
  <core name="core0" instanceDir="core0/"/>
</cores>
</solr>

The CREATE core command works fine with absolute paths, but I have a
requirement to use relative paths. I want to be able to create a new core
like this:

http://localhost:8080/solr/admin/cores
?action=CREATE
&name=core1
&instanceDir=core1
&config=core0/conf/solrconfig.xml
&schema=core0/conf/schema.xml
(core1 is the name for the new core to be created, and I want to use the
config and schema from core0 to create the new core).

but the error is always due to the servlet container thinking
$TOMCAT_HOME/bin is the current working directory:
Caused by: java.lang.RuntimeException: *Can't find resource
'core0/conf/solrconfig.xml'* in classpath or '/opt/solr/core1/conf/', *
cwd=/opt/tomcat/bin
*
Does anyone know how to make this happen?

Thanks,
-Jay

Re: Creating new Solr cores using relative paths

[Jan Høydahl / Cominvent <ja...@cominvent.com>]

Yes, this is really a pain sometimes.

I'd prefer a well defined base path, which could be assumed everywhere unless otherwise documented.
SolrHome is one natural choice. For backward compat we could add a config in solr(config).xml to easily switch to old behaviour.

Also, it makes sense to define some pre-initialized variables which could be used everywhere:
${pwd} - startup directory of Tomcat
${core.name} - name of current core
${zk.home} - root of ZooKeeper config tree (or perhaps simply a zk:// notation?)

Another feature which is pretty useless without a stable CWD is the XInclude syntax to embed XML snippets into solrconfig.xml. Since it does not support variable substitution, without a stable base path it's very hard to use.

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Training in Europe - www.solrtraining.com

On 27. aug. 2010, at 00.47, Chris Hostetter wrote:

> 
> : http://localhost:8080/solr/admin/cores
> : ?action=CREATE
> : &name=core1
> : &instanceDir=core1
> : &config=core0/conf/solrconfig.xml
> : &schema=core0/conf/schema.xml
> : (core1 is the name for the new core to be created, and I want to use the
> : config and schema from core0 to create the new core).
> : 
> : but the error is always due to the servlet container thinking
> : $TOMCAT_HOME/bin is the current working directory:
> : Caused by: java.lang.RuntimeException: *Can't find resource
> : 'core0/conf/solrconfig.xml'* in classpath or '/opt/solr/core1/conf/', *
> : cwd=/opt/tomcat/bin
> 
> 
> A major pain point is the Solr code base is that various pieces of code 
> assume paths are relative to varoius other paths -- frequently the CWD -- 
> and it's not always clear what the right "fix" is (ie: when should it be 
> the SolrHome dire? when should it be the instanceDir for the current core? 
> when should it be the conf dir for hte current core? etc...)
> 
> Even if we were startig from scratch right now, it's not clear how paths 
> like the "schema" and "Config" params should be interpreted in a CREATE 
> command (should they be relative the SolrHome? or relative the 
> "instanceDir" param?)
> 
> Even in cases where it should be obvious, and it would be nice to "fix" 
> these paths, it could easily break things for existing users with 
> code/configs that have come to expect the particular excentricities (in 
> "read only" cases, we can check multiple paths, but that doesn't work for 
> files/dirs that Solr is being asked to create)
> 
> The easiest workarround: configure tomcat to use your SolrHome dir (ie: 
> "/opt/solr" in your example) as it's Working Directory.
> 
> -Hoss
> 
> --
> http://lucenerevolution.org/  ...  October 7-8, Boston
> http://bit.ly/stump-hoss      ...  Stump The Chump!
> 

Re: Creating new Solr cores using relative paths

[Chris Hostetter <ho...@fucit.org>]

: http://localhost:8080/solr/admin/cores
: ?action=CREATE
: &name=core1
: &instanceDir=core1
: &config=core0/conf/solrconfig.xml
: &schema=core0/conf/schema.xml
: (core1 is the name for the new core to be created, and I want to use the
: config and schema from core0 to create the new core).
: 
: but the error is always due to the servlet container thinking
: $TOMCAT_HOME/bin is the current working directory:
: Caused by: java.lang.RuntimeException: *Can't find resource
: 'core0/conf/solrconfig.xml'* in classpath or '/opt/solr/core1/conf/', *
: cwd=/opt/tomcat/bin


A major pain point is the Solr code base is that various pieces of code 
assume paths are relative to varoius other paths -- frequently the CWD -- 
and it's not always clear what the right "fix" is (ie: when should it be 
the SolrHome dire? when should it be the instanceDir for the current core? 
when should it be the conf dir for hte current core? etc...)

Even if we were startig from scratch right now, it's not clear how paths 
like the "schema" and "Config" params should be interpreted in a CREATE 
command (should they be relative the SolrHome? or relative the 
"instanceDir" param?)

Even in cases where it should be obvious, and it would be nice to "fix" 
these paths, it could easily break things for existing users with 
code/configs that have come to expect the particular excentricities (in 
"read only" cases, we can check multiple paths, but that doesn't work for 
files/dirs that Solr is being asked to create)

The easiest workarround: configure tomcat to use your SolrHome dir (ie: 
"/opt/solr" in your example) as it's Working Directory.

-Hoss

--
http://lucenerevolution.org/  ...  October 7-8, Boston
http://bit.ly/stump-hoss      ...  Stump The Chump!