You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Shawn Heisey <so...@elyograg.org> on 2010/04/05 19:53:16 UTC
including external files in config by corename
Is it possible to access the core name in a config file (such as
solrconfig.xml) so I can include core-specific configlets into a common
config file? I would like to pull in different configurations for
things like shards and replication, but have all the cores otherwise use
an identical config file.
Also, I have been looking for the syntax to include a snippet and
haven't turned anything up yet.
Thanks,
Shawn
Re: including external files in config by corename
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/8/2010 1:15 PM, Chris Hostetter wrote:
> ...i suspect you want something like...
>
> <xi:include href="handlers.xml" xpointer="//requestHandler" />
>
> where handlers.xml looks like...
>
> <anyThingYouWant>
> <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
> <requestHandler name="/update/javabin" class="solr.BinaryUpdateRequestHandler" />
> <requestHandler name="/analysis/document" class="solr.DocumentAnalysisRequestHandler" />
> <requestHandler name="/analysis/field" class="solr.FieldAnalysisRequestHandler" />
> <requestHandler name="/update/csv" class="solr.CSVRequestHandler" startup="lazy" />
> <requestHandler name="/admin/" class="org.apache.solr.handler.admin.AdminHandlers" />
> </anyThingYouWant>
>
The xpointer you mentioned above didn't work. I finally found something
that did, though:
<xi:include
href="/index/solr/config/requestHandlers.xml#xpointer(/*/node())" />
I wouldn't have found this without your help. A thousand thanks.
Shawn
Re: including external files in config by corename
Posted by Chris Hostetter <ho...@fucit.org>.
: I'm having a problem with this idea. It seems that what you include with
: XInclude can only be a single XML element, not "big chunks" as I had hoped.
: If you have more than one, it dies. It took running it through xmllint to
: figure out why, the Solr exception was not informative. This will make it
: choke if included in a single file:
First off: all of the files need to be wellformed XML, or the XML parser
will complain before Solr ever even gets a chance at a good error message.
Second: in order to include the sub-elements of the root tag from your
included file, w/o including the root element itself (ie: so you can have
a bunch of request handlers declared in an XML file with an arbitrary root
tag to make the file well formed) i think you need to use the xpointer
option when doing the XInclude. Here's an example in the XInclude
docs...
http://www.w3.org/TR/xinclude/#fragment-example
...i suspect you want something like...
<xi:include href="handlers.xml" xpointer="//requestHandler" />
where handlers.xml looks like...
<anyThingYouWant>
<requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
<requestHandler name="/update/javabin" class="solr.BinaryUpdateRequestHandler" />
<requestHandler name="/analysis/document" class="solr.DocumentAnalysisRequestHandler" />
<requestHandler name="/analysis/field" class="solr.FieldAnalysisRequestHandler" />
<requestHandler name="/update/csv" class="solr.CSVRequestHandler" startup="lazy" />
<requestHandler name="/admin/" class="org.apache.solr.handler.admin.AdminHandlers" />
</anyThingYouWant>
-Hoss
Re: including external files in config by corename
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/7/2010 9:16 AM, Shawn Heisey wrote:
> On 4/5/2010 8:12 PM, Chris Hostetter wrote:
>> what you cna do however, is have a distinct solrconfig.xml for each
>> core,
>> which is just a thin shell that uses XInclude to include big chunkcs of
>> frequently reused declarations, and some cores can exclude some of thes
>> includes. (ie: turn the problem inside out)
> When I first read this, I didn't really understand it. Now that I am
> trying to actually use XInclude, it makes sense, and the more I think
> about it, the more I like the idea. Thank you!
>
> Shawn
>
I'm having a problem with this idea. It seems that what you include
with XInclude can only be a single XML element, not "big chunks" as I
had hoped. If you have more than one, it dies. It took running it
through xmllint to figure out why, the Solr exception was not
informative. This will make it choke if included in a single file:
<requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
<requestHandler name="/update/javabin"
class="solr.BinaryUpdateRequestHandler" />
<requestHandler name="/analysis/document"
class="solr.DocumentAnalysisRequestHandler" />
<requestHandler name="/analysis/field"
class="solr.FieldAnalysisRequestHandler" />
<requestHandler name="/update/csv" class="solr.CSVRequestHandler"
startup="lazy" />
<requestHandler name="/admin/"
class="org.apache.solr.handler.admin.AdminHandlers" />
Is there any way with XML to group multiple elements for the purposes of
inclusion? Can solrconfig.xml handle multiple <config> elements, or
just the one?
Thanks,
Shawn
Re: including external files in config by corename
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/5/2010 8:12 PM, Chris Hostetter wrote:
> what you cna do however, is have a distinct solrconfig.xml for each core,
> which is just a thin shell that uses XInclude to include big chunkcs of
> frequently reused declarations, and some cores can exclude some of thes
> includes. (ie: turn the problem inside out)
>
When I first read this, I didn't really understand it. Now that I am
trying to actually use XInclude, it makes sense, and the more I think
about it, the more I like the idea. Thank you!
Shawn
Re: including external files in config by corename
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/5/2010 8:43 PM, Mark Miller wrote:
> On 04/05/2010 10:12 PM, Chris Hostetter wrote:
>> : The best you have to work with at the moment is Xincludes:
>> :
>> : http://wiki.apache.org/solr/SolrConfigXml#XInclude
>> :
>> : and System Property Substitution:
>> :
>> : http://wiki.apache.org/solr/SolrConfigXml#System_property_substitution
>>
>> Except that XInclude is a feature of hte XML parser, while property
>> substitution is soemthing Solr does after the XML has been parsed into a
>> DOM -- so you can't have an XInclude of a fle whose name is
>> determined by
>> a property (like the core name
> Didn't suggest he could - just giving him the features he has to work
> with.
>
Thanks for the ideas. I actually have a separate solrconfig.xml for
each core, but I've been wanting to make them all the same except when
testing. I think what I will do is have them include subsections from a
local directory and maintain a central repository of subsections for
each host that will be compared and updated with scripts run by cron.
Shawn
Re: including external files in config by corename
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/5/2010 8:43 PM, Mark Miller wrote:
> On 04/05/2010 10:12 PM, Chris Hostetter wrote:
>> : The best you have to work with at the moment is Xincludes:
>> :
>> : http://wiki.apache.org/solr/SolrConfigXml#XInclude
>> :
>> : and System Property Substitution:
>> :
>> : http://wiki.apache.org/solr/SolrConfigXml#System_property_substitution
>>
>> Except that XInclude is a feature of hte XML parser, while property
>> substitution is soemthing Solr does after the XML has been parsed into a
>> DOM -- so you can't have an XInclude of a fle whose name is
>> determined by
>> a property (like the core name
> Didn't suggest he could - just giving him the features he has to work
> with.
>
I'm finally reaching the point where I have to implement this. I'm
working on replication, and have realized that I need the name of the
core to be substituted into the config snippet after it's incorporated
into the DOM. I searched and could not figure out what to use for
property substitution. For the Java-challenged like me ... is there a
way to get a list of all properties that can be used?
Thanks,
Shawn
Re: including external files in config by corename
Posted by Mark Miller <ma...@gmail.com>.
On 04/05/2010 10:12 PM, Chris Hostetter wrote:
> : The best you have to work with at the moment is Xincludes:
> :
> : http://wiki.apache.org/solr/SolrConfigXml#XInclude
> :
> : and System Property Substitution:
> :
> : http://wiki.apache.org/solr/SolrConfigXml#System_property_substitution
>
> Except that XInclude is a feature of hte XML parser, while property
> substitution is soemthing Solr does after the XML has been parsed into a
> DOM -- so you can't have an XInclude of a fle whose name is determined by
> a property (like the core name
Didn't suggest he could - just giving him the features he has to work with.
--
- Mark
http://www.lucidimagination.com
Re: including external files in config by corename
Posted by Chris Hostetter <ho...@fucit.org>.
: The best you have to work with at the moment is Xincludes:
:
: http://wiki.apache.org/solr/SolrConfigXml#XInclude
:
: and System Property Substitution:
:
: http://wiki.apache.org/solr/SolrConfigXml#System_property_substitution
Except that XInclude is a feature of hte XML parser, while property
substitution is soemthing Solr does after the XML has been parsed into a
DOM -- so you can't have an XInclude of a fle whose name is determined by
a property (like the core name)
what you cna do however, is have a distinct solrconfig.xml for each core,
which is just a thin shell that uses XInclude to include big chunkcs of
frequently reused declarations, and some cores can exclude some of thes
includes. (ie: turn the problem inside out)
-Hoss
Re: including external files in config by corename
Posted by Mark Miller <ma...@gmail.com>.
On 04/05/2010 01:53 PM, Shawn Heisey wrote:
> Is it possible to access the core name in a config file (such as
> solrconfig.xml) so I can include core-specific configlets into a
> common config file? I would like to pull in different configurations
> for things like shards and replication, but have all the cores
> otherwise use an identical config file.
>
> Also, I have been looking for the syntax to include a snippet and
> haven't turned anything up yet.
>
> Thanks,
> Shawn
>
The best you have to work with at the moment is Xincludes:
http://wiki.apache.org/solr/SolrConfigXml#XInclude
and System Property Substitution:
http://wiki.apache.org/solr/SolrConfigXml#System_property_substitution
--
- Mark
http://www.lucidimagination.com
Re: including external files in config by corename
Posted by Lance Norskog <go...@gmail.com>.
Making snippets is part of highlighting.
http://www.lucidimagination.com/search/s:lucid/li:cdrg?q=snippet
On Mon, Apr 5, 2010 at 10:53 AM, Shawn Heisey <so...@elyograg.org> wrote:
> Is it possible to access the core name in a config file (such as
> solrconfig.xml) so I can include core-specific configlets into a common
> config file? I would like to pull in different configurations for things
> like shards and replication, but have all the cores otherwise use an
> identical config file.
>
> Also, I have been looking for the syntax to include a snippet and haven't
> turned anything up yet.
>
> Thanks,
> Shawn
>
>
--
Lance Norskog
goksron@gmail.com