You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Julian Hille <ju...@netimpact.de> on 2010/02/09 13:21:14 UTC
Replication and querying
Hi,
id like to know if its possible to have a solr Server with a schema and lets say 10 fields indexed.
I know want to replicate this whole index to another solr server which has a slightly different schema.
There are additional 6 fields these fields change the sort order for a product which base is our solr database.
Is this kind of replication possible?
Is there another way to interact with data in solr? We'd like to calculate some fields when they will be added.
I cant seem to find a good documentation about the possible calls in the query itself nor documentaion about queries/calculation which should be done on update.
so far,
Julian Hille
---------------------------------------------------------------
NetImpact KG
Altonaer Straße 8
20357 Hamburg
Tel: 040 / 6738363 2
Mail: julian@netimpact.de
Geschäftsführer: Tarek Müller
Re: Replication and querying
Posted by Jan Høydahl / Cominvent <ja...@cominvent.com>.
Hi again,
I would still keep all fields in the original schema of the global Solr, just for the sake of simplicity.
For custom sort order, you can look at ExternalFileField which is a text file that you can add to your local Solr index independently of the pre-built index. However, this only supports float and cannot be returned in result. http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html
The Solr replication does a binary copy of the index - i.e. no way to change docs in input.
But if you instead replicate source XML feed from master to slaves, you can hook into that stream to modify/add fields (see http://wiki.apache.org/solr/SolrConfigXml#UpdateRequestProcessorChain_section). But then you need to index locally of course. 1.5 mill docs isn't that much, so why not?
--
Jan Høydahl - search architect
Cominvent AS - www.cominvent.com
On 10. feb. 2010, at 10.22, Julian Hille wrote:
> Hi,
>
> its would be possible to add that to the main solr but the problem is:
> Lets face it (example):
> We have kind of 1.5 million documents in the solr master. These Documents are books.
> These books have fields like title, ids, numbers and authors and more.
> This solr is global.
>
> Now: The slave solr is for a local library which has all these books, but want to sort in another way,
> and wants to add their own fields. For sorting and output (these fields doesnt need to be indexed or searched through).
>
> So we try to replicate the whole database but have a slightly differen schema.xml in the slaves.
>
>
> Secondly we need for another Project to know if its possible to change data "oninsert", "onupdate".
> So that the replicationed data gets edited before its really inserted. Is there some kind of hook?
> As an exmaple lets take the book example from top:
> On replication the slave gets a updated document set. But before updated on the the slaves db
> we like to add fields which come from another database or we like to replace strings in some fields and such things.
>
> Is that possible?
>
> Thanks for any answers.
>
>
>
> Am 09.02.2010 um 16:53 schrieb Jan Høydahl / Cominvent:
>
>> Hi,
>>
>> Index replication in Solr makes an exact copy of the original index.
>> Is it not possible to add the 6 extra fields to both instances?
>> An alternative to replication is to feed two independent Solr instances -> full control :)
>> Please elaborate on your specific use case if this is not useful answer to you.
>>
>> --
>> Jan Høydahl - search architect
>> Cominvent AS - www.cominvent.com
>>
>> On 9. feb. 2010, at 13.21, Julian Hille wrote:
>>
>>> Hi,
>>>
>>> id like to know if its possible to have a solr Server with a schema and lets say 10 fields indexed.
>>> I know want to replicate this whole index to another solr server which has a slightly different schema.
>>> There are additional 6 fields these fields change the sort order for a product which base is our solr database.
>>>
>>> Is this kind of replication possible?
>>>
>>> Is there another way to interact with data in solr? We'd like to calculate some fields when they will be added.
>>> I cant seem to find a good documentation about the possible calls in the query itself nor documentaion about queries/calculation which should be done on update.
>>>
>>>
>>> so far,
>>> Julian Hille
>>>
>>>
>>> ---------------------------------------------------------------
>>> NetImpact KG
>>> Altonaer Straße 8
>>> 20357 Hamburg
>>>
>>> Tel: 040 / 6738363 2
>>> Mail: julian@netimpact.de
>>>
>>> Geschäftsführer: Tarek Müller
>>>
>
> Mit freundlichen Grüßen,
> Julian Hille
>
>
> ---------------------------------------------------------------
> NetImpact KG
> Altonaer Straße 8
> 20357 Hamburg
>
> Tel: 040 / 6738363 2
> Mail: julian@netimpact.de
>
> Geschäftsführer: Tarek Müller
>
Re: Replication and querying
Posted by Julian Hille <ju...@netimpact.de>.
Hi,
its would be possible to add that to the main solr but the problem is:
Lets face it (example):
We have kind of 1.5 million documents in the solr master. These Documents are books.
These books have fields like title, ids, numbers and authors and more.
This solr is global.
Now: The slave solr is for a local library which has all these books, but want to sort in another way,
and wants to add their own fields. For sorting and output (these fields doesnt need to be indexed or searched through).
So we try to replicate the whole database but have a slightly differen schema.xml in the slaves.
Secondly we need for another Project to know if its possible to change data "oninsert", "onupdate".
So that the replicationed data gets edited before its really inserted. Is there some kind of hook?
As an exmaple lets take the book example from top:
On replication the slave gets a updated document set. But before updated on the the slaves db
we like to add fields which come from another database or we like to replace strings in some fields and such things.
Is that possible?
Thanks for any answers.
Am 09.02.2010 um 16:53 schrieb Jan Høydahl / Cominvent:
> Hi,
>
> Index replication in Solr makes an exact copy of the original index.
> Is it not possible to add the 6 extra fields to both instances?
> An alternative to replication is to feed two independent Solr instances -> full control :)
> Please elaborate on your specific use case if this is not useful answer to you.
>
> --
> Jan Høydahl - search architect
> Cominvent AS - www.cominvent.com
>
> On 9. feb. 2010, at 13.21, Julian Hille wrote:
>
>> Hi,
>>
>> id like to know if its possible to have a solr Server with a schema and lets say 10 fields indexed.
>> I know want to replicate this whole index to another solr server which has a slightly different schema.
>> There are additional 6 fields these fields change the sort order for a product which base is our solr database.
>>
>> Is this kind of replication possible?
>>
>> Is there another way to interact with data in solr? We'd like to calculate some fields when they will be added.
>> I cant seem to find a good documentation about the possible calls in the query itself nor documentaion about queries/calculation which should be done on update.
>>
>>
>> so far,
>> Julian Hille
>>
>>
>> ---------------------------------------------------------------
>> NetImpact KG
>> Altonaer Straße 8
>> 20357 Hamburg
>>
>> Tel: 040 / 6738363 2
>> Mail: julian@netimpact.de
>>
>> Geschäftsführer: Tarek Müller
>>
Mit freundlichen Grüßen,
Julian Hille
---------------------------------------------------------------
NetImpact KG
Altonaer Straße 8
20357 Hamburg
Tel: 040 / 6738363 2
Mail: julian@netimpact.de
Geschäftsführer: Tarek Müller
Re: Replication and querying
Posted by Jan Høydahl / Cominvent <ja...@cominvent.com>.
Hi,
Index replication in Solr makes an exact copy of the original index.
Is it not possible to add the 6 extra fields to both instances?
An alternative to replication is to feed two independent Solr instances -> full control :)
Please elaborate on your specific use case if this is not useful answer to you.
--
Jan Høydahl - search architect
Cominvent AS - www.cominvent.com
On 9. feb. 2010, at 13.21, Julian Hille wrote:
> Hi,
>
> id like to know if its possible to have a solr Server with a schema and lets say 10 fields indexed.
> I know want to replicate this whole index to another solr server which has a slightly different schema.
> There are additional 6 fields these fields change the sort order for a product which base is our solr database.
>
> Is this kind of replication possible?
>
> Is there another way to interact with data in solr? We'd like to calculate some fields when they will be added.
> I cant seem to find a good documentation about the possible calls in the query itself nor documentaion about queries/calculation which should be done on update.
>
>
> so far,
> Julian Hille
>
>
> ---------------------------------------------------------------
> NetImpact KG
> Altonaer Straße 8
> 20357 Hamburg
>
> Tel: 040 / 6738363 2
> Mail: julian@netimpact.de
>
> Geschäftsführer: Tarek Müller
>