You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Timo Schmidt <ti...@gmx.net> on 2013/04/22 20:31:47 UTC
Support of field variants in solr
Hi together,
i am timo and work for a solr implementation company. During the last projects we came to know that we need to be able to generate different variants of a document.
Example 1 (Language):
To handle all documents in one solr core, we need a field variant for each language.
content for spanish content
<field name="content" type="text_es" indexed="true" stored="true" variant=“es“ />
content for german content
<field name="content" type="text_de" indexed="true" stored="true" variant=“de“ />
Each of these fields can be configured in the solr schema to act optimal for the specific taget language.
Example 2 (Stores):
We have customers who want to sell the same product in different stores for different prices.
price in frankfurt
<field name="price" type="sfloat" indexed="true" stored="true" variant=“fr“ />
price in paris
<field name="price" type="sfloat" indexed="true" stored="true" variant=“pr“ />
To solve this in an optimal way it would be nice when this works complely transparent inside solr by definig a „variantQuery“
A select query could look like this:
select?variantQuery=fr&qf=price,content
Additional the following is possible. No variant is present, behavious should be as before, so it should be relevant for all queries.
The setting variant=“*“ would mean: There can be several wildcard variant defined in a commited document. This makes sence when the data type would be the same for all variants and you will have many variants (like in the price example).
The same as during query time should be possible during indexing time.
I know, that we can do somthing like this also with dynamic fields but then we need to resolve the concrete fields during index and querytime on the application level, what is possible but it would be nicer to have a concept like this in solr, also working with facets is easier with this approach when the concrete fieldname does not need to be populated in the application.
So my questions are:
What do you think about this approach?
Is it better to work with dynamic fields? Is it reasonable when you have 200 variants or more of a document?
What needs to be done in solr to have something like this variant attribute for fields?
Do you have other approaches?
Re: Re: Support of field variants in solr
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
You can certainly specify all your aliases in the request. The request
handler is just there to simplify the client by allowing it to specify
a different URL with everything else mapped on the server. And, of
course, with request handler you can lock the parameters to force
them.
Regarding language detection during indexing, there is a module for
that: http://wiki.apache.org/solr/LanguageDetection . Hopefully that
would be sufficient.
Regards,
Alex.
Personal blog: http://blog.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all
at once. Lately, it doesn't seem to be working. (Anonymous - via GTD
book)
On Tue, Apr 23, 2013 at 4:45 PM, Timo Schmidt <ti...@gmx.net> wrote:
> Ok, thanks for this hint i have two further questions to understand it completly.
>
> Settingup custom request handler makes it easier to avoid all the mapping parameters in the query but it
> would also be possible with one request handler and all mapping in the request arguments right?
>
> What about indexing, i there also a mechanism like this or should the application deside with target field to use?
>
>
> Gesendet: Dienstag, 23. April 2013 um 02:32 Uhr
> Von: "Alexandre Rafalovitch" <ar...@gmail.com>
> An: solr-user@lucene.apache.org
> Betreff: Re: Support of field variants in solr
> To route different languages, you could use different request handlers
> and do different alias mapping. There are two alias mapping:
> On the way in for eDisMax:
> https://wiki.apache.org/solr/ExtendedDisMax#Field_aliasing_.2BAC8_renaming
> On the way out: https://wiki.apache.org/solr/CommonQueryParameters#Field_alias[https://wiki.apache.org/solr/CommonQueryParameters#Field_alias]
>
> Between the two, you can make sure that all searches to /searchES map
> 'content' field to 'content_es' and for /searchDE map 'content' to
> 'content_de'.
>
> Hope this helps,
> Alex.
>
> Personal blog: http://blog.outerthoughts.com/[http://blog.outerthoughts.com/]
> LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch[http://www.linkedin.com/in/alexandrerafalovitch]
> - Time is the quality of nature that keeps events from happening all
> at once. Lately, it doesn't seem to be working. (Anonymous - via GTD
> book)
>
>
> On Mon, Apr 22, 2013 at 2:31 PM, Timo Schmidt <ti...@gmx.net> wrote:
>> Hi together,
>>
>> i am timo and work for a solr implementation company. During the last projects we came to know that we need to be able to generate different variants of a document.
>>
>> Example 1 (Language):
>>
>> To handle all documents in one solr core, we need a field variant for each language.
>>
>>
>> content for spanish content
>>
>> <field name="content" type="text_es" indexed="true" stored="true" variant=“es“ />
>>
>> content for german content
>>
>> <field name="content" type="text_de" indexed="true" stored="true" variant=“de“ />
>>
>>
>> Each of these fields can be configured in the solr schema to act optimal for the specific taget language.
>>
>> Example 2 (Stores):
>>
>> We have customers who want to sell the same product in different stores for different prices.
>>
>>
>> price in frankfurt
>>
>> <field name="price" type="sfloat" indexed="true" stored="true" variant=“fr“ />
>>
>> price in paris
>>
>> <field name="price" type="sfloat" indexed="true" stored="true" variant=“pr“ />
>>
>> To solve this in an optimal way it would be nice when this works complely transparent inside solr by definig a „variantQuery“
>>
>> A select query could look like this:
>>
>> select?variantQuery=fr&qf=price,content
>>
>> Additional the following is possible. No variant is present, behavious should be as before, so it should be relevant for all queries.
>>
>> The setting variant=“*“ would mean: There can be several wildcard variant defined in a commited document. This makes sence when the data type would be the same for all variants and you will have many variants (like in the price example).
>>
>> The same as during query time should be possible during indexing time.
>>
>> I know, that we can do somthing like this also with dynamic fields but then we need to resolve the concrete fields during index and querytime on the application level, what is possible but it would be nicer to have a concept like this in solr, also working with facets is easier with this approach when the concrete fieldname does not need to be populated in the application.
>>
>> So my questions are:
>>
>> What do you think about this approach?
>> Is it better to work with dynamic fields? Is it reasonable when you have 200 variants or more of a document?
>> What needs to be done in solr to have something like this variant attribute for fields?
>> Do you have other approaches?
Aw: Re: Support of field variants in solr
Posted by Timo Schmidt <ti...@gmx.net>.
Ok, thanks for this hint i have two further questions to understand it completly.
Settingup custom request handler makes it easier to avoid all the mapping parameters in the query but it
would also be possible with one request handler and all mapping in the request arguments right?
What about indexing, i there also a mechanism like this or should the application deside with target field to use?
Gesendet: Dienstag, 23. April 2013 um 02:32 Uhr
Von: "Alexandre Rafalovitch" <ar...@gmail.com>
An: solr-user@lucene.apache.org
Betreff: Re: Support of field variants in solr
To route different languages, you could use different request handlers
and do different alias mapping. There are two alias mapping:
On the way in for eDisMax:
https://wiki.apache.org/solr/ExtendedDisMax#Field_aliasing_.2BAC8_renaming
On the way out: https://wiki.apache.org/solr/CommonQueryParameters#Field_alias[https://wiki.apache.org/solr/CommonQueryParameters#Field_alias]
Between the two, you can make sure that all searches to /searchES map
'content' field to 'content_es' and for /searchDE map 'content' to
'content_de'.
Hope this helps,
Alex.
Personal blog: http://blog.outerthoughts.com/[http://blog.outerthoughts.com/]
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch[http://www.linkedin.com/in/alexandrerafalovitch]
- Time is the quality of nature that keeps events from happening all
at once. Lately, it doesn't seem to be working. (Anonymous - via GTD
book)
On Mon, Apr 22, 2013 at 2:31 PM, Timo Schmidt <ti...@gmx.net> wrote:
> Hi together,
>
> i am timo and work for a solr implementation company. During the last projects we came to know that we need to be able to generate different variants of a document.
>
> Example 1 (Language):
>
> To handle all documents in one solr core, we need a field variant for each language.
>
>
> content for spanish content
>
> <field name="content" type="text_es" indexed="true" stored="true" variant=“es“ />
>
> content for german content
>
> <field name="content" type="text_de" indexed="true" stored="true" variant=“de“ />
>
>
> Each of these fields can be configured in the solr schema to act optimal for the specific taget language.
>
> Example 2 (Stores):
>
> We have customers who want to sell the same product in different stores for different prices.
>
>
> price in frankfurt
>
> <field name="price" type="sfloat" indexed="true" stored="true" variant=“fr“ />
>
> price in paris
>
> <field name="price" type="sfloat" indexed="true" stored="true" variant=“pr“ />
>
> To solve this in an optimal way it would be nice when this works complely transparent inside solr by definig a „variantQuery“
>
> A select query could look like this:
>
> select?variantQuery=fr&qf=price,content
>
> Additional the following is possible. No variant is present, behavious should be as before, so it should be relevant for all queries.
>
> The setting variant=“*“ would mean: There can be several wildcard variant defined in a commited document. This makes sence when the data type would be the same for all variants and you will have many variants (like in the price example).
>
> The same as during query time should be possible during indexing time.
>
> I know, that we can do somthing like this also with dynamic fields but then we need to resolve the concrete fields during index and querytime on the application level, what is possible but it would be nicer to have a concept like this in solr, also working with facets is easier with this approach when the concrete fieldname does not need to be populated in the application.
>
> So my questions are:
>
> What do you think about this approach?
> Is it better to work with dynamic fields? Is it reasonable when you have 200 variants or more of a document?
> What needs to be done in solr to have something like this variant attribute for fields?
> Do you have other approaches?
Re: Support of field variants in solr
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
To route different languages, you could use different request handlers
and do different alias mapping. There are two alias mapping:
On the way in for eDisMax:
https://wiki.apache.org/solr/ExtendedDisMax#Field_aliasing_.2BAC8_renaming
On the way out: https://wiki.apache.org/solr/CommonQueryParameters#Field_alias
Between the two, you can make sure that all searches to /searchES map
'content' field to 'content_es' and for /searchDE map 'content' to
'content_de'.
Hope this helps,
Alex.
Personal blog: http://blog.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all
at once. Lately, it doesn't seem to be working. (Anonymous - via GTD
book)
On Mon, Apr 22, 2013 at 2:31 PM, Timo Schmidt <ti...@gmx.net> wrote:
> Hi together,
>
> i am timo and work for a solr implementation company. During the last projects we came to know that we need to be able to generate different variants of a document.
>
> Example 1 (Language):
>
> To handle all documents in one solr core, we need a field variant for each language.
>
>
> content for spanish content
>
> <field name="content" type="text_es" indexed="true" stored="true" variant=“es“ />
>
> content for german content
>
> <field name="content" type="text_de" indexed="true" stored="true" variant=“de“ />
>
>
> Each of these fields can be configured in the solr schema to act optimal for the specific taget language.
>
> Example 2 (Stores):
>
> We have customers who want to sell the same product in different stores for different prices.
>
>
> price in frankfurt
>
> <field name="price" type="sfloat" indexed="true" stored="true" variant=“fr“ />
>
> price in paris
>
> <field name="price" type="sfloat" indexed="true" stored="true" variant=“pr“ />
>
> To solve this in an optimal way it would be nice when this works complely transparent inside solr by definig a „variantQuery“
>
> A select query could look like this:
>
> select?variantQuery=fr&qf=price,content
>
> Additional the following is possible. No variant is present, behavious should be as before, so it should be relevant for all queries.
>
> The setting variant=“*“ would mean: There can be several wildcard variant defined in a commited document. This makes sence when the data type would be the same for all variants and you will have many variants (like in the price example).
>
> The same as during query time should be possible during indexing time.
>
> I know, that we can do somthing like this also with dynamic fields but then we need to resolve the concrete fields during index and querytime on the application level, what is possible but it would be nicer to have a concept like this in solr, also working with facets is easier with this approach when the concrete fieldname does not need to be populated in the application.
>
> So my questions are:
>
> What do you think about this approach?
> Is it better to work with dynamic fields? Is it reasonable when you have 200 variants or more of a document?
> What needs to be done in solr to have something like this variant attribute for fields?
> Do you have other approaches?