You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by ra...@thomsonreuters.com on 2012/03/21 21:16:47 UTC
copyField question
Hi:
Is it it possible to store a value and a corresponding score in Solr as part of a single Field definition. And Can this field be a multivalued field ?
I have several terms that are score. I would like to store them as part of a single field definition rather than having to create two different fields (one storing score and the other the value).
However, If the multivalued complex data field is not possible. Is it possible to use copyField directive to copy fields if a certain score is higher than a threshold ?
Thanks
Ramdev
Re: copyField question
Posted by ra...@thomsonreuters.com.
Hi Tomas:
These fields are for searching only.
Currently we have around 1.8M docs indexed.and Assuming each Doc has about
20 of these additional fields to be created as dynamic fields (worst case
scenario), and also there are about 6K if these different values (I.e. If
we were to create static fields defs, there would be 6K fields).
I did create dynamic fields as you suggested, but only on a subset of docs
(10K). I have not extensive performance analysis on it or anything. (its a
rather simple schema/index structure).
Thanks
Ramdev
On 3/22/12 7:42 AM, "Tomás Fernández Löbbe" <to...@gmail.com> wrote:
>I meant, how many values in total? A single document may have 20, but are
>those 20 shared with other document (even if they have different score) or
>each document will have 10-20 completely different values? I think Solr
>could handle a couple hundred of fields, but I don't know how it would
>behave with thousands (really, I don't know you should test it).
>
>You should be using a dynamic field for creating those fields dynamically,
>and make sure you have the omitNorms attribute set to true.
>
>What do you need to use those fields for? searching? displaying?
>
>
>On Wed, Mar 21, 2012 at 5:49 PM, <ra...@thomsonreuters.com> wrote:
>
>> Hi Tomás:
>> I think there is simplicity in your solution ;) A document would have
>> Tens of different values. (at the most 20)Š
>>
>> So If were to follow your suggestion of naming a dynamic field with the
>> value as the name of the field and the corresponding Score as the value.
>> How would I go about changing the schema ?
>>
>> Thanks
>>
>> Ramdev
>>
>>
>> On 3/21/12 3:24 PM, "Tomás Fernández Löbbe" <to...@gmail.com>
>>wrote:
>>
>> >> However, If the multivalued complex data field is not possible. Is it
>> >possible to use copyField directive to copy fields if a certain score
>>is
>> >higher than a threshold ?
>> >I don't think that's possible out of the box, but you could use custom
>> >UpdateRequestProcessor for for that.
>> >
>> >How many different values do you have? tens? hundreds? thousands?...
>> >millions? If those are not too many, you could use dynamic fields,
>>using
>> >the value as field name and the score as field value. Unless I'm
>> >oversimplifying your problem.
>> >
>> >Tomás
>> >
>> >
>> >On Wed, Mar 21, 2012 at 5:16 PM, <ra...@thomsonreuters.com>
>> wrote:
>> >
>> >> Hi:
>> >> Is it it possible to store a value and a corresponding score in
>>Solr
>> >>as
>> >> part of a single Field definition. And Can this field be a
>>multivalued
>> >> field ?
>> >> I have several terms that are score. I would like to store them as
>>part
>> >>of
>> >> a single field definition rather than having to create two different
>> >>fields
>> >> (one storing score and the other the value).
>> >>
>> >> However, If the multivalued complex data field is not possible. Is it
>> >> possible to use copyField directive to copy fields if a certain score
>> >>is
>> >> higher than a threshold ?
>> >>
>> >>
>> >> Thanks
>> >>
>> >> Ramdev
>> >>
>>
>>
Re: copyField question
Posted by Tomás Fernández Löbbe <to...@gmail.com>.
I meant, how many values in total? A single document may have 20, but are
those 20 shared with other document (even if they have different score) or
each document will have 10-20 completely different values? I think Solr
could handle a couple hundred of fields, but I don't know how it would
behave with thousands (really, I don't know you should test it).
You should be using a dynamic field for creating those fields dynamically,
and make sure you have the omitNorms attribute set to true.
What do you need to use those fields for? searching? displaying?
On Wed, Mar 21, 2012 at 5:49 PM, <ra...@thomsonreuters.com> wrote:
> Hi Tomás:
> I think there is simplicity in your solution ;) A document would have
> Tens of different values. (at the most 20)Š
>
> So If were to follow your suggestion of naming a dynamic field with the
> value as the name of the field and the corresponding Score as the value.
> How would I go about changing the schema ?
>
> Thanks
>
> Ramdev
>
>
> On 3/21/12 3:24 PM, "Tomás Fernández Löbbe" <to...@gmail.com> wrote:
>
> >> However, If the multivalued complex data field is not possible. Is it
> >possible to use copyField directive to copy fields if a certain score is
> >higher than a threshold ?
> >I don't think that's possible out of the box, but you could use custom
> >UpdateRequestProcessor for for that.
> >
> >How many different values do you have? tens? hundreds? thousands?...
> >millions? If those are not too many, you could use dynamic fields, using
> >the value as field name and the score as field value. Unless I'm
> >oversimplifying your problem.
> >
> >Tomás
> >
> >
> >On Wed, Mar 21, 2012 at 5:16 PM, <ra...@thomsonreuters.com>
> wrote:
> >
> >> Hi:
> >> Is it it possible to store a value and a corresponding score in Solr
> >>as
> >> part of a single Field definition. And Can this field be a multivalued
> >> field ?
> >> I have several terms that are score. I would like to store them as part
> >>of
> >> a single field definition rather than having to create two different
> >>fields
> >> (one storing score and the other the value).
> >>
> >> However, If the multivalued complex data field is not possible. Is it
> >> possible to use copyField directive to copy fields if a certain score
> >>is
> >> higher than a threshold ?
> >>
> >>
> >> Thanks
> >>
> >> Ramdev
> >>
>
>
Re: copyField question
Posted by ra...@thomsonreuters.com.
Hi Tomás:
I think there is simplicity in your solution ;) A document would have
Tens of different values. (at the most 20)Š
So If were to follow your suggestion of naming a dynamic field with the
value as the name of the field and the corresponding Score as the value.
How would I go about changing the schema ?
Thanks
Ramdev
On 3/21/12 3:24 PM, "Tomás Fernández Löbbe" <to...@gmail.com> wrote:
>> However, If the multivalued complex data field is not possible. Is it
>possible to use copyField directive to copy fields if a certain score is
>higher than a threshold ?
>I don't think that's possible out of the box, but you could use custom
>UpdateRequestProcessor for for that.
>
>How many different values do you have? tens? hundreds? thousands?...
>millions? If those are not too many, you could use dynamic fields, using
>the value as field name and the score as field value. Unless I'm
>oversimplifying your problem.
>
>Tomás
>
>
>On Wed, Mar 21, 2012 at 5:16 PM, <ra...@thomsonreuters.com> wrote:
>
>> Hi:
>> Is it it possible to store a value and a corresponding score in Solr
>>as
>> part of a single Field definition. And Can this field be a multivalued
>> field ?
>> I have several terms that are score. I would like to store them as part
>>of
>> a single field definition rather than having to create two different
>>fields
>> (one storing score and the other the value).
>>
>> However, If the multivalued complex data field is not possible. Is it
>> possible to use copyField directive to copy fields if a certain score
>>is
>> higher than a threshold ?
>>
>>
>> Thanks
>>
>> Ramdev
>>
Re: copyField question
Posted by Tomás Fernández Löbbe <to...@gmail.com>.
> However, If the multivalued complex data field is not possible. Is it
possible to use copyField directive to copy fields if a certain score is
higher than a threshold ?
I don't think that's possible out of the box, but you could use custom
UpdateRequestProcessor for for that.
How many different values do you have? tens? hundreds? thousands?...
millions? If those are not too many, you could use dynamic fields, using
the value as field name and the score as field value. Unless I'm
oversimplifying your problem.
Tomás
On Wed, Mar 21, 2012 at 5:16 PM, <ra...@thomsonreuters.com> wrote:
> Hi:
> Is it it possible to store a value and a corresponding score in Solr as
> part of a single Field definition. And Can this field be a multivalued
> field ?
> I have several terms that are score. I would like to store them as part of
> a single field definition rather than having to create two different fields
> (one storing score and the other the value).
>
> However, If the multivalued complex data field is not possible. Is it
> possible to use copyField directive to copy fields if a certain score is
> higher than a threshold ?
>
>
> Thanks
>
> Ramdev
>