You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Xavier Schepler <xa...@sciences-po.fr> on 2010/02/08 17:17:32 UTC
Dynamic fields with more than 100 fields inside
Hey,
I'm thinking about using dynamic fields.
I need one or more user specific field in my schema,
for example, "concept_user_*", and I will have maybe more than 200 users
using this feature.
One user will send and retrieve values from its field. It will then be
used to filter result.
How would it impact query performance ?
Thanks,
Xavier S.
Re: Dynamic fields with more than 100 fields inside
Posted by Mat Brown <ma...@patch.com>.
On Thu, Feb 11, 2010 at 15:41, gdeconto <ge...@topproducer.com> wrote:
>
>
> Xavier Schepler wrote:
>>
>> for example, "concept_user_*", and I will have maybe more than 200 users
>> using this feature.
>>
>
> I've done tests with many hundred dynamically created fields (ie foo_1 thru
> f_400). generally speaking, I havent noticed any noticeable performance
> issues from having that many fields
>
> the only exception: I have noticed performance issues if you try to query
> large numbers of fields (ie q=(foo_1:123 AND foo_2:234 ... AND foo_400:912)
> --
> View this message in context: http://old.nabble.com/Dynamic-fields-with-more-than-100-fields-inside-tp27502271p27554402.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Since Lucene is schema-less, my assumption would be that using dynamic
fields wouldn't have any effect on performance within the search stack
itself - presumably the only penalty would be matching a given field
name against the appropriate dynamic field (which I would assume would
be cached/memoized).
Lots of assumptions there, of course. Feel free to correct me if I'm wrong.
Re: Dynamic fields with more than 100 fields inside
Posted by gdeconto <ge...@topproducer.com>.
Xavier Schepler wrote:
>
> for example, "concept_user_*", and I will have maybe more than 200 users
> using this feature.
>
I've done tests with many hundred dynamically created fields (ie foo_1 thru
f_400). generally speaking, I havent noticed any noticeable performance
issues from having that many fields
the only exception: I have noticed performance issues if you try to query
large numbers of fields (ie q=(foo_1:123 AND foo_2:234 ... AND foo_400:912)
--
View this message in context: http://old.nabble.com/Dynamic-fields-with-more-than-100-fields-inside-tp27502271p27554402.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Dynamic fields with more than 100 fields inside
Posted by Xavier Schepler <xa...@sciences-po.fr>.
Shalin Shekhar Mangar a écrit :
> On Tue, Feb 9, 2010 at 2:43 PM, Xavier Schepler <
> xavier.schepler@sciences-po.fr> wrote:
>
>
>> Shalin Shekhar Mangar a écrit :
>>
>> On Mon, Feb 8, 2010 at 9:47 PM, Xavier Schepler <
>>
>>> xavier.schepler@sciences-po.fr> wrote:
>>>
>>>
>>>
>>>
>>>> Hey,
>>>>
>>>> I'm thinking about using dynamic fields.
>>>>
>>>> I need one or more user specific field in my schema, for example,
>>>> "concept_user_*", and I will have maybe more than 200 users using this
>>>> feature.
>>>> One user will send and retrieve values from its field. It will then be
>>>> used
>>>> to filter result.
>>>>
>>>> How would it impact query performance ?
>>>>
>>>>
>>>>
>>>>
>>>>
>>> Can you give an example of such a query?
>>>
>>>
>>>
>>>
>> Hi,
>>
>> it could be queries such as :
>>
>> allFr: état-unis AND concept_researcher_99 = 303
>>
>> modalitiesFr: exactement AND questionFr: correspond AND
>> concept_researcher_2 = 101
>>
>> and facetting like this :
>>
>>
>> q=%2A%3A%2A&fl=variableXMLFr,lang&start=0&rows=10&facet=true&facet.field=concept_researcher_2&facet.field=studyDateAndStudyTitle&facet.sort=lex
>>
>>
>>
> It doesn't impact query performance any more than filtering on other fields.
> Is there a performance problem or were you just asking generally?
>
>
I was asking generally, thanks for your response.
Re: Dynamic fields with more than 100 fields inside
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Tue, Feb 9, 2010 at 2:43 PM, Xavier Schepler <
xavier.schepler@sciences-po.fr> wrote:
> Shalin Shekhar Mangar a écrit :
>
> On Mon, Feb 8, 2010 at 9:47 PM, Xavier Schepler <
>> xavier.schepler@sciences-po.fr> wrote:
>>
>>
>>
>>> Hey,
>>>
>>> I'm thinking about using dynamic fields.
>>>
>>> I need one or more user specific field in my schema, for example,
>>> "concept_user_*", and I will have maybe more than 200 users using this
>>> feature.
>>> One user will send and retrieve values from its field. It will then be
>>> used
>>> to filter result.
>>>
>>> How would it impact query performance ?
>>>
>>>
>>>
>>>
>> Can you give an example of such a query?
>>
>>
>>
> Hi,
>
> it could be queries such as :
>
> allFr: état-unis AND concept_researcher_99 = 303
>
> modalitiesFr: exactement AND questionFr: correspond AND
> concept_researcher_2 = 101
>
> and facetting like this :
>
>
> q=%2A%3A%2A&fl=variableXMLFr,lang&start=0&rows=10&facet=true&facet.field=concept_researcher_2&facet.field=studyDateAndStudyTitle&facet.sort=lex
>
>
It doesn't impact query performance any more than filtering on other fields.
Is there a performance problem or were you just asking generally?
--
Regards,
Shalin Shekhar Mangar.
Re: Dynamic fields with more than 100 fields inside
Posted by Xavier Schepler <xa...@sciences-po.fr>.
Shalin Shekhar Mangar a écrit :
> On Mon, Feb 8, 2010 at 9:47 PM, Xavier Schepler <
> xavier.schepler@sciences-po.fr> wrote:
>
>
>> Hey,
>>
>> I'm thinking about using dynamic fields.
>>
>> I need one or more user specific field in my schema, for example,
>> "concept_user_*", and I will have maybe more than 200 users using this
>> feature.
>> One user will send and retrieve values from its field. It will then be used
>> to filter result.
>>
>> How would it impact query performance ?
>>
>>
>>
> Can you give an example of such a query?
>
>
Hi,
it could be queries such as :
allFr: état-unis AND concept_researcher_99 = 303
modalitiesFr: exactement AND questionFr: correspond AND
concept_researcher_2 = 101
and facetting like this :
q=%2A%3A%2A&fl=variableXMLFr,lang&start=0&rows=10&facet=true&facet.field=concept_researcher_2&facet.field=studyDateAndStudyTitle&facet.sort=lex
Thanks in advance,
Xavier S.
Re: Dynamic fields with more than 100 fields inside
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Mon, Feb 8, 2010 at 9:47 PM, Xavier Schepler <
xavier.schepler@sciences-po.fr> wrote:
> Hey,
>
> I'm thinking about using dynamic fields.
>
> I need one or more user specific field in my schema, for example,
> "concept_user_*", and I will have maybe more than 200 users using this
> feature.
> One user will send and retrieve values from its field. It will then be used
> to filter result.
>
> How would it impact query performance ?
>
>
Can you give an example of such a query?
--
Regards,
Shalin Shekhar Mangar.