Solr performance using fq with multiple values

[Odysci <od...@gmail.com>]

Hi,

We are seeing significant performance degradation on single queries that
use fq with multiple values as in:

fq=field1_name:(V1 V2 V3 ...)

If we use only one value in the fq (say only V1) we get Qtime = T ms
As we increase the number of values, say to 5 values, Qtime more than
triples, even if the number of results is small. In my tests I made sure
cache was not an issue and nothing else was using the cpu.

We commonly need to use fq with multiple values (on the same field name,
which is normally a long).
Is this performance hit to be expected?
Is there a better way to do this?

We use Solr Cloud 8.3, and the field that we use fq on is defined as:

    <dynamicField name="*_long" type="long" indexed="false"
        stored="false" required="false" multiValued="false"
docValues="true" />

Thanks

Reinaldo

Re: Solr performance using fq with multiple values

[Shawn Heisey <ap...@elyograg.org>]

On 4/18/2020 12:20 PM, Odysci wrote:
> We don't used this field for general queries (q:*), only for fq and
> faceting.
> Do you think making it indexed="true" would make a difference in fq
> performance?

fq means "filter query".  It's still a query.  So yes, the field should 
be indexed.  The query you're doing only works because docValues is true 
... but queries using docValues have terrible performance.

Thanks,
Shawn

Re: Solr performance using fq with multiple values

[Odysci <od...@gmail.com>]

We don't used this field for general queries (q:*), only for fq and
faceting.
Do you think making it indexed="true" would make a difference in fq
performance?
Thanks

Reinaldo

On Sat, Apr 18, 2020 at 3:06 PM Sylvain James <sy...@gmail.com>
wrote:

> Hi Reinaldo,
>
> Involved fields should be indexed for better performance ?
>
> <dynamicField name="*_long" type="long"* indexed="false"*
>         stored="false" required="false" multiValued="false"
> docValues="true" />
>
> Sylvain
>
> Le sam. 18 avr. 2020 à 18:46, Odysci <od...@gmail.com> a écrit :
>
> > Hi,
> >
> > We are seeing significant performance degradation on single queries that
> > use fq with multiple values as in:
> >
> > fq=field1_name:(V1 V2 V3 ...)
> >
> > If we use only one value in the fq (say only V1) we get Qtime = T ms
> > As we increase the number of values, say to 5 values, Qtime more than
> > triples, even if the number of results is small. In my tests I made sure
> > cache was not an issue and nothing else was using the cpu.
> >
> > We commonly need to use fq with multiple values (on the same field name,
> > which is normally a long).
> > Is this performance hit to be expected?
> > Is there a better way to do this?
> >
> > We use Solr Cloud 8.3, and the field that we use fq on is defined as:
> >
> >     <dynamicField name="*_long" type="long" indexed="false"
> >         stored="false" required="false" multiValued="false"
> > docValues="true" />
> >
> > Thanks
> >
> > Reinaldo
> >
>

Re: Solr performance using fq with multiple values

[Sylvain James <sy...@gmail.com>]

Hi Reinaldo,

Involved fields should be indexed for better performance ?

<dynamicField name="*_long" type="long"* indexed="false"*
        stored="false" required="false" multiValued="false"
docValues="true" />

Sylvain

Le sam. 18 avr. 2020 à 18:46, Odysci <od...@gmail.com> a écrit :

> Hi,
>
> We are seeing significant performance degradation on single queries that
> use fq with multiple values as in:
>
> fq=field1_name:(V1 V2 V3 ...)
>
> If we use only one value in the fq (say only V1) we get Qtime = T ms
> As we increase the number of values, say to 5 values, Qtime more than
> triples, even if the number of results is small. In my tests I made sure
> cache was not an issue and nothing else was using the cpu.
>
> We commonly need to use fq with multiple values (on the same field name,
> which is normally a long).
> Is this performance hit to be expected?
> Is there a better way to do this?
>
> We use Solr Cloud 8.3, and the field that we use fq on is defined as:
>
>     <dynamicField name="*_long" type="long" indexed="false"
>         stored="false" required="false" multiValued="false"
> docValues="true" />
>
> Thanks
>
> Reinaldo
>