Result grouping performance

[Kempelen, Ákos <ak...@wolterskluwer.com>]

Hello,

I am migrating our codebase from Solr 4.7 to 7.0.1 but the performance of result grouping seems very poor using the newer Solr.
For example a simple MatchAllDocsQuery takes 5 sec on Solr4.7, and 21 sec on Solr7.
I wonder what causes the x4 difference in time? We hoped that newer Solr versions will provide better performances...
Using Field collapsing could would be a solution, but it produces different facet counts.
Thanks,
Akos 

Re: Result grouping performance

[Erick Erickson <er...@gmail.com>]

Have you enabled docValues (and reindexed from scratch) on the field
you're grouping on?

Best,
Erick

On Wed, Nov 22, 2017 at 5:13 AM, Kempelen, Ákos
<ak...@wolterskluwer.com> wrote:
> Hello,
>
> I am migrating our codebase from Solr 4.7 to 7.0.1 but the performance of result grouping seems very poor using the newer Solr.
> For example a simple MatchAllDocsQuery takes 5 sec on Solr4.7, and 21 sec on Solr7.
> I wonder what causes the x4 difference in time? We hoped that newer Solr versions will provide better performances...
> Using Field collapsing could would be a solution, but it produces different facet counts.
> Thanks,
> Akos
>
>

RE: Result grouping performance

[Kempelen, Ákos <ak...@wolterskluwer.com>]

Hi Mikhail,

group.facet=false, but group.truncate=true.
I found out that if the group.truncate parameter is true, the query will be much slower (x5 times), regardless of faceting.
I thought, that the group.truncate param is meaningless without activating the facet system, but this is not the case.

Testing environment: Solr 7.0.1, 1 core with 70GB data (~1.000.000 documents), no shards

Akos



-----Original Message-----
From: Mikhail Khludnev [mailto:mkhl@apache.org] 
Sent: Thursday, November 23, 2017 8:38 AM
To: solr-user <so...@lucene.apache.org>
Subject: Re: Result grouping performance

Akos,
Can you provide your request params? Do you just group and/or count grouped facets?
Can you clarify how field collapsing is different from grouping, just make it unambiguous?


On Wed, Nov 22, 2017 at 4:13 PM, Kempelen, Ákos < akos.kempelen@wolterskluwer.com> wrote:

> Hello,
>
> I am migrating our codebase from Solr 4.7 to 7.0.1 but the performance 
> of result grouping seems very poor using the newer Solr.
> For example a simple MatchAllDocsQuery takes 5 sec on Solr4.7, and 21 
> sec on Solr7.
> I wonder what causes the x4 difference in time? We hoped that newer 
> Solr versions will provide better performances...
> Using Field collapsing could would be a solution, but it produces 
> different facet counts.
> Thanks,
> Akos
>
>
>


--
Sincerely yours
Mikhail Khludnev

Re: Result grouping performance

[Mikhail Khludnev <mk...@apache.org>]

Akos,
Can you provide your request params? Do you just group and/or count grouped
facets?
Can you clarify how field collapsing is different from grouping, just make
it unambiguous?


On Wed, Nov 22, 2017 at 4:13 PM, Kempelen, Ákos <
akos.kempelen@wolterskluwer.com> wrote:

> Hello,
>
> I am migrating our codebase from Solr 4.7 to 7.0.1 but the performance of
> result grouping seems very poor using the newer Solr.
> For example a simple MatchAllDocsQuery takes 5 sec on Solr4.7, and 21 sec
> on Solr7.
> I wonder what causes the x4 difference in time? We hoped that newer Solr
> versions will provide better performances...
> Using Field collapsing could would be a solution, but it produces
> different facet counts.
> Thanks,
> Akos
>
>
>


-- 
Sincerely yours
Mikhail Khludnev