You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Marek Tichy <ma...@gn.apc.org> on 2017/05/31 11:16:13 UTC

Grouping and group.facet performance disaster

Hi,

I'm getting a very slow response times on grouping, especially on facet
grouping.

Without grouping, the query takes 14ms, faceting 57ms.

With grouping, the query time goes up to 1131ms, with facet grouping,
the faceting goes up to the unbearable 12103 ms.

Single solr instance, 927086docs, 518.23 MB size, solr 6.4.1.

Is this really the price of grouping ? Are there any magic
tricks/tips/techniques to improve the speed ?
The query params below.

Many thanks for any help, much appreciated.

Best
 Marek Tichy








fq=((type:knihy) OR (type:defekty))
fl=*
start=0
f.ebook_formats.facet.mincount=1
f.authorid.facet.mincount=1
f.thematicgroupid.facet.mincount=1
f.articleparts.facet.mincount=1
f.type.facet.mincount=1
f.languageid.facet.mincount=1
f.showwindow.facet.mincount=1
f.articletypeid_grouped.facet.mincount=1
f.languageid.facet.limit=10
f.ebook_formats.facet.limit=10
f.authorid.facet.limit=10
f.type.facet.limit=10
f.articleparts.facet.limit=10
f.thematicgroupid.facet.limit=10
f.articletypeid_grouped.facet.limit=10
f.showwindow.facet.limit=100
version=2.2
group.limit=30
rows=30
echoParams=all
sort=date desc,planneddate asc
group.field=edition
facet.method=enum
group.truncate=false
group.format=grouped
group=true
group.ngroups=true
stats=true
facet=true
group.facet=true
stats.field={!distinctValues=true}categoryid
facet.field={!ex=at}articletypeid_grouped
facet.field={!ex=at}type
facet.field={!ex=author}authorid
facet.field={!ex=format}articleparts
facet.field={!ex=format}ebook_formats
facet.field={!ex=lang}languageid
facet.field={!ex=sw}showwindow
facet.field={!ex=tema}thematicgroupid
stats.field={!min=true max=true}price
stats.field={!min=true max=true}yearout

Re: Grouping and group.facet performance disaster

Posted by Susheel Kumar <su...@gmail.com>.
Did you try sub-facets ( http://yonik.com/json-facet-api/ )  if that meets
your facet grouping requirements or try Collapse/Expand Results.
https://cwiki.apache.org/confluence/display/solr/Collapse+and+Expand+Results


Thnx

On Wed, May 31, 2017 at 7:16 AM, Marek Tichy <ma...@gn.apc.org> wrote:

> Hi,
>
> I'm getting a very slow response times on grouping, especially on facet
> grouping.
>
> Without grouping, the query takes 14ms, faceting 57ms.
>
> With grouping, the query time goes up to 1131ms, with facet grouping,
> the faceting goes up to the unbearable 12103 ms.
>
> Single solr instance, 927086docs, 518.23 MB size, solr 6.4.1.
>
> Is this really the price of grouping ? Are there any magic
> tricks/tips/techniques to improve the speed ?
> The query params below.
>
> Many thanks for any help, much appreciated.
>
> Best
>  Marek Tichy
>
>
>
>
>
>
>
>
> fq=((type:knihy) OR (type:defekty))
> fl=*
> start=0
> f.ebook_formats.facet.mincount=1
> f.authorid.facet.mincount=1
> f.thematicgroupid.facet.mincount=1
> f.articleparts.facet.mincount=1
> f.type.facet.mincount=1
> f.languageid.facet.mincount=1
> f.showwindow.facet.mincount=1
> f.articletypeid_grouped.facet.mincount=1
> f.languageid.facet.limit=10
> f.ebook_formats.facet.limit=10
> f.authorid.facet.limit=10
> f.type.facet.limit=10
> f.articleparts.facet.limit=10
> f.thematicgroupid.facet.limit=10
> f.articletypeid_grouped.facet.limit=10
> f.showwindow.facet.limit=100
> version=2.2
> group.limit=30
> rows=30
> echoParams=all
> sort=date desc,planneddate asc
> group.field=edition
> facet.method=enum
> group.truncate=false
> group.format=grouped
> group=true
> group.ngroups=true
> stats=true
> facet=true
> group.facet=true
> stats.field={!distinctValues=true}categoryid
> facet.field={!ex=at}articletypeid_grouped
> facet.field={!ex=at}type
> facet.field={!ex=author}authorid
> facet.field={!ex=format}articleparts
> facet.field={!ex=format}ebook_formats
> facet.field={!ex=lang}languageid
> facet.field={!ex=sw}showwindow
> facet.field={!ex=tema}thematicgroupid
> stats.field={!min=true max=true}price
> stats.field={!min=true max=true}yearout
>

Re: [EXTERNAL] Grouping and group.facet performance disaster

Posted by "Sunil.Srinivasan" <Su...@target.com>.
Use group.cache.percent – for your index size, it might work well.

Thanks,

On 5/31/17, 4:16 AM, "Marek Tichy" <ma...@gn.apc.org> wrote:

    Hi,
    
    I'm getting a very slow response times on grouping, especially on facet
    grouping.
    
    Without grouping, the query takes 14ms, faceting 57ms.
    
    With grouping, the query time goes up to 1131ms, with facet grouping,
    the faceting goes up to the unbearable 12103 ms.
    
    Single solr instance, 927086docs, 518.23 MB size, solr 6.4.1.
    
    Is this really the price of grouping ? Are there any magic
    tricks/tips/techniques to improve the speed ?
    The query params below.
    
    Many thanks for any help, much appreciated.
    
    Best
     Marek Tichy
    
    
    
    
    
    
    
    
    fq=((type:knihy) OR (type:defekty))
    fl=*
    start=0
    f.ebook_formats.facet.mincount=1
    f.authorid.facet.mincount=1
    f.thematicgroupid.facet.mincount=1
    f.articleparts.facet.mincount=1
    f.type.facet.mincount=1
    f.languageid.facet.mincount=1
    f.showwindow.facet.mincount=1
    f.articletypeid_grouped.facet.mincount=1
    f.languageid.facet.limit=10
    f.ebook_formats.facet.limit=10
    f.authorid.facet.limit=10
    f.type.facet.limit=10
    f.articleparts.facet.limit=10
    f.thematicgroupid.facet.limit=10
    f.articletypeid_grouped.facet.limit=10
    f.showwindow.facet.limit=100
    version=2.2
    group.limit=30
    rows=30
    echoParams=all
    sort=date desc,planneddate asc
    group.field=edition
    facet.method=enum
    group.truncate=false
    group.format=grouped
    group=true
    group.ngroups=true
    stats=true
    facet=true
    group.facet=true
    stats.field={!distinctValues=true}categoryid
    facet.field={!ex=at}articletypeid_grouped
    facet.field={!ex=at}type
    facet.field={!ex=author}authorid
    facet.field={!ex=format}articleparts
    facet.field={!ex=format}ebook_formats
    facet.field={!ex=lang}languageid
    facet.field={!ex=sw}showwindow
    facet.field={!ex=tema}thematicgroupid
    stats.field={!min=true max=true}price
    stats.field={!min=true max=true}yearout

Re: Grouping and group.facet performance disaster

Posted by Ere Maijala <er...@helsinki.fi>.
While I can't say whether it affects you in this case, Solr 6.4.1 has 
serious performance issues. I'd suggest upgrading to at least 6.4.2.

--Ere

31.5.2017, 14.16, Marek Tichy kirjoitti:
> Hi,
>
> I'm getting a very slow response times on grouping, especially on facet
> grouping.
>
> Without grouping, the query takes 14ms, faceting 57ms.
>
> With grouping, the query time goes up to 1131ms, with facet grouping,
> the faceting goes up to the unbearable 12103 ms.
>
> Single solr instance, 927086docs, 518.23 MB size, solr 6.4.1.
>
> Is this really the price of grouping ? Are there any magic
> tricks/tips/techniques to improve the speed ?
> The query params below.
>
> Many thanks for any help, much appreciated.
>
> Best
>  Marek Tichy
>
>
>
>
>
>
>
>
> fq=((type:knihy) OR (type:defekty))
> fl=*
> start=0
> f.ebook_formats.facet.mincount=1
> f.authorid.facet.mincount=1
> f.thematicgroupid.facet.mincount=1
> f.articleparts.facet.mincount=1
> f.type.facet.mincount=1
> f.languageid.facet.mincount=1
> f.showwindow.facet.mincount=1
> f.articletypeid_grouped.facet.mincount=1
> f.languageid.facet.limit=10
> f.ebook_formats.facet.limit=10
> f.authorid.facet.limit=10
> f.type.facet.limit=10
> f.articleparts.facet.limit=10
> f.thematicgroupid.facet.limit=10
> f.articletypeid_grouped.facet.limit=10
> f.showwindow.facet.limit=100
> version=2.2
> group.limit=30
> rows=30
> echoParams=all
> sort=date desc,planneddate asc
> group.field=edition
> facet.method=enum
> group.truncate=false
> group.format=grouped
> group=true
> group.ngroups=true
> stats=true
> facet=true
> group.facet=true
> stats.field={!distinctValues=true}categoryid
> facet.field={!ex=at}articletypeid_grouped
> facet.field={!ex=at}type
> facet.field={!ex=author}authorid
> facet.field={!ex=format}articleparts
> facet.field={!ex=format}ebook_formats
> facet.field={!ex=lang}languageid
> facet.field={!ex=sw}showwindow
> facet.field={!ex=tema}thematicgroupid
> stats.field={!min=true max=true}price
> stats.field={!min=true max=true}yearout
>

-- 
Ere Maijala
Kansalliskirjasto / The National Library of Finland