large query producing graph error ... maybe?

[John Blythe <jo...@gmail.com>]

hey all!

i'm having an issue w large queries. one of our use cases is for users to
drop in an untold amount of product skus. we previously had our
maxBooleanClause limit set to 20k (eek!). but it worked phenomenally well
and i think our record amount from a user was ~19k items.

we're now on 7.4 Cloud. i'm getting this error when testing with a measly
600 skus:

org.apache.lucene.util.graph.GraphTokenStreamFiniteStrings.articulationPointsRecurse(GraphTokenStreamFiniteStrings.java:278)\n\tat


there's a lot more to the error message but that is the tail end of it all
and is repeated a lot of times (maybe 600? idk).

no mention of maxBooleanClause issues specifically in the output, shows as
a stack overflow error.

is this something we can solve in our solr/cloud/zk configuration or is it
somewhere else to be solved?

thanks!

--
John Blythe

Re: large query producing graph error ... maybe?

[John Blythe <jo...@gmail.com>]

well, it's our general text field that things get dumped into. this special
use case that is sku specific just ends up being done on the general input.

ended up raising the Xss value and i'm able to get results :)

i imagine this is a n00b or stupid question but imma go for it: what would
be the value add on fq + cache=false variation?

thanks for the help!




--
John Blythe


On Wed, Sep 12, 2018 at 1:02 PM Erick Erickson <er...@gmail.com>
wrote:

> Looks like your SKU field is points-based? Strings would probably be
> better, if you switched to points-based it's new code.
>
> And maxBooleanClauses is sooooo old-school ;) You're better off with
> TermsQueryParser, especially if you pre-sort the tokens. see:
> https://lucene.apache.org/solr/guide/6_6/other-parsers.html
>
> Although IIRC this is automagic in recent Solr's. I'd also put it in
> an fq clause with cache=false...
>
> Best,
> Erick
> On Wed, Sep 12, 2018 at 8:27 AM John Blythe <jo...@gmail.com> wrote:
> >
> > hey all!
> >
> > i'm having an issue w large queries. one of our use cases is for users to
> > drop in an untold amount of product skus. we previously had our
> > maxBooleanClause limit set to 20k (eek!). but it worked phenomenally well
> > and i think our record amount from a user was ~19k items.
> >
> > we're now on 7.4 Cloud. i'm getting this error when testing with a measly
> > 600 skus:
> >
> >
> org.apache.lucene.util.graph.GraphTokenStreamFiniteStrings.articulationPointsRecurse(GraphTokenStreamFiniteStrings.java:278)\n\tat
> >
> >
> > there's a lot more to the error message but that is the tail end of it
> all
> > and is repeated a lot of times (maybe 600? idk).
> >
> > no mention of maxBooleanClause issues specifically in the output, shows
> as
> > a stack overflow error.
> >
> > is this something we can solve in our solr/cloud/zk configuration or is
> it
> > somewhere else to be solved?
> >
> > thanks!
> >
> > --
> > John Blythe
>

Re: large query producing graph error ... maybe?

[Erick Erickson <er...@gmail.com>]

Looks like your SKU field is points-based? Strings would probably be
better, if you switched to points-based it's new code.

And maxBooleanClauses is sooooo old-school ;) You're better off with
TermsQueryParser, especially if you pre-sort the tokens. see:
https://lucene.apache.org/solr/guide/6_6/other-parsers.html

Although IIRC this is automagic in recent Solr's. I'd also put it in
an fq clause with cache=false...

Best,
Erick
On Wed, Sep 12, 2018 at 8:27 AM John Blythe <jo...@gmail.com> wrote:
>
> hey all!
>
> i'm having an issue w large queries. one of our use cases is for users to
> drop in an untold amount of product skus. we previously had our
> maxBooleanClause limit set to 20k (eek!). but it worked phenomenally well
> and i think our record amount from a user was ~19k items.
>
> we're now on 7.4 Cloud. i'm getting this error when testing with a measly
> 600 skus:
>
> org.apache.lucene.util.graph.GraphTokenStreamFiniteStrings.articulationPointsRecurse(GraphTokenStreamFiniteStrings.java:278)\n\tat
>
>
> there's a lot more to the error message but that is the tail end of it all
> and is repeated a lot of times (maybe 600? idk).
>
> no mention of maxBooleanClause issues specifically in the output, shows as
> a stack overflow error.
>
> is this something we can solve in our solr/cloud/zk configuration or is it
> somewhere else to be solved?
>
> thanks!
>
> --
> John Blythe