facet.offset issue (previosly: [solr 3.4] anomaly during distributed facet query with 102 shards)

[Dmitry Kan <so...@gmail.com>]

Hi list,

We have encountered a weird bug related to the facet.offset parameter. In
short: the more general query is, that generates lots of hits, the higher
the risk of the facet.offset parameter to stop working.

In more detail:

1. Since getting all facets we need (facet.limit=1000) from around 100
shards didn't work for some broad query terms, like "the" (yes, we index
and search those too), we decided to paginate.

2. The facet page size is set to 100 for all pages starting the second one.
We start with: facet.offset=0&facet.limit=30, then continue with
facet.offset=30&facet.limit=100, then facet.offset=100&facet.limit=100 and
so on, until we get facet.offset=900.

All facets work just fine, until we hit facet.offset=700.

Debugging showed, that in the class HttpCommComponent static Executor
instance is created with a setting to terminate idle threads after 5 sec.
Our belief, is that this setting way too low for our billion document
scenario and broad searches. Setting this to 5 min seems to improve the
situation a bit, but not solve fully. This same class is no longer used in
4.2.1 (can anyone tell what's used instead in distributed faceting?) so it
isn't easy to compare these parts of the code.

Anyhow, playing now with this value in the hope to see some light in the
tunnel (would be good, if it is not the train).

One more question: can this be related to RAM allocation on the router and
/ or shards? If RAM isn't enough for some operations, why the router or
shards wouldn't just crash with OOM?

If anyone has other ideas for what to try / look into, that'll be much
appreciated.

Dmitry

Re: facet.offset issue (previosly: [solr 3.4] anomaly during distributed facet query with 102 shards)

[Dmitry Kan <so...@gmail.com>]

It is logical to use facet pagination for us, if it would work. It just
doesn't, probably due the data amount we store and imposed RAM settings.




On Tue, Apr 30, 2013 at 10:39 AM, Toke Eskildsen <te...@statsbiblioteket.dk>wrote:

> On Mon, 2013-04-29 at 18:22 +0200, Dmitry Kan wrote:
> > Does it even make sense to paginate in facet searches, if we require deep
> > paging?
>
> Whether it makes sense logically is up to you.
>
> Technically deep paging in facets could be implemented in approximately
> the same way as deep paging in search results: By providing previously
> received maximum count and value, the faceting module could skip ahead
> to the subsequent entries and only return those.
>
> It would be fairly simple to implement for single core, but tricky for
> distributed search. I am not aware of any initiatives in that direction.
>
> - Toke Eskildsen, State and University Library
>
>

Re: facet.offset issue (previosly: [solr 3.4] anomaly during distributed facet query with 102 shards)

[Toke Eskildsen <te...@statsbiblioteket.dk>]

On Mon, 2013-04-29 at 18:22 +0200, Dmitry Kan wrote:
> Does it even make sense to paginate in facet searches, if we require deep
> paging?

Whether it makes sense logically is up to you.

Technically deep paging in facets could be implemented in approximately
the same way as deep paging in search results: By providing previously
received maximum count and value, the faceting module could skip ahead
to the subsequent entries and only return those.

It would be fairly simple to implement for single core, but tricky for
distributed search. I am not aware of any initiatives in that direction.

- Toke Eskildsen, State and University Library

Re: facet.offset issue (previosly: [solr 3.4] anomaly during distributed facet query with 102 shards)

[Dmitry Kan <so...@gmail.com>]

Hi guys,

do you have any ideas?

Does it even make sense to paginate in facet searches, if we require deep
paging?

Dmitry


On Fri, Apr 26, 2013 at 11:09 PM, Dmitry Kan <so...@gmail.com> wrote:

> Hi list,
>
> We have encountered a weird bug related to the facet.offset parameter. In
> short: the more general query is, that generates lots of hits, the higher
> the risk of the facet.offset parameter to stop working.
>
> In more detail:
>
> 1. Since getting all facets we need (facet.limit=1000) from around 100
> shards didn't work for some broad query terms, like "the" (yes, we index
> and search those too), we decided to paginate.
>
> 2. The facet page size is set to 100 for all pages starting the second
> one. We start with: facet.offset=0&facet.limit=30, then continue with
> facet.offset=30&facet.limit=100, then facet.offset=100&facet.limit=100 and
> so on, until we get facet.offset=900.
>
> All facets work just fine, until we hit facet.offset=700.
>
> Debugging showed, that in the class HttpCommComponent static Executor
> instance is created with a setting to terminate idle threads after 5 sec.
> Our belief, is that this setting way too low for our billion document
> scenario and broad searches. Setting this to 5 min seems to improve the
> situation a bit, but not solve fully. This same class is no longer used in
> 4.2.1 (can anyone tell what's used instead in distributed faceting?) so it
> isn't easy to compare these parts of the code.
>
> Anyhow, playing now with this value in the hope to see some light in the
> tunnel (would be good, if it is not the train).
>
> One more question: can this be related to RAM allocation on the router and
> / or shards? If RAM isn't enough for some operations, why the router or
> shards wouldn't just crash with OOM?
>
> If anyone has other ideas for what to try / look into, that'll be much
> appreciated.
>
> Dmitry
>