Query in Solr plugin across shards

[Kevin Osborn <ko...@centraldesktop.com>]

I have created a PostFilter. PostFilter creates a DelegatingCollector,
which provides a Lucene IndexSearcher.

However, I need to query for an object that may or may not be located on
the shard that I am filtering on.

Normally, I would do something like:

searcher.search(new TermQuery(new Term("field", "value").scoreDocs

But this does not work across shards. So, if the document I am looking for
is on a different shard, I get no results.

Any idea how I would best do my search across all shards from within my
plugin?

Re: Query in Solr plugin across shards

[Kevin Osborn <ko...@centraldesktop.com>]

Thanks.

It is a fairly large ACL, so I am hoping to avoid any sort of application
redirect. That is sort of the problem we are trying to solve actually. Our
list was getting too large and we were maxing out maxBooleanQueries.

And I don't know which shard the user document is located on, just its
unique key. Although I suppose we could put all user objects on a single
shard. Not an ideal solution though.

I did see an option of using SolrJ CloudServer from within a plugin, but
that didn't seem very desirable to me.

-Kevin

On Mon, Mar 23, 2015 at 10:41 AM, Erick Erickson <er...@gmail.com>
wrote:

> How much information do you need from this document? If it's a reasonably
> small
> amount, can you read it at the application layer and attach it as a
> set of parameters
> to the query that are then available to the post filter. Or is it a
> huge ACL list of something....
>
> In this latter case, if you know the URL of a shard with the doc, you
> could send a query
> (that you perhaps cache in your postFilter code) to that shard with
> &distrib=false and
> get the doc. Hmmm, I suppose if this is a well-known doc ID you don't
> even have to know
> what the shard is, just send the request....
>
> BTW, there's a "userCache" that you can configure in solrconfig.xml
> that you might want
> to use, the advantage here is that it gets notified whenever a new
> searcher is opened
> so it can "do the right thing" in terms of refreshing itself.
>
> FWIW,
> Erick
>
> On Mon, Mar 23, 2015 at 9:23 AM, Kevin Osborn
> <ko...@centraldesktop.com> wrote:
> > I have created a PostFilter. PostFilter creates a DelegatingCollector,
> > which provides a Lucene IndexSearcher.
> >
> > However, I need to query for an object that may or may not be located on
> > the shard that I am filtering on.
> >
> > Normally, I would do something like:
> >
> > searcher.search(new TermQuery(new Term("field", "value").scoreDocs
> >
> > But this does not work across shards. So, if the document I am looking
> for
> > is on a different shard, I get no results.
> >
> > Any idea how I would best do my search across all shards from within my
> > plugin?
>

Re: Query in Solr plugin across shards

[Erick Erickson <er...@gmail.com>]

How much information do you need from this document? If it's a reasonably small
amount, can you read it at the application layer and attach it as a
set of parameters
to the query that are then available to the post filter. Or is it a
huge ACL list of something....

In this latter case, if you know the URL of a shard with the doc, you
could send a query
(that you perhaps cache in your postFilter code) to that shard with
&distrib=false and
get the doc. Hmmm, I suppose if this is a well-known doc ID you don't
even have to know
what the shard is, just send the request....

BTW, there's a "userCache" that you can configure in solrconfig.xml
that you might want
to use, the advantage here is that it gets notified whenever a new
searcher is opened
so it can "do the right thing" in terms of refreshing itself.

FWIW,
Erick

On Mon, Mar 23, 2015 at 9:23 AM, Kevin Osborn
<ko...@centraldesktop.com> wrote:
> I have created a PostFilter. PostFilter creates a DelegatingCollector,
> which provides a Lucene IndexSearcher.
>
> However, I need to query for an object that may or may not be located on
> the shard that I am filtering on.
>
> Normally, I would do something like:
>
> searcher.search(new TermQuery(new Term("field", "value").scoreDocs
>
> But this does not work across shards. So, if the document I am looking for
> is on a different shard, I get no results.
>
> Any idea how I would best do my search across all shards from within my
> plugin?