Querying Different Index Types in a single request

[Todd Benge <to...@gmail.com>]

Hi,

We're in the early stages of converting a large scale lucene deployment to
Solr.  The primary reason is to take advantage of the distributed search
capabilities.

We have multiple sets of indices that serve different types of documents.
In our existing deployment, we use MultiSearcher and custom servlets to load
those indices and serve the response in a single request.

An example of the request would be searching for some media type and we'll
respond with the search results with that type as well as suggested other
media of other types that may be applicable.

In looking through the process for sharding data across multiple indices, I
don't see a way to combine results from mutliple cores into a single
response.  It looks like any data that is added to the SolrQueryResponse
object that is not part of the standard "response" key is filtered out
during the merging of documents.

Is there any other way to query multiple index types in a single query
across a cluster?

Thanks for any help.

Todd

Re: Querying Different Index Types in a single request

[Todd Benge <to...@gmail.com>]

Hi,

There are 4 different sets of indices - 2 of which have a similar schema and
the other 2 have completely different schemas.

The intent would be to use a combination of a federated & distributed search
if possible.  The initial user query - is only for data from a  distributed
search.  Ideally, we execute additional queries in the same request against
other schemas so that we can return suggested results as well.

We were looking at writing a custom handler that would execute the incoming
query against other cores so that we can query other indices and then append
those results to the response.  After looking at the code, it seems that if
any results are added and are not in the expected document list are filtered
out during the merging.

In answer to you question for information are we missing, we have multiple
media types with different schemas and we'd like the results to be sorted &
scored separately.

Any suggestions are helpful.  I'm sure we can find a solution to the problem
but are just trying to find the best architectural solution.

Todd

On Mon, Mar 30, 2009 at 10:53 PM, Yonik Seeley
<yo...@lucidimagination.com>wrote:

> On Mon, Mar 30, 2009 at 7:07 PM, Todd Benge <to...@gmail.com> wrote:
> > Is there any other way to query multiple index types in a single query
> > across a cluster?
>
> Splitting a homogeneous index into shards, querying across them, and
> combining the results is distributed search.  If the shards are
> heterogeneous (have different "schemas") then it's closer to federated
> search.  Solr's distributed search may work for federated search if
> you combine all the schemas into one though.  What information are you
> missing from the distributed request that you can get from a single
> distributed Solr shard?
>
> -Yonik
> http://www.lucidimagination.com
>

Re: Querying Different Index Types in a single request

[Yonik Seeley <yo...@lucidimagination.com>]

On Mon, Mar 30, 2009 at 7:07 PM, Todd Benge <to...@gmail.com> wrote:
> Is there any other way to query multiple index types in a single query
> across a cluster?

Splitting a homogeneous index into shards, querying across them, and
combining the results is distributed search.  If the shards are
heterogeneous (have different "schemas") then it's closer to federated
search.  Solr's distributed search may work for federated search if
you combine all the schemas into one though.  What information are you
missing from the distributed request that you can get from a single
distributed Solr shard?

-Yonik
http://www.lucidimagination.com