You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Hector Castro <he...@gmail.com> on 2012/01/10 01:15:51 UTC

Solr core as a dispatcher

Hi,

Has anyone had success with multicore single node Solr configurations that have one core acting solely as a dispatcher for the other cores?  For example, say you had 4 populated Solr cores – configure a 5th to be the definitive endpoint with `shards` containing cores 1-4.  

Is there any advantage to this setup over simply having requests distributed randomly across the 4 populated cores (all with `shards` equal to cores 1-4)?  Is it even worth distributing requests across the cores over always hitting the same one?

Thanks,  

--  
Hector


Re: Solr core as a dispatcher

Posted by shlomi java <sh...@gmail.com>.
My point is this - *everyone *using shards needs to handle that uniqueness.
Why not make it as a mechanism in Solr, with the ability to plugin our own
implementation of the hashing method?

Or is that what Solr Cloud is all about?

SJ

On Wed, Jan 11, 2012 at 4:33 PM, Hector Castro <he...@gmail.com> wrote:

> In our setup, we handle the document distribution and uniqueness across
> cores outside of Solr.
>
> --
> Hector
>
> On Jan 11, 2012, at 1:53 AM, shlomi java wrote:
>
> > Straying a bit from the subject,
> >
> > don't you think it will be useful to have the shards parameter used also
> in
> > the index, in order to maintain document uniqueness?
> > I mean as an out of the box feature of Solr.
> >
> > Because the situation today is that a Solr's client working with a
> sharded
> > Solr is responsible for keeping a document uniqueness across all shards.
> >
> > *Solution *- let Solr decide in which shard to index a document, using a
> > plugable hashing method.
> >
> > What do you think?
> >
> > ShlomiJ
> >
> > On Tue, Jan 10, 2012 at 6:15 PM, Shawn Heisey <so...@elyograg.org> wrote:
> >
> >> On 1/9/2012 5:15 PM, Hector Castro wrote:
> >>
> >>> Hi,
> >>>
> >>> Has anyone had success with multicore single node Solr configurations
> >>> that have one core acting solely as a dispatcher for the other cores?
>  For
> >>> example, say you had 4 populated Solr cores – configure a 5th to be the
> >>> definitive endpoint with `shards` containing cores 1-4.
> >>>
> >>> Is there any advantage to this setup over simply having requests
> >>> distributed randomly across the 4 populated cores (all with `shards`
> equal
> >>> to cores 1-4)?  Is it even worth distributing requests across the cores
> >>> over always hitting the same one?
> >>>
> >>
> >> I've got a setup where a single index chain consists of seven cores
> across
> >> two servers.  Those seven cores do not have the shards parameter in
> them.
> >> I have what you call a dispatcher core (I call it a broker core) that
> >> contains the shards parameter, but has no index data.
> >>
> >> If you use a dispatcher core, your application does not need to be
> >> concerned with the makeup of your index, so you don't need to include a
> >> shards parameter with your request.  You can change the index
> distribution
> >> and not have to worry about your application configuration.  This is the
> >> major advantage to doing it this way.  Distributed search has some
> overhead
> >> and not all Solr features work with it, so if your application already
> >> knows which core will contain the data it is trying to find, it is
> better
> >> to query the right core directly.
> >>
> >> Be careful that you aren't adding a shards parameter to a core
> >> configuration that points at itself.  Solr will, as of the last time I
> >> checked, try to complete the recursion and will fail.
> >>
> >> Thanks,
> >> Shawn
> >>
> >>
>
>

Re: Solr core as a dispatcher

Posted by Hector Castro <he...@gmail.com>.
In our setup, we handle the document distribution and uniqueness across cores outside of Solr.

--
Hector

On Jan 11, 2012, at 1:53 AM, shlomi java wrote:

> Straying a bit from the subject,
> 
> don't you think it will be useful to have the shards parameter used also in
> the index, in order to maintain document uniqueness?
> I mean as an out of the box feature of Solr.
> 
> Because the situation today is that a Solr's client working with a sharded
> Solr is responsible for keeping a document uniqueness across all shards.
> 
> *Solution *- let Solr decide in which shard to index a document, using a
> plugable hashing method.
> 
> What do you think?
> 
> ShlomiJ
> 
> On Tue, Jan 10, 2012 at 6:15 PM, Shawn Heisey <so...@elyograg.org> wrote:
> 
>> On 1/9/2012 5:15 PM, Hector Castro wrote:
>> 
>>> Hi,
>>> 
>>> Has anyone had success with multicore single node Solr configurations
>>> that have one core acting solely as a dispatcher for the other cores?  For
>>> example, say you had 4 populated Solr cores – configure a 5th to be the
>>> definitive endpoint with `shards` containing cores 1-4.
>>> 
>>> Is there any advantage to this setup over simply having requests
>>> distributed randomly across the 4 populated cores (all with `shards` equal
>>> to cores 1-4)?  Is it even worth distributing requests across the cores
>>> over always hitting the same one?
>>> 
>> 
>> I've got a setup where a single index chain consists of seven cores across
>> two servers.  Those seven cores do not have the shards parameter in them.
>> I have what you call a dispatcher core (I call it a broker core) that
>> contains the shards parameter, but has no index data.
>> 
>> If you use a dispatcher core, your application does not need to be
>> concerned with the makeup of your index, so you don't need to include a
>> shards parameter with your request.  You can change the index distribution
>> and not have to worry about your application configuration.  This is the
>> major advantage to doing it this way.  Distributed search has some overhead
>> and not all Solr features work with it, so if your application already
>> knows which core will contain the data it is trying to find, it is better
>> to query the right core directly.
>> 
>> Be careful that you aren't adding a shards parameter to a core
>> configuration that points at itself.  Solr will, as of the last time I
>> checked, try to complete the recursion and will fail.
>> 
>> Thanks,
>> Shawn
>> 
>> 


Re: Solr core as a dispatcher

Posted by shlomi java <sh...@gmail.com>.
Straying a bit from the subject,

don't you think it will be useful to have the shards parameter used also in
the index, in order to maintain document uniqueness?
I mean as an out of the box feature of Solr.

Because the situation today is that a Solr's client working with a sharded
Solr is responsible for keeping a document uniqueness across all shards.

*Solution *- let Solr decide in which shard to index a document, using a
plugable hashing method.

What do you think?

ShlomiJ

On Tue, Jan 10, 2012 at 6:15 PM, Shawn Heisey <so...@elyograg.org> wrote:

> On 1/9/2012 5:15 PM, Hector Castro wrote:
>
>> Hi,
>>
>> Has anyone had success with multicore single node Solr configurations
>> that have one core acting solely as a dispatcher for the other cores?  For
>> example, say you had 4 populated Solr cores – configure a 5th to be the
>> definitive endpoint with `shards` containing cores 1-4.
>>
>> Is there any advantage to this setup over simply having requests
>> distributed randomly across the 4 populated cores (all with `shards` equal
>> to cores 1-4)?  Is it even worth distributing requests across the cores
>> over always hitting the same one?
>>
>
> I've got a setup where a single index chain consists of seven cores across
> two servers.  Those seven cores do not have the shards parameter in them.
>  I have what you call a dispatcher core (I call it a broker core) that
> contains the shards parameter, but has no index data.
>
> If you use a dispatcher core, your application does not need to be
> concerned with the makeup of your index, so you don't need to include a
> shards parameter with your request.  You can change the index distribution
> and not have to worry about your application configuration.  This is the
> major advantage to doing it this way.  Distributed search has some overhead
> and not all Solr features work with it, so if your application already
> knows which core will contain the data it is trying to find, it is better
> to query the right core directly.
>
> Be careful that you aren't adding a shards parameter to a core
> configuration that points at itself.  Solr will, as of the last time I
> checked, try to complete the recursion and will fail.
>
> Thanks,
> Shawn
>
>

Re: Solr core as a dispatcher

Posted by Shawn Heisey <so...@elyograg.org>.
On 1/9/2012 5:15 PM, Hector Castro wrote:
> Hi,
>
> Has anyone had success with multicore single node Solr configurations that have one core acting solely as a dispatcher for the other cores?  For example, say you had 4 populated Solr cores – configure a 5th to be the definitive endpoint with `shards` containing cores 1-4.
>
> Is there any advantage to this setup over simply having requests distributed randomly across the 4 populated cores (all with `shards` equal to cores 1-4)?  Is it even worth distributing requests across the cores over always hitting the same one?

I've got a setup where a single index chain consists of seven cores 
across two servers.  Those seven cores do not have the shards parameter 
in them.  I have what you call a dispatcher core (I call it a broker 
core) that contains the shards parameter, but has no index data.

If you use a dispatcher core, your application does not need to be 
concerned with the makeup of your index, so you don't need to include a 
shards parameter with your request.  You can change the index 
distribution and not have to worry about your application 
configuration.  This is the major advantage to doing it this way.  
Distributed search has some overhead and not all Solr features work with 
it, so if your application already knows which core will contain the 
data it is trying to find, it is better to query the right core directly.

Be careful that you aren't adding a shards parameter to a core 
configuration that points at itself.  Solr will, as of the last time I 
checked, try to complete the recursion and will fail.

Thanks,
Shawn


Re: Solr core as a dispatcher

Posted by Hector Castro <he...@gmail.com>.
In my case the cores are populated with different records that adhere to the same schema. The question about randomly distributing requests is because each core has the `shards` parameter populated so that it can hit the other core's indexes.

My question is more about the advantages (if any) of utilizing a dispatcher core vs. simply querying the populated cores. 

--
Hector

On Jan 10, 2012, at 1:57 AM, shlomi java <sh...@gmail.com> wrote:

> If you want to randomly distribute requests across shards, then I think
> it's a case of Replication.
> 
> In Replication setup, all cores have the same schema AND data, so query any
> core should return the same result. It is used to support heavy load. Of
> course such setup will required some kind of load balancer.
> 
> In Distributed Search the shards have the same schema, but NOT the same
> data. So there is no point of randomly querying a shard, because we will
> get randomly different results.
> 
> ShlomiJ
> 
> On Tue, Jan 10, 2012 at 2:15 AM, Hector Castro <he...@gmail.com> wrote:
> 
>> Hi,
>> 
>> Has anyone had success with multicore single node Solr configurations that
>> have one core acting solely as a dispatcher for the other cores?  For
>> example, say you had 4 populated Solr cores – configure a 5th to be the
>> definitive endpoint with `shards` containing cores 1-4.
>> 
>> Is there any advantage to this setup over simply having requests
>> distributed randomly across the 4 populated cores (all with `shards` equal
>> to cores 1-4)?  Is it even worth distributing requests across the cores
>> over always hitting the same one?
>> 
>> Thanks,
>> 
>> --
>> Hector
>> 
>> 

Re: Solr core as a dispatcher

Posted by shlomi java <sh...@gmail.com>.
If you want to randomly distribute requests across shards, then I think
it's a case of Replication.

In Replication setup, all cores have the same schema AND data, so query any
core should return the same result. It is used to support heavy load. Of
course such setup will required some kind of load balancer.

In Distributed Search the shards have the same schema, but NOT the same
data. So there is no point of randomly querying a shard, because we will
get randomly different results.

ShlomiJ

On Tue, Jan 10, 2012 at 2:15 AM, Hector Castro <he...@gmail.com> wrote:

> Hi,
>
> Has anyone had success with multicore single node Solr configurations that
> have one core acting solely as a dispatcher for the other cores?  For
> example, say you had 4 populated Solr cores – configure a 5th to be the
> definitive endpoint with `shards` containing cores 1-4.
>
> Is there any advantage to this setup over simply having requests
> distributed randomly across the 4 populated cores (all with `shards` equal
> to cores 1-4)?  Is it even worth distributing requests across the cores
> over always hitting the same one?
>
> Thanks,
>
> --
> Hector
>
>

Re: Solr core as a dispatcher

Posted by Hector Castro <he...@gmail.com>.
Thanks for the reply, Ken – it was your training session that brought the dispatcher core approach to my attention in the first place.  

Regarding your deep query point, if you're in a situation where numFound=5000 and you're trying to output all 5000 records at once – your point seems to suggest that you're better off setting rows=5000 instead of chunking by 100.  Is that correct?   

--  
Hector


On Wednesday, January 11, 2012 at 7:10 PM, Ken Krugler wrote:

> Hi Hector,
>  
> On Jan 9, 2012, at 4:15pm, Hector Castro wrote:
>  
> > Hi,
> >  
> > Has anyone had success with multicore single node Solr configurations that have one core acting solely as a dispatcher for the other cores? For example, say you had 4 populated Solr cores – configure a 5th to be the definitive endpoint with `shards` containing cores 1-4.  
> >  
> > Is there any advantage to this setup over simply having requests distributed randomly across the 4 populated cores (all with `shards` equal to cores 1-4)? Is it even worth distributing requests across the cores over always hitting the same one?
>  
> If you have low query rates, then using a shards approach can improve performance on a multi-core (CPUs here, not Solr cores) setup.
>  
> By distributing the requests, you effectively use all CPU cores in parallel on one request.
>  
> And if you spread your shards across spindles, then you're also maximizing I/O throughput.
>  
> But there are a few issues with this approach:
>  
> - binary fields don't work. The results come back as "@B[<hex address>]", versus the actual data.
> - short fields get "java.lang.Short" text prefixed on every value.
> - deep queries result in lots of extra load. E.g. if you want the 5000th hit then you'll get (5000 * # of shards) hits being collected/returned to the dispatcher. Though only the unique id & score is returned in this case, followed by the second request to get the actual top N hits from the shards.
>  
> And there's something wonky with the way that distributed HTTP requests are queued up & processed - under load, I see IOExceptions where it's always N-1 shards that succeed, and one shard request fails. But I don't have a good reproducible case yet to debug.
>  
> -- Ken
>  
> --------------------------
> Ken Krugler
> http://www.scaleunlimited.com
> custom big data solutions & training
> Hadoop, Cascading, Mahout & Solr
>  
>  



Re: Solr core as a dispatcher

Posted by Chris Hostetter <ho...@fucit.org>.
: - binary fields don't work. The results come back as "@B[<hex address>]", versus the actual data.
: - short fields get "java.lang.Short" text prefixed on every value.

these sound like they must be bugs in the javabin codec, certianly not 
intentional

Ken: did you file jiras about these?


-Hoss

Re: Solr core as a dispatcher

Posted by Ken Krugler <kk...@transpac.com>.
Hi Hector,

On Jan 9, 2012, at 4:15pm, Hector Castro wrote:

> Hi,
> 
> Has anyone had success with multicore single node Solr configurations that have one core acting solely as a dispatcher for the other cores?  For example, say you had 4 populated Solr cores – configure a 5th to be the definitive endpoint with `shards` containing cores 1-4.  
> 
> Is there any advantage to this setup over simply having requests distributed randomly across the 4 populated cores (all with `shards` equal to cores 1-4)?  Is it even worth distributing requests across the cores over always hitting the same one?

If you have low query rates, then using a shards approach can improve performance on a multi-core (CPUs here, not Solr cores) setup.

By distributing the requests, you effectively use all CPU cores in parallel on one request.

And if you spread your shards across spindles, then you're also maximizing I/O throughput.

But there are a few issues with this approach:

- binary fields don't work. The results come back as "@B[<hex address>]", versus the actual data.
- short fields get "java.lang.Short" text prefixed on every value.
- deep queries result in lots of extra load. E.g. if you want the 5000th hit then you'll get (5000 * # of shards) hits being collected/returned to the dispatcher. Though only the unique id & score is returned in this case, followed by the second request to get the actual top N hits from the shards.

And there's something wonky with the way that distributed HTTP requests are queued up & processed - under load, I see IOExceptions where it's always N-1 shards that succeed, and one shard request fails. But I don't have a good reproducible case yet to debug.

-- Ken

--------------------------
Ken Krugler
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Mahout & Solr