You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Mark <st...@gmail.com> on 2013/11/07 21:58:30 UTC
Sharding and replicas (Solr Cloud)
If I create my collection via the ZkCLI (https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities) how do I configure the number of shards and replicas?
Thanks
Re: Sharding and replicas (Solr Cloud)
Posted by Shawn Heisey <so...@elyograg.org>.
On 11/7/2013 4:34 PM, Software Dev wrote:
> I too want to be in control of everything that is created.
>
> Here is what I'm trying to do.
>
> 1) Start up a cluster of 5 Solr Instances
> 2) Import the configuration to Zookeeper
> 3) Manually create a collection via the collections api with number of
> shards and replication factor
>
> Now there are some issues with step 3. After creating the collection reload
> the GUI I always see:
>
> - *collection1:*
> org.apache.solr.common.cloud.ZooKeeperException:org.apache.solr.common.cloud.ZooKeeperException:
> Could not find configName for collection collection1 found:null
>
> until I restart the cluster. Is there a way around this?
>
> Also after creating the collection it creates a directory in
> $SOLR_HOME/home. So in this example it created
> ${SOLR_HOME}/collection1_shard1_replica1 and
> ${SOLR_HOME}/collection1_shard1_replica2. What happens when I rename both
> of these to the same in the core admin?
Don't rename the cores. That is a feature that's been around forever,
but many of the core-level operations (like renaming and swapping) will
break SolrCloud. Why do you want to rename them?
How many Solr servers (or virtual machines) do you have, how many Solr
instances are running on each server, and what was the exact collections
API URL that you called, with all the parameters that you used? You
should never end up with two replicas of the same shard on the same Solr
instance unless something is done incorrectly or goes very wrong.
Thanks,
Shawn
Re: Sharding and replicas (Solr Cloud)
Posted by Software Dev <st...@gmail.com>.
I too want to be in control of everything that is created.
Here is what I'm trying to do.
1) Start up a cluster of 5 Solr Instances
2) Import the configuration to Zookeeper
3) Manually create a collection via the collections api with number of
shards and replication factor
Now there are some issues with step 3. After creating the collection reload
the GUI I always see:
- *collection1:*
org.apache.solr.common.cloud.ZooKeeperException:org.apache.solr.common.cloud.ZooKeeperException:
Could not find configName for collection collection1 found:null
until I restart the cluster. Is there a way around this?
Also after creating the collection it creates a directory in
$SOLR_HOME/home. So in this example it created
${SOLR_HOME}/collection1_shard1_replica1 and
${SOLR_HOME}/collection1_shard1_replica2. What happens when I rename both
of these to the same in the core admin?
On Thu, Nov 7, 2013 at 3:15 PM, Shawn Heisey <so...@elyograg.org> wrote:
> On 11/7/2013 2:52 PM, Software Dev wrote:
>
>> Sorry about the confusion. I meant I created my config via the ZkCLI and
>> then I wanted to create my core via the CollectionsAPI. I *think* I have
>> it
>> working but was wondering why there are a crazy amount of core names under
>> the admin "Core Selector"?
>>
>> When I create X amount of shards via the bootstrap command I think it only
>> creates 1 core. Am I missing something?
>>
>
> If you create it with numShards=1 and replicationFactor=2, you'll end up
> with a total of 2 cores across all your Solr instances. For my simple
> cloud install, these are the numbers that I'm using. One shard, a total of
> two copies.
>
> If you create it with the numbers given on the wiki page, numShards=3 and
> replicationFactor=4, there would be a total of 12 cores created across all
> your servers. The maxShardsPerNode parameter defaults to 1, which means
> that only 1 core per instance (SolrCloud node) is allowed for that
> collection. If there aren't enough Solr instances for the numbers you have
> entered, the creation will fail.
>
> I don't know any details about what the bootstrap_conf parameter actually
> does when it creates collections. I've never used it - I want to be in
> control of the configs and collections that get created.
>
> Thanks,
> Shawn
>
>
Re: Sharding and replicas (Solr Cloud)
Posted by Shawn Heisey <so...@elyograg.org>.
On 11/7/2013 2:52 PM, Software Dev wrote:
> Sorry about the confusion. I meant I created my config via the ZkCLI and
> then I wanted to create my core via the CollectionsAPI. I *think* I have it
> working but was wondering why there are a crazy amount of core names under
> the admin "Core Selector"?
>
> When I create X amount of shards via the bootstrap command I think it only
> creates 1 core. Am I missing something?
If you create it with numShards=1 and replicationFactor=2, you'll end up
with a total of 2 cores across all your Solr instances. For my simple
cloud install, these are the numbers that I'm using. One shard, a total
of two copies.
If you create it with the numbers given on the wiki page, numShards=3
and replicationFactor=4, there would be a total of 12 cores created
across all your servers. The maxShardsPerNode parameter defaults to 1,
which means that only 1 core per instance (SolrCloud node) is allowed
for that collection. If there aren't enough Solr instances for the
numbers you have entered, the creation will fail.
I don't know any details about what the bootstrap_conf parameter
actually does when it creates collections. I've never used it - I want
to be in control of the configs and collections that get created.
Thanks,
Shawn
Re: Sharding and replicas (Solr Cloud)
Posted by Software Dev <st...@gmail.com>.
Sorry about the confusion. I meant I created my config via the ZkCLI and
then I wanted to create my core via the CollectionsAPI. I *think* I have it
working but was wondering why there are a crazy amount of core names under
the admin "Core Selector"?
When I create X amount of shards via the bootstrap command I think it only
creates 1 core. Am I missing something?
On Thu, Nov 7, 2013 at 1:06 PM, Shawn Heisey <el...@elyograg.org> wrote:
> On 11/7/2013 1:58 PM, Mark wrote:
>
>> If I create my collection via the ZkCLI (https://cwiki.apache.org/
>> confluence/display/solr/Command+Line+Utilities) how do I configure the
>> number of shards and replicas?
>>
>
> I was not aware that you could create collections with zkcli. I did not
> think that was possible. Use the collections API:
>
> http://wiki.apache.org/solr/SolrCloud#Managing_collections_via_the_
> Collections_API
>
> Thanks,
> Shawn
>
>
Re: Sharding and replicas (Solr Cloud)
Posted by Shawn Heisey <el...@elyograg.org>.
On 11/7/2013 1:58 PM, Mark wrote:
> If I create my collection via the ZkCLI (https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities) how do I configure the number of shards and replicas?
I was not aware that you could create collections with zkcli. I did
not think that was possible. Use the collections API:
http://wiki.apache.org/solr/SolrCloud#Managing_collections_via_the_Collections_API
Thanks,
Shawn