You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by spillane <sp...@aol.com> on 2015/03/02 19:39:57 UTC
SolrCloud default shard assignment order not correct
I'm trying to build a SolrCloud with 4 leader/replica pairs. Starting my
first host with this tomcat6.conf setting
CATALINA_OPTS="${CATALINA_OPTS} -DnumShards=4
-Dbootstrap_confdir=/data/solr/rockbands/conf
-Dcollection.configName=rockbands_conf -DzkHost=zk1:2181,zk2:2181,zk3:2181"
and then starting up host2, host3 and host4 in order I am expecting to see a
cloud graph like this
shard1 - host1 ip
rockbands --> shard2 - host2 ip
shard3 - host3 ip
shard4 - host4 ip
But the cloud graph shows this order after the first host is started
shard1 - host1 ip
rockbands --> shard4
shard2
shard3
Clearing out the /var/lib/zookeeper directory (excepting myid), restarting
all zookeeper hosts and restarting all SOLR hosts does not help. I would
like to know why I cannot get these in correct order.
This is SOLR 4.2
--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-default-shard-assignment-order-not-correct-tp4190426.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud default shard assignment order not correct
Posted by spillane <sp...@aol.com>.
Shawn,
The 4.2 cloud graph ordering turned out not to be a problem, after the first
startup of my 5 leaders and 5 replicas their shard assignments were 'fixed'
in Zookeeper. I can now start them in any order and get the same graph.
<http://lucene.472066.n3.nabble.com/file/n4202702/cloudview.jpg>
I haven't gotten to 5.0 yet but will revisit your comments on that soon.
--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-default-shard-assignment-order-not-correct-tp4190426p4202702.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud default shard assignment order not correct
Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/6/2015 1:34 AM, Shawn Heisey wrote:
> In Solr 5.0, the cloud graph is sorting the collections by name. The
> shard names also appear to be sorted -- all the collections I have on
> the example cloud setup only have two shards, so I really can't be sure.
> It might also be sorting the replicas within each shard.
I built a collection that would tell me what exactly is sorted in Solr
5.0. The collections are sorted and the shards are sorted, but the
replicas are NOT sorted. Because there are normally only a few replicas
and the leader is clearly marked, I don't see that as a problem, but if
you really want them sorted, feel free to open an issue in Jira.
Screenshot:
https://www.dropbox.com/s/yzkubdbj86dbkda/solr5-cloud-graph-sorting.png?dl=0
SOLR project in Jira:
https://issues.apache.org/jira/browse/SOLR
Thanks,
Shawn
Re: SolrCloud default shard assignment order not correct
Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/2/2015 2:12 PM, spillane wrote:
> Since the order is consistently 1,4,2,3 it sounds like I can start the
> leaders in 1,4,2,3 order and then replicas in 1,4,2,3 order and expect the
> relationships to stick
>
> leader1 -> replica1
> leader4 -> replica4
> leader2 -> replica2
> leader3 -> replica3
In Solr 5.0, the cloud graph is sorting the collections by name. The
shard names also appear to be sorted -- all the collections I have on
the example cloud setup only have two shards, so I really can't be sure.
It might also be sorting the replicas within each shard.
I looked for an issue so I would know what version first included the
sort, but I could not find one. I only know that 4.2 does not have the
sort, and 5.0 does.
Thanks,
Shawn
Re: SolrCloud default shard assignment order not correct
Posted by spillane <sp...@aol.com>.
Since the order is consistently 1,4,2,3 it sounds like I can start the
leaders in 1,4,2,3 order and then replicas in 1,4,2,3 order and expect the
relationships to stick
leader1 -> replica1
leader4 -> replica4
leader2 -> replica2
leader3 -> replica3
--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-default-shard-assignment-order-not-correct-tp4190426p4190487.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud default shard assignment order not correct
Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/2/2015 11:39 AM, spillane wrote:
> and then starting up host2, host3 and host4 in order I am expecting to see a
> cloud graph like this
>
> shard1 - host1 ip
> rockbands --> shard2 - host2 ip
> shard3 - host3 ip
> shard4 - host4 ip
>
> But the cloud graph shows this order after the first host is started
>
> shard1 - host1 ip
> rockbands --> shard4
> shard2
> shard3
I have not looked at the code, so I could be completely wrong about
this, but I am guessing that the results you see are because of the code
being implemented in a specific way for performance reasons.
It's very likely that these pieces of information are stored in a Map
structure in the java code, and it is most likely a HashMap, because
that is a very fast Map implementation. When it is time to return that
information to the admin UI cloud graph, it probably uses the "keySet()"
method on the Map object. HashMap makes no guarantees about the order
of the elements in the map, and adding more entries very likely will
change the order:
http://docs.oracle.com/javase/7/docs/api/java/util/HashMap.html
There should probably be one or more options available for the API calls
that the admin UI uses, which can specify that the list should be
sorted, and possibly how to sort it. Our UI expert will tell us whether
the UI should have options to specify sort parameters, or whether it
will simply sort by default. The UI work may require a separate issue
in Jira.
Thanks,
Shawn