Achieve replication in shards for SolrCloud cluster

[Deeksha Sharma <ds...@palamida.com>]

I am new to Solr and already have Lucene indexes that I want to serve through SolrCloud.

I have SolrCloud setup with external zookeeper and 2 Solr Instances - solr1 and solr2registered with this zookeeper.

On solr1 I add a symlink to my existing Lucene indexes. (and not on solr2)

I create the collection with following command​

$ bin/solr create_collection -c sharedCollection -d /conf -shards 2 -p 8983

Connecting to ZooKeeper at 172.17.0.2:2181 ...
Uploading /conf for config sharedCollection to ZooKeeper at 172.17.0.2:2181

Creating new collection 'sharedCollection' using command:
http://localhost:8983/solr/admin/collections?action=CREATE&name=sharedCollection&numShards=2&replicationFactor=1&maxShardsPerNode=1&collection.configName=sharedCollection

{
  "responseHeader":{
    "status":0,
    "QTime":9905},
  "success":{
    "172.17.0.4:8983_solr":{
      "responseHeader":{
        "status":0,
        "QTime":7680},
      "core":"sharedCollection_shard1_replica1"},
    "172.17.0.3:8983_solr":{
      "responseHeader":{
        "status":0,
        "QTime":7945},
      "core":"sharedCollection_shard2_replica1"}}}

Now when I go to solr1 Admin Page on Browser, I see that solr1 has all the documents and solr2 has none.


Then I try to split shard using

http://192.168.99.100:8983/solr/admin/collections?action=SPLITSHARD&collection=sharedCollection&shard=shard1


<?xml version="1.0" encoding="UTF-8"?>
<response>
    <lst name="responseHeader">
        <int name="status">500</int>
        <int name="QTime">3525</int>
    </lst>
    <lst name="failure">
        <str name="172.17.0.4:8983_solr">org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error from server at http://172.17.0.4:8983/solr: Error CREATEing SolrCore 'sharedCollection_shard1_0_replica1': Unable to create core [sharedCollection_shard1_0_replica1] Caused by: Lock held by this virtual machine: /external_index/index/write.lock</str>
    </lst>
    <str name="Operation splitshard caused exception:">org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: ADDREPLICA failed to create replica</str>
    <lst name="exception">
        <str name="msg">ADDREPLICA failed to create replica</str>
        <int name="rspCode">500</int>
    </lst>
    <lst name="error">
        <lst name="metadata">
            <str name="error-class">org.apache.solr.common.SolrException</str>
            <str name="root-error-class">org.apache.solr.common.SolrException</str>
        </lst>
        <str name="msg">ADDREPLICA failed to create replica</str>
        <str name="trace">org.apache.solr.common.SolrException: ADDREPLICA failed to create replica
    at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:266)
    at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:197)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
    at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:658)
    at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
    at org.eclipse.jetty.server.Server.handle(Server.java:518)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
    at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
    at java.lang.Thread.run(Thread.java:745)
</str>
        <int name="code">500</int>
    </lst>


What I want to do
- I want to split these Lucene indexes into different shards on different nodes so that there is replication of indexes.
- But all the data still resides on solr1 that is hosting shard1 and there are no documents on solr2 that hosts shard2.

Can anyone please suggest how to achieve replication?​