You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Joshi, Shital" <Sh...@gs.com> on 2013/06/21 22:26:04 UTC
SPLITSHARD throws error
Hi,
We have 5 shards with replication factor 2 (total 10 jvm instances). Our shards are named (shardid) shard1,shard2,shard3,shar4 and shar5 and collection name is collection1. When we execute this command:
curl --proxy '' "http://$HOST_NAME:8983/solr/admin/collections?action=SPLITSHARD&collection=collection1&shard=shard1"
We get this error:
<lst name="failure">
<str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error CREATEing SolrCore 'collection1_shard1_0_replica1': Core with same data dir 'collection1' already exists.</str>
<str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error CREATEing SolrCore 'collection1_shard1_1_replica1': Core with same data dir 'collection1' already exists.</str>
<str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:core not found:collection1_shard1_0_replica1</str>
<str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:core not found:collection1_shard1_1_replica1</str>
<str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Server at http://$HOST:8983/solr returned non ok status:500, message:Server Error</str>
<str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Server at http://$HOST:8983/solr returned non ok status:500, message:Server Error</str>
<str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Server at http://$HOST:8983/solr returned non ok status:500, message:Server Error</str>
<str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error CREATEing SolrCore 'collection1_shard1_1_replica2': Unable to create core: collection1_shard1_1_replica2</str>
<str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error CREATEing SolrCore 'collection1_shard1_0_replica2': Core with same data dir 'collection1' already exists.</str>
<str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:core not found:collection1_shard1_0_replica1</str>
<str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:core not found:collection1_shard1_1_replica1</str>
</lst>
</response>
Does the collection name has to be different than existing collection? If yes, why is that? We're not creating a new collection. We just want to split existing shard.
Thanks much!
Re: SPLITSHARD throws error
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
The UI will still show shard1 after the split for two reasons:
1. The UI is not aware of shard states yet so even though shard1 is
inactive and won't process any requests, it shows up as "green" in the
Admin UI
2. A shard is *not* deleted automatically after a split so if you want
it to be thrown away, you must manually call unload on each of its
nodes. A "deleteshard" API will be released with Solr 4.4
On Tue, Jun 25, 2013 at 4:27 AM, Joshi, Shital <Sh...@gs.com> wrote:
> Hi Shawn,
>
> Thanks for your reply. I removed hardcoded dataDir from solr.xml and created different solr.xml per Solr instance. After these changes, the SPLITSHARD command returned successfully.
>
> Before splitting, shard1 was pointing to HOST1 and HOST2 (leader and replica) and shard2 was pointing to HOST3 and HOST4. I splited shard1 and now sharc1_0 and shard1_1 got created. But these two new shards are pointing to HOST1 and HOST3. I think it should point to HOST1 and HOST2 since the shard1 was originally pointing to HOST1 and HOST2.
>
> The solr cloud still shows me shard1 after splitting it. I will restart the cloud and see if it goes away.
>
> Thanks!
>
> -----Original Message-----
> From: Shawn Heisey [mailto:solr@elyograg.org]
> Sent: Friday, June 21, 2013 5:38 PM
> To: solr-user@lucene.apache.org
> Subject: Re: SPLITSHARD throws error
>
> On 6/21/2013 3:06 PM, Joshi, Shital wrote:
>>
>> This is what we have in solr.xml.
>>
>>
>> <solr persistent="true">
>> <cores adminPath="/admin/cores" defaultCoreName="collection1" host="${host:}" hostPort="${jetty.port:8983}" hostContext="${hostContext:solr}" zkClientTimeout="${zkClientTimeout:15000}">
>> <core name="collection1" instanceDir="collection1" dataDir="${solr.data.dir:}" shard="${shard:}" />
>> </cores>
>> </solr>
>>
>>
>> We use one solr.xml and pass dataDir, shard as java system property. (e.g.: -Dsolr.data.dir=$HOME/solr_data/solr6
>> -Dsolr.ulog.dir=$HOME/solr_data/solr6_tranlog -DnumShards=5 -Dshard=shard1)
>
> There's the problem. You can't hard-code dataDir with SolrCloud unless
> it's a relative path. It's trying to share the same dataDir between the
> two cores that it's creating from the old one. dataDir defaults to
> ${instanceDir}/data.
>
> The solr.xml file will get modified when you use the collections and
> core admin APIs, and each server will get different modifications, so
> sharing it between different Solr servers is not a good idea.
>
> Thanks,
> Shawn
>
--
Regards,
Shalin Shekhar Mangar.
RE: SPLITSHARD throws error
Posted by "Joshi, Shital" <Sh...@gs.com>.
Hi Shawn,
Thanks for your reply. I removed hardcoded dataDir from solr.xml and created different solr.xml per Solr instance. After these changes, the SPLITSHARD command returned successfully.
Before splitting, shard1 was pointing to HOST1 and HOST2 (leader and replica) and shard2 was pointing to HOST3 and HOST4. I splited shard1 and now sharc1_0 and shard1_1 got created. But these two new shards are pointing to HOST1 and HOST3. I think it should point to HOST1 and HOST2 since the shard1 was originally pointing to HOST1 and HOST2.
The solr cloud still shows me shard1 after splitting it. I will restart the cloud and see if it goes away.
Thanks!
-----Original Message-----
From: Shawn Heisey [mailto:solr@elyograg.org]
Sent: Friday, June 21, 2013 5:38 PM
To: solr-user@lucene.apache.org
Subject: Re: SPLITSHARD throws error
On 6/21/2013 3:06 PM, Joshi, Shital wrote:
>
> This is what we have in solr.xml.
>
>
> <solr persistent="true">
> <cores adminPath="/admin/cores" defaultCoreName="collection1" host="${host:}" hostPort="${jetty.port:8983}" hostContext="${hostContext:solr}" zkClientTimeout="${zkClientTimeout:15000}">
> <core name="collection1" instanceDir="collection1" dataDir="${solr.data.dir:}" shard="${shard:}" />
> </cores>
> </solr>
>
>
> We use one solr.xml and pass dataDir, shard as java system property. (e.g.: -Dsolr.data.dir=$HOME/solr_data/solr6
> -Dsolr.ulog.dir=$HOME/solr_data/solr6_tranlog -DnumShards=5 -Dshard=shard1)
There's the problem. You can't hard-code dataDir with SolrCloud unless
it's a relative path. It's trying to share the same dataDir between the
two cores that it's creating from the old one. dataDir defaults to
${instanceDir}/data.
The solr.xml file will get modified when you use the collections and
core admin APIs, and each server will get different modifications, so
sharing it between different Solr servers is not a good idea.
Thanks,
Shawn
Re: SPLITSHARD throws error
Posted by Shawn Heisey <so...@elyograg.org>.
On 6/21/2013 3:06 PM, Joshi, Shital wrote:
>
> This is what we have in solr.xml.
>
>
> <solr persistent="true">
> <cores adminPath="/admin/cores" defaultCoreName="collection1" host="${host:}" hostPort="${jetty.port:8983}" hostContext="${hostContext:solr}" zkClientTimeout="${zkClientTimeout:15000}">
> <core name="collection1" instanceDir="collection1" dataDir="${solr.data.dir:}" shard="${shard:}" />
> </cores>
> </solr>
>
>
> We use one solr.xml and pass dataDir, shard as java system property. (e.g.: -Dsolr.data.dir=$HOME/solr_data/solr6
> -Dsolr.ulog.dir=$HOME/solr_data/solr6_tranlog -DnumShards=5 -Dshard=shard1)
There's the problem. You can't hard-code dataDir with SolrCloud unless
it's a relative path. It's trying to share the same dataDir between the
two cores that it's creating from the old one. dataDir defaults to
${instanceDir}/data.
The solr.xml file will get modified when you use the collections and
core admin APIs, and each server will get different modifications, so
sharing it between different Solr servers is not a good idea.
Thanks,
Shawn
RE: SPLITSHARD throws error
Posted by "Joshi, Shital" <Sh...@gs.com>.
This is what we have in solr.xml.
<solr persistent="true">
<cores adminPath="/admin/cores" defaultCoreName="collection1" host="${host:}" hostPort="${jetty.port:8983}" hostContext="${hostContext:solr}" zkClientTimeout="${zkClientTimeout:15000}">
<core name="collection1" instanceDir="collection1" dataDir="${solr.data.dir:}" shard="${shard:}" />
</cores>
</solr>
We use one solr.xml and pass dataDir, shard as java system property. (e.g.: -Dsolr.data.dir=$HOME/solr_data/solr6
-Dsolr.ulog.dir=$HOME/solr_data/solr6_tranlog -DnumShards=5 -Dshard=shard1)
-----Original Message-----
From: Shawn Heisey [mailto:solr@elyograg.org]
Sent: Friday, June 21, 2013 4:45 PM
To: solr-user@lucene.apache.org
Subject: Re: SPLITSHARD throws error
On 6/21/2013 2:26 PM, Joshi, Shital wrote:
> Hi,
>
> We have 5 shards with replication factor 2 (total 10 jvm instances). Our shards are named (shardid) shard1,shard2,shard3,shar4 and shar5 and collection name is collection1. When we execute this command:
>
> curl --proxy '' "http://$HOST_NAME:8983/solr/admin/collections?action=SPLITSHARD&collection=collection1&shard=shard1"
>
> We get this error:
>
> <lst name="failure">
> <str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error CREATEing SolrCore 'collection1_shard1_0_replica1': Core with same data dir 'collection1' already exists.</str>
What's in your solr.xml file? Except for the shard names, is the
solr.xml from your other server(s) radically different?
Thanks,
Shawn
Re: SPLITSHARD throws error
Posted by Shawn Heisey <so...@elyograg.org>.
On 6/21/2013 2:26 PM, Joshi, Shital wrote:
> Hi,
>
> We have 5 shards with replication factor 2 (total 10 jvm instances). Our shards are named (shardid) shard1,shard2,shard3,shar4 and shar5 and collection name is collection1. When we execute this command:
>
> curl --proxy '' "http://$HOST_NAME:8983/solr/admin/collections?action=SPLITSHARD&collection=collection1&shard=shard1"
>
> We get this error:
>
> <lst name="failure">
> <str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error CREATEing SolrCore 'collection1_shard1_0_replica1': Core with same data dir 'collection1' already exists.</str>
What's in your solr.xml file? Except for the shard names, is the
solr.xml from your other server(s) radically different?
Thanks,
Shawn