You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Nimesh Aghera <na...@asite.com> on 2017/12/12 09:15:00 UTC
Implicit routing changes to Composite while re-deploy changes and
make Upconfig command
Hello,
I have implemented implicit routing with below configuration.
Created one default collection manually 'AMS_Config' which contains configurations files schema,solrconfig etc.
Using 'AMS_Config' I have created 2 collections model,workset respectively with below command which created 2 shard for each collection containing 2 node for each shard where nodes are each solr instance where collection created.
Command: /admin/collections?action=CREATE&name=model&replicationFactor=2&router.name=implicit&router.field=dr&shards=shard1,shard2&maxShardsPerNode=2&collection.configName=AMS_Config
Collection Detail:
Model = Shard1,Shard2
Shard1 = node1,node2[leader]
Shard2 =node1[leader],node2
Configuration in admin UI on solr for model collection:
Shard count:2
configName:AMS_Config
replicationFactor:2
maxShardsPerNode:2
router:implicit
autoAddReplicas:false
after this I have index document to particular shard using set value (router.field) dr = shard1
Issue: After indexing the document made changes in schema files and redeploy using below command for set latest configs
zkcli.bat -cmd upconfig -confdir ../../solr/AMS_Config/conf -confname AMS_Config -z <zk Ips>
It will changes the router value implicit to compositeId and now my document are index across all shard so why this should happens.How to avoid this.
Please do the needful.
Regards,
Nimesh Aghera
[CC Award Winners!]
Re: Implicit routing changes to Composite while re-deploy changes and
make Upconfig command
Posted by Erick Erickson <er...@gmail.com>.
No images come through in e-mail so I can't see what you're actually doing.
On Wed, Dec 13, 2017 at 3:06 AM, Nimesh Aghera <na...@asite.com> wrote:
>
>
> Hi Erik,
>
>
>
> Thanks for the reply.
>
>
>
> Currently using solr cloud 6.1.0 and below is the solr admin UI which
> shows that routing changes after make upconfig command
>
>
>
> Below image are before make re-upconfig where all data are indexed into
> particular shard.
>
>
>
> AMS_Config:
>
>
>
>
>
> Model Collection: Created collection using Collection API and also data
> index into particular shard
>
>
>
>
>
> But after that I have make schema changes (Not Mandatory) and do upconfig
> as mention in previsou mail and start the solr instance its shows my model
> collection as below and also it index data across all shards
>
>
>
>
>
>
>
> Its causes issues that if any how its required to make upconfig than it
> change the routing from implicit to compositeId.
>
>
>
> Please do needful.
>
>
>
> Regards,
>
> Nimesh Aghera
>
>
>
>
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com
> <er...@gmail.com>]
> Sent: Tuesday, December 12, 2017 9:29 PM
> To: solr-user
> Subject: Re: Implicit routing changes to Composite while re-deploy changes
> and make Upconfig command
>
>
>
> bq: It will changes the router value implicit to compositeId
>
>
>
> What evidence do you have of that? This better not happen. The routing
> scheme is part of the collection creation command and is stored in
> ZooKeeper in the collection's znode, it has nothing to do with the
> configset so updating the configs shouldn't change that at all.
>
>
>
> Best,
>
> Erick
>
>
>
> On Tue, Dec 12, 2017 at 1:15 AM, Nimesh Aghera <na...@asite.com> wrote:
>
> > Hello,
>
> >
>
> > I have implemented implicit routing with below configuration.
>
> > Created one default collection manually 'AMS_Config' which contains
> configurations files schema,solrconfig etc.
>
> >
>
> > Using 'AMS_Config' I have created 2 collections model,workset
> respectively with below command which created 2 shard for each collection
> containing 2 node for each shard where nodes are each solr instance where
> collection created.
>
> >
>
> > Command:
>
> > /admin/collections?action=CREATE&name=model&replicationFactor=2&router
>
> > .name=implicit&router.field=dr&shards=shard1,shard2&maxShardsPerNode=2
>
> > &collection.configName=AMS_Config
>
> >
>
> > Collection Detail:
>
> > Model = Shard1,Shard2
>
> > Shard1 = node1,node2[leader]
>
> > Shard2 =node1[leader],node2
>
> >
>
> > Configuration in admin UI on solr for model collection:
>
> > Shard count:2
>
> > configName:AMS_Config
>
> > replicationFactor:2
>
> > maxShardsPerNode:2
>
> > router:implicit
>
> > autoAddReplicas:false
>
> >
>
> > after this I have index document to particular shard using set value
>
> > (router.field) dr = shard1
>
> >
>
> > Issue: After indexing the document made changes in schema files and
>
> > redeploy using below command for set latest configs zkcli.bat -cmd
>
> > upconfig -confdir ../../solr/AMS_Config/conf -confname AMS_Config -z
>
> > <zk Ips>
>
> >
>
> > It will changes the router value implicit to compositeId and now my
> document are index across all shard so why this should happens.How to avoid
> this.
>
> >
>
> > Please do the needful.
>
> >
>
> > Regards,
>
> > Nimesh Aghera
>
> >
>
> > [CC Award Winners!]
>
> >
>
RE: Implicit routing changes to Composite while re-deploy changes
and make Upconfig command
Posted by Nimesh Aghera <na...@asite.com>.
Hi Erik,
Thanks for the reply.
Currently using solr cloud 6.1.0 and below is the solr admin UI which shows that routing changes after make upconfig command
Below image are before make re-upconfig where all data are indexed into particular shard.
AMS_Config:
[cid:image001.png@01D37430.873F2130]
Model Collection: Created collection using Collection API and also data index into particular shard
[cid:image002.png@01D37430.873F2130]
But after that I have make schema changes (Not Mandatory) and do upconfig as mention in previsou mail and start the solr instance its shows my model collection as below and also it index data across all shards
[cid:image003.png@01D37430.873F2130]
Its causes issues that if any how its required to make upconfig than it change the routing from implicit to compositeId.
Please do needful.
Regards,
Nimesh Aghera
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Tuesday, December 12, 2017 9:29 PM
To: solr-user
Subject: Re: Implicit routing changes to Composite while re-deploy changes and make Upconfig command
bq: It will changes the router value implicit to compositeId
What evidence do you have of that? This better not happen. The routing scheme is part of the collection creation command and is stored in ZooKeeper in the collection's znode, it has nothing to do with the configset so updating the configs shouldn't change that at all.
Best,
Erick
On Tue, Dec 12, 2017 at 1:15 AM, Nimesh Aghera <na...@asite.com>> wrote:
> Hello,
>
> I have implemented implicit routing with below configuration.
> Created one default collection manually 'AMS_Config' which contains configurations files schema,solrconfig etc.
>
> Using 'AMS_Config' I have created 2 collections model,workset respectively with below command which created 2 shard for each collection containing 2 node for each shard where nodes are each solr instance where collection created.
>
> Command:
> /admin/collections?action=CREATE&name=model&replicationFactor=2&router
> .name=implicit&router.field=dr&shards=shard1,shard2&maxShardsPerNode=2
> &collection.configName=AMS_Config
>
> Collection Detail:
> Model = Shard1,Shard2
> Shard1 = node1,node2[leader]
> Shard2 =node1[leader],node2
>
> Configuration in admin UI on solr for model collection:
> Shard count:2
> configName:AMS_Config
> replicationFactor:2
> maxShardsPerNode:2
> router:implicit
> autoAddReplicas:false
>
> after this I have index document to particular shard using set value
> (router.field) dr = shard1
>
> Issue: After indexing the document made changes in schema files and
> redeploy using below command for set latest configs zkcli.bat -cmd
> upconfig -confdir ../../solr/AMS_Config/conf -confname AMS_Config -z
> <zk Ips>
>
> It will changes the router value implicit to compositeId and now my document are index across all shard so why this should happens.How to avoid this.
>
> Please do the needful.
>
> Regards,
> Nimesh Aghera
>
> [CC Award Winners!]
>
Re: Implicit routing changes to Composite while re-deploy changes and
make Upconfig command
Posted by Shawn Heisey <ap...@elyograg.org>.
On 12/18/2017 7:21 AM, Nimesh Aghera wrote:
> Thanks Shawn for reply,
> Need to do below step.
>
> After creating collection using the Collection API clear the version-2 folder from zoo_data and after that I will start server and make upconfig the collection is up but the routing changes from implicit to explicit with solr 6.1.0 while I do same to replicate in solr 7.1.0 while up server it will gives org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Replica core_node5 is not present in cluster state: null So unable to replicate in solr 7.1.0 and also while start server it will create the 'gettingstarted' collection every time which is already exists.
I'm pretty sure that deleting the zoodata/version-2 folder deletes all
zookeeper data. Don't do that.
SolrCloud can't recover from all of its ZK data being deleted. I'm
surprised anything works at all.
Thanks,
Shawn
RE: Implicit routing changes to Composite while re-deploy changes
and make Upconfig command
Posted by Nimesh Aghera <na...@asite.com>.
Hi
Thanks Shawn for reply,
Need to do below step.
After creating collection using the Collection API clear the version-2 folder from zoo_data and after that I will start server and make upconfig the collection is up but the routing changes from implicit to explicit with solr 6.1.0 while I do same to replicate in solr 7.1.0 while up server it will gives org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Replica core_node5 is not present in cluster state: null So unable to replicate in solr 7.1.0 and also while start server it will create the 'gettingstarted' collection every time which is already exists.
So will you please try above scenario.
Please do needful.
Regards,
Nimesh Aghera
-----Original Message-----
From: Shawn Heisey [mailto:apache@elyograg.org]
Sent: Thursday, December 14, 2017 8:55 PM
To: solr-user@lucene.apache.org
Subject: Re: Implicit routing changes to Composite while re-deploy changes and make Upconfig command
On 12/13/2017 10:08 PM, Nimesh Aghera wrote:
> Currently using solr cloud 6.1.0 and below is the solr admin UI which
> shows that routing changes after make upconfig command
>
> Below values shown in Solr Admin UI are before make re-upconfig where all data are indexed into particular shard.
I could not reproduce this on Solr 7.1.0. If there *is* a problem in version 6.1.0, it's not going to get fixed. Versions that old will never see an update.
First, I started up the cloud example with "bin\solr start -e cloud -nomprompt". This was on a Windows 7 system with Oracle Java 8u144.
Then I made a Collections API call like yours to create a collection:
http://localhost:8983/solr/admin/collections?action=CREATE&name=model&replicationFactor=2&router.name=implicit&router.field=title&shards=shard1,shard2&maxShardsPerNode=2&collection.configName=gettingstarted
This is the response from that call:
{
"responseHeader":{
"status":0,
"QTime":4971},
"success":{
"192.168.1.4:7574_solr":{
"responseHeader":{
"status":0,
"QTime":2843},
"core":"model_shard2_replica_n6"},
"192.168.1.4:7574_solr":{
"responseHeader":{
"status":0,
"QTime":3078},
"core":"model_shard1_replica_n2"},
"192.168.1.4:8983_solr":{
"responseHeader":{
"status":0,
"QTime":3382},
"core":"model_shard2_replica_n4"},
"192.168.1.4:8983_solr":{
"responseHeader":{
"status":0,
"QTime":3618},
"core":"model_shard1_replica_n1"}}}
This is a screenshot of the collection overview, showing the implicit
router:
https://www.dropbox.com/s/x5gtpkkl6lv18g8/test1-collection-info-after-create.png?dl=0
Then I made a whitespace-only change to solrconfig.xml in the _default configset, and re-uploaded it to replace the configuration I used for the collection:
bin\solr zk upconfig -d server\solr\configsets\_default -n gettingstarted -z localhost:9983
There was no difference at all in the collection details. I even did a collection reload, and still did not see anything change.
Thanks,
Shawn
Re: Implicit routing changes to Composite while re-deploy changes and
make Upconfig command
Posted by Shawn Heisey <ap...@elyograg.org>.
On 12/13/2017 10:08 PM, Nimesh Aghera wrote:
> Currently using solr cloud 6.1.0 and below is the solr admin UI which shows that routing changes after make upconfig command
>
> Below values shown in Solr Admin UI are before make re-upconfig where all data are indexed into particular shard.
I could not reproduce this on Solr 7.1.0. If there *is* a problem in
version 6.1.0, it's not going to get fixed. Versions that old will
never see an update.
First, I started up the cloud example with "bin\solr start -e cloud
-nomprompt". This was on a Windows 7 system with Oracle Java 8u144.
Then I made a Collections API call like yours to create a collection:
http://localhost:8983/solr/admin/collections?action=CREATE&name=model&replicationFactor=2&router.name=implicit&router.field=title&shards=shard1,shard2&maxShardsPerNode=2&collection.configName=gettingstarted
This is the response from that call:
{
"responseHeader":{
"status":0,
"QTime":4971},
"success":{
"192.168.1.4:7574_solr":{
"responseHeader":{
"status":0,
"QTime":2843},
"core":"model_shard2_replica_n6"},
"192.168.1.4:7574_solr":{
"responseHeader":{
"status":0,
"QTime":3078},
"core":"model_shard1_replica_n2"},
"192.168.1.4:8983_solr":{
"responseHeader":{
"status":0,
"QTime":3382},
"core":"model_shard2_replica_n4"},
"192.168.1.4:8983_solr":{
"responseHeader":{
"status":0,
"QTime":3618},
"core":"model_shard1_replica_n1"}}}
This is a screenshot of the collection overview, showing the implicit
router:
https://www.dropbox.com/s/x5gtpkkl6lv18g8/test1-collection-info-after-create.png?dl=0
Then I made a whitespace-only change to solrconfig.xml in the _default
configset, and re-uploaded it to replace the configuration I used for
the collection:
bin\solr zk upconfig -d server\solr\configsets\_default -n
gettingstarted -z localhost:9983
There was no difference at all in the collection details. I even did a
collection reload, and still did not see anything change.
Thanks,
Shawn
RE: Implicit routing changes to Composite while re-deploy changes
and make Upconfig command
Posted by Nimesh Aghera <na...@asite.com>.
Hi Erik,
Thanks for the reply.
Currently using solr cloud 6.1.0 and below is the solr admin UI which shows that routing changes after make upconfig command
Below values shown in Solr Admin UI are before make re-upconfig where all data are indexed into particular shard.
AMS_Config:
----------------
Shard count:2
configName:AMS_Config
replicationFactor:1
maxShardsPerNode:1
router:compositeId
autoAddReplicas:false
Model Collection: Created collection using Collection API and also data index into particular shard
---------------------
Shard count:2
configName:AMS_Config
replicationFactor:2
maxShardsPerNode:2
router:implicit
autoAddReplicas:false
But after that I have make schema changes (Not Mandatory) and do upconfig as mention in previsou mail and start the solr instance its shows my model collection as below and also it index data across all shards
Shard count:2
configName:AMS_Config
replicationFactor:1
maxShardsPerNode:1
router:compositeId
autoAddReplicas:false
Its causes issues that if any how its required to make upconfig than it change the routing from implicit to compositeId.
Please do needful.
Regards,
Nimesh Aghera
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Tuesday, December 12, 2017 9:29 PM
To: solr-user
Subject: Re: Implicit routing changes to Composite while re-deploy changes and make Upconfig command
bq: It will changes the router value implicit to compositeId
What evidence do you have of that? This better not happen. The routing scheme is part of the collection creation command and is stored in ZooKeeper in the collection's znode, it has nothing to do with the configset so updating the configs shouldn't change that at all.
Best,
Erick
On Tue, Dec 12, 2017 at 1:15 AM, Nimesh Aghera <na...@asite.com>> wrote:
> Hello,
>
> I have implemented implicit routing with below configuration.
> Created one default collection manually 'AMS_Config' which contains configurations files schema,solrconfig etc.
>
> Using 'AMS_Config' I have created 2 collections model,workset respectively with below command which created 2 shard for each collection containing 2 node for each shard where nodes are each solr instance where collection created.
>
> Command:
> /admin/collections?action=CREATE&name=model&replicationFactor=2&router
> .name=implicit&router.field=dr&shards=shard1,shard2&maxShardsPerNode=2
> &collection.configName=AMS_Config
>
> Collection Detail:
> Model = Shard1,Shard2
> Shard1 = node1,node2[leader]
> Shard2 =node1[leader],node2
>
> Configuration in admin UI on solr for model collection:
> Shard count:2
> configName:AMS_Config
> replicationFactor:2
> maxShardsPerNode:2
> router:implicit
> autoAddReplicas:false
>
> after this I have index document to particular shard using set value
> (router.field) dr = shard1
>
> Issue: After indexing the document made changes in schema files and
> redeploy using below command for set latest configs zkcli.bat -cmd
> upconfig -confdir ../../solr/AMS_Config/conf -confname AMS_Config -z
> <zk Ips>
>
> It will changes the router value implicit to compositeId and now my document are index across all shard so why this should happens.How to avoid this.
>
> Please do the needful.
>
> Regards,
> Nimesh Aghera
>
> [CC Award Winners!]
>
Re: Implicit routing changes to Composite while re-deploy changes and
make Upconfig command
Posted by Erick Erickson <er...@gmail.com>.
bq: It will changes the router value implicit to compositeId
What evidence do you have of that? This better not happen. The routing
scheme is part of the collection creation command and is stored in
ZooKeeper in the collection's znode, it has nothing to do with the
configset so updating the configs shouldn't change that at all.
Best,
Erick
On Tue, Dec 12, 2017 at 1:15 AM, Nimesh Aghera <na...@asite.com> wrote:
> Hello,
>
> I have implemented implicit routing with below configuration.
> Created one default collection manually 'AMS_Config' which contains configurations files schema,solrconfig etc.
>
> Using 'AMS_Config' I have created 2 collections model,workset respectively with below command which created 2 shard for each collection containing 2 node for each shard where nodes are each solr instance where collection created.
>
> Command: /admin/collections?action=CREATE&name=model&replicationFactor=2&router.name=implicit&router.field=dr&shards=shard1,shard2&maxShardsPerNode=2&collection.configName=AMS_Config
>
> Collection Detail:
> Model = Shard1,Shard2
> Shard1 = node1,node2[leader]
> Shard2 =node1[leader],node2
>
> Configuration in admin UI on solr for model collection:
> Shard count:2
> configName:AMS_Config
> replicationFactor:2
> maxShardsPerNode:2
> router:implicit
> autoAddReplicas:false
>
> after this I have index document to particular shard using set value (router.field) dr = shard1
>
> Issue: After indexing the document made changes in schema files and redeploy using below command for set latest configs
> zkcli.bat -cmd upconfig -confdir ../../solr/AMS_Config/conf -confname AMS_Config -z <zk Ips>
>
> It will changes the router value implicit to compositeId and now my document are index across all shard so why this should happens.How to avoid this.
>
> Please do the needful.
>
> Regards,
> Nimesh Aghera
>
> [CC Award Winners!]
>