[jira] [Comment Edited] (SOLR-6249) Schema API changes return success before all cores are updated

["Gregory Chanan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-6249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097812#comment-14097812 ] 

Gregory Chanan edited comment on SOLR-6249 at 8/14/14 10:29 PM:
----------------------------------------------------------------

bq. Seems to me like less coordination here is best? In other words, instead of the core that accepts the update request worrying about all the other cores that are going to see the update, it just assumes it will be successful. If any of the remote cores fail in processing the update, then they mark themselves as "down". This seems to solve the root of the problem raised by this ticket, namely a replica using the wrong version of a managed schema. It's likely that a replica that can't process a good update successfully will need some manual intervention anyway (such as missing a JAR file or something).

Sure, less coordination is best.  I think there are multiple issues here, though:
1) how do we handle Schema API modifications failing on some replicas?
2) if i'm a client and I want to update the schema and send some updates that are dependent on the updated schema, how do I do that?  Right now, it's a pain (and without a client-visible version it may get more complicated if we add more schema APIs, e.g. removing fields).

I don't think the "wait for one replica" idea solves either of those issues.


was (Author: gchanan):
Seems to me like less coordination here is best? In other words, instead of the core that accepts the update request worrying about all the other cores that are going to see the update, it just assumes it will be successful. If any of the remote cores fail in processing the update, then they mark themselves as "down". This seems to solve the root of the problem raised by this ticket, namely a replica using the wrong version of a managed schema. It's likely that a replica that can't process a good update successfully will need some manual intervention anyway (such as missing a JAR file or something).

Sure, less coordination is best.  I think there are multiple issues here, though:
1) how do we handle Schema API modifications failing on some replicas?
2) if i'm a client and I want to update the schema and send some updates that are dependent on the updated schema, how do I do that?  Right now, it's a pain (and without a client-visible version it may get more complicated if we add more schema APIs, e.g. removing fields).

I don't think the "wait for one replica" idea solves either of those issues.

> Schema API changes return success before all cores are updated
> --------------------------------------------------------------
>
>                 Key: SOLR-6249
>                 URL: https://issues.apache.org/jira/browse/SOLR-6249
>             Project: Solr
>          Issue Type: Improvement
>          Components: Schema and Analysis, SolrCloud
>            Reporter: Gregory Chanan
>
> See SOLR-6137 for more details.
> The basic issue is that Schema API changes return success when the first core is updated, but other cores asynchronously read the updated schema from ZooKeeper.
> So a client application could make a Schema API change and then index some documents based on the new schema that may fail on other nodes.
> Possible fixes:
> 1) Make the Schema API calls synchronous
> 2) Give the client some ability to track the state of the schema.  They can already do this to a certain extent by checking the Schema API on all the replicas and verifying that the field has been added, though this is pretty cumbersome.  Maybe it makes more sense to do this sort of thing on the collection level, i.e. Schema API changes return the zk version to the client.  We add an API to return the current zk version.  On a replica, if the zk version is >= the version the client has, the client knows that replica has at least seen the schema change.  We could also provide an API to do the distribution and checking across the different replicas of the collection so that clients don't need ot do that themselves.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org