You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Shawn Heisey (JIRA)" <ji...@apache.org> on 2015/10/02 02:50:26 UTC
[jira] [Commented] (SOLR-7282) Cache config or index schema objects
by configset and share them across cores
[ https://issues.apache.org/jira/browse/SOLR-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940653#comment-14940653 ]
Shawn Heisey commented on SOLR-7282:
------------------------------------
Something to think about, and this might affect the existing non-cloud feature "shareSchema" too:
If the actual Java object is shared, we need to be careful about behavior on reload. Users expect that updating the config in zookeeper will not cause immediate usage of that config, that they will have to do a reload before it will take effect. Say that one machine, updated with a Solr version that has a fix for this issue, hosts ten cores that are replicas for various shards from Collection A. Those ten cores are all sharing Java objects for config and schema.
When the user updates the config in zookeeper that for collection A, here's how I think everything should play out:
* The objects that those ten cores are sharing will NOT be updated by the change to zookeeper -- they will still contain info from the config before modification.
* When one of those cores is reloaded, it should create new objects holding the new information in zookeeper, but the other cores for that collection will still have the old objects with the old config.
* When a second and subsequent cores are reloaded, they should share the objects created during the first core reload.
The coding challenge will be to figure out when the existing set of config objects (probably at the CoreContainer level) need to be thrown away (but still held by existing cores) and replaced with a new set of objects. I'm guessing that the config znode in zookeeper contains some info that will only change when the config is updated, that info could probably be used.
> Cache config or index schema objects by configset and share them across cores
> -----------------------------------------------------------------------------
>
> Key: SOLR-7282
> URL: https://issues.apache.org/jira/browse/SOLR-7282
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
> Reporter: Shalin Shekhar Mangar
> Assignee: Shalin Shekhar Mangar
> Fix For: 5.2, Trunk
>
> Attachments: SOLR-7282.patch
>
>
> Sharing schema and config objects has been known to improve startup performance when a large number of cores are on the same box (See http://wiki.apache.org/solr/LotsOfCores).Damien also saw improvements to cluster startup speed upon caching the index schema in SOLR-7191.
> Now that SolrCloud configuration is based on config sets in ZK, we should explore how we can minimize config/schema parsing for each core in a way that is compatible with the recent/planned changes in the config and schema APIs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org