You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by GitBox <gi...@apache.org> on 2022/10/19 18:37:30 UTC
[GitHub] [solr] patsonluk opened a new pull request, #1093: SOLR-16446: Avoid redundant state.json update for PRS collection creation
patsonluk opened a new pull request, #1093:
URL: https://github.com/apache/solr/pull/1093
Copied from 8.8 PR https://github.com/fullstorydev/lucene-solr/pull/173
https://issues.apache.org/jira/browse/SOLR-16446
# Description
It is found that the existing `CreateCollectionCmd` would update the [`state.json` per new replica](https://github.com/fullstorydev/lucene-solr/blob/fec2ac23fb5068fddf1a2c3d51956128719ec7a5/solr/core/src/java/org/apache/solr/cloud/api/collections/CreateCollectionCmd.java#L281-L282). This potentially creates a lot of unnecessary writes as only the last state.json with all the replicas matters.
Take note that this actually is a problem for non PRS collection as well, but probably less problematic for non PRS as even if it does very similar operation per replica by sending the update to OS queue `ccc.offerStateUpdate(Utils.toJSON(props));`, at least OS would use the `ClusterStateUpdater` which uses `ZkStateWriter#enqueueUpdate` that likely batches up state.json update (only writes the latest entry every 2 secs by default) ; while for PRS, we use directly update which each call translates to an actual call to ZK.
# Solution
We will move the `ZkClient#setData` statement out of the replica loop and only call it once outside
# Tests
There is already an integration test `PerReplicaStatesIntegrationTest#testZkNodeVersions`, we have updated the count to reflect the reduction of `setData` calls
# Checklist
Please review the following and check all that apply:
- [ ] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability.
- [ ] I have created a Jira issue and added the issue ID to my pull request title.
- [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended)
- [ ] I have developed this patch against the `main` branch.
- [ ] I have run `./gradlew check`.
- [ ] I have added tests for my changes.
- [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] patsonluk commented on pull request #1093: SOLR-16446: Avoid redundant state.json update for PRS collection creation
Posted by GitBox <gi...@apache.org>.
patsonluk commented on PR #1093:
URL: https://github.com/apache/solr/pull/1093#issuecomment-1289818691
@noblepaul @chatman please help to review and merge. Many thanks! 🙇🏼
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] noblepaul closed pull request #1093: SOLR-16446: Avoid redundant state.json update for PRS collection creation
Posted by GitBox <gi...@apache.org>.
noblepaul closed pull request #1093: SOLR-16446: Avoid redundant state.json update for PRS collection creation
URL: https://github.com/apache/solr/pull/1093
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] sonatype-lift[bot] commented on pull request #1093: SOLR-16446: Avoid redundant state.json update for PRS collection creation
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on PR #1093:
URL: https://github.com/apache/solr/pull/1093#issuecomment-1284472278
:warning: **313 God Classes** were detected by Lift in this project. [Visit the Lift web console](https://lift.sonatype.com/results/github.com/apache/solr/01GFRRJJF5YYD3V2X1S81YZY60?tab=technical-debt&utm_source=github.com&utm_campaign=lift-comment&utm_content=apache\%20solr) for more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org