[GitHub] [solr] justinrsweeney commented on pull request #1267: SOLR-16608: Ability to compress the collection state

[GitBox <gi...@apache.org>]

justinrsweeney commented on PR #1267:
URL: https://github.com/apache/solr/pull/1267#issuecomment-1373888188

   > This would be a pretty scary feature to use for me (since there are times where modifying cluster state manually can be unfortunately necessary).
   > 
   > But overall I have a few comments.
   > 
   > Is there a full e2e test here? I see all tests generally use a ZkStateWriter. But it would be good to see a test just start a cloud with compression enabled, create a collection and make sure it is compressed by fetching the data from ZK.
   > 
   > Also for upgrade notes, this feature should not be enabled until all Solr Nodes are running a version of Solr that supports it.
   
   Thanks for taking a look @HoustonPutman, one of the reasons we chose ZLib compression is because tools like [zk-shell](https://github.com/rgs1/zk_shell) can handle it natively when doing a get on a compressed state.json. Also since this is done in ZKStateReader it is handled across Solr including Admin pages and Solr CLI, which hopefully helps alleviate some of the concern. There is certainly some tradeoff here in terms of manual modification of state, but we've found value in this for very large collections to keep files in ZK small.
   
   +1 on upgrade notes and I'll follow up on additional tests and the other comments you provided.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org