[GitHub] [solr] bruno-roustant commented on a diff in pull request #1469: SOLR-16706: Reduce the ZLibCompressor compression buffer size.

["bruno-roustant (via GitHub)" <gi...@apache.org>]

bruno-roustant commented on code in PR #1469:
URL: https://github.com/apache/solr/pull/1469#discussion_r1140341672


##########
solr/core/src/java/org/apache/solr/cloud/overseer/ZkStateWriter.java:
##########
@@ -286,7 +286,8 @@ public ClusterState writePendingUpdates(
           } else {
             byte[] data = Utils.toJSON(singletonMap(c.getName(), c));
             if (minStateByteLenForCompression > -1 && data.length > minStateByteLenForCompression) {
-              data = compressor.compressBytes(data);
+              // When compressing state.json, we expect at least a 10:1 compression ratio.
+              data = compressor.compressBytes(data, data.length / 10);

Review Comment:
   The json format is quite verbose, and state.json even more.
   I tested with some state.json, including the one in ZkStateWriterTest.testSingleExternalCollectionCompressedState() which triggers the compression at 500KB. I got a compression ratio of 20:1. That's why I chose 10:1 here to have a high probability of not resizing the array, and saving significant memory at the same time.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org