[GitHub] [cassandra] maedhroz commented on a change in pull request #651: Fix entire sstable streaming leak and reduce buffer allocation

[GitBox <gi...@apache.org>]

maedhroz commented on a change in pull request #651:
URL: https://github.com/apache/cassandra/pull/651#discussion_r445149854



##########
File path: src/java/org/apache/cassandra/net/AsyncStreamingOutputPlus.java
##########
@@ -170,36 +170,50 @@ public int writeToChannel(Write write, StreamRateLimiter limiter) throws IOExcep
      */
     public long writeFileToChannel(FileChannel file, StreamRateLimiter limiter) throws IOException
     {
-        // write files in 1MiB chunks, since there may be blocking work performed to fetch it from disk,
-        // the data is never brought in process and is gated by the wire anyway
         if (channel.pipeline().get(SslHandler.class) != null)
-            return writeFileToChannel(file, limiter, 1 << 20, 1 << 20, 2 << 20);
+            // each batch is loaded into ByteBuffer, 64kb is more BufferPool friendly.
+            return writeFileToChannel(file, limiter, 1 << 16);

Review comment:
       @jasonstack Would it be less brittle here if we just made the batch size `BufferPool.NORMAL_CHUNK_SIZE`? That seems like it would still avoid passing the threshold where we have to allocate new buffers, and would adjust automatically if we ever had to change the normal chunk size. The thing I don't know is what the optimal buffer size actually is. If we want to stay as close as possible to the existing behavior, using `BufferPool.NORMAL_CHUNK_SIZE` obviously makes sense.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org