[GitHub] [pinot] siddharthteotia commented on a diff in pull request #8753: Streamed segment download & untar with rate limiter to control disk usage `feature`

[GitBox <gi...@apache.org>]

siddharthteotia commented on code in PR #8753:
URL: https://github.com/apache/pinot/pull/8753#discussion_r893020624


##########
pinot-common/src/main/java/org/apache/pinot/common/utils/TarGzCompressionUtils.java:
##########
@@ -198,4 +238,30 @@ public static void untarOneFile(File inputFile, String fileName, File outputFile
       throw new IOException(String.format("Failed to find file: %s in: %s", fileName, inputFile));
     }
   }
+
+  public static long streamCopyWithRateLimiter(InputStream inputStream, FileOutputStream outputStream, long rateLimit)

Review Comment:
   As suggested offline, the function name / config use is confusing. We need to choose better naming to clearly expose the 2 flavors
   
   - non-streaming download (existing)
   - streaming download
     -- with rate limiter
     -- without rate limiter



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org