[GitHub] [ozone] sodonnel commented on a diff in pull request #3882: HDDS-7407. EC: Block allocation should not be stripped across the EC group

[GitBox <gi...@apache.org>]

sodonnel commented on code in PR #3882:
URL: https://github.com/apache/ozone/pull/3882#discussion_r1004338314


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableECContainerProvider.java:
##########
@@ -93,16 +93,13 @@ public WritableECContainerProvider(ConfigurationSource conf,
   public ContainerInfo getContainer(final long size,
       ECReplicationConfig repConfig, String owner, ExcludeList excludeList)
       throws IOException, TimeoutException {
-    // Bound this at a minimum of 1 byte in case a request is made for a very
-    // small size, which when divided by EC DataNum is zero.
-    long requiredSpace = Math.max(1, size / repConfig.getData());
     synchronized (this) {
       int openPipelineCount = pipelineManager.getPipelineCount(repConfig,
           Pipeline.PipelineState.OPEN);
       if (openPipelineCount < providerConfig.getMinimumPipelines()) {
         try {
           return allocateContainer(
-              repConfig, requiredSpace, owner, excludeList);
+              repConfig, size, owner, excludeList);

Review Comment:
   I'm not sure this is correct. I remember looking into what this "size" is that is passed into this method. In the Java doc, I wrote
   
   ```
      * @param size The max size of block in bytes which will be written. This
      *             comes from Ozone Manager and will be the block size configured
      *             for the cluster. The client cannot pass any arbitrary value
      *             from this setting.
   ```
   
   So if the block size gets passed in (256MB) and we track an EC container as having a 5GB limit for each replica index, then the space we need in the container is 256 / dataNum.
   
   Now if the size at OM is divided by dataNum already and then passed down to SCM, then we don't need to divide it again, but I don't think it is, at least it did not used to be.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org