[GitHub] [incubator-druid] clintropolis commented on a change in pull request #8201: Simple memory allocation for CliIndexer tasks

[GitBox <gi...@apache.org>]

clintropolis commented on a change in pull request #8201: Simple memory allocation for CliIndexer tasks
URL: https://github.com/apache/incubator-druid/pull/8201#discussion_r309110797
 
 

 ##########
 File path: server/src/main/java/org/apache/druid/segment/realtime/appenderator/UnifiedIndexerAppenderatorsManager.java
 ##########
 @@ -196,4 +236,288 @@ public boolean shouldTaskMakeNodeAnnouncements()
   {
     return false;
   }
+
+  @Override
+  public void shutdown()
+  {
+    if (mergeExecutor != null) {
+      mergeExecutor.shutdownNow();
+    }
+  }
+
+  private AppenderatorConfig rewriteAppenderatorConfigMemoryLimits(AppenderatorConfig baseConfig)
+  {
+    long perWorkerLimit = workerConfig.getGlobalIngestionHeapLimitBytes() / workerConfig.getCapacity();
+    return new MemoryParameterOverridingAppenderatorConfig(baseConfig, perWorkerLimit);
+  }
+
+  /**
+   * This is a wrapper around AppenderatorConfig that overrides the `maxRowsInMemory` and `maxBytesInMemory`
+   * parameters. Row-based limits are disabled by setting `maxRowsInMemory` to an essentially unlimited value.
+   * `maxBytesInMemory` is overridden with the provided value.
 
 Review comment:
   This javadoc could maybe a little more clearly state that this means that these will override any value the user specifies on the tasks. Also could these by chance be javadoc links to the configs instead of using '`'?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org