[GitHub] [samza] shanthoosh commented on a change in pull request #1011: SAMZA-2172: Make High Level applications respect job.container.thread.pool.size

[GitBox <gi...@apache.org>]

shanthoosh commented on a change in pull request #1011: SAMZA-2172: Make High Level applications respect job.container.thread.pool.size
URL: https://github.com/apache/samza/pull/1011#discussion_r284499127
 
 

 ##########
 File path: samza-core/src/main/java/org/apache/samza/task/RunLoop.java
 ##########
 @@ -457,17 +457,25 @@ private void process() {
       final IncomingMessageEnvelope envelope = state.fetchEnvelope();
       log.trace("Process ssp {} offset {}", envelope.getSystemStreamPartition(), envelope.getOffset());
 
-      final ReadableCoordinator coordinator = new ReadableCoordinator(task.taskName());
-      TaskCallbackFactory callbackFactory = new TaskCallbackFactory() {
-        @Override
-        public TaskCallback createCallback() {
-          state.startProcess();
-          containerMetrics.processes().inc();
-          return callbackManager.createCallback(task.taskName(), envelope, coordinator);
-        }
+      Runnable processWorker = () -> {
+        final ReadableCoordinator coordinator = new ReadableCoordinator(task.taskName());
+        TaskCallbackFactory callbackFactory = new TaskCallbackFactory() {
+          @Override
+          public TaskCallback createCallback() {
+            state.startProcess();
+            containerMetrics.processes().inc();
+            return callbackManager.createCallback(task.taskName(), envelope, coordinator);
+          }
 
 Review comment:
   This is not related to your change. Currently the life-cycle of the executor service is controlled two levels up at the `SamzaContainer` layer, i.e, the executor service is stopped as a part of the `SamzaContainer` shutdown sequence. 
   
   This makes reasoning about life-cycle of the executor service harder. It would be better to manage the life-cycle entirely within the RunLoop and stop it as a part of `RunLoop` shutdown sequence.
   
   What do you think?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services