[GitHub] [samza] cameronlee314 commented on a change in pull request #1213: SAMZA-2305: Stream processor should ensure previous container is stopped during a rebalance

[GitBox <gi...@apache.org>]

cameronlee314 commented on a change in pull request #1213: SAMZA-2305: Stream processor should ensure previous container is stopped during a rebalance
URL: https://github.com/apache/samza/pull/1213#discussion_r348684851
 
 

 ##########
 File path: samza-core/src/main/java/org/apache/samza/processor/StreamProcessor.java
 ##########
 @@ -461,8 +482,25 @@ public void onJobModelExpired() {
             } else {
               LOGGER.info("Container: {} shutdown completed for stream processor: {}.", container, processorId);
             }
+          } else if (state == State.IN_REBALANCE) {
+            if (container != null) {
+              boolean hasContainerShutdown = interruptContainerAndShutdownExecutorService();
+              if (!hasContainerShutdown) {
+                LOGGER.warn("Job model expire unsuccessful. Failed to interrupt container: {} safely. "
+                    + "Stopping the stream processor: {}", container, processorId);
+                state = State.STOPPING;
+                jobCoordinator.stop();
+              } else {
+                Preconditions.checkState(containerExecutorService.isShutdown(),
 
 Review comment:
   1) Are you trying to ensure here that all of the tasks are actually finished? It looks like `isShutdown` might just check if you called `shutdown` (and you know you already did that within `interruptContainerAndShutdownExecutorService`), while `isTerminated` actually checks that the tasks are done. That said, if you use `isTerminated`, then is there a race condition between the latch being decremented and the task actually completing.
   2) What happens when this throws an exception and causes the `onJobModelExpired` callback to fail? Will the processor die/restart?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services