[GitHub] [samza] mynameborat commented on a diff in pull request #1652: Fix deadlock in StreamAppender

["mynameborat (via GitHub)" <gi...@apache.org>]

mynameborat commented on code in PR #1652:
URL: https://github.com/apache/samza/pull/1652#discussion_r1091377933


##########
samza-log4j2/src/main/java/org/apache/samza/logging/log4j2/StreamAppender.java:
##########
@@ -186,13 +190,20 @@ public void append(LogEvent event) {
       if (!systemInitialized) {
         // configs are needed to set up producer system, so check that before actually initializing
         if (this.loggingContextHolder.getConfig() != null) {
-          synchronized (this) {
-            if (!systemInitialized) {
-              setupSystem();
-              systemInitialized = true;
+          if (setUpSystemLock.tryLock(SET_UP_SYSTEM_TIMEOUT_MILLI_SECONDS, TimeUnit.MILLISECONDS)) {
+            try {
+              if (!systemInitialized) {
+                setupSystem();

Review Comment:
   Is this supposed to be run only once? I am thinking if we should also set it to `true` within finally block? 
   Alternatively, use `AtomicBoolean` and check and set in ln:190.



##########
samza-log4j2/src/test/java/org/apache/samza/logging/log4j2/TestStreamAppender.java:
##########
@@ -444,6 +486,36 @@ private void logRecursivelyAndVerifyMessages(List<String> messages) throws Inter
     assertEquals(messages.size(), MockSystemProducer.messagesReceived.size());
   }
 
+  private void SetupSystemTimeoutAndVerifyMessages(long setUpSystemTime) throws InterruptedException {

Review Comment:
   nit: camelCasing 
   s/SetupSystemTimeoutAndVerifyMessages/setupSystemTImoutAndVerifyMessages 



##########
samza-log4j2/src/test/java/org/apache/samza/logging/log4j2/TestStreamAppender.java:
##########
@@ -444,6 +486,36 @@ private void logRecursivelyAndVerifyMessages(List<String> messages) throws Inter
     assertEquals(messages.size(), MockSystemProducer.messagesReceived.size());
   }
 
+  private void SetupSystemTimeoutAndVerifyMessages(long setUpSystemTime) throws InterruptedException {
+    MockSystemProducer.listeners.clear();
+    MockSystemAdmin.listeners.clear();
+    List<String> messages = Lists.newArrayList("testing1", "testing2");
+    // Set up latch
+    final CountDownLatch allMessagesSent = new CountDownLatch(messages.size());
+    MockSystemProducer.listeners.add((source, envelope) -> allMessagesSent.countDown());
+    MockSystemAdmin.listeners.add(streamSpec -> {
+      try {
+        // mock setUpSystem time during createStream() call
+        sleep(setUpSystemTime);
+      } catch (InterruptedException e) {
+        e.printStackTrace();
+      }
+    });
+    ExecutorService service0 = Executors.newFixedThreadPool(1);
+    ExecutorService service1 = Executors.newFixedThreadPool(1);
+    // Log the messages with two threads
+    service0.submit(()->LOG.info(messages.get(0)));
+    service1.submit(()->LOG.info(messages.get(1)));
+    // Wait for messages
+    allMessagesSent.await(setUpSystemTime * 4, TimeUnit.MILLISECONDS);
+    // If the setUpSystem time out, verify only one message sent. otherwise, verify two messages sent
+    if (setUpSystemTime >= SET_UP_SYSTEM_TIMEOUT_MILLI_SECONDS) {
+      assertEquals(messages.size() - 1, MockSystemProducer.messagesReceived.size());
+    } else {
+      assertEquals(messages.size(), MockSystemProducer.messagesReceived.size());
+    }
+  }
+
   private static Config baseConfig() {

Review Comment:
   perhaps add another test case for `setupSystem` throw an exception. Not sure what the expected behavior, but it will be good have a test with the expected behavior so that I won't have to ask next time :)



##########
samza-log4j2/src/test/java/org/apache/samza/logging/log4j2/TestStreamAppender.java:
##########
@@ -444,6 +486,36 @@ private void logRecursivelyAndVerifyMessages(List<String> messages) throws Inter
     assertEquals(messages.size(), MockSystemProducer.messagesReceived.size());
   }
 
+  private void SetupSystemTimeoutAndVerifyMessages(long setUpSystemTime) throws InterruptedException {
+    MockSystemProducer.listeners.clear();
+    MockSystemAdmin.listeners.clear();
+    List<String> messages = Lists.newArrayList("testing1", "testing2");
+    // Set up latch
+    final CountDownLatch allMessagesSent = new CountDownLatch(messages.size());
+    MockSystemProducer.listeners.add((source, envelope) -> allMessagesSent.countDown());
+    MockSystemAdmin.listeners.add(streamSpec -> {
+      try {
+        // mock setUpSystem time during createStream() call
+        sleep(setUpSystemTime);
+      } catch (InterruptedException e) {
+        e.printStackTrace();
+      }
+    });
+    ExecutorService service0 = Executors.newFixedThreadPool(1);
+    ExecutorService service1 = Executors.newFixedThreadPool(1);
+    // Log the messages with two threads
+    service0.submit(()->LOG.info(messages.get(0)));
+    service1.submit(()->LOG.info(messages.get(1)));

Review Comment:
   Introduce a jitter so that there is no guarantees on the ordering of `setup`. Meaning `service1.submit(() -> LOG.info(messages.get(1)))` can also trigger `setup`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org