[GitHub] [pulsar] Jason918 commented on a change in pull request #13130: [Issue 13129] [pulsar-metadata] Add watchdog thread in metadata store and track long running tasks.

[GitBox <gi...@apache.org>]

Jason918 commented on a change in pull request #13130:
URL: https://github.com/apache/pulsar/pull/13130#discussion_r762377508



##########
File path: pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/AbstractMetadataStore.java
##########
@@ -317,7 +346,33 @@ public void invalidateAll() {
      */
     protected void execute(Runnable task, CompletableFuture<?> future) {
         try {
-            executor.execute(task);
+            // Wrap the original task, so we can record the thread on which it is running
+            TaskWrapper taskWrapper = new TaskWrapper(task);
+            executorWatchDog.execute(() -> {

Review comment:
       Agreed on we should dump the stack trace of all threads. 
   In previous deadlock cases, we got deadlock because metadata-store thread depends on it self. But it's possible that, the depend path is like `metadata-store thread` --> async task in other pulsar-executor --> `metadata-store thread`.  So we need to see the whole picture.
   
   Futher more, I think just dumping the stack trace is not enough, We need the lock info of these threads to help locating the deadlock.  We can use `ThreadDumpUtil#buildThreadDiagnosticString` metioned in #13124




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org