You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by GitBox <gi...@apache.org> on 2020/08/07 05:29:19 UTC
[GitHub] [helix] jiajunwang commented on a change in pull request #1227: Implement thread leakage check for each test class
jiajunwang commented on a change in pull request #1227:
URL: https://github.com/apache/helix/pull/1227#discussion_r466827098
##########
File path: helix-core/src/test/java/org/apache/helix/common/ZkTestBase.java
##########
@@ -719,6 +720,12 @@ public void cleanupLiveInstanceOwners() {
clientMap.clear();
}
_liveInstanceOwners.clear();
+
+ boolean status = TestHelper.afterClassCheck(name);
+ // Assert here does not work.
+ if (!status) {
+ System.out.println("---------- Test Class " + name + " thread leakage detected! ---------------");
Review comment:
Shall we just fail the test? And why it is only applied to ZkTestBase? There are other test classes which are not based on this one.
##########
File path: helix-core/src/test/java/org/apache/helix/TestHelper.java
##########
@@ -847,4 +850,161 @@ public static void printZkListeners(HelixZkClient client) throws Exception {
}
System.out.println("}");
}
+
+ // Thread dump related utility function
+ private static ThreadGroup getRootThreadGroup() {
+ ThreadGroup candidate = Thread.currentThread().getThreadGroup();
+ while (candidate.getParent() != null) {
+ candidate = candidate.getParent();
+ }
+ return candidate;
+ }
+
+ public static List<Thread> getAllThreads() {
+ ThreadGroup rootThreadGroup = getRootThreadGroup();
+ Thread[] threads = new Thread[32];
+ int count = rootThreadGroup.enumerate(threads);
+ while (count == threads.length) {
+ threads = new Thread[threads.length * 2];
+ count = rootThreadGroup.enumerate(threads);
+ }
+ return Arrays.asList(Arrays.copyOf(threads, count));
+ }
+
+ static public enum ThreadCategory {
+ ZkServer("zookeeper server threads", 4, 100,
+ new String[]{"SessionTracker", "NIOServerCxn", "SyncThread:", "ProcessThread"}),
+ ZkSession("zkclient/zooKeeper session threads", 12, 12,
+ new String[] {"ZkClient-EventThread", "ZkClient-AsyncCallback", "-EventThread", "-SendThread"}),
+ ForkJoin( "fork join pool threads", 2, 10, new String[]{"ForkJoinPool"}),
+ Timer( "timer threads", 0, 2, new String[]{"time"}),
+ TaskStateModel("TaskStateModel threads", 0, 0, new String[]{"TaskStateModel"}),
+ Other("Other threads", 0, 3, new String[]{""});
+
+ private String _description;
+ private List<String> _pattern;
+ private int _warningLimit;
+ private int _limit;
+
+ public String getDescription() {
+ return _description;
+ }
+
+ public Predicate<String> getMatchPred() {
+ if (this.name() != ThreadCategory.Other.name()) {
+ Predicate<String> pred = target -> {
+ for (String p : _pattern) {
+ if (target.toLowerCase().contains(p.toLowerCase())) {
+ return true;
+ }
+ }
+ return false;
+ };
+ return pred;
+ }
+
+ List<Predicate<String>> predicateList = new ArrayList<>();
+ for (ThreadCategory threadCategory : ThreadCategory.values()) {
+ if (threadCategory == ThreadCategory.Other) {
+ continue;
+ }
+ predicateList.add(threadCategory.getMatchPred());
+ }
+ Predicate<String> pred = target -> {
+ for (Predicate<String> p : predicateList) {
+ if (p.test(target)) {
+ return false;
+ }
+ }
+ return true;
+ };
+
+ return pred;
+ }
+
+ public int getWarningLimit() {
+ return _warningLimit;
+ }
+
+ public int getLimit() {
+ return _limit;
+ }
+
+ private ThreadCategory(String description, int warningLimit, int limit, String[] patterns ) {
+ _description = description;
+ _pattern = Arrays.asList(patterns);
+ _warningLimit = warningLimit;
+ _limit = limit;
+ }
+ }
+
+ public static boolean afterClassCheck(String classname) {
Review comment:
This method seems to be a required check instead of Test Helper.
It would be cleaner if you can create a new class for it.
##########
File path: helix-core/src/test/java/org/apache/helix/TestHelper.java
##########
@@ -847,4 +850,161 @@ public static void printZkListeners(HelixZkClient client) throws Exception {
}
System.out.println("}");
}
+
+ // Thread dump related utility function
+ private static ThreadGroup getRootThreadGroup() {
+ ThreadGroup candidate = Thread.currentThread().getThreadGroup();
+ while (candidate.getParent() != null) {
+ candidate = candidate.getParent();
+ }
+ return candidate;
+ }
+
+ public static List<Thread> getAllThreads() {
+ ThreadGroup rootThreadGroup = getRootThreadGroup();
+ Thread[] threads = new Thread[32];
+ int count = rootThreadGroup.enumerate(threads);
+ while (count == threads.length) {
+ threads = new Thread[threads.length * 2];
+ count = rootThreadGroup.enumerate(threads);
+ }
+ return Arrays.asList(Arrays.copyOf(threads, count));
+ }
+
+ static public enum ThreadCategory {
+ ZkServer("zookeeper server threads", 4, 100,
Review comment:
The hard coded patterns seem hard to maintain. If any thread names are changed, then their leakage won't be detected, right? Can we make the pattern string defined in some constants and refer to them in here and also in the place where thread names are specified?
##########
File path: helix-core/src/test/java/org/apache/helix/common/ZkTestBase.java
##########
@@ -711,6 +711,7 @@ protected Message createMessage(Message.MessageType type, String msgId, String f
@AfterClass
Review comment:
nit, I believe you can define multiple AfterClass methods so we don't need to mix things up.
##########
File path: helix-core/src/test/java/org/apache/helix/TestHelper.java
##########
@@ -847,4 +850,161 @@ public static void printZkListeners(HelixZkClient client) throws Exception {
}
System.out.println("}");
}
+
+ // Thread dump related utility function
+ private static ThreadGroup getRootThreadGroup() {
+ ThreadGroup candidate = Thread.currentThread().getThreadGroup();
+ while (candidate.getParent() != null) {
+ candidate = candidate.getParent();
+ }
+ return candidate;
+ }
+
+ public static List<Thread> getAllThreads() {
+ ThreadGroup rootThreadGroup = getRootThreadGroup();
+ Thread[] threads = new Thread[32];
+ int count = rootThreadGroup.enumerate(threads);
+ while (count == threads.length) {
+ threads = new Thread[threads.length * 2];
+ count = rootThreadGroup.enumerate(threads);
+ }
+ return Arrays.asList(Arrays.copyOf(threads, count));
+ }
+
+ static public enum ThreadCategory {
+ ZkServer("zookeeper server threads", 4, 100,
+ new String[]{"SessionTracker", "NIOServerCxn", "SyncThread:", "ProcessThread"}),
+ ZkSession("zkclient/zooKeeper session threads", 12, 12,
+ new String[] {"ZkClient-EventThread", "ZkClient-AsyncCallback", "-EventThread", "-SendThread"}),
+ ForkJoin( "fork join pool threads", 2, 10, new String[]{"ForkJoinPool"}),
+ Timer( "timer threads", 0, 2, new String[]{"time"}),
+ TaskStateModel("TaskStateModel threads", 0, 0, new String[]{"TaskStateModel"}),
+ Other("Other threads", 0, 3, new String[]{""});
+
+ private String _description;
+ private List<String> _pattern;
+ private int _warningLimit;
+ private int _limit;
+
+ public String getDescription() {
+ return _description;
+ }
+
+ public Predicate<String> getMatchPred() {
+ if (this.name() != ThreadCategory.Other.name()) {
+ Predicate<String> pred = target -> {
+ for (String p : _pattern) {
+ if (target.toLowerCase().contains(p.toLowerCase())) {
+ return true;
+ }
+ }
+ return false;
+ };
+ return pred;
+ }
+
+ List<Predicate<String>> predicateList = new ArrayList<>();
+ for (ThreadCategory threadCategory : ThreadCategory.values()) {
+ if (threadCategory == ThreadCategory.Other) {
+ continue;
+ }
+ predicateList.add(threadCategory.getMatchPred());
+ }
+ Predicate<String> pred = target -> {
+ for (Predicate<String> p : predicateList) {
+ if (p.test(target)) {
+ return false;
+ }
+ }
+ return true;
+ };
+
+ return pred;
+ }
+
+ public int getWarningLimit() {
+ return _warningLimit;
+ }
+
+ public int getLimit() {
+ return _limit;
+ }
+
+ private ThreadCategory(String description, int warningLimit, int limit, String[] patterns ) {
+ _description = description;
+ _pattern = Arrays.asList(patterns);
+ _warningLimit = warningLimit;
+ _limit = limit;
+ }
+ }
+
+ public static boolean afterClassCheck(String classname) {
+ // step 1: get all active threads
+ List<Thread> threads = TestHelper.getAllThreads();
+ System.out.println(classname + " has active threads cnt:" + threads.size());
+
+
+ // step 2: caegorize threads
Review comment:
typo?
##########
File path: helix-core/src/test/java/org/apache/helix/TestHelper.java
##########
@@ -847,4 +850,161 @@ public static void printZkListeners(HelixZkClient client) throws Exception {
}
System.out.println("}");
}
+
+ // Thread dump related utility function
+ private static ThreadGroup getRootThreadGroup() {
+ ThreadGroup candidate = Thread.currentThread().getThreadGroup();
+ while (candidate.getParent() != null) {
+ candidate = candidate.getParent();
+ }
+ return candidate;
+ }
+
+ public static List<Thread> getAllThreads() {
Review comment:
private?
##########
File path: helix-core/src/test/java/org/apache/helix/TestHelper.java
##########
@@ -847,4 +850,161 @@ public static void printZkListeners(HelixZkClient client) throws Exception {
}
System.out.println("}");
}
+
+ // Thread dump related utility function
+ private static ThreadGroup getRootThreadGroup() {
+ ThreadGroup candidate = Thread.currentThread().getThreadGroup();
+ while (candidate.getParent() != null) {
+ candidate = candidate.getParent();
+ }
+ return candidate;
+ }
+
+ public static List<Thread> getAllThreads() {
+ ThreadGroup rootThreadGroup = getRootThreadGroup();
+ Thread[] threads = new Thread[32];
+ int count = rootThreadGroup.enumerate(threads);
+ while (count == threads.length) {
+ threads = new Thread[threads.length * 2];
+ count = rootThreadGroup.enumerate(threads);
+ }
+ return Arrays.asList(Arrays.copyOf(threads, count));
+ }
+
+ static public enum ThreadCategory {
Review comment:
private?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org