You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by GitBox <gi...@apache.org> on 2021/09/22 16:11:08 UTC
[GitHub] [solr] markrmiller opened a new pull request #300: SOLR-15644: Test thread leak management.
markrmiller opened a new pull request #300:
URL: https://github.com/apache/solr/pull/300
Add the ability to interrupt and wait for threads for problematic tests.
https://issues.apache.org/jira/browse/SOLR-15644
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] anshumg commented on pull request #300: SOLR-15644: Test thread leak management.
Posted by GitBox <gi...@apache.org>.
anshumg commented on pull request #300:
URL: https://github.com/apache/solr/pull/300#issuecomment-946252363
@markrmiller - do you plan to merge parts of this in, as @madrob suggested i.e. directly to Randomized Testing ? If not, should we close this out ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] madrob commented on a change in pull request #300: SOLR-15644: Test thread leak management.
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #300:
URL: https://github.com/apache/solr/pull/300#discussion_r718870309
##########
File path: solr/test-framework/src/java/org/apache/solr/SolrIgnoredThreadsFilter.java
##########
@@ -52,7 +56,23 @@ public boolean reject(Thread t) {
}
// These is a java pool for the collection stream api
- if (threadName.startsWith("ForkJoinPool.")) {
+ if (threadName.startsWith("ForkJoinPool.") || threadName.startsWith("pool-")) {
+ return true;
+ }
+
+ // load balancer is leaky
+ if (threadName.startsWith("aliveCheckExecutor")) {
+ return true;
+ }
+
+
+ // we don't handle zk shutdown well, but these threads are harmless and will shortly go away
+ if (threadName.startsWith("SessionTracker")) {
+ return true;
+ }
+
+ // tools
+ if (threadName.startsWith("Reference Handler") && threadName.startsWith("Signal Dispatcher") && threadName.startsWith("Monitor") && threadName.startsWith("YJPAgent-RequestListener")) {
Review comment:
Some of these might be useful to commit to randomized testing directly? Like I did in https://github.com/randomizedtesting/randomizedtesting/pull/284
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] markrmiller commented on pull request #300: SOLR-15644: Test thread leak management.
Posted by GitBox <gi...@apache.org>.
markrmiller commented on pull request #300:
URL: https://github.com/apache/solr/pull/300#issuecomment-948387105
No I'm not, I dont think mike understands what this pr is about.
I say what I'm doing above in the comment right above yours.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] markrmiller closed pull request #300: SOLR-15644: Test thread leak management.
Posted by GitBox <gi...@apache.org>.
markrmiller closed pull request #300:
URL: https://github.com/apache/solr/pull/300
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] markrmiller commented on pull request #300: SOLR-15644: Test thread leak management.
Posted by GitBox <gi...@apache.org>.
markrmiller commented on pull request #300:
URL: https://github.com/apache/solr/pull/300#issuecomment-958959533
Won't do. Will avoid ideal solution for these rarer of cases, will do waits / hacks / alternatives in line with current approaches in individual issues as necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] markrmiller closed pull request #300: SOLR-15644: Test thread leak management.
Posted by GitBox <gi...@apache.org>.
markrmiller closed pull request #300:
URL: https://github.com/apache/solr/pull/300
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] markrmiller commented on pull request #300: SOLR-15644: Test thread leak management.
Posted by GitBox <gi...@apache.org>.
markrmiller commented on pull request #300:
URL: https://github.com/apache/solr/pull/300#issuecomment-938669235
It turns out I may end up ditching this PR. The primary test it would be used for is an HDFS test, and then also the rest of the HDFS tests (which can be dealt with alternatively, but I don't recall all that is necessary and in the past it took quite a bit of digging). I won't be looking at the HDFS tests though.
There is still a possible need in the S3 tests, but that may still end up surmountable depending on some remaining efforts If that is the case, I won't make the remaining changes necessary to this code that I hacked and chopped out and in quickly just to get compile friendly and into a base I could leverage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] markrmiller commented on pull request #300: SOLR-15644: Test thread leak management.
Posted by GitBox <gi...@apache.org>.
markrmiller commented on pull request #300:
URL: https://github.com/apache/solr/pull/300#issuecomment-958959533
Won't do. Will avoid ideal solution for these rarer of cases, will do waits / hacks / alternatives in line with current approaches in individual issues as necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] dweiss commented on a change in pull request #300: SOLR-15644: Test thread leak management.
Posted by GitBox <gi...@apache.org>.
dweiss commented on a change in pull request #300:
URL: https://github.com/apache/solr/pull/300#discussion_r714536837
##########
File path: solr/test-framework/src/java/org/apache/solr/SolrTestCase.java
##########
@@ -144,12 +152,107 @@ public static void checkSyspropForceBeforeClassAssumptionFailure() {
public void checkSyspropForceBeforeAssumptionFailure() {
// ant test -Dargs="-Dtests.force.assumption.failure.before=true"
final String PROP = "tests.force.assumption.failure.before";
- assumeFalse(PROP + " == true",
- systemPropertyAsBoolean(PROP, false));
+ assumeFalse(PROP + " == true", systemPropertyAsBoolean(PROP, false));
}
-
+
@AfterClass
- public static void shutdownLogger() throws Exception {
+ public static void shutdownLogger() {
StartupLoggingUtils.shutdown();
}
+
+ protected static void interruptThreadsOnTearDown() {
Review comment:
This can be done (or maybe is, didn't look at the current annotation on SolrTestCase/LuceneTestCase) automatically by the test framework -- see thread lingering/ actions on these annotations.
https://github.com/randomizedtesting/randomizedtesting/blob/master/examples/maven/src/main/java/com/carrotsearch/examples/randomizedrunner/Test010Lingering.java#L40-L43
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] madrob commented on a change in pull request #300: SOLR-15644: Test thread leak management.
Posted by GitBox <gi...@apache.org>.
madrob commented on a change in pull request #300:
URL: https://github.com/apache/solr/pull/300#discussion_r718870309
##########
File path: solr/test-framework/src/java/org/apache/solr/SolrIgnoredThreadsFilter.java
##########
@@ -52,7 +56,23 @@ public boolean reject(Thread t) {
}
// These is a java pool for the collection stream api
- if (threadName.startsWith("ForkJoinPool.")) {
+ if (threadName.startsWith("ForkJoinPool.") || threadName.startsWith("pool-")) {
+ return true;
+ }
+
+ // load balancer is leaky
+ if (threadName.startsWith("aliveCheckExecutor")) {
+ return true;
+ }
+
+
+ // we don't handle zk shutdown well, but these threads are harmless and will shortly go away
+ if (threadName.startsWith("SessionTracker")) {
+ return true;
+ }
+
+ // tools
+ if (threadName.startsWith("Reference Handler") && threadName.startsWith("Signal Dispatcher") && threadName.startsWith("Monitor") && threadName.startsWith("YJPAgent-RequestListener")) {
Review comment:
Some of these might be useful to commit to randomized testing directly? Like I did in https://github.com/randomizedtesting/randomizedtesting/pull/284
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] markrmiller commented on pull request #300: SOLR-15644: Test thread leak management.
Posted by GitBox <gi...@apache.org>.
markrmiller commented on pull request #300:
URL: https://github.com/apache/solr/pull/300#issuecomment-958959533
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] markrmiller closed pull request #300: SOLR-15644: Test thread leak management.
Posted by GitBox <gi...@apache.org>.
markrmiller closed pull request #300:
URL: https://github.com/apache/solr/pull/300
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] markrmiller commented on pull request #300: SOLR-15644: Test thread leak management.
Posted by GitBox <gi...@apache.org>.
markrmiller commented on pull request #300:
URL: https://github.com/apache/solr/pull/300#issuecomment-935292836
I only just learned of the draft option. This is ripped from other code, I will update it after it's modified for use in an issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org