[GitHub] [netbeans] mbien commented on a change in pull request #3781: exile job for unreliable tests.

[GitBox <gi...@apache.org>]

mbien commented on a change in pull request #3781:
URL: https://github.com/apache/netbeans/pull/3781#discussion_r827006238



##########
File path: .travis.yml
##########
@@ -723,6 +721,18 @@ matrix:
             - export JAVA_HOME=$TEST_JDK
             - hide-logs.sh ant $OPTS commit-validation
 
+#   exile: for tests which don't behave. Job should be kept short to be easily restartable
+        - name: Test unreliable tests on Java 8
+          jdk: openjdk8
+          env:
+            - OPTS="-Dmetabuild.jsonurl=https://raw.githubusercontent.com/apache/netbeans-jenkins-lib/master/meta/netbeansrelease.json -quiet -Dcluster.config=java -Djavac.compilerargs=-nowarn -Dbuild.compiler.deprecation=false -Dtest-unit-sys-prop.ignore.random.failures=true"
+          before_script:
+            - nbbuild/travis/ant.sh $OPTS clean
+            - nbbuild/travis/ant.sh $OPTS build
+          script:
+            - travis_retry ant $OPTS -f java/java.lsp.server test
+            - travis_retry hide-logs.sh ant $OPTS -f java/gradle.java test

Review comment:
       I don't know actually. `hide-logs.sh` was there before, i just moved the line to a different job. I assumed it being a streaming filter which removes some spammy log lines to make CI not error - but I haven't actually looked at the script.
   
   Part of this test is me figuring out how practical `travis_retry` is. If you think this is a problem i can remove it. Main intention was to isolate bad behaving tests so that they would be in a short(er) restartable job, separated from reliable tests.
   
   This is also somewhat of a contingency plan in case the tests can't be stabilized despite all efforts. I didn't plan to merge this if it turns out that the pending reliability PRs fix the problem.
   
   But if you want I can merge it anyway (after your PRs are integrated).




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists