You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@netbeans.apache.org by Michael Bien <mb...@gmail.com> on 2022/07/04 10:52:04 UTC
Re: Kill ValidateLayerConsistencyTest?
the test is actually crashing the JVM, I am not quite sure why.
I noticed this while trying to add an action which displays test reports
on github. At first I thought it started when we enabled the xserver,
however, i just tested it while turning the xserver off again (temporary
in a PR) and it was still crashing the jvm.
So I went backwards through
https://github.com/apache/netbeans/commits/master and tried to find
"errors: 1" in the logs. And it is already happening for >24 days (thats
when i gave up looking at logs).
stopped at this commit:
https://github.com/apache/netbeans/commit/5b822a305e61bdb4ae3cb7177a2a21a8209c357b
I believe this is proof that most (including me) are not looking at log
output of unreliable tests when they fail before restarting or ignoring
them. That is basically as if CI wouldn't run them in the first place.
2022-06-24T19:36:52.4792402Z [junit] Testsuite: org.netbeans.core.validation.ValidateLayerConsistencyTest
2022-06-24T19:36:52.4906892Z [junit] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec
2022-06-24T19:36:52.4907315Z [junit]
2022-06-24T19:36:52.4908198Z [junit] Testcase: org.netbeans.core.validation.ValidateLayerConsistencyTest:testActionInstancesOnlyInActionsFolder: Caused an ERROR
2022-06-24T19:36:52.4908858Z [junit] Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit.
2022-06-24T19:36:52.4909459Z [junit] junit.framework.AssertionFailedError: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit.
2022-06-24T19:36:52.4910056Z [junit] at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
2022-06-24T19:36:52.4910660Z [junit] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-24T19:36:52.4911206Z [junit] at java.base/java.util.Vector.forEach(Vector.java:1388)
2022-06-24T19:36:52.4911716Z [junit] at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
2022-06-24T19:36:52.4912366Z [junit] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-24T19:36:52.4912984Z [junit] at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
2022-06-24T19:36:52.4913634Z [junit] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-24T19:36:52.4914270Z [junit] at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
2022-06-24T19:36:52.4914927Z [junit] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-24T19:36:52.4916082Z [junit] at java.base/java.util.Vector.forEach(Vector.java:1388)
2022-06-24T19:36:52.4916699Z [junit] at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
2022-06-24T19:36:52.4917293Z [junit] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-24T19:36:52.4917891Z [junit] at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
2022-06-24T19:36:52.4918462Z [junit] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-24T19:36:52.4919023Z [junit] at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
2022-06-24T19:36:52.4920176Z [junit] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-24T19:36:52.4920629Z [junit]
2022-06-24T19:36:52.4921187Z [junit]
2022-06-24T19:36:52.5374861Z [junit] Test org.netbeans.core.validation.ValidateLayerConsistencyTest FAILED (crashed)
here is my +1 for turning this test off again. I see no benefit in
running this right now.
-mbien
On 29.05.22 19:19, Matthias Bläsing wrote:
> Hi all,
>
> for me the testing of NetBeans shows its ugly dark side and I'm
> seriously thinking about killing the ValidateLayerConsistencyTest. It
> is flaky and causes untraceable errors.
>
> This is the PR:
>
> https://github.com/apache/netbeans/pull/4058
>
> The flakiness can be observed when comparing the commit validation
> runs. 1 fails whiel 3 work ok. In this instance the failing instance is
> JDK8, but I saw others.
>
> I use this PR to try to fix this:
>
> https://github.com/apache/netbeans/tree/try_fix_commit_validation2
>
> One problem could be tracked down, that the maven module has a entry in
> the configuration, that is a template, but is checked for instablility.
> Fixed here:
>
> https://github.com/apache/netbeans/commit/40a60e311000977b490935c6bf762464d3ddaa96
>
> Currently it fails with an error that also seems to be a race condition
> (line 23766-2769):
>
> https://pipelines.actions.githubusercontent.com/serviceHosts/3cf2747f-b5d6-4e8e-a999-150ff8ae61e1/_apis/pipelines/1/runs/14001/signedlogcontent/7?urlExpires=2022-05-29T17%3A16%3A59.2143803Z&urlSigningMethod=HMACV1&urlSignature=f%2B%2BJVlE0n3fHCqA4wJkq4iKPdD%2B502hC2csZJFs5keE%3D
>
> 2022-05-29T17:06:31.9092439Z [junit] java.lang.AssertionError: Has to be NbRepository: org.openide.filesystems.Repository@744b325b
> 2022-05-29T17:06:31.9093232Z [junit] at org.netbeans.core.startup.Main.start(Main.java:298)
> 2022-05-29T17:06:31.9094120Z [junit] at org.netbeans.core.startup.TopThreadGroup.run(TopThreadGroup.java:98)
> 2022-05-29T17:06:31.9094571Z [junit] at java.lang.Thread.run(Thread.java:750)
> 2022-05-29T17:06:31.9095235Z [junit] -------------------------------------------------------------------------------
> 2022-05-29T17:06:31.9095602Z [junit] >Log Session: Sunday, May 29, 2022 5:06:31 PM UTC
> 2022-05-29T17:06:31.9095875Z [junit] >System Info:
> 2022-05-29T17:06:31.9096375Z [junit] Product Version = Apache NetBeans Platform Dev (Build dev-40a60e311000977b490935c6bf762464d3ddaa96)
>
> Given that I changed code in the JS area, this does not seem to related
> to my changes.
>
> So at this point this Test does harm and so is a candidate to be
> removed.
>
> I would like to fix this tough, but I put literally days into this and
> am at the end of my ideas. Anyone wants to chime in?
>
> Greetings
>
> Matthias
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail:dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>