You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@harmony.apache.org by "Alexey Varlamov (JIRA)" <ji...@apache.org> on 2007/09/24 11:06:50 UTC
[jira] Created: (HARMONY-4842) [bti] Unreliable status calculation
for snapshot testing results
[bti] Unreliable status calculation for snapshot testing results
----------------------------------------------------------------
Key: HARMONY-4842
URL: https://issues.apache.org/jira/browse/HARMONY-4842
Project: Harmony
Issue Type: Bug
Components: build - test - ci
Reporter: Alexey Varlamov
The snapshot status page [1] wrongly accounts some suites PASSED. It happens when suite is expected to generate JUnit-style report and some tests crash or cannot compile and build run aborts after running only part of suite. In this case those few successfully run tests are published and suite status is shown green on the page [1].
This in particular explains wrong data for DRLVM tests on Windows [2], which in reality is caused by compilation errors in cunit tests.
The most reliable way would be to generate status.txt always and rely on it when calculating status, while look for JUnit statistics only to get pass rate.
[1] http://people.apache.org/~mloenko/snapshot_testing/script/snapshots_summary.html
[2] http://people.apache.org/~mloenko/snapshot_testing/script/r577501/index.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HARMONY-4842) [bti] Unreliable status calculation
for snapshot testing results
Posted by "Stepan Mishura (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HARMONY-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stepan Mishura reassigned HARMONY-4842:
---------------------------------------
Assignee: Stepan Mishura
> [bti] Unreliable status calculation for snapshot testing results
> ----------------------------------------------------------------
>
> Key: HARMONY-4842
> URL: https://issues.apache.org/jira/browse/HARMONY-4842
> Project: Harmony
> Issue Type: Bug
> Components: build - test - ci
> Reporter: Alexey Varlamov
> Assignee: Stepan Mishura
>
> The snapshot status page [1] wrongly accounts some suites PASSED. It happens when suite is expected to generate JUnit-style report and some tests crash or cannot compile and build run aborts after running only part of suite. In this case those few successfully run tests are published and suite status is shown green on the page [1].
> This in particular explains wrong data for DRLVM tests on Windows [2], which in reality is caused by compilation errors in cunit tests.
> The most reliable way would be to generate status.txt always and rely on it when calculating status, while look for JUnit statistics only to get pass rate.
> [1] http://people.apache.org/~mloenko/snapshot_testing/script/snapshots_summary.html
> [2] http://people.apache.org/~mloenko/snapshot_testing/script/r577501/index.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.