[jira] [Created] (KAFKA-12216) Improve flaky test reporting

["Ismael Juma (Jira)" <ji...@apache.org>]

Ismael Juma created KAFKA-12216:
-----------------------------------

             Summary: Improve flaky test reporting
                 Key: KAFKA-12216
                 URL: https://issues.apache.org/jira/browse/KAFKA-12216
             Project: Kafka
          Issue Type: Improvement
            Reporter: Ismael Juma


Since Gradle 6.8, setting the following improves flaky test reporting by Gradle:
{code:groovy}
 reports.junitXml.mergeReruns = true
{code}
When this new option is enabled, if a test fails but is then retried and succeeds, its failures will be recorded as <flakyFailure> instead of <failure>, within one <testcase> (same as Maven when reruns are enabled).

For Jenkins to understand this notation, we need to use the `JUnitFlakyTestDataPublisher`:
{code:java}
junit testResults: '**/build/test-results/**/TEST-*.xml', testDataPublishers: [[$class: 'JUnitFlakyTestDataPublisher']]
{code}
But this plugin doesn't seem to be installed in Apache Jenkins and is generally not actively maintained (last release was 5 years ago). I filed an issue on the core junit plugin for Jenkins:

https://github.com/jenkinsci/junit-plugin/issues/237

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)