You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Yongjun Zhang (JIRA)" <ji...@apache.org> on 2014/09/02 04:22:20 UTC
[jira] [Updated] (HADOOP-11045) Introducing a tool to detect flaky
tests of hadoop jenkins test job
[ https://issues.apache.org/jira/browse/HADOOP-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yongjun Zhang updated HADOOP-11045:
-----------------------------------
Description:
File this jira to introduce a tool to detect flaky tests of hadoop jenkins test jobs.
I developed the tool on top of some initial work [~tlipcon] did. We find it quite useful. With Todd's agreement, I'd like to push it to upstream so all of us can share (thanks Todd for the initial work and support). I hope you find the tool useful.
This is a tool for hadoop contributors rather than hadoop users. Thanks [~tedyu] for the advice to put to dev-support dir.
Description of the tool:
{code}
#
# Given a jenkins test job, this script examines all runs of the job done
# within specified period of time (number of days prior to the execution
# time of this script), and reports all failed tests.
#
# The output of this script includes a section for each run that has failed
# tests, with each failed test name listed.
#
# More importantly, at the end, it outputs a summary section to list all failed
# tests within all examined runs, and indicate how many runs a same test
# failed, and sorted all failed tests by how many runs each test failed in.
#
# This way, when we see failed tests in PreCommit build, we can quickly tell
# whether a failed test is a new failure or it failed before, and it may just
# be a flaky test.
#
# Of course, to be 100% sure about the reason of a failed test, closer look
# at the failed test for the specific run is necessary.
#
{code}
was:
File this jira to introduce a tool to detect flaky tests of hadoop jenkins test jobs.
I developed the tool on top of some initial work [~tlipcon] did. We find it quite useful. With Todd's agreement, I'd like to push it to upstream so all of us can share (thanks Todd for the initial work and support). I hope you find the tool useful.
This is a tool for hadoop contributors rather than hadoop users. Thanks [~tedyu] for the advice to put to dev-support dir.
Description of the tool:
#
# Given a jenkins test job, this script examines all runs of the job done
# within specified period of time (number of days prior to the execution
# time of this script), and reports all failed tests.
#
# The output of this script includes a section for each run that has failed
# tests, with each failed test name listed.
#
# More importantly, at the end, it outputs a summary section to list all failed
# tests within all examined runs, and indicate how many runs a same test
# failed, and sorted all failed tests by how many runs each test failed in.
#
# This way, when we see failed tests in PreCommit build, we can quickly tell
# whether a failed test is a new failure or it failed before, and it may just
# be a flaky test.
#
# Of course, to be 100% sure about the reason of a failed test, closer look
# at the failed test for the specific run is necessary.
#
> Introducing a tool to detect flaky tests of hadoop jenkins test job
> -------------------------------------------------------------------
>
> Key: HADOOP-11045
> URL: https://issues.apache.org/jira/browse/HADOOP-11045
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build, tools
> Reporter: Yongjun Zhang
> Assignee: Yongjun Zhang
>
> File this jira to introduce a tool to detect flaky tests of hadoop jenkins test jobs.
> I developed the tool on top of some initial work [~tlipcon] did. We find it quite useful. With Todd's agreement, I'd like to push it to upstream so all of us can share (thanks Todd for the initial work and support). I hope you find the tool useful.
> This is a tool for hadoop contributors rather than hadoop users. Thanks [~tedyu] for the advice to put to dev-support dir.
> Description of the tool:
> {code}
> #
> # Given a jenkins test job, this script examines all runs of the job done
> # within specified period of time (number of days prior to the execution
> # time of this script), and reports all failed tests.
> #
> # The output of this script includes a section for each run that has failed
> # tests, with each failed test name listed.
> #
> # More importantly, at the end, it outputs a summary section to list all failed
> # tests within all examined runs, and indicate how many runs a same test
> # failed, and sorted all failed tests by how many runs each test failed in.
> #
> # This way, when we see failed tests in PreCommit build, we can quickly tell
> # whether a failed test is a new failure or it failed before, and it may just
> # be a flaky test.
> #
> # Of course, to be 100% sure about the reason of a failed test, closer look
> # at the failed test for the specific run is necessary.
> #
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)