You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by mfenes <gi...@git.apache.org> on 2017/09/15 12:54:44 UTC
[GitHub] zookeeper pull request #375: ZOOKEEPER-1363: Categorise unit tests by 'test-...
GitHub user mfenes opened a pull request:
https://github.com/apache/zookeeper/pull/375
ZOOKEEPER-1363: Categorise unit tests by 'test-commit', 'full-test' etc
ZOOKEEPER-1363: Categorise unit tests by 'test-commit', 'full-test' etc
Added new capability of defining test categories in test category configuration files. Test category configuration files can contain list of test file names and/or patterns. It is also possible to exclude tests from a test category. Test categories are added to ${test.src.dir}/category directory. Current test categories are defined so that they remained API compatible with the previous build.xml in regards to how ‘ant test’ should be run (options are the same and they behave the same way).
Func and Perf categories are added to keep the previous FuncTest and PerfTest categories which were coded into the test file names.
Slow test category is added and contains tests which require more than 30 seconds to complete on a local developer machine.
Command ‘ant test -Dtest.quick=yes’ runs tests minus tests in the Slow category.
Command ‘ant test -Dtest.category=<category name>’ runs tests from the specified category.
It is possible to combine these two options to run a test category minus the Slow tests.
Test category file name must match the name of the test category (e.g. category/Commit).
List of excluded tests/patterns should be added to a .exclude file having the same name as the category (e.g. category/Commit.exclude).
Added README.txt which describes how to run unit tests in detail.
Benefits:
- test categories do not need to be included in the test file names (a test can be part of more than just one category).
- Slow test category is more flexible to exclude slow tests than the hardcoded exclusion of “**/*HammerTest.java” in build.xml.
- new test categorization can also exclude test file names/patterns from a category.
- test category can be added for a patch.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mfenes/zookeeper ZOOKEEPER-1363
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zookeeper/pull/375.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #375
----
commit cd51fe57cd839f49bf7babdae4ebded3c92f4b26
Author: Mark Fenes <mf...@cloudera.com>
Date: 2017-09-08T15:00:29Z
ZOOKEEPER-1363: Categorise unit tests by 'test-commit', 'full-test' etc
----
---
[GitHub] zookeeper pull request #375: ZOOKEEPER-1363: Categorise unit tests by 'test-...
Posted by mfenes <gi...@git.apache.org>.
GitHub user mfenes reopened a pull request:
https://github.com/apache/zookeeper/pull/375
ZOOKEEPER-1363: Categorise unit tests by 'test-commit', 'full-test' etc
ZOOKEEPER-1363: Categorise unit tests by 'test-commit', 'full-test' etc
Added new capability of defining test categories in test category configuration files. Test category configuration files can contain list of test file names and/or patterns. It is also possible to exclude tests from a test category. Test categories are added to ${test.src.dir}/category directory. Current test categories are defined so that they remained API compatible with the previous build.xml in regards to how ‘ant test’ should be run (options are the same and they behave the same way).
Func and Perf categories are added to keep the previous FuncTest and PerfTest categories which were coded into the test file names.
Slow test category is added and contains tests which require more than 30 seconds to complete on a local developer machine.
Command ‘ant test -Dtest.quick=yes’ runs tests minus tests in the Slow category.
Command ‘ant test -Dtest.category=<category name>’ runs tests from the specified category.
It is possible to combine these two options to run a test category minus the Slow tests.
Test category file name must match the name of the test category (e.g. category/Commit).
List of excluded tests/patterns should be added to a .exclude file having the same name as the category (e.g. category/Commit.exclude).
Added README.txt which describes how to run unit tests in detail.
Benefits:
- test categories do not need to be included in the test file names (a test can be part of more than just one category).
- Slow test category is more flexible to exclude slow tests than the hardcoded exclusion of “**/*HammerTest.java” in build.xml.
- new test categorization can also exclude test file names/patterns from a category.
- test category can be added for a patch.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mfenes/zookeeper ZOOKEEPER-1363
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zookeeper/pull/375.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #375
----
commit cd51fe57cd839f49bf7babdae4ebded3c92f4b26
Author: Mark Fenes <mf...@cloudera.com>
Date: 2017-09-08T15:00:29Z
ZOOKEEPER-1363: Categorise unit tests by 'test-commit', 'full-test' etc
commit 30414528e8550fa4835beb7f8d901f8cad773d6e
Author: Mark Fenes <mf...@cloudera.com>
Date: 2017-10-24T14:37:29Z
Trigger notification
Change-Id: I38ad5771552ff2bad16271dfb05fa00a683c3228
----
---
[GitHub] zookeeper pull request #375: ZOOKEEPER-1363: Categorise unit tests by 'test-...
Posted by mfenes <gi...@git.apache.org>.
Github user mfenes closed the pull request at:
https://github.com/apache/zookeeper/pull/375
---
[GitHub] zookeeper pull request #375: ZOOKEEPER-1363: Categorise unit tests by 'test-...
Posted by mfenes <gi...@git.apache.org>.
Github user mfenes closed the pull request at:
https://github.com/apache/zookeeper/pull/375
---
[GitHub] zookeeper issue #375: ZOOKEEPER-1363: Categorise unit tests by 'test-commit'...
Posted by afine <gi...@git.apache.org>.
Github user afine commented on the issue:
https://github.com/apache/zookeeper/pull/375
verified that jenkins is running the same number of tests with this patch as it did before. The list of slow tests also appears to be reasonable.
small nitpick, wouldn't it make more sense to have a test category called "fast" and use an exclusion list rather than include special logic for "slow"?
---
[GitHub] zookeeper pull request #375: ZOOKEEPER-1363: Categorise unit tests by 'test-...
Posted by mfenes <gi...@git.apache.org>.
Github user mfenes closed the pull request at:
https://github.com/apache/zookeeper/pull/375
---
[GitHub] zookeeper pull request #375: ZOOKEEPER-1363: Categorise unit tests by 'test-...
Posted by mfenes <gi...@git.apache.org>.
Github user mfenes closed the pull request at:
https://github.com/apache/zookeeper/pull/375
---
[GitHub] zookeeper pull request #375: ZOOKEEPER-1363: Categorise unit tests by 'test-...
Posted by mfenes <gi...@git.apache.org>.
GitHub user mfenes reopened a pull request:
https://github.com/apache/zookeeper/pull/375
ZOOKEEPER-1363: Categorise unit tests by 'test-commit', 'full-test' etc
ZOOKEEPER-1363: Categorise unit tests by 'test-commit', 'full-test' etc
Added new capability of defining test categories in test category configuration files. Test category configuration files can contain list of test file names and/or patterns. It is also possible to exclude tests from a test category. Test categories are added to ${test.src.dir}/category directory. Current test categories are defined so that they remained API compatible with the previous build.xml in regards to how ‘ant test’ should be run (options are the same and they behave the same way).
Func and Perf categories are added to keep the previous FuncTest and PerfTest categories which were coded into the test file names.
Slow test category is added and contains tests which require more than 30 seconds to complete on a local developer machine.
Command ‘ant test -Dtest.quick=yes’ runs tests minus tests in the Slow category.
Command ‘ant test -Dtest.category=<category name>’ runs tests from the specified category.
It is possible to combine these two options to run a test category minus the Slow tests.
Test category file name must match the name of the test category (e.g. category/Commit).
List of excluded tests/patterns should be added to a .exclude file having the same name as the category (e.g. category/Commit.exclude).
Added README.txt which describes how to run unit tests in detail.
Benefits:
- test categories do not need to be included in the test file names (a test can be part of more than just one category).
- Slow test category is more flexible to exclude slow tests than the hardcoded exclusion of “**/*HammerTest.java” in build.xml.
- new test categorization can also exclude test file names/patterns from a category.
- test category can be added for a patch.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mfenes/zookeeper ZOOKEEPER-1363
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zookeeper/pull/375.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #375
----
commit cd51fe57cd839f49bf7babdae4ebded3c92f4b26
Author: Mark Fenes <mf...@cloudera.com>
Date: 2017-09-08T15:00:29Z
ZOOKEEPER-1363: Categorise unit tests by 'test-commit', 'full-test' etc
----
---
[GitHub] zookeeper pull request #375: ZOOKEEPER-1363: Categorise unit tests by 'test-...
Posted by mfenes <gi...@git.apache.org>.
GitHub user mfenes reopened a pull request:
https://github.com/apache/zookeeper/pull/375
ZOOKEEPER-1363: Categorise unit tests by 'test-commit', 'full-test' etc
ZOOKEEPER-1363: Categorise unit tests by 'test-commit', 'full-test' etc
Added new capability of defining test categories in test category configuration files. Test category configuration files can contain list of test file names and/or patterns. It is also possible to exclude tests from a test category. Test categories are added to ${test.src.dir}/category directory. Current test categories are defined so that they remained API compatible with the previous build.xml in regards to how ‘ant test’ should be run (options are the same and they behave the same way).
Func and Perf categories are added to keep the previous FuncTest and PerfTest categories which were coded into the test file names.
Slow test category is added and contains tests which require more than 30 seconds to complete on a local developer machine.
Command ‘ant test -Dtest.quick=yes’ runs tests minus tests in the Slow category.
Command ‘ant test -Dtest.category=<category name>’ runs tests from the specified category.
It is possible to combine these two options to run a test category minus the Slow tests.
Test category file name must match the name of the test category (e.g. category/Commit).
List of excluded tests/patterns should be added to a .exclude file having the same name as the category (e.g. category/Commit.exclude).
Added README.txt which describes how to run unit tests in detail.
Benefits:
- test categories do not need to be included in the test file names (a test can be part of more than just one category).
- Slow test category is more flexible to exclude slow tests than the hardcoded exclusion of “**/*HammerTest.java” in build.xml.
- new test categorization can also exclude test file names/patterns from a category.
- test category can be added for a patch.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mfenes/zookeeper ZOOKEEPER-1363
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zookeeper/pull/375.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #375
----
commit cd51fe57cd839f49bf7babdae4ebded3c92f4b26
Author: Mark Fenes <mf...@cloudera.com>
Date: 2017-09-08T15:00:29Z
ZOOKEEPER-1363: Categorise unit tests by 'test-commit', 'full-test' etc
----
---
[GitHub] zookeeper issue #375: ZOOKEEPER-1363: Categorise unit tests by 'test-commit'...
Posted by anmolnar <gi...@git.apache.org>.
Github user anmolnar commented on the issue:
https://github.com/apache/zookeeper/pull/375
Hi,
This patch would be extremely useful for analysing flaky tests.
Any plans for merging it?
---
[GitHub] zookeeper issue #375: ZOOKEEPER-1363: Categorise unit tests by 'test-commit'...
Posted by mfenes <gi...@git.apache.org>.
Github user mfenes commented on the issue:
https://github.com/apache/zookeeper/pull/375
The reason why I've created a test category for "slow" and not for "fast" is because this way one can specify a test category by ant test -Dtest.category=... and can exclude the "slow" tests from any category by using the -Dtest.quick=yes option.
The special logic for "slow" is also needed for backwards compatibility with the current ant test options.
---