You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org> on 2012/10/23 04:30:12 UTC
[jira] [Created] (HADOOP-8964) test-patch.sh doesn't run all tests
Vinod Kumar Vavilapalli created HADOOP-8964:
-----------------------------------------------
Summary: test-patch.sh doesn't run all tests
Key: HADOOP-8964
URL: https://issues.apache.org/jira/browse/HADOOP-8964
Project: Hadoop Common
Issue Type: Bug
Components: build
Reporter: Vinod Kumar Vavilapalli
test-patch.sh tries to be smart but with bad results. When it runs tests, say on each commit, it only runs tests only from modules which have corresponding changes. This is a cause of concern for downstream modules. We ran into multiple issues in the past because of this, the most recent of it being YARN-179.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8964) test-patch.sh doesn't run all
tests
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482757#comment-13482757 ]
Vinod Kumar Vavilapalli commented on HADOOP-8964:
-------------------------------------------------
I am definitely for not running mapreduce tests when a common patch gets checked in. But if YARN common lib changes, I definitely want to run RM and NM tests. I understand that not running everything speeds things up, but we have correctness issues when we don't run downstream tests.
Is it okay to just run all tests in case of YARN for the shorter term, they don't take more than 10 mins for now? Separating integration tests into their own suite is a bigger effort.
> test-patch.sh doesn't run all tests
> -----------------------------------
>
> Key: HADOOP-8964
> URL: https://issues.apache.org/jira/browse/HADOOP-8964
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
>
> test-patch.sh tries to be smart but with bad results. When it runs tests, say on each commit, it only runs tests only from modules which have corresponding changes. This is a cause of concern for downstream modules. We ran into multiple issues in the past because of this, the most recent of it being YARN-179.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8964) test-patch.sh doesn't run all
tests
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482392#comment-13482392 ]
Robert Joseph Evans commented on HADOOP-8964:
---------------------------------------------
We had a lot of speedup in the tests by doing this. The problem is that running all tests takes hours. This means that JIRAs start to back up and fill up the queues. I am fine with changing it so it runs all of the tests for the top level project, like if it is in YARN it runs everything in YARN. But if we want to enable all of the tests to run we first need to classify the tests between true unit tests that should be run as part of check-in/pre-commit and integration tests that take longer but should not be run as part of the pre-commit. i.e. anything that uses a minicluster.
> test-patch.sh doesn't run all tests
> -----------------------------------
>
> Key: HADOOP-8964
> URL: https://issues.apache.org/jira/browse/HADOOP-8964
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
>
> test-patch.sh tries to be smart but with bad results. When it runs tests, say on each commit, it only runs tests only from modules which have corresponding changes. This is a cause of concern for downstream modules. We ran into multiple issues in the past because of this, the most recent of it being YARN-179.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-8964) test-patch.sh doesn't run all tests
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli reassigned HADOOP-8964:
-----------------------------------------------
Assignee: Vinod Kumar Vavilapalli
This may have been done to avoid running all tests which can take a long time, but under each project root (common/hdfs/mapreduce/yarn), we should run all tests from all sub-modules in the whole tree?
If nothing, I'd like to modify to run all tests under all modules of yarn.
> test-patch.sh doesn't run all tests
> -----------------------------------
>
> Key: HADOOP-8964
> URL: https://issues.apache.org/jira/browse/HADOOP-8964
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
>
> test-patch.sh tries to be smart but with bad results. When it runs tests, say on each commit, it only runs tests only from modules which have corresponding changes. This is a cause of concern for downstream modules. We ran into multiple issues in the past because of this, the most recent of it being YARN-179.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8964) test-patch.sh doesn't run all
tests
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484413#comment-13484413 ]
Tom White commented on HADOOP-8964:
-----------------------------------
Vinod - that seems reasonable to me.
> test-patch.sh doesn't run all tests
> -----------------------------------
>
> Key: HADOOP-8964
> URL: https://issues.apache.org/jira/browse/HADOOP-8964
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
>
> test-patch.sh tries to be smart but with bad results. When it runs tests, say on each commit, it only runs tests only from modules which have corresponding changes. This is a cause of concern for downstream modules. We ran into multiple issues in the past because of this, the most recent of it being YARN-179.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8964) test-patch.sh doesn't run all
tests
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482412#comment-13482412 ]
Tom White commented on HADOOP-8964:
-----------------------------------
There was some discussion on this on http://mail-archives.aurora.apache.org/mod_mbox/hadoop-common-dev/201204.mbox/%3CCAF-WD4TvKwYpuuQ9ibxv4UZ8B2BEhXnpfKb5mq3D-pwVKSHOzA@mail.gmail.com%3E and HADOOP-8308.
I like the direction that Bobby is suggesting. We could use Maven's distinction between unit and integration tests ({{mvn test}} vs {{mvn integration-test}}).
> test-patch.sh doesn't run all tests
> -----------------------------------
>
> Key: HADOOP-8964
> URL: https://issues.apache.org/jira/browse/HADOOP-8964
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
>
> test-patch.sh tries to be smart but with bad results. When it runs tests, say on each commit, it only runs tests only from modules which have corresponding changes. This is a cause of concern for downstream modules. We ran into multiple issues in the past because of this, the most recent of it being YARN-179.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira