You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Enis Soztutar (JIRA)" <ji...@apache.org> on 2012/06/12 04:34:42 UTC
[jira] [Created] (HBASE-6201) HBase integration/system tests
Enis Soztutar created HBASE-6201:
------------------------------------
Summary: HBase integration/system tests
Key: HBASE-6201
URL: https://issues.apache.org/jira/browse/HBASE-6201
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.96.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
Initially, here is what I have in mind:
1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
{code}
mvn verify
{code}
2. Add ability to run all integration/system tests on a given cluster. Smt like:
{code}
mvn verify -Dconf=/etc/hbase/conf/
{code}
should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
3. Port candidate unit tests to the integration tests module. Some of the candidates are:
- TestAcidGuarantees / TestAtomicOperation
- TestRegionBalancing (HBASE-6053)
- TestFullLogReconstruction
- TestMasterFailover
- TestImportExport
- TestMultiVersions / TestKeepDeletes
- TestFromClientSide
- TestShell and src/test/ruby
- TestRollingRestart
- Test**OnCluster
- Balancer tests
These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295455#comment-13295455 ]
Andrew Purtell commented on HBASE-6201:
---------------------------------------
bq. On IT tests and mvn verify, I concluded it a lost cause.
That is disappointing.
And we should also consider the potential benefits of importing and using iTest: http://cloudera.github.com/bigtop/iTest/userguide.html It provides some useful functionality.
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13293981#comment-13293981 ]
Enis Soztutar commented on HBASE-6201:
--------------------------------------
bq. Bigtop provides a framework for integration tests that is, essentially, 'mvn verify'.
Thanks for bringing this up. I know that bigtop provides a test framework for integration tests. From my perspective, I see hbase and bigtop sharing responsibility on the testing side, and we can work to define best practices for this, and would love to hear Bigtop's perspective as well.
I completely agree that HBase code, should not bother with deployments, cluster management services, smoke testing, nor integration with other components (hive, pig, etc). Those kind of functionality can belong in BigTop or similar projects.
However, some core testing functionality, is better managed by the HBase project. Lets consider the TestMasterFailover test. Right now it is a unit test, testing the internal state transitions, when the master fails. However, we can extend this test to run from the client side, and see whether the transition is transparent when we kill the active master on an actual cluster. That kind of testing, should be managed by HBase itself, because, although they would run from the client side, these kind of tests are hbase-specific, and better managed by Hbase devs. Also, I do not expect BigTop to host a large number of test cases for all of the stack (right now 8 projects).
Having said that, in this issue, we can come up with a way to interface with BigTop (and other projects, custom jenkins jobs, etc) so that, these tests can use the underlying deployment, server management, etc services, and BigTop, and others can just execute the HBase internal integration tests on the cluster. A simple way for this is that HBase to offer {{mvn verify}} to be consumed by BigTop, and those tests will use HBase's own scripts (and SSH, etc) for cluster/server management. Since BigTop configures the cluster to be usable by those, it should be ok.
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6201) HBase integration/system tests
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13293954#comment-13293954 ]
Andrew Purtell edited comment on HBASE-6201 at 6/12/12 10:04 PM:
-----------------------------------------------------------------
Bigtop provides a framework for integration tests that is, essentially, 'mvn verify'. If we are discussing an integration test framework that must interact with a cluster, perhaps on demand, then there is potentially one already under development for that.
was (Author: apurtell):
Bigtop provides a framework for integration tests that is, essentially, 'mvn verify'. If we are discussing an integration test framework that must interact with a cluster, perhaps on demand, then there is potentially a framework already under development for that.
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396326#comment-13396326 ]
Enis Soztutar commented on HBASE-6201:
--------------------------------------
I think your categorization, and my comments above are telling the same thing, no confusion there. This umbrella issue is all about maintaining #2 kind of tests inside HBase. Now, the problem is how to best interface between HBase and Bigtop.
My proposal is that depends on itest-common, and uses it to interact with the servers. My understanding is that, even if you are not deploying the cluster with bigtop, as long as /etc/init.d/ scripts are there, you should be fine. At this point, we only need starting / stopping deamons kind of functionality, assuming the cluster is already deployed.
On the other side, if we provide a "mvn verify" in hbase-it module to run the tests on the actual cluster, I assume BigTop can leverage this to carry out the tests.
For refactoring, once the module, and other bits are ready, we can move select tests from Bigtop to HBase. I'll open a subtask for that.
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456349#comment-13456349 ]
Hudson commented on HBASE-6201:
-------------------------------
Integrated in HBase-TRUNK #3336 (See [https://builds.apache.org/job/HBase-TRUNK/3336/])
HBASE-6201 HBaseCluster interface for interacting with the cluster from system tests (Revision 1385024)
Result = FAILURE
stack :
Files :
* /hbase/trunk/bin/hbase
* /hbase/trunk/hbase-it/pom.xml
* /hbase/trunk/hbase-it/src
* /hbase/trunk/hbase-it/src/test
* /hbase/trunk/hbase-it/src/test/java
* /hbase/trunk/hbase-it/src/test/java/org
* /hbase/trunk/hbase-it/src/test/java/org/apache
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/ClusterManager.java
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/DistributedHBaseCluster.java
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/HBaseClusterManager.java
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestDataIngestWithChaosMonkey.java
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestingUtility.java
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestsDriver.java
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/util
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/util/ChaosMonkey.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/AbstractHBaseTool.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/Threads.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseCluster.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/IntegrationTests.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/LargeTests.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/MediumTests.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/SmallTests.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestCheckTestClasses.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/RestartMetaTest.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/StoppableImplementation.java
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506421#comment-13506421 ]
Hudson commented on HBASE-6201:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #279 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/279/])
HBASE-6201 Document how to run integration tests (Revision 1415055)
Result = FAILURE
stack :
Files :
* /hbase/trunk/src/docbkx/developer.xml
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396383#comment-13396383 ]
Roman Shaposhnik commented on HBASE-6201:
-----------------------------------------
@Enis,
in that case I'd strongly encourage the community to take a serious look at BIGTOP-635. This is exactly what we're hoping to achieve with it.
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396329#comment-13396329 ]
Roman Shaposhnik commented on HBASE-6201:
-----------------------------------------
bq. I think your categorization, and my comments above are telling the same thing, no confusion there.
Great to hear -- in that case the implementation is the next level to go to.
bq. At this point, we only need starting / stopping deamons kind of functionality, assuming the cluster is already deployed.
bq. On the other side, if we provide a "mvn verify" in hbase-it module to run the tests on the actual cluster, I assume BigTop can leverage this to carry out the tests.
Basically, all that Bigtop needs is a maven artifact that we can plug in our infrastructure. We won't be using the 'mvn verify' as it is implemented in HBase's pom.xml but rather hooking the very same artifact to our Maven execution framework.
Are you saying that you would like the tests themeselves to get involved in the lifecycle of each service? Like bringing them up and down, etc?
This an area I'm really interested in providing a framework for in Bigtop. I'm about to open up a JIRA for that with the hope that we can cook something usable up rather quickly. The key point is that I don't want tests to have any logic that concerns itself with ssh/etc. It needs to be a sort of an agent-based framework that allows tests to query the topology of the cluster and also perform actions on the nodes of that topology in a most generic sense.
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456369#comment-13456369 ]
Hudson commented on HBASE-6201:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #175 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/175/])
HBASE-6201 HBaseCluster interface for interacting with the cluster from system tests (Revision 1385024)
Result = FAILURE
stack :
Files :
* /hbase/trunk/bin/hbase
* /hbase/trunk/hbase-it/pom.xml
* /hbase/trunk/hbase-it/src
* /hbase/trunk/hbase-it/src/test
* /hbase/trunk/hbase-it/src/test/java
* /hbase/trunk/hbase-it/src/test/java/org
* /hbase/trunk/hbase-it/src/test/java/org/apache
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/ClusterManager.java
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/DistributedHBaseCluster.java
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/HBaseClusterManager.java
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestDataIngestWithChaosMonkey.java
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestingUtility.java
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestsDriver.java
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/util
* /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/util/ChaosMonkey.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/AbstractHBaseTool.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/Threads.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseCluster.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/IntegrationTests.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/LargeTests.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/MediumTests.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/SmallTests.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestCheckTestClasses.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/RestartMetaTest.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/StoppableImplementation.java
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396347#comment-13396347 ]
Enis Soztutar commented on HBASE-6201:
--------------------------------------
bq. Are you saying that you would like the tests themeselves to get involved in the lifecycle of each service? Like bringing them up and down, etc?
Yes. At the current state, most of our unit tests, which are candidates to be upgraded to be system tests does start a mini-cluster of n-nodes, load some data, kill a few nodes, verify, etc. We are converting/reimplementing them to do the same things on the actual cluster. A particular test case, for example, starts 4 region servers, put some data, kills 1 RS, checks whether the regions are balanced, kills one more, checks agains, etc.
Some basic functionality we can use from itest are:
- Starting / stopping / sending a signal to daemons (start a region server on host1, kill master on host2, etc). For both HBase and Hadoop processes.
- Basic cluster/node discovery (give me the nodes running hmaster)
- Run this command on host3 (SSH)
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13293995#comment-13293995 ]
Andrew Purtell commented on HBASE-6201:
---------------------------------------
Makes sense, thanks Enis.
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13293954#comment-13293954 ]
Andrew Purtell commented on HBASE-6201:
---------------------------------------
Bigtop provides a framework for integration tests that is, essentially, 'mvn verify'. If we are discussing an integration test framework that must interact with a cluster, perhaps on demand, then there is potentially a framework already under development for that.
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396346#comment-13396346 ]
Roman Shaposhnik commented on HBASE-6201:
-----------------------------------------
Feel free to chime in: BIGTOP-635
This is meant to cover HBase and HDFS use cases to begin with, but also should be generic enough to tackle just about anything. Given the level of ambition -- the more code we can reuse the better. Hence anybody familiar with real Cluster Management frameworks are welcome to chime in and give us feedback on how realistic such a reuse really is.
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295354#comment-13295354 ]
stack commented on HBASE-6201:
------------------------------
Thanks for doing this Enis. +1 on making subjiras off this one.
On IT tests and mvn verify, I concluded it a lost cause. Maybe you can figure how to make it change its behavior when you pass a -Dconf=xxxx and have it instead run on a real cluster (I found the mvn verify targets insufficient IIRC).
The above is a comment on the mvn verify mechanism only, not on the intent of this issue. I'm rah-rah on your intent. Just suggesting we may need to do it atop a different vehicle (BT would be coolio. On hearing BT perspective on this, we may have blown that for a while. We've gotten Roman excited on multiple occasions but have failed to come through. I'd say lets deliver something first, then Roman might think us serious this time).
bq. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
I agree. Need to make it so test is same whether on mini or real cluster.
+1 on #3 and #4 points above.
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396272#comment-13396272 ]
Roman Shaposhnik commented on HBASE-6201:
-----------------------------------------
Sorry for chiming in late, but here's how I see it after quite a bit of internal discussions with some of the HBase and HDFS devs. First of all, lets not got caught up in terminology of what is a unit test, what is a functional test, etc. If we assume this stance than all the tests, essentially, fall under the 3 main categories:
# tests that muck about with the internals of the particular single project (HDFS, HBase, etc) using things like private APIs (or sometimes even things like reflections, etc to really get into the guts of the system)
# tests that concern themselves with a single project (HDFS, HBase, etc) but use only public APIs AND don't use
# tests that concern themselves with multiple projects at the same time (imagine a test that submits an Oozie workflow that has some Pig and Hive actions actively manipulating data in HBase) but only using public APIs
It is pretty clear that #3 definitely belongs to Bigtop while #1 definitely belongs to individual projects. For quite some time I was thinking that #2 belongs to Bigtop testbase as well, but I've changed my mind. I now believe that such tests should reside in individual projects and:
# be clearly marked as such not to be confused with class #1 (test suites, test lists, naming convention work, etc)
# be written/refactored in such a way that doesn't tie them to a particular deployment strategy. IOW they should assume the subsystem to be deployed.
# be hooked up to the project build's system in such a way that takes care of deploying the least amount of a system to make them run (e.g. MiniDFS, MiniMR, etc.)
Thus if HBase can follow these rules and have a subset of tests that can be executed in different envs. both HBase core devs and bigger Bigtop dev community win, since we can leverage each other's work.
Makes sense? If it does I can help with re-factoring.
> HBase integration/system tests
> ------------------------------
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the conclusion is that we need to unify how we do "release candidate" testing (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open subtickets for execution so that we can carry out most of the tests in HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454 (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some of the tests (TestAcidGuarantees) from command line). Most of the system tests will be client side, and interface with the cluster through public APIs. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility, so that tests can interface with the mini cluster or the actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in semantics. However, given an actual cluster, they should use that, instead of spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira