You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "Stephen Chu (JIRA)" <ji...@apache.org> on 2012/06/06 23:07:23 UTC
[jira] [Created] (BIGTOP-614) Add HDFS HA tests
Stephen Chu created BIGTOP-614:
----------------------------------
Summary: Add HDFS HA tests
Key: BIGTOP-614
URL: https://issues.apache.org/jira/browse/BIGTOP-614
Project: Bigtop
Issue Type: Test
Components: Tests
Reporter: Stephen Chu
Assignee: Stephen Chu
In this patch, I am adding 2 tests and a test util class.
1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-614) Add HDFS HA tests
Posted by "Sujay Rau (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13296102#comment-13296102 ]
Sujay Rau commented on BIGTOP-614:
----------------------------------
I'd definitely be interested in working on this problem.. but need help figuring out a starting point.
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Attachments: HATests.patch, bigtop-614.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-614) Add HDFS HA tests
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295844#comment-13295844 ]
Roman Shaposhnik commented on BIGTOP-614:
-----------------------------------------
Sujay, I agree with your comments above. Also, please include the following clean up into your new patch:
# there doesn't seem to be any need for a shell class in TestHAFailover and TestKillNNFailover
# the test shouldn't assume that it is executed under any specific user account (e.g. assumptions that it is root). If something requires root privileges it needs to be executed via a root shell. Something like shRoot = new Shell("/bin/bash -s", "root")
# please get rid of System.out.println calls around shell calls. all shell tracing is done via log4j
# it would be nice to get rid of the assertNotNull("JAVA_HOME has to be set to run this test", JAVA_HOME) and assertNotNull("HADOOP_HOME has to be set to run this test", HADOOP_HOME) in the HATestUtil
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Attachments: HATests.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-614) Add HDFS HA tests
Posted by "Sujay Rau (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13296035#comment-13296035 ]
Sujay Rau commented on BIGTOP-614:
----------------------------------
Roman, on my dfsadmin test you said that the test can't assume it is being run from the namenode. This is hard to do for HA tests since failovers need to be done from the namenode (I think...?). Do you have any recommendations for dealing with this, as I am not sure how to make HA tests more general to any node in the cluster.
Also, I'll fix the Shell user thing for the tests I have submitted so far.. makes much more sense now. Thanks
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Attachments: HATests.patch, bigtop-614.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (BIGTOP-614) Add HDFS HA tests
Posted by "Sujay Rau (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294731#comment-13294731 ]
Sujay Rau edited comment on BIGTOP-614 at 6/13/12 11:51 PM:
------------------------------------------------------------
Namenodes in HA might always be specified from hdfs-site.xml, so changing the following to your tests in the setUp() might get rid of the need to specify -Dservice.id.1=nn1 and -Dservice.id.2=nn2 from the command line.
import org.apache.hadoop.conf.Configuration;
static void setUp() {
Configuration conf = new Configuration();
conf.addResource('hdfs-site.xml');
String namenodes = conf.get("dfs.ha.namenodes.ha-nn-uri");
service_id_1 = namenodes.split(",")[0];
service_id_2 = namenodes.split(",")[1];
assertTrue("Unspecified service id 1", service_id_1 != null);
assertTrue("Unspecified service id 2", service_id_2 != null);
num_failover = Integer.parseInt(System.getProperty("num.failover", "10"));
}
^ sorry about formatting
was (Author: sujay.rau):
Namenodes in HA might always be specified from hdfs-site.xml, so changing the following to your tests in the setUp() might get rid of the need to specify -Dservice.id.1=nn1 and -Dservice.id.2=nn2 from the command line.
import org.apache.hadoop.conf.Configuration;
static void setUp() {
Configuration conf = new Configuration();
conf.addResource('hdfs-site.xml');
String namenodes = conf.get("dfs.ha.namenodes.ha-nn-uri");
service_id_1 = namenodes.split(",")[0];
service_id_2 = namenodes.split(",")[1];
assertTrue("Unspecified service id 1", service_id_1 != null);
assertTrue("Unspecified service id 2", service_id_2 != null);
num_failover = Integer.parseInt(System.getProperty("num.failover", "10"));
}
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Attachments: HATests.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-614) Add HDFS HA tests
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Shaposhnik updated BIGTOP-614:
------------------------------------
Fix Version/s: 0.4.0
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Fix For: 0.4.0
>
> Attachments: HATests.patch, bigtop-614.patch, bigtop-614.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-614) Add HDFS HA tests
Posted by "Stephen Chu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stephen Chu updated BIGTOP-614:
-------------------------------
Attachment: HATests.patch
Submitted a patch to trunk.
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Attachments: HATests.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (BIGTOP-614) Add HDFS HA tests
Posted by "Stephen Chu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13296044#comment-13296044 ]
Stephen Chu edited comment on BIGTOP-614 at 6/16/12 12:53 AM:
--------------------------------------------------------------
Yes, definitely worth working on building this infrastructure now. It will be very helpful to expand other tests, too (e.g. fault injection HBase tests).
I am interested in being involved as well.
Also, is there a way for me to re-assign this issue to Sujay? I don't see him as a user I can assign issues to.
was (Author: schu):
Yes, definitely worth working on building this infrastructure now. It will be very helpful to expand other tests, too (e.g. fault injection HBase tests).
I am interested in being involved as well.
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Attachments: HATests.patch, bigtop-614.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-614) Add HDFS HA tests
Posted by "Sujay Rau (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sujay Rau updated BIGTOP-614:
-----------------------------
Attachment: bigtop-614.patch
last patch had an error
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Attachments: HATests.patch, bigtop-614.patch, bigtop-614.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-614) Add HDFS HA tests
Posted by "Stephen Chu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13296044#comment-13296044 ]
Stephen Chu commented on BIGTOP-614:
------------------------------------
Yes, definitely worth working on building this infrastructure now. It will be very helpful to expand other tests, too (e.g. fault injection HBase tests).
I am interested in being involved as well.
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Attachments: HATests.patch, bigtop-614.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-614) Add HDFS HA tests
Posted by "Sujay Rau (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sujay Rau updated BIGTOP-614:
-----------------------------
Attachment: (was: bigtop-614.patch)
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Attachments: HATests.patch, bigtop-614.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-614) Add HDFS HA tests
Posted by "Sujay Rau (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sujay Rau updated BIGTOP-614:
-----------------------------
Attachment: bigtop-614.patch
Updated the util file from previous patch I submitted
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Attachments: HATests.patch, bigtop-614.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-614) Add HDFS HA tests
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13296038#comment-13296038 ]
Roman Shaposhnik commented on BIGTOP-614:
-----------------------------------------
Sujay, this is a pretty perplexing problem that we've been trying not to notice so far -- we have to have an infrastructure for doing what you want be we don't have any so far. Let me file a separate JIRA over the weekend that will aim at (finally!) addressing this issue.
The trouble is, though, that I won't be able to work on it for quite some time. Do you feel like you'd be interested in tackling this?
Meanwhile, we can still check the test code in, but it'll have to be disable during actual test runs I suppose. Hm. Let me think about this over the weekend.
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Attachments: HATests.patch, bigtop-614.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-614) Add HDFS HA tests
Posted by "Sujay Rau (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sujay Rau updated BIGTOP-614:
-----------------------------
Attachment: bigtop-614.patch
Updated patch for three desribed files.
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Attachments: HATests.patch, bigtop-614.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-614) Add HDFS HA tests
Posted by "Sujay Rau (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294731#comment-13294731 ]
Sujay Rau commented on BIGTOP-614:
----------------------------------
Namenodes in HA might always be specified from hdfs-site.xml, so changing the following to your tests in the setUp() might get rid of the need to specify -Dservice.id.1=nn1 and -Dservice.id.2=nn2 from the command line.
import org.apache.hadoop.conf.Configuration;
static void setUp() {
Configuration conf = new Configuration();
conf.addResource('hdfs-site.xml');
String namenodes = conf.get("dfs.ha.namenodes.ha-nn-uri");
service_id_1 = namenodes.split(",")[0];
service_id_2 = namenodes.split(",")[1];
assertTrue("Unspecified service id 1", service_id_1 != null);
assertTrue("Unspecified service id 2", service_id_2 != null);
num_failover = Integer.parseInt(System.getProperty("num.failover", "10"));
}
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Attachments: HATests.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-614) Add HDFS HA tests
Posted by "Sujay Rau (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295809#comment-13295809 ]
Sujay Rau commented on BIGTOP-614:
----------------------------------
Currently modifying these to get rid of necessary command line options and making it so the test can be run from any namenode (not just the active one... right now the test doesn't work if it is run from the standby namenode)
> Add HDFS HA tests
> -----------------
>
> Key: BIGTOP-614
> URL: https://issues.apache.org/jira/browse/BIGTOP-614
> Project: Bigtop
> Issue Type: Test
> Components: Tests
> Reporter: Stephen Chu
> Assignee: Stephen Chu
> Attachments: HATests.patch
>
>
> In this patch, I am adding 2 tests and a test util class.
> 1) TestHAFailover: Fails over between the NameNodes a configurable number of times. Checks the service state of each NN after each failover to ensure that the service state is correct.
> 2) TestKilLNNFailover: Kills the Active NN. Waits and fails over to the standby NN. Waits and checks that the former standby is now active. Restarts the killed NN and checks that it is now standby. Goes through this process a configurable number of times.
> 3) HATestUtil: Helper methods to fail over and kill the NameNodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira