You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/09/01 01:58:55 UTC
[GitHub] [hadoop-ozone] runzhiwang opened a new pull request #1370: HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException
runzhiwang opened a new pull request #1370:
URL: https://github.com/apache/hadoop-ozone/pull/1370
## What changes were proposed in this pull request?
What's the problem ?
What's the reason of the failed ut ?
When cluster create 9 nodes, the following code try to shutdown the follower of the pipeline. Actually, the pipeline only exist in 3 nodes, but the following code check 9 nodes, so the pipeline can not found in the other 6 nodes.
```
for (HddsDatanodeService dn : cluster.getHddsDatanodes()) {
// shutdown the ratis follower
if (ContainerTestHelper.isRatisFollower(dn, pipeline)) {
cluster.shutdownHddsDatanode(dn.getDatanodeDetails());
break;
}
}
```
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-4176
## How was this patch tested?
Existed ut.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai merged pull request #1370: HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException
Posted by GitBox <gi...@apache.org>.
adoroszlai merged pull request #1370:
URL: https://github.com/apache/hadoop-ozone/pull/1370
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1370: HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException
Posted by GitBox <gi...@apache.org>.
amaliujia commented on a change in pull request #1370:
URL: https://github.com/apache/hadoop-ozone/pull/1370#discussion_r480881190
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestWatchForCommit.java
##########
@@ -297,9 +298,11 @@ public void test2WayCommitForTimeoutException() throws Exception {
xceiverClient.getPipeline()));
reply.getResponse().get();
Assert.assertEquals(3, ratisClient.getCommitInfoMap().size());
+ List<DatanodeDetails> datanodeDetails = pipeline.getNodes();
for (HddsDatanodeService dn : cluster.getHddsDatanodes()) {
// shutdown the ratis follower
- if (ContainerTestHelper.isRatisFollower(dn, pipeline)) {
+ if (datanodeDetails.contains(dn.getDatanodeDetails())
+ && ContainerTestHelper.isRatisFollower(dn, pipeline)) {
Review comment:
Oops. You are right. I was looking at a wrong file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1370: HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException
Posted by GitBox <gi...@apache.org>.
runzhiwang commented on a change in pull request #1370:
URL: https://github.com/apache/hadoop-ozone/pull/1370#discussion_r481493152
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestWatchForCommit.java
##########
@@ -297,9 +298,11 @@ public void test2WayCommitForTimeoutException() throws Exception {
xceiverClient.getPipeline()));
reply.getResponse().get();
Assert.assertEquals(3, ratisClient.getCommitInfoMap().size());
+ List<DatanodeDetails> datanodeDetails = pipeline.getNodes();
Review comment:
@adoroszlai Thanks for review. I have updated the patch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1370: HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException
Posted by GitBox <gi...@apache.org>.
amaliujia commented on a change in pull request #1370:
URL: https://github.com/apache/hadoop-ozone/pull/1370#discussion_r480794854
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestWatchForCommit.java
##########
@@ -297,9 +298,11 @@ public void test2WayCommitForTimeoutException() throws Exception {
xceiverClient.getPipeline()));
reply.getResponse().get();
Assert.assertEquals(3, ratisClient.getCommitInfoMap().size());
+ List<DatanodeDetails> datanodeDetails = pipeline.getNodes();
for (HddsDatanodeService dn : cluster.getHddsDatanodes()) {
// shutdown the ratis follower
- if (ContainerTestHelper.isRatisFollower(dn, pipeline)) {
+ if (datanodeDetails.contains(dn.getDatanodeDetails())
+ && ContainerTestHelper.isRatisFollower(dn, pipeline)) {
Review comment:
This logic looks right to me.
Can you bump up the number of DN to make tests run against higher number DN: https://github.com/apache/hadoop-ozone/blob/34ee8311b0d0a37878fe1fd2e5d8c1b91aa8cc8f/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/Test2WayCommitInRatis.java#L109
Which seems will cover your change?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] runzhiwang commented on pull request #1370: HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException
Posted by GitBox <gi...@apache.org>.
runzhiwang commented on pull request #1370:
URL: https://github.com/apache/hadoop-ozone/pull/1370#issuecomment-684175039
@elek Could you help review this patch ? Thank you very much.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1370: HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on pull request #1370:
URL: https://github.com/apache/hadoop-ozone/pull/1370#issuecomment-685243200
Thanks @runzhiwang for the fix and @amaliujia for the review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1370: HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #1370:
URL: https://github.com/apache/hadoop-ozone/pull/1370#discussion_r481426506
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestWatchForCommit.java
##########
@@ -297,9 +298,11 @@ public void test2WayCommitForTimeoutException() throws Exception {
xceiverClient.getPipeline()));
reply.getResponse().get();
Assert.assertEquals(3, ratisClient.getCommitInfoMap().size());
+ List<DatanodeDetails> datanodeDetails = pipeline.getNodes();
Review comment:
Nit: `nodesInPipeline` may be a better name.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1370: HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException
Posted by GitBox <gi...@apache.org>.
runzhiwang commented on a change in pull request #1370:
URL: https://github.com/apache/hadoop-ozone/pull/1370#discussion_r480831186
##########
File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestWatchForCommit.java
##########
@@ -297,9 +298,11 @@ public void test2WayCommitForTimeoutException() throws Exception {
xceiverClient.getPipeline()));
reply.getResponse().get();
Assert.assertEquals(3, ratisClient.getCommitInfoMap().size());
+ List<DatanodeDetails> datanodeDetails = pipeline.getNodes();
for (HddsDatanodeService dn : cluster.getHddsDatanodes()) {
// shutdown the ratis follower
- if (ContainerTestHelper.isRatisFollower(dn, pipeline)) {
+ if (datanodeDetails.contains(dn.getDatanodeDetails())
+ && ContainerTestHelper.isRatisFollower(dn, pipeline)) {
Review comment:
@amaliujia Thanks for your review. TestWatchForCommit already startCluster with 9 nodes, maybe it is enough.
https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestWatchForCommit.java#L124
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org