You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/03/22 20:20:58 UTC
[GitHub] [ozone] adoroszlai opened a new pull request #3223: HDDS-6488. Intermittent failure in ozone debug read-replicas with one datanode STALE
adoroszlai opened a new pull request #3223:
URL: https://github.com/apache/ozone/pull/3223
## What changes were proposed in this pull request?
Smoketest for `read-replicas` [assumes](https://github.com/apache/ozone/pull/3187#discussion_r825649889) `chunkinfo` and `read-replicas` return datanodes in the same order. This turned out to be false, causing intermittent failures.
This PR eliminates this assumption, improving the test to not rely on replica order.
https://issues.apache.org/jira/browse/HDDS-6488
## How was this patch tested?
Temporarily changed `execute_debug_tests` to corrupt replica=2 instead of replica=0.
Excerpt from `read-replicas` manifest:
```
"replicas": [
{
"hostname": "ozone_datanode_1.ozone_default",
"uuid": "00fab883-5d74-403c-ad54-2d31da07e445"
},
{
"hostname": "ozone_datanode_5.ozone_default",
"uuid": "ff218756-b571-41e9-9d4b-65f79dee30f0"
},
{
"hostname": "ozone_datanode_3.ozone_default",
"uuid": "5025a57f-8178-42d5-b18c-5731b71cc678",
"exception": "java.util.concurrent.ExecutionException: org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: UNAVAILABLE: io exception"
}
]
```
Order of datanodes in `chunkinfo` output:
```
$ cat result/testfile-blocks-10827 | jq -r '.KeyLocations[0][]["Datanode-HostName"]'
ozone_datanode_1.ozone_default
ozone_datanode_5.ozone_default
ozone_datanode_3.ozone_default
```
Test still passed.
CI with only this test:
https://github.com/adoroszlai/hadoop-ozone/runs/5649813090#step:5:127
Regular CI:
https://github.com/adoroszlai/hadoop-ozone/runs/5650040811#step:5:887
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #3223: HDDS-6488. Intermittent failure in ozone debug read-replicas with one datanode STALE
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on pull request #3223:
URL: https://github.com/apache/ozone/pull/3223#issuecomment-1076225776
Thanks @dombizita and @sadanand48 for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] avijayanhwx commented on pull request #3223: HDDS-6488. Intermittent failure in ozone debug read-replicas with one datanode STALE
Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on pull request #3223:
URL: https://github.com/apache/ozone/pull/3223#issuecomment-1075617710
cc @dombizita for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai merged pull request #3223: HDDS-6488. Intermittent failure in ozone debug read-replicas with one datanode STALE
Posted by GitBox <gi...@apache.org>.
adoroszlai merged pull request #3223:
URL: https://github.com/apache/ozone/pull/3223
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org