You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "adoroszlai (via GitHub)" <gi...@apache.org> on 2023/03/06 13:48:10 UTC
[GitHub] [ozone] adoroszlai opened a new pull request, #4353: HDDS-8087. Intermittent crash in TestHddsDatanodeService
adoroszlai opened a new pull request, #4353:
URL: https://github.com/apache/ozone/pull/4353
## What changes were proposed in this pull request?
`TestHddsDatanodeService` intermittently exits with:
```
[Datanode State Machine Task Thread - 0] ERROR datanode.InitDatanodeState (InitDatanodeState.java:call(84)) - Failed to get SCM addresses: ozone.scm.names need to be a set of valid DNS names or IP addresses. Empty address list found.
[Datanode State Machine Daemon Thread] ERROR statemachine.StateContext (StateContext.java:execute(675)) - Critical error occurred in StateMachine, setting shutDownMachine
[Datanode State Machine Daemon Thread] ERROR statemachine.DatanodeStateMachine (DatanodeStateMachine.java:startStateMachineThread(334)) - DatanodeStateMachine Shutdown due to an critical error
[Datanode State Machine Daemon Thread] INFO util.ExitUtil (ExitUtil.java:terminate(210)) - Exiting with status 1: ExitException
```
Since the test is fast, the background thread usually does not get a chance to trigger the exit. It can be reproduced consistently by adding some sleep after:
https://github.com/apache/ozone/blob/888c6dca5b9c6cc3690b063666fa67a39f1b2a69/hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/TestHddsDatanodeService.java#L77
https://issues.apache.org/jira/browse/HDDS-8087
## How was this patch tested?
Passed locally with 1 second sleep, 100 repetitions:
```
Tests run: 100, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 126.957 s - in org.apache.hadoop.ozone.TestHddsDatanodeService
```
Regular CI:
https://github.com/adoroszlai/hadoop-ozone/actions/runs/4343223145
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] fapifta commented on pull request #4353: HDDS-8087. Intermittent crash in TestHddsDatanodeService
Posted by "fapifta (via GitHub)" <gi...@apache.org>.
fapifta commented on PR #4353:
URL: https://github.com/apache/ozone/pull/4353#issuecomment-1461904752
right. I am fine with that ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #4353: HDDS-8087. Intermittent crash in TestHddsDatanodeService
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #4353:
URL: https://github.com/apache/ozone/pull/4353#issuecomment-1461531789
Thanks @fapifta for the review. I think this documents it:
https://github.com/apache/ozone/blob/3de525c60c40441a938f84ffda61ebd2b8bad50b/hadoop-hdds/common/src/main/resources/ozone-default.xml#L1168-L1171
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kerneltime commented on pull request #4353: HDDS-8087. Intermittent crash in TestHddsDatanodeService
Posted by "kerneltime (via GitHub)" <gi...@apache.org>.
kerneltime commented on PR #4353:
URL: https://github.com/apache/ozone/pull/4353#issuecomment-1456538790
@xBis7 can you please take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #4353: HDDS-8087. Intermittent crash in TestHddsDatanodeService
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #4353:
URL: https://github.com/apache/ozone/pull/4353#issuecomment-1462155086
Thanks @fapifta, @xBis7 for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai merged pull request #4353: HDDS-8087. Intermittent crash in TestHddsDatanodeService
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #4353:
URL: https://github.com/apache/ozone/pull/4353
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org