You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/02/25 06:59:55 UTC
[GitHub] [ozone] cchenax opened a new pull request #1962: Ozone datanode initraftlog fail due to bad disk so can not communicat…
cchenax opened a new pull request #1962:
URL: https://github.com/apache/ozone/pull/1962
…e to SCM
## What changes were proposed in this pull request?
When datanode initraftlog fail due to bad disk,the datanode can not connect to SCM,and the datanode will always throw the exception,so we should let datanode throw the runtime exception once,and SCM will remove the raftgroup
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-4868?orderby=affectedVersion+ASC%2C+priority+DESC%2C+updated+DESC
## How was this patch tested?
I will mock the datanode initraftlog fail and the process of the datanode will not always throw the exception
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ChenSammi closed pull request #1962: HDDS-4868. Ozone datanode initraftlog fail due to bad disk so can not communicate to SCM
Posted by GitBox <gi...@apache.org>.
ChenSammi closed pull request #1962:
URL: https://github.com/apache/ozone/pull/1962
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] cchenax commented on pull request #1962: HDDS-4868. Ozone datanode initraftlog fail due to bad disk so can not communicate to SCM
Posted by GitBox <gi...@apache.org>.
cchenax commented on pull request #1962:
URL: https://github.com/apache/ozone/pull/1962#issuecomment-799996458
> Thanks @cchenax for proposing a fix. However, simply catching the exception does not work, because the datanode would not be functional without its container services. (You can simulate the error by throwing an `IOException` from `XceiverServerRatis.start()`).
when the datanode was failed,the scm would through heartbeat to detect the datanode which failed.So the scm would remove the container.This datanode really not work.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #1962: HDDS-4868. Ozone datanode initraftlog fail due to bad disk so can not communicate to SCM
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on pull request #1962:
URL: https://github.com/apache/ozone/pull/1962#issuecomment-800108569
> when the datanode was failed,the scm would through heartbeat to detect the datanode which failed.So the scm would remove the container.This datanode really not work.
SCM is expected to mark the datanode first `STALE`, then `DEAD`, after not receiving heartbeat for some time. I don't see why keeping the datanode alive as a zombie would be necessary.
Also, I think 8a0b530 is unrelated and intended for another issue (HDDS-4983).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] cchenax commented on pull request #1962: HDDS-4868. Ozone datanode initraftlog fail due to bad disk so can not communicate to SCM
Posted by GitBox <gi...@apache.org>.
cchenax commented on pull request #1962:
URL: https://github.com/apache/ozone/pull/1962#issuecomment-803524676
> > when the datanode was failed,the scm would through heartbeat to detect the datanode which failed.So the scm would remove the container.This datanode really not work.
>
> SCM is expected to mark the datanode first `STALE`, then `DEAD`, after not receiving heartbeat for some time. I don't see why keeping the datanode alive as a zombie would be necessary.
>
> Also, I think [8a0b530](https://github.com/apache/ozone/commit/8a0b5308032ae91d37ce5620c6cee501992b3515) is unrelated and intended for another issue ([HDDS-4983](https://issues.apache.org/jira/browse/HDDS-4983)).
Ok，I know what you mean，the datanode initraftgroup would skip the init failed raft group,the other normal pipeline raft group would init success，and the datanode would not fail.I would change the ratis code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ChenSammi commented on pull request #1962: HDDS-4868. Ozone datanode initraftlog fail due to bad disk so can not communicate to SCM
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on pull request #1962:
URL: https://github.com/apache/ozone/pull/1962#issuecomment-811732377
Close this PR. It issue will be handled in ratis RATIS-1350.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] cchenax closed pull request #1962: HDDS-4868. Ozone datanode initraftlog fail due to bad disk so can not communicate to SCM
Posted by GitBox <gi...@apache.org>.
cchenax closed pull request #1962:
URL: https://github.com/apache/ozone/pull/1962
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] cchenax commented on pull request #1962: HDDS-4868. Ozone datanode initraftlog fail due to bad disk so can not communicate to SCM
Posted by GitBox <gi...@apache.org>.
cchenax commented on pull request #1962:
URL: https://github.com/apache/ozone/pull/1962#issuecomment-800109971
> > when the datanode was failed,the scm would through heartbeat to detect the datanode which failed.So the scm would remove the container.This datanode really not work.
>
> SCM is expected to mark the datanode first `STALE`, then `DEAD`, after not receiving heartbeat for some time. I don't see why keeping the datanode alive as a zombie would be necessary.
>
> Also, I think [8a0b530](https://github.com/apache/ozone/commit/8a0b5308032ae91d37ce5620c6cee501992b3515) is unrelated and intended for another issue ([HDDS-4983](https://issues.apache.org/jira/browse/HDDS-4983)).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] cchenax edited a comment on pull request #1962: HDDS-4868. Ozone datanode initraftlog fail due to bad disk so can not communicate to SCM
Posted by GitBox <gi...@apache.org>.
cchenax edited a comment on pull request #1962:
URL: https://github.com/apache/ozone/pull/1962#issuecomment-799996458
> Thanks @cchenax for proposing a fix. However, simply catching the exception does not work, because the datanode would not be functional without its container services. (You can simulate the error by throwing an `IOException` from `XceiverServerRatis.start()`).
when the datanode was failed,the scm would through heartbeat to detect the datanode which failed.So the scm would remove the container.This datanode really not work.But this fix would catch the exception.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org