You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/08/31 07:14:36 UTC
[GitHub] [ozone] wzhallright opened a new pull request #2599: HDDS-5701. Add auditLog when DN register fail
wzhallright opened a new pull request #2599:
URL: https://github.com/apache/ozone/pull/2599
## What changes were proposed in this pull request?
I recently upgraded the scm, after upgrade I found the dn can not register to scm.
So, I checked the audit log and want to know why it failed，the audit log like this
`2021-08-31 14:47:07,920 | INFO | SCMAudit | user=ozone | ip=localhost | op=REGISTER {datanodeDetails=161de39c-0ffe-4d39-8f81-f66fbd8148a7{ip: localhost, host: , ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default-rack, certSerialId: null, persistedOpState: null, persistedOpStateExpiryEpochSec: 0}} | ret=SUCCESS |`
the audit log is still register success, but the real thing is that the dn registration failed.
The register dn layout version lower than scm version, it may be that we need to log the failure in the audit.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5701
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] wzhallright commented on pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
wzhallright commented on pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#issuecomment-910122292
> @wzhallright , what is the log after the patch, could you paste an example here?
Thanks for review.
The fail log is
`2021-09-01 17:41:09,660 | ERROR | SCMAudit | user=ozone | ip=localhost | op=REGISTER {datanodeDetails=d68d781c-e720-4107-8b47-eb59a5e6cfd7{ip: localhost, host: , ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default-rack, certSerialId: null, persistedOpState: IN_SERVICE, persistedOpStateExpiryEpochSec: 0}} | ret=FAILURE |`
If I modify it like this PR，when getRegisterResponse(registeredCommand) throw exception, the audit will log twice.
So, I'm wondering if I need to add a boolean parameter for ErrorCode.errorNodeNotPermitted when log audit in finally.
Do you have any suggestions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a change in pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#discussion_r793922628
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMDatanodeProtocolServer.java
##########
@@ -241,6 +241,9 @@ public SCMRegisteredResponseProto register(
pipelineReportsProto));
}
try {
+ Preconditions.checkState(registeredCommand.getError()
+ == SCMRegisteredResponseProto.ErrorCode.success,
+ "DataNode has higher Software Layout Version than SCM.");
Review comment:
@wzhallright Would you mind adding a slightly different check, which does not log an exception unnecessarily?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] wzhallright commented on pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
wzhallright commented on pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#issuecomment-915226909
@ayushtkn Thank you so much for review, I ignored the dn will throw exception anyway when we don't have a success.
I resubmitted a pr, could you help review again, thank you very much.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ChenSammi edited a comment on pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
ChenSammi edited a comment on pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#issuecomment-910093997
@wzhallright , what is the log after the patch, could you paste an example here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] wzhallright removed a comment on pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
wzhallright removed a comment on pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#issuecomment-930722793
> @ChenSammi do you have any further comments?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] wzhallright edited a comment on pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
wzhallright edited a comment on pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#issuecomment-910122292
Thanks for review. @ChenSammi
The fail log is
`2021-09-01 17:41:09,660 | ERROR | SCMAudit | user=ozone | ip=localhost | op=REGISTER {datanodeDetails=d68d781c-e720-4107-8b47-eb59a5e6cfd7{ip: localhost, host: , ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default-rack, certSerialId: null, persistedOpState: IN_SERVICE, persistedOpStateExpiryEpochSec: 0}} | ret=FAILURE |`
If I modify it like this PR，when getRegisterResponse(registeredCommand) throw exception, the audit will log twice.
So, I'm wondering if I need to add a boolean parameter for ErrorCode.errorNodeNotPermitted when log audit in finally.
Do you have any suggestions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#issuecomment-1032716105
/pending
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a change in pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#discussion_r768608184
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMDatanodeProtocolServer.java
##########
@@ -241,6 +241,9 @@ public SCMRegisteredResponseProto register(
pipelineReportsProto));
}
try {
+ Preconditions.checkState(registeredCommand.getError()
+ == SCMRegisteredResponseProto.ErrorCode.success,
+ "DataNode has higher Software Layout Version than SCM.");
Review comment:
While this one-liner is elegant from code perspective, I think we should avoid polluting the audit log with the exception.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] github-actions[bot] commented on pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#issuecomment-1055996764
Thank you very much for the patch. I am closing this PR __temporarily__ as there was no activity recently and it is waiting for response from its author.
It doesn't mean that this PR is not important or ignored: feel free to reopen the PR at any time.
It only means that attention of committers is not required. We prefer to keep the review queue clean. This ensures PRs in need of review are more visible, which results in faster feedback for all PRs.
If you need ANY help to finish this PR, please [contact the community](https://github.com/apache/hadoop-ozone#contact) on the mailing list or the slack channel."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] wzhallright edited a comment on pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
wzhallright edited a comment on pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#issuecomment-910122292
Thanks for review. @ChenSammi
The fail log is
`2021-09-01 17:41:09,660 | ERROR | SCMAudit | user=ozone | ip=localhost | op=REGISTER {datanodeDetails=d68d781c-e720-4107-8b47-eb59a5e6cfd7{ip: localhost, host: , ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default-rack, certSerialId: null, persistedOpState: IN_SERVICE, persistedOpStateExpiryEpochSec: 0}} | ret=FAILURE |`
If I modify it like this PR，when getRegisterResponse(registeredCommand) throw exception, the audit will log twice.
So, I'm wondering if I need to add a boolean parameter for ErrorCode.errorNodeNotPermitted when log audit in finally.
Do you have any suggestions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] wzhallright edited a comment on pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
wzhallright edited a comment on pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#issuecomment-910122292
Thanks for review. @ChenSammi
The fail log is
`2021-09-01 17:41:09,660 | ERROR | SCMAudit | user=ozone | ip=localhost | op=REGISTER {datanodeDetails=d68d781c-e720-4107-8b47-eb59a5e6cfd7{ip: localhost, host: , ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default-rack, certSerialId: null, persistedOpState: IN_SERVICE, persistedOpStateExpiryEpochSec: 0}} | ret=FAILURE |`
If I modify it like this PR，when getRegisterResponse(registeredCommand) throw exception, the audit will log twice.
So, I'm wondering if I need to add a boolean parameter for ErrorCode.errorNodeNotPermitted when log audit in finally.
Do you have any suggestions?
Maybe can modify like this?
`2021-09-01 18:21:09,672 | ERROR | SCMAudit | user=ozone | ip=localhost | op=REGISTER {datanodeDetails=d68d781c-e720-4107-8b47-eb59a5e6cfd7{ip: localhost, host: , ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default-rack, certSerialId: null, persistedOpState: IN_SERVICE, persistedOpStateExpiryEpochSec: 0}} | ret=FAILURE | java.lang.Exception: The register dn layout version lower than scm version
at org.apache.hadoop.hdds.scm.server.SCMDatanodeProtocolServer.register(SCMDatanodeProtocolServer.java:260)
at org.apache.hadoop.ozone.protocolPB.StorageContainerDatanodeProtocolServerSideTranslatorPB.register(StorageContainerDatanodeProtocolServerSideTranslatorPB.java:84)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] wzhallright commented on pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
wzhallright commented on pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#issuecomment-915096231
@ChenSammi @ayushtkn Could you help review this PR? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] wzhallright edited a comment on pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
wzhallright edited a comment on pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#issuecomment-910122292
Thanks for review. @ChenSammi
The fail log is
`2021-09-01 17:41:09,660 | ERROR | SCMAudit | user=ozone | ip=localhost | op=REGISTER {datanodeDetails=d68d781c-e720-4107-8b47-eb59a5e6cfd7{ip: localhost, host: , ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default-rack, certSerialId: null, persistedOpState: IN_SERVICE, persistedOpStateExpiryEpochSec: 0}} | ret=FAILURE |`
If I modify it like this PR，when getRegisterResponse(registeredCommand) throw exception, the audit will log twice.
So, I'm wondering if I need to add a boolean parameter for ErrorCode.errorNodeNotPermitted when log audit in finally.
Do you have any suggestions?
Maybe can modify like this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ChenSammi commented on pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#issuecomment-910093997
@wzhallright , what is the log after the patch, would you paste an example here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ayushtkn commented on pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
ayushtkn commented on pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#issuecomment-919813992
@ChenSammi do you have any further comments?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] wzhallright commented on pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
wzhallright commented on pull request #2599:
URL: https://github.com/apache/ozone/pull/2599#issuecomment-930722793
> @ChenSammi do you have any further comments?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] wzhallright closed pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
wzhallright closed pull request #2599:
URL: https://github.com/apache/ozone/pull/2599
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] github-actions[bot] closed pull request #2599: HDDS-5701. Add auditLog when DN register fail
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #2599:
URL: https://github.com/apache/ozone/pull/2599
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org