You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/02/26 22:05:11 UTC
[GitHub] [hadoop-ozone] bharatviswa504 opened a new pull request #609:
HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
bharatviswa504 opened a new pull request #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609
## What changes were proposed in this pull request?
Donot have precondition check of updateID in non-HA OM cluster.
// Because in non-HA, we have multiple rpc handler threads and
// transactionID is generated in OzoneManagerServerSideTranslatorPB.
// Lets take T1 -> Set Bucket Property
// T2 -> Set Bucket Acl
// Now T2 got lock first, so updateID will be set to 2. Now when T1 gets
// executed we will hit the precondition exception. So for OM non-HA with
// out ratis we should not have this check.
// Same can happen after OM restart also.
// OM Start
// T1 -> Create Bucket
// T2 -> Set Bucket Property
// OM restart
// T1 -> Set Bucket Acl
// So when T1 is executing, Bucket will have updateID 2 which is set by T2
// execution before restart.
// Main reason, in non-HA transaction Index after restart starts from 0.
// And also because of this same reason we don't do replay checks in non-HA.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-3069
## How was this patch tested?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #609: HDDS-3069.
UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-593562110
Test failures are related, this has caused due to force push, and one of the commits which is directly done in GitHub is missed. I think now tests should pass.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 edited a comment on issue #609:
HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 edited a comment on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-593681874
This time it failed in it-freon step with time out. Not related to this PR.
```
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test) on project hadoop-ozone-integration-test: There was a timeout or other error in the fork -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #609: HDDS-3069.
UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-592759499
Test failures are related, fixing them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] arp7 commented on issue #609: HDDS-3069. UpdateID
check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
arp7 commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-593220067
Retriggered checks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] hanishakoneru commented on issue #609: HDDS-3069.
UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
hanishakoneru commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-593613662
+1 pending CI.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 merged pull request #609: HDDS-3069.
UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 merged pull request #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #609: HDDS-3069.
UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-593681874
This time it failed in it-freon step with time out.
```
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test) on project hadoop-ozone-integration-test: There was a timeout or other error in the fork -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 edited a comment on issue #609:
HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 edited a comment on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-594339134
> Let's commit it then. Is it possible that this PR has somehow slowed down things causing the occasional timeouts?
In this PR, the code change is if ratisEnabled check to do Precondition, and it just sets updateID. I think that should not cause any regression and cause performance to degrade.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #609: HDDS-3069.
UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-594339821
Thinking more we can remove this check once we have only one code path and by default ratis is used for both HA and non-HA
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #609: HDDS-3069.
UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-594273358
Retriggered CI.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #609: HDDS-3069.
UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-592798237
/retest
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 edited a comment on issue #609:
HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 edited a comment on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-594339134
> Let's commit it then. Is it possible that this PR has somehow slowed down things causing the occasional timeouts?
This PR in non-HA added an if ratisEnabled check to do Precondition, and it just sets updateID. I think that should not cause any regression and cause performance degrade.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] hanishakoneru commented on issue #609: HDDS-3069.
UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
hanishakoneru commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-592778743
+1 pending CI.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #609: HDDS-3069.
UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-594339134
> Let's commit it then. Is it possible that this PR has somehow slowed down things causing the occasional timeouts?
This PR in non-HA added an if ratisEnabled check to do Precondition, and it just sets updateID. I think that should not regress any performance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] hanishakoneru commented on issue #609: HDDS-3069.
UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
hanishakoneru commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-592246403
Thanks Bharat for fixing this. +1 pending CI.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] arp7 commented on issue #609: HDDS-3069. UpdateID
check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
arp7 commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-594336794
Let's commit it then. Is it possible that this PR has somehow slowed down things causing the occasional timeouts?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #609: HDDS-3069.
UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-593793574
This time
`##[error]The operation was canceled.`
Before run has all acceptance tests passed, but in this run, it is timed out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #609: HDDS-3069.
UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-592778091
As in KeyManagerImpl om instance is passed null and the use of ozoneManager.isRatisEnabled() is failing with NPE. Fixed that bypassing false, as any way for write requests old code is not used anymore.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #609: HDDS-3069.
UpdateID check should be skipped for non-HA OzoneManager.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on issue #609: HDDS-3069. UpdateID check should be skipped for non-HA OzoneManager.
URL: https://github.com/apache/hadoop-ozone/pull/609#issuecomment-594336220
Now time out in it-freon step.
Acceptance test time out in previous run, passed in this run. it-freon which passed in the previous run timed out in this run.
The failures are unrelated to this PR.
```
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test) on project hadoop-ozone-integration-test: There was a timeout or other error in the fork -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org