You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2022/03/21 03:34:03 UTC
[GitHub] [hadoop] tomscut opened a new pull request #4087: HDFS-16513. [SBN read] Observer Namenode does not trigger the edits r…
tomscut opened a new pull request #4087:
URL: https://github.com/apache/hadoop/pull/4087
JIRA: [HDFS-16513](https://issues.apache.org/jira/browse/HDFS-16513).
To avoid frequent edtis rolling, we should disable OBN from triggering the edits rolling of active Namenode.
It is sufficient to retain only the `triggering of SNN` and the `auto rolling of ANN`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #4087: HDFS-16513. [SBN read] Observer Namenode should not trigger the edits rolling of active Namenode
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #4087:
URL: https://github.com/apache/hadoop/pull/4087#issuecomment-1084285726
> I think this is pretty similar to https://issues.apache.org/jira/browse/HDFS-14378. If I remember correctly, the better approach would be to have ANN roll its own edit logs.
>
> Even though we address the observer issue here, in a real scenario there could still be multiple SNNs.
Thank you @sunchao very much for your review.
Active Namenode does automatically roll logs periodically. It might be risky (but I think it's ok) by simply disabling all SNN to trigger active roll edits log. However, disabling OBN rolle active edits has no side effects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut edited a comment on pull request #4087: HDFS-16513. [SBN read] Observer Namenode should not trigger the edits rolling of active Namenode
Posted by GitBox <gi...@apache.org>.
tomscut edited a comment on pull request #4087:
URL: https://github.com/apache/hadoop/pull/4087#issuecomment-1084285726
> I think this is pretty similar to https://issues.apache.org/jira/browse/HDFS-14378. If I remember correctly, the better approach would be to have ANN roll its own edit logs.
>
> Even though we address the observer issue here, in a real scenario there could still be multiple SNNs.
Thank you @sunchao very much for your review.
Active Namenode does automatically roll logs periodically. It might be risky(we can look at here HDFS-2737) by simply disabling all SNN to trigger active roll edits log. However, disabling OBN rolle active edits has no side effects. What do you think of this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #4087: HDFS-16513. [SBN read] Observer Namenode does not trigger the edits r…
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #4087:
URL: https://github.com/apache/hadoop/pull/4087#issuecomment-1074579167
Hi @xkrogen @sunchao , please take a look at this. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut removed a comment on pull request #4087: HDFS-16513. [SBN read] Observer Namenode should not trigger the edits rolling of active Namenode
Posted by GitBox <gi...@apache.org>.
tomscut removed a comment on pull request #4087:
URL: https://github.com/apache/hadoop/pull/4087#issuecomment-1074579167
Hi @xkrogen @sunchao , please take a look at this. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #4087: HDFS-16513. [SBN read] Observer Namenode should not trigger the edits rolling of active Namenode
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #4087:
URL: https://github.com/apache/hadoop/pull/4087#issuecomment-1082513955
To avoid frequent edtis rolling, we should disable OBN from triggering the edits rolling of active Namenode.
Hi @sunchao, please have a look at this. Thank you very much.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4087: HDFS-16513. [SBN read] Observer Namenode does not trigger the edits r…
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #4087:
URL: https://github.com/apache/hadoop/pull/4087#issuecomment-1073667887
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 35m 29s | | trunk passed |
| +1 :green_heart: | compile | 1m 31s | | trunk passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 26s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 32s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 9s | | trunk passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 34s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 44s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 22s | | the patch passed |
| +1 :green_heart: | compile | 1m 25s | | the patch passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 15s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 52s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 23s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 55s | | the patch passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 21s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 29s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 1s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 237m 34s | | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 46s | | The patch does not generate ASF License warnings. |
| | | 343m 30s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4087/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4087 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 956ddb79f286 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0cf9128bf5f6a2ce5fc9842c6966e374c7aae7fd |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4087/1/testReport/ |
| Max. process+thread count | 2880 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4087/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #4087: HDFS-16513. [SBN read] Observer Namenode should not trigger the edits rolling of active Namenode
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #4087:
URL: https://github.com/apache/hadoop/pull/4087#issuecomment-1080317480
Hi @xkrogen @sunchao @tamaashu @ayushtkn @ferhui @virajjasani , please take a look at this. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut edited a comment on pull request #4087: HDFS-16513. [SBN read] Observer Namenode should not trigger the edits rolling of active Namenode
Posted by GitBox <gi...@apache.org>.
tomscut edited a comment on pull request #4087:
URL: https://github.com/apache/hadoop/pull/4087#issuecomment-1084285726
> I think this is pretty similar to https://issues.apache.org/jira/browse/HDFS-14378. If I remember correctly, the better approach would be to have ANN roll its own edit logs.
>
> Even though we address the observer issue here, in a real scenario there could still be multiple SNNs.
Thank you @sunchao very much for your review.
Active Namenode does automatically roll logs periodically. It might be risky(we can look at here HDFS-2737) by simply disabling all SNN to trigger active roll edits log. However, disabling OBN rolle active edits has no side effects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org