You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Hadoop QA (Jira)" <ji...@apache.org> on 2020/12/18 11:25:00 UTC
[jira] [Commented] (HDFS-15738) Forbid the transition to Observer
state when NameNode is in StartupSafeMode
[ https://issues.apache.org/jira/browse/HDFS-15738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17251710#comment-17251710 ]
Hadoop QA commented on HDFS-15738:
----------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 46s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m 0s{color} | {color:green}{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green}{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color} | {color:green}test4tests{color} | {color:green} The patch appears to include 1 new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m 18s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 19s{color} | {color:green}{color} | {color:green} trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 50s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 23s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 17m 18s{color} | {color:green}{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s{color} | {color:green}{color} | {color:green} trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 21s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 3m 8s{color} | {color:blue}{color} | {color:blue} Used deprecated FindBugs config; considering switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 8s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 12s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s{color} | {color:green}{color} | {color:green} the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 15s{color} | {color:red}https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/364/artifact/out/diff-compile-javac-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt{color} | {color:red} hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 generated 1 new + 599 unchanged - 0 fixed = 600 total (was 599) {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s{color} | {color:green}{color} | {color:green} the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 3s{color} | {color:red}https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/364/artifact/out/diff-compile-javac-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt{color} | {color:red} hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 generated 1 new + 583 unchanged - 0 fixed = 584 total (was 583) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 41s{color} | {color:orange}https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/364/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 158 unchanged - 0 fixed = 159 total (was 158) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 14s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green}{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 18m 10s{color} | {color:green}{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s{color} | {color:green}{color} | {color:green} the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 29s{color} | {color:green}{color} | {color:green} the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 40s{color} | {color:green}{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} || ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 16m 52s{color} | {color:red}https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/364/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt{color} | {color:red} hadoop-hdfs in the patch passed. {color} |
| {color:blue}0{color} | {color:blue} asflicense {color} | {color:blue} 0m 26s{color} | {color:blue}{color} | {color:blue} ASF License check generated no output? {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 96m 22s{color} | {color:black}{color} | {color:black}{color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.fs.TestUnbuffer |
| | hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader |
| | hadoop.fs.TestSWebHdfsFileContextMainOperations |
| | hadoop.fs.contract.hdfs.TestHDFSContractSeek |
| | hadoop.fs.contract.hdfs.TestHDFSContractConcat |
| | hadoop.fs.contract.hdfs.TestHDFSContractOpen |
| | hadoop.fs.contract.hdfs.TestHDFSContractAppend |
| | hadoop.fs.contract.hdfs.TestHDFSContractCreate |
| | hadoop.fs.contract.hdfs.TestHDFSContractRename |
| | hadoop.fs.contract.hdfs.TestHDFSContractPathHandle |
| | hadoop.fs.contract.hdfs.TestHDFSContractDelete |
| | hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory |
| | hadoop.fs.contract.hdfs.TestHDFSContractUnbuffer |
| | hadoop.fs.contract.hdfs.TestHDFSContractMkdir |
| | hadoop.fs.contract.hdfs.TestHDFSContractSetTimes |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/364/artifact/out/Dockerfile |
| JIRA Issue | HDFS-15738 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/13017364/HDFS-15738.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux d9d6e553461d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 7a88f453667 |
| Default Java | Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
| Test Results | https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/364/testReport/ |
| Max. process+thread count | 820 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/364/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Forbid the transition to Observer state when NameNode is in StartupSafeMode
> ---------------------------------------------------------------------------
>
> Key: HDFS-15738
> URL: https://issues.apache.org/jira/browse/HDFS-15738
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Janus Chow
> Assignee: Janus Chow
> Priority: Major
> Attachments: HDFS-15738.001.patch
>
>
> Currently when a _getBlockLocation_ request comes to an Observer Namenode which is in safemode, NameNode will have a check that if the result is empty, it will reply to the client with a _RetriableException_, noting the client to retry the request later.
> And If the Observer Namenode is in startup safe mode, the client would have to wait for the Observer NameNode to leave the safe mode. For a big cluster, it may cause a long time of waiting for the client. In our cluster, we met this problem, and the client needs to wait for about 30 minutes before the service back to normal.
> The reason for this situation is that the NameNode becomes the state of Observer when it's still in safe mode getting Datanode's block reports. And here are two solutions for this issue:
> # Throw _ObserverRetryOnActiveException_ when the Observer NameNode is in startup safe mode, redirecting the user's requests to active NN.
> # Forbid the transition to Observer state when the cluster maintainer is trying to do the transition operation.
> We choose the second solution because the first one would abet the bad operation of transition NN to Observers while it's not ready for real service.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org