You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by ch huang <ju...@gmail.com> on 2013/07/26 04:54:10 UTC

issure about config namenode HA

when i config all things and format namenodes, it's error ,it seems all
journal node is unavailable, should the journal node must be start before
format name node?


13/07/26 18:45:39 INFO ipc.Client: Retrying connect to server: node3/
192.168.142.131:8485. Already tried 8 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
13/07/26 18:45:39 INFO ipc.Client: Retrying connect to server: node2/
192.168.142.130:8485. Already tried 8 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server: node1/
192.168.142.129:8485. Already tried 9 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server: node3/
192.168.142.131:8485. Already tried 9 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server: node2/
192.168.142.130:8485. Already tried 9 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
13/07/26 18:45:40 FATAL namenode.NameNode: Exception in namenode join
org.apache.hadoop.hdfs.qjournal.client.QuorumException: Unable to check if
JNs are ready for formatting. 2 exceptions thrown:
192.168.142.131:8485: Call From node1/192.168.142.129 to node3:8485 failed
on connection exception: java.net.ConnectException: Connection refused; For
more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
192.168.142.130:8485: Call From node1/192.168.142.129 to node2:8485 failed
on connection exception: java.net.ConnectException: Connection refused; For
more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
        at
org.apache.hadoop.hdfs.qjournal.client.QuorumException.create(QuorumException.java:81)
        at
org.apache.hadoop.hdfs.qjournal.client.QuorumCall.rethrowException(QuorumCall.java:213)
        at
org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.hasSomeData(QuorumJournalManager.java:218)
        at
org.apache.hadoop.hdfs.server.common.Storage.confirmFormat(Storage.java:831)
        at
org.apache.hadoop.hdfs.server.namenode.FSImage.confirmFormat(FSImage.java:170)
        at
org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:749)
        at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1121)
        at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1226)
13/07/26 18:45:40 INFO util.ExitUtil: Exiting with status 1
13/07/26 18:45:40 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at node1/192.168.142.129
************************************************************/

Re: issure about config namenode HA

Posted by Harsh J <ha...@cloudera.com>.
If you have already configured QJM for NN's edits storage, then yes,
it needs to be available.

On Fri, Jul 26, 2013 at 8:24 AM, ch huang <ju...@gmail.com> wrote:
> when i config all things and format namenodes, it's error ,it seems all
> journal node is unavailable, should the journal node must be start before
> format name node?
>
>
> 13/07/26 18:45:39 INFO ipc.Client: Retrying connect to server:
> node3/192.168.142.131:8485. Already tried 8 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:39 INFO ipc.Client: Retrying connect to server:
> node2/192.168.142.130:8485. Already tried 8 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server:
> node1/192.168.142.129:8485. Already tried 9 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server:
> node3/192.168.142.131:8485. Already tried 9 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server:
> node2/192.168.142.130:8485. Already tried 9 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 FATAL namenode.NameNode: Exception in namenode join
> org.apache.hadoop.hdfs.qjournal.client.QuorumException: Unable to check if
> JNs are ready for formatting. 2 exceptions thrown:
> 192.168.142.131:8485: Call From node1/192.168.142.129 to node3:8485 failed
> on connection exception: java.net.ConnectException: Connection refused; For
> more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
> 192.168.142.130:8485: Call From node1/192.168.142.129 to node2:8485 failed
> on connection exception: java.net.ConnectException: Connection refused; For
> more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
>         at
> org.apache.hadoop.hdfs.qjournal.client.QuorumException.create(QuorumException.java:81)
>         at
> org.apache.hadoop.hdfs.qjournal.client.QuorumCall.rethrowException(QuorumCall.java:213)
>         at
> org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.hasSomeData(QuorumJournalManager.java:218)
>         at
> org.apache.hadoop.hdfs.server.common.Storage.confirmFormat(Storage.java:831)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSImage.confirmFormat(FSImage.java:170)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:749)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1121)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1226)
> 13/07/26 18:45:40 INFO util.ExitUtil: Exiting with status 1
> 13/07/26 18:45:40 INFO namenode.NameNode: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down NameNode at node1/192.168.142.129
> ************************************************************/



-- 
Harsh J

Re: issure about config namenode HA

Posted by Harsh J <ha...@cloudera.com>.
If you have already configured QJM for NN's edits storage, then yes,
it needs to be available.

On Fri, Jul 26, 2013 at 8:24 AM, ch huang <ju...@gmail.com> wrote:
> when i config all things and format namenodes, it's error ,it seems all
> journal node is unavailable, should the journal node must be start before
> format name node?
>
>
> 13/07/26 18:45:39 INFO ipc.Client: Retrying connect to server:
> node3/192.168.142.131:8485. Already tried 8 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:39 INFO ipc.Client: Retrying connect to server:
> node2/192.168.142.130:8485. Already tried 8 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server:
> node1/192.168.142.129:8485. Already tried 9 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server:
> node3/192.168.142.131:8485. Already tried 9 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server:
> node2/192.168.142.130:8485. Already tried 9 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 FATAL namenode.NameNode: Exception in namenode join
> org.apache.hadoop.hdfs.qjournal.client.QuorumException: Unable to check if
> JNs are ready for formatting. 2 exceptions thrown:
> 192.168.142.131:8485: Call From node1/192.168.142.129 to node3:8485 failed
> on connection exception: java.net.ConnectException: Connection refused; For
> more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
> 192.168.142.130:8485: Call From node1/192.168.142.129 to node2:8485 failed
> on connection exception: java.net.ConnectException: Connection refused; For
> more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
>         at
> org.apache.hadoop.hdfs.qjournal.client.QuorumException.create(QuorumException.java:81)
>         at
> org.apache.hadoop.hdfs.qjournal.client.QuorumCall.rethrowException(QuorumCall.java:213)
>         at
> org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.hasSomeData(QuorumJournalManager.java:218)
>         at
> org.apache.hadoop.hdfs.server.common.Storage.confirmFormat(Storage.java:831)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSImage.confirmFormat(FSImage.java:170)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:749)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1121)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1226)
> 13/07/26 18:45:40 INFO util.ExitUtil: Exiting with status 1
> 13/07/26 18:45:40 INFO namenode.NameNode: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down NameNode at node1/192.168.142.129
> ************************************************************/



-- 
Harsh J

Re: issure about config namenode HA

Posted by Harsh J <ha...@cloudera.com>.
If you have already configured QJM for NN's edits storage, then yes,
it needs to be available.

On Fri, Jul 26, 2013 at 8:24 AM, ch huang <ju...@gmail.com> wrote:
> when i config all things and format namenodes, it's error ,it seems all
> journal node is unavailable, should the journal node must be start before
> format name node?
>
>
> 13/07/26 18:45:39 INFO ipc.Client: Retrying connect to server:
> node3/192.168.142.131:8485. Already tried 8 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:39 INFO ipc.Client: Retrying connect to server:
> node2/192.168.142.130:8485. Already tried 8 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server:
> node1/192.168.142.129:8485. Already tried 9 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server:
> node3/192.168.142.131:8485. Already tried 9 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server:
> node2/192.168.142.130:8485. Already tried 9 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 FATAL namenode.NameNode: Exception in namenode join
> org.apache.hadoop.hdfs.qjournal.client.QuorumException: Unable to check if
> JNs are ready for formatting. 2 exceptions thrown:
> 192.168.142.131:8485: Call From node1/192.168.142.129 to node3:8485 failed
> on connection exception: java.net.ConnectException: Connection refused; For
> more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
> 192.168.142.130:8485: Call From node1/192.168.142.129 to node2:8485 failed
> on connection exception: java.net.ConnectException: Connection refused; For
> more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
>         at
> org.apache.hadoop.hdfs.qjournal.client.QuorumException.create(QuorumException.java:81)
>         at
> org.apache.hadoop.hdfs.qjournal.client.QuorumCall.rethrowException(QuorumCall.java:213)
>         at
> org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.hasSomeData(QuorumJournalManager.java:218)
>         at
> org.apache.hadoop.hdfs.server.common.Storage.confirmFormat(Storage.java:831)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSImage.confirmFormat(FSImage.java:170)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:749)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1121)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1226)
> 13/07/26 18:45:40 INFO util.ExitUtil: Exiting with status 1
> 13/07/26 18:45:40 INFO namenode.NameNode: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down NameNode at node1/192.168.142.129
> ************************************************************/



-- 
Harsh J

Re: issure about config namenode HA

Posted by Harsh J <ha...@cloudera.com>.
If you have already configured QJM for NN's edits storage, then yes,
it needs to be available.

On Fri, Jul 26, 2013 at 8:24 AM, ch huang <ju...@gmail.com> wrote:
> when i config all things and format namenodes, it's error ,it seems all
> journal node is unavailable, should the journal node must be start before
> format name node?
>
>
> 13/07/26 18:45:39 INFO ipc.Client: Retrying connect to server:
> node3/192.168.142.131:8485. Already tried 8 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:39 INFO ipc.Client: Retrying connect to server:
> node2/192.168.142.130:8485. Already tried 8 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server:
> node1/192.168.142.129:8485. Already tried 9 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server:
> node3/192.168.142.131:8485. Already tried 9 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 INFO ipc.Client: Retrying connect to server:
> node2/192.168.142.130:8485. Already tried 9 time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
> 13/07/26 18:45:40 FATAL namenode.NameNode: Exception in namenode join
> org.apache.hadoop.hdfs.qjournal.client.QuorumException: Unable to check if
> JNs are ready for formatting. 2 exceptions thrown:
> 192.168.142.131:8485: Call From node1/192.168.142.129 to node3:8485 failed
> on connection exception: java.net.ConnectException: Connection refused; For
> more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
> 192.168.142.130:8485: Call From node1/192.168.142.129 to node2:8485 failed
> on connection exception: java.net.ConnectException: Connection refused; For
> more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
>         at
> org.apache.hadoop.hdfs.qjournal.client.QuorumException.create(QuorumException.java:81)
>         at
> org.apache.hadoop.hdfs.qjournal.client.QuorumCall.rethrowException(QuorumCall.java:213)
>         at
> org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.hasSomeData(QuorumJournalManager.java:218)
>         at
> org.apache.hadoop.hdfs.server.common.Storage.confirmFormat(Storage.java:831)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSImage.confirmFormat(FSImage.java:170)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:749)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1121)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1226)
> 13/07/26 18:45:40 INFO util.ExitUtil: Exiting with status 1
> 13/07/26 18:45:40 INFO namenode.NameNode: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down NameNode at node1/192.168.142.129
> ************************************************************/



-- 
Harsh J