Small mistake (?) in doc about HA with Journal Nodes

[Alberto Chiusole <al...@gmail.com>]

Hi all,
I'm Alberto Chiusole, an Italian computer science student and 
open-source fan.
I'm currently performing a small research to expose to my fellow 
students the Hadoop project, and this is my first post in this ML.

I think I spotted I small mistake in the HDFS documentation regarding 
achieving HA with the Quorum Journal Manager [1], section "Hardware 
resources", paragraph "JournalNode machines": it's stated:
"""
The JournalNode daemon is relatively lightweight, so these daemons may 
reasonably be collocated on machines with other Hadoop daemons, for 
example NameNodes, the JobTracker, (...)
"""

Is "NameNodes" a typo and you meant "DataNode" instead? Aren't the 
JournalNodes meant to survive in case of a failure of the NameNodes? Why 
should I place a JournalNode on the same machine that contains the log I 
need to synchronize?


Moreover I have a quick question on the same topic: why do you suggest 
to place an odd numbers of machines as JournalNodes in order to increase 
the Fault Tolerance?


Regards,
Alberto Chiusole


[1]: 
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html#Hardware_resources 


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org

Re: Small mistake (?) in doc about HA with Journal Nodes

[Ravi Prakash <ra...@gmail.com>]

Hi Alberto!

The assumption is that *multiple* machines could be running the Namenode
process. Only one of them would be active, while the other Namenode
processes would be in Standby mode.

The number of machines is suggested to be odd so that its easier to form
consensus. To handle the failure of k machines, 2k+1 is usually the number
of QJMs you'd need.

HTH
Ravi

On Mon, Dec 5, 2016 at 11:32 AM, Alberto Chiusole <
alberto.chiusole95@gmail.com> wrote:

> Hi all,
> I'm Alberto Chiusole, an Italian computer science student and open-source
> fan.
> I'm currently performing a small research to expose to my fellow students
> the Hadoop project, and this is my first post in this ML.
>
> I think I spotted I small mistake in the HDFS documentation regarding
> achieving HA with the Quorum Journal Manager [1], section "Hardware
> resources", paragraph "JournalNode machines": it's stated:
> """
> The JournalNode daemon is relatively lightweight, so these daemons may
> reasonably be collocated on machines with other Hadoop daemons, for example
> NameNodes, the JobTracker, (...)
> """
>
> Is "NameNodes" a typo and you meant "DataNode" instead? Aren't the
> JournalNodes meant to survive in case of a failure of the NameNodes? Why
> should I place a JournalNode on the same machine that contains the log I
> need to synchronize?
>
>
> Moreover I have a quick question on the same topic: why do you suggest to
> place an odd numbers of machines as JournalNodes in order to increase the
> Fault Tolerance?
>
>
> Regards,
> Alberto Chiusole
>
>
> [1]: https://hadoop.apache.org/docs/stable/hadoop-project-dist/
> hadoop-hdfs/HDFSHighAvailabilityWithQJM.html#Hardware_resources
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: user-help@hadoop.apache.org
>
>