You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by SP <sa...@gmail.com> on 2015/01/31 21:13:11 UTC
how does datanodes know where to send the block report in HA
Hi All,
How does datanodes know where to send the block report and heart beat info
in HA cluster implemented using Quorum Based Storage?
What are the properties that need to set for the data node in the
*-site.xml ?
Will the data node send report directly to Name Node or will it send to
Journal Nodes / ZKFC?
Thanks
SP
Re: how does datanodes know where to send the block report in HA
Posted by Harsh J <ha...@cloudera.com>.
The JournalNodes are only spoken to by the NameNode - they are part of
the shared edits storage. The ZKFC is a controller that talks to the
NN for failover activities. Neither of these roles care/know about
DataNodes.
The DataNodes send the block reports to the NameNodes just like they
send their heartbeats. You don't need to set any extra properties to
allow these - if your DNs are registering onto the HDFS cluster
properly, they are also sending their block reports properly as an
automatic result. The DNs seek the NN address via the rpc-address or
servicerpc-address configs for the NNs.
On Sun, Feb 1, 2015 at 1:43 AM, SP <sa...@gmail.com> wrote:
> Hi All,
>
> How does datanodes know where to send the block report and heart beat info
> in HA cluster implemented using Quorum Based Storage?
>
> What are the properties that need to set for the data node in the *-site.xml
> ?
>
> Will the data node send report directly to Name Node or will it send to
> Journal Nodes / ZKFC?
>
> Thanks
> SP
>
>
>
--
Harsh J
Re: how does datanodes know where to send the block report in HA
Posted by Harsh J <ha...@cloudera.com>.
The JournalNodes are only spoken to by the NameNode - they are part of
the shared edits storage. The ZKFC is a controller that talks to the
NN for failover activities. Neither of these roles care/know about
DataNodes.
The DataNodes send the block reports to the NameNodes just like they
send their heartbeats. You don't need to set any extra properties to
allow these - if your DNs are registering onto the HDFS cluster
properly, they are also sending their block reports properly as an
automatic result. The DNs seek the NN address via the rpc-address or
servicerpc-address configs for the NNs.
On Sun, Feb 1, 2015 at 1:43 AM, SP <sa...@gmail.com> wrote:
> Hi All,
>
> How does datanodes know where to send the block report and heart beat info
> in HA cluster implemented using Quorum Based Storage?
>
> What are the properties that need to set for the data node in the *-site.xml
> ?
>
> Will the data node send report directly to Name Node or will it send to
> Journal Nodes / ZKFC?
>
> Thanks
> SP
>
>
>
--
Harsh J
Re: how does datanodes know where to send the block report in HA
Posted by Harsh J <ha...@cloudera.com>.
The JournalNodes are only spoken to by the NameNode - they are part of
the shared edits storage. The ZKFC is a controller that talks to the
NN for failover activities. Neither of these roles care/know about
DataNodes.
The DataNodes send the block reports to the NameNodes just like they
send their heartbeats. You don't need to set any extra properties to
allow these - if your DNs are registering onto the HDFS cluster
properly, they are also sending their block reports properly as an
automatic result. The DNs seek the NN address via the rpc-address or
servicerpc-address configs for the NNs.
On Sun, Feb 1, 2015 at 1:43 AM, SP <sa...@gmail.com> wrote:
> Hi All,
>
> How does datanodes know where to send the block report and heart beat info
> in HA cluster implemented using Quorum Based Storage?
>
> What are the properties that need to set for the data node in the *-site.xml
> ?
>
> Will the data node send report directly to Name Node or will it send to
> Journal Nodes / ZKFC?
>
> Thanks
> SP
>
>
>
--
Harsh J
Re: how does datanodes know where to send the block report in HA
Posted by Harsh J <ha...@cloudera.com>.
The JournalNodes are only spoken to by the NameNode - they are part of
the shared edits storage. The ZKFC is a controller that talks to the
NN for failover activities. Neither of these roles care/know about
DataNodes.
The DataNodes send the block reports to the NameNodes just like they
send their heartbeats. You don't need to set any extra properties to
allow these - if your DNs are registering onto the HDFS cluster
properly, they are also sending their block reports properly as an
automatic result. The DNs seek the NN address via the rpc-address or
servicerpc-address configs for the NNs.
On Sun, Feb 1, 2015 at 1:43 AM, SP <sa...@gmail.com> wrote:
> Hi All,
>
> How does datanodes know where to send the block report and heart beat info
> in HA cluster implemented using Quorum Based Storage?
>
> What are the properties that need to set for the data node in the *-site.xml
> ?
>
> Will the data node send report directly to Name Node or will it send to
> Journal Nodes / ZKFC?
>
> Thanks
> SP
>
>
>
--
Harsh J