Active Name Nodes behavior difference in HA Cluster

[oula alshiekh <ou...@gmail.com>]

iam using hadoop apache 2.7.1 with zookeeper  quorum in order to achieve
automatic failover on centos 7

and every thing is done ok but i am facing this strange problem

supposing my active two name nodes are mn1 and mn2

when mn1 is off so because of automatic failover mn2 is active

and iam using webhdfs requests which are directed to mn2 by client and the
performance is greate

but in the reverse status i mean when mn2 is off and mn1 becomes active and
webhdfs requests are directed to mn1 the performance got stack until mn2 is
on again ,even though mn2 is turned on again with no services

notice1: mn1 and mn2 has same configurations and all cluster nodes are vms
which are connected to virtual switch so there are no network issues to
worry about

notice2:there are no difference between two active name nodes performance
if we issue hdfs commands the difference only in webhdfs interface requests
(in curl requests)

what could be possible reasons of this difference in performance between
two active name nodes ?

Re: Active Name Nodes behavior difference in HA Cluster

[Philippe Kernévez <pk...@octo.com>]

Hi,

WebHdfs do not redirect request to active nn (like Yarn Resource Manager).
You have to pu a proxy on top : Knox or HttpFS (yes it's quite complex) the
will be aware of the state of the cluster .
https://knox.apache.org/
https://hadoop.apache.org/docs/r2.7.1/hadoop-hdfs-httpfs

Regards,
Philippe

On Tue, Jul 11, 2017 at 9:23 AM, oula alshiekh <ou...@gmail.com>
wrote:

> iam using hadoop apache 2.7.1 with zookeeper  quorum in order to achieve
> automatic failover on centos 7
>
> and every thing is done ok but i am facing this strange problem
>
> supposing my active two name nodes are mn1 and mn2
>
> when mn1 is off so because of automatic failover mn2 is active
>
> and iam using webhdfs requests which are directed to mn2 by client and the
> performance is greate
>
> but in the reverse status i mean when mn2 is off and mn1 becomes active
> and webhdfs requests are directed to mn1 the performance got stack until
> mn2 is on again ,even though mn2 is turned on again with no services
>
> notice1: mn1 and mn2 has same configurations and all cluster nodes are vms
> which are connected to virtual switch so there are no network issues to
> worry about
>
> notice2:there are no difference between two active name nodes performance
> if we issue hdfs commands the difference only in webhdfs interface requests
> (in curl requests)
>
> what could be possible reasons of this difference in performance between
> two active name nodes ?
>



-- 
Philippe Kernévez



Directeur technique (Suisse),
pkernevez@octo.com
+41 79 888 33 32

Retrouvez OCTO sur OCTO Talk : http://blog.octo.com
OCTO Technology http://www.octo.ch