You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Levi Nie <le...@gmail.com> on 2016/09/13 09:19:13 UTC
how about syncing namenode metadata to glusterfs?
Use glusterfs instead of disk to keep fsimage, will it work without HA?
Re: how about syncing namenode metadata to glusterfs?
Posted by Kihwal Lee <ki...@yahoo-inc.com.INVALID>.
We do. :) Yes, it needs other fencing method.
Kihwal
From: Andrew Wang <an...@cloudera.com>
To: Zhe Zhang <zh...@gmail.com>
Cc: Levi Nie <le...@gmail.com>; "common-dev@hadoop.apache.org" <co...@hadoop.apache.org>
Sent: Tuesday, September 13, 2016 12:46 PM
Subject: Re: how about syncing namenode metadata to glusterfs?
This is a similar idea to NFS HA, which you can read about here:
https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html
This is definitely less preferred compared to using QJM, since you need an
external mechanism for fencing writes. FWIW, I don't know of anyone who is
still using NFS HA.
On Tue, Sep 13, 2016 at 7:31 AM, Zhe Zhang <zh...@gmail.com>
wrote:
> Hi Levi,
>
> This clearly should go to hdfs-dev instead of common-dev. Just using an HA
> storage for fsimage won't allow HDFS work without the current NN-HA
> framework. After all fsimage is only a checkpoint of NameNode state.
>
> On Tue, Sep 13, 2016 at 2:19 AM Levi Nie <le...@gmail.com> wrote:
>
> > Use glusterfs instead of disk to keep fsimage, will it work without HA?
> >
> --
> Zhe Zhang
> Apache Hadoop Committer
> http://zhe-thoughts.github.io/about/ | @oldcap
>
Re: how about syncing namenode metadata to glusterfs?
Posted by Andrew Wang <an...@cloudera.com>.
This is a similar idea to NFS HA, which you can read about here:
https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html
This is definitely less preferred compared to using QJM, since you need an
external mechanism for fencing writes. FWIW, I don't know of anyone who is
still using NFS HA.
On Tue, Sep 13, 2016 at 7:31 AM, Zhe Zhang <zh...@gmail.com>
wrote:
> Hi Levi,
>
> This clearly should go to hdfs-dev instead of common-dev. Just using an HA
> storage for fsimage won't allow HDFS work without the current NN-HA
> framework. After all fsimage is only a checkpoint of NameNode state.
>
> On Tue, Sep 13, 2016 at 2:19 AM Levi Nie <le...@gmail.com> wrote:
>
> > Use glusterfs instead of disk to keep fsimage, will it work without HA?
> >
> --
> Zhe Zhang
> Apache Hadoop Committer
> http://zhe-thoughts.github.io/about/ | @oldcap
>
Re: how about syncing namenode metadata to glusterfs?
Posted by Zhe Zhang <zh...@gmail.com>.
Hi Levi,
This clearly should go to hdfs-dev instead of common-dev. Just using an HA
storage for fsimage won't allow HDFS work without the current NN-HA
framework. After all fsimage is only a checkpoint of NameNode state.
On Tue, Sep 13, 2016 at 2:19 AM Levi Nie <le...@gmail.com> wrote:
> Use glusterfs instead of disk to keep fsimage, will it work without HA?
>
--
Zhe Zhang
Apache Hadoop Committer
http://zhe-thoughts.github.io/about/ | @oldcap