[Updated] RE: current direction in namenode HA

[Uma Maheswara Rao G <ma...@huawei.com>]

My previous reply messed up the options numbering, It may confuse some people. I am resending it again by correcting that. Appologies for confusions.

>>>1) rely on external HA filer
    If you can invest on NFS filers, it is one option in HA.  If you want to try this, you can use released Hadoop-2 version and try. but below #3 and #4 will avoid this external hardware dependency.
  
>>>2) multiple edit directories
    HDFS already supports, multiple edit directory configuration. I am not exactly sure what option you are referring here.

>>>3) book keeper
   Please take a look at https://issues.apache.org/jira/browse/HDFS-3399
  We are testing with BookeKeeper from last 2/3 months and going well. BK is progressing on autorecovery and security  parts. Almost auto recoverry done (BOOKKEEPER-237) and will be released in BK 4.2 version very soon. BK already started work on security part as well. Also this integration part will come out with next hadoop-2 release as well. Also attached tested scenarios in HDFS-3399 for your reference if you want to take a look. Also there is one subTask in that umbrella  JIRA for user manual  information. BookKeeper Wiki https://cwiki.apache.org/confluence/display/BOOKKEEPER/BookKeeper
     

>>>4) keep edits in HDFS / quorum based
    you can take a look at HDFS-3077. In this umbrella JIRA, work is going actively.
    https://issues.apache.org/jira/browse/HDFS-3077 you can get more design details in that JIRA.

Regards,
Uma

________________________________________
From: Sujee Maniyam [sujee@sujee.net]
Sent: Wednesday, September 05, 2012 4:07 AM
To: hdfs-dev@hadoop.apache.org
Subject: current direction in namenode HA

Hello devs,

I am trying to understand the current state / direction of  namenode
HA implementation.

For using shared directory, I see the following options
(from http://www.cloudera.com/blog/2012/03/high-availability-for-the-hadoop-distributed-file-system-hdfs/
  and  https://issues.apache.org/jira/browse/HDFS-3278)

1) rely on external HA filer
2) multiple edit directories
3) book keeper
4) keep edits in HDFS / quorum based

is there going to be an 'official / supported' method, or it is going
to be a configurable choice when setting up a cluster?

thanks
Sujee
http://sujee.net

Re: [Updated] RE: current direction in namenode HA

[Todd Lipcon <to...@cloudera.com>]

The quorum-based journal implementation is in HDFS-3077, soon to be
merged to trunk.

The BookKeeper-based implementation has been developed on trunk under
a contrib/ module.

Thanks
-Todd

On Tue, Sep 25, 2012 at 3:34 PM, Sujee Maniyam <su...@sujee.net> wrote:
> and what branch is this work going into?
> thanks
> Sujee
>
>
> On Mon, Sep 10, 2012 at 4:18 PM, Sujee Maniyam <su...@sujee.net> wrote:
>
>> Thanks very much for all your input!  Much appreciated...
>>



-- 
Todd Lipcon
Software Engineer, Cloudera

Re: [Updated] RE: current direction in namenode HA

[Sujee Maniyam <su...@sujee.net>]

and what branch is this work going into?
thanks
Sujee


On Mon, Sep 10, 2012 at 4:18 PM, Sujee Maniyam <su...@sujee.net> wrote:

> Thanks very much for all your input!  Much appreciated...
>

Re: [Updated] RE: current direction in namenode HA

[Sujee Maniyam <su...@sujee.net>]

Thanks very much for all your input!  Much appreciated...