You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@hadoop.apache.org by Andrew Wharton <an...@nokia.com> on 2009/06/19 18:10:58 UTC
HDFS name node HA roadmap
Hello,
My understanding is that the HDFS name node single point of failure will
be removed in Hadoop 0.21. Is that still the case? Also, what is the
expected delivery date of that release?
Thanks.
-- Andrew
Re: HDFS name node HA roadmap
Posted by Owen O'Malley <ow...@gmail.com>.
On Fri, Jun 19, 2009 at 9:23 AM, Jie Qiu <qi...@cn.ibm.com> wrote:
However, our approach may be different with shv. Let me discuss with shv
> about how to combine our thoughts and implementation together.
If you want a major change like this to go into the real code base, you need
to open a jira, and let people comment on your architecture, design and test
plans. If you present it as a huge code drop, it is unlikely to go in.
-- Owen
Re: HDFS name node HA roadmap
Posted by Konstantin Shvachko <sh...@yahoo-inc.com>.
Hi Jie Qiu,
It would be interesting to see what your approach to HA is.
Indeed, I had some ideas on how to implement it.
And I would be happy to talk to you about it.
Which jira do you refer to?
Thanks,
--Konstantin
Jie Qiu wrote:
>
> Hi, Dhruba,
>
> Yes, we plan to open a JIRA to let community to review our
> implementation and give comments. Recently, we are going with IBM
> internal process before submitting to the community.
>
> I also want to discuss with shv whether we need to another separate JIRA
> or work on the same integrated JIRA.
>
> Do you any suggestion on this?
>
> ==================================================
> True insight comes from within!
>
> Qiu Jie
>
> Autonomic Middleware and Service Delivery
> IBM China Research Lab.
> Tel: 8610-58748086 Tieline: 905-8086
> FAX: 8610-58748330
> Add.: Building 19 Zhongguancun Software Park, 8 Dongbeiwang WestRoad,
> Haidian District, Beijing,P.R.C.100094
> Email: qiujie@cn.ibm.com
> ==================================================
>
>
> *Dhruba Borthakur <dh...@gmail.com>*
>
> 2009-06-20 01:19
> Please respond to
> general@hadoop.apache.org
>
>
>
> To
> general@hadoop.apache.org
> cc
> shv@yahoo-inc.com, andrew.wharton@nokia.com
> Subject
> Re: HDFS name node HA roadmap
>
>
>
>
>
>
>
>
> Hi Hie,
>
> If you are working on this one, it woudl be nice if you can open a JIRA and
> let the community comment/review your design. This process usually takes a
> while and it is good to get a an early start on it.
>
> thanks,
> dhruba
>
>
> On Fri, Jun 19, 2009 at 9:23 AM, Jie Qiu <qi...@cn.ibm.com> wrote:
>
> > Hi Andrew,
> >
> > Yes, our IBM is implementing HA for Hadoop which will be open sourced
> > soon. I think shv from Yahoo also post JIRA to describe this issue.
> >
> > However, our approach may be different with shv. Let me discuss with shv
> > about how to combine our thoughts and implementation together.
> >
> > ==================================================
> > True insight comes from within!
> >
> > Qiu Jie
> >
> > Autonomic Middleware and Service Delivery
> > IBM China Research Lab.
> > Tel: 8610-58748086 Tieline: 905-8086
> > FAX: 8610-58748330
> > Add.: Building 19 Zhongguancun Software Park, 8 Dongbeiwang WestRoad,
> > Haidian District, Beijing,P.R.C.100094
> > Email: qiujie@cn.ibm.com
> > ==================================================
> >
> >
> >
> > Andrew Wharton <an...@nokia.com>
> > 2009-06-20 00:10
> > Please respond to
> > general@hadoop.apache.org
> >
> >
> > To
> > general@hadoop.apache.org
> > cc
> >
> > Subject
> > HDFS name node HA roadmap
> >
> >
> >
> >
> >
> >
> > Hello,
> >
> > My understanding is that the HDFS name node single point of failure will
> > be removed in Hadoop 0.21. Is that still the case? Also, what is the
> > expected delivery date of that release?
> >
> > Thanks.
> >
> > -- Andrew
> >
> >
> >
>
Re: HDFS name node HA roadmap
Posted by Andrew Wharton <an...@nokia.com>.
I asked on the HBase list and they pointed me to this ticket, which is
more to my liking: https://issues.apache.org/jira/browse/HADOOP-4539
Re: HDFS name node HA roadmap
Posted by Jie Qiu <qi...@cn.ibm.com>.
https://issues.apache.org/jira/browse/HADOOP-5651
==================================================
True insight comes from within!
Qiu Jie
Autonomic Middleware and Service Delivery
IBM China Research Lab.
Tel: 8610-58748086 Tieline: 905-8086
FAX: 8610-58748330
Add.: Building 19 Zhongguancun Software Park, 8 Dongbeiwang WestRoad,
Haidian District, Beijing,P.R.C.100094
Email: qiujie@cn.ibm.com
==================================================
Mahadev Konar <ma...@yahoo-inc.com>
2009-06-20 01:44
Please respond to
general@hadoop.apache.org
To
<ge...@hadoop.apache.org>
cc
<an...@nokia.com>, <sh...@yahoo-inc.com>
Subject
Re: HDFS name node HA roadmap
Does anyone know the jira that shv (konstantin) has opened?
mahadev
On 6/19/09 10:28 AM, "Jie Qiu" <qi...@cn.ibm.com> wrote:
> Hi, Dhruba,
>
> Yes, we plan to open a JIRA to let community to review our
implementation
> and give comments. Recently, we are going with IBM internal process
before
> submitting to the community.
>
> I also want to discuss with shv whether we need to another separate JIRA
> or work on the same integrated JIRA.
>
> Do you any suggestion on this?
>
> ==================================================
> True insight comes from within!
>
> Qiu Jie
>
> Autonomic Middleware and Service Delivery
> IBM China Research Lab.
> Tel: 8610-58748086 Tieline: 905-8086
> FAX: 8610-58748330
> Add.: Building 19 Zhongguancun Software Park, 8 Dongbeiwang WestRoad,
> Haidian District, Beijing,P.R.C.100094
> Email: qiujie@cn.ibm.com
> ==================================================
>
>
>
> Dhruba Borthakur <dh...@gmail.com>
> 2009-06-20 01:19
> Please respond to
> general@hadoop.apache.org
>
>
> To
> general@hadoop.apache.org
> cc
> shv@yahoo-inc.com, andrew.wharton@nokia.com
> Subject
> Re: HDFS name node HA roadmap
>
>
>
>
>
>
> Hi Hie,
>
> If you are working on this one, it woudl be nice if you can open a JIRA
> and
> let the community comment/review your design. This process usually takes
a
> while and it is good to get a an early start on it.
>
> thanks,
> dhruba
>
>
> On Fri, Jun 19, 2009 at 9:23 AM, Jie Qiu <qi...@cn.ibm.com> wrote:
>
>> Hi Andrew,
>>
>> Yes, our IBM is implementing HA for Hadoop which will be open sourced
>> soon. I think shv from Yahoo also post JIRA to describe this issue.
>>
>> However, our approach may be different with shv. Let me discuss with
shv
>> about how to combine our thoughts and implementation together.
>>
>> ==================================================
>> True insight comes from within!
>>
>> Qiu Jie
>>
>> Autonomic Middleware and Service Delivery
>> IBM China Research Lab.
>> Tel: 8610-58748086 Tieline: 905-8086
>> FAX: 8610-58748330
>> Add.: Building 19 Zhongguancun Software Park, 8 Dongbeiwang WestRoad,
>> Haidian District, Beijing,P.R.C.100094
>> Email: qiujie@cn.ibm.com
>> ==================================================
>>
>>
>>
>> Andrew Wharton <an...@nokia.com>
>> 2009-06-20 00:10
>> Please respond to
>> general@hadoop.apache.org
>>
>>
>> To
>> general@hadoop.apache.org
>> cc
>>
>> Subject
>> HDFS name node HA roadmap
>>
>>
>>
>>
>>
>>
>> Hello,
>>
>> My understanding is that the HDFS name node single point of failure
will
>> be removed in Hadoop 0.21. Is that still the case? Also, what is the
>> expected delivery date of that release?
>>
>> Thanks.
>>
>> -- Andrew
>>
>>
>>
>
Re: HDFS name node HA roadmap
Posted by Mahadev Konar <ma...@yahoo-inc.com>.
Does anyone know the jira that shv (konstantin) has opened?
mahadev
On 6/19/09 10:28 AM, "Jie Qiu" <qi...@cn.ibm.com> wrote:
> Hi, Dhruba,
>
> Yes, we plan to open a JIRA to let community to review our implementation
> and give comments. Recently, we are going with IBM internal process before
> submitting to the community.
>
> I also want to discuss with shv whether we need to another separate JIRA
> or work on the same integrated JIRA.
>
> Do you any suggestion on this?
>
> ==================================================
> True insight comes from within!
>
> Qiu Jie
>
> Autonomic Middleware and Service Delivery
> IBM China Research Lab.
> Tel: 8610-58748086 Tieline: 905-8086
> FAX: 8610-58748330
> Add.: Building 19 Zhongguancun Software Park, 8 Dongbeiwang WestRoad,
> Haidian District, Beijing,P.R.C.100094
> Email: qiujie@cn.ibm.com
> ==================================================
>
>
>
> Dhruba Borthakur <dh...@gmail.com>
> 2009-06-20 01:19
> Please respond to
> general@hadoop.apache.org
>
>
> To
> general@hadoop.apache.org
> cc
> shv@yahoo-inc.com, andrew.wharton@nokia.com
> Subject
> Re: HDFS name node HA roadmap
>
>
>
>
>
>
> Hi Hie,
>
> If you are working on this one, it woudl be nice if you can open a JIRA
> and
> let the community comment/review your design. This process usually takes a
> while and it is good to get a an early start on it.
>
> thanks,
> dhruba
>
>
> On Fri, Jun 19, 2009 at 9:23 AM, Jie Qiu <qi...@cn.ibm.com> wrote:
>
>> Hi Andrew,
>>
>> Yes, our IBM is implementing HA for Hadoop which will be open sourced
>> soon. I think shv from Yahoo also post JIRA to describe this issue.
>>
>> However, our approach may be different with shv. Let me discuss with shv
>> about how to combine our thoughts and implementation together.
>>
>> ==================================================
>> True insight comes from within!
>>
>> Qiu Jie
>>
>> Autonomic Middleware and Service Delivery
>> IBM China Research Lab.
>> Tel: 8610-58748086 Tieline: 905-8086
>> FAX: 8610-58748330
>> Add.: Building 19 Zhongguancun Software Park, 8 Dongbeiwang WestRoad,
>> Haidian District, Beijing,P.R.C.100094
>> Email: qiujie@cn.ibm.com
>> ==================================================
>>
>>
>>
>> Andrew Wharton <an...@nokia.com>
>> 2009-06-20 00:10
>> Please respond to
>> general@hadoop.apache.org
>>
>>
>> To
>> general@hadoop.apache.org
>> cc
>>
>> Subject
>> HDFS name node HA roadmap
>>
>>
>>
>>
>>
>>
>> Hello,
>>
>> My understanding is that the HDFS name node single point of failure will
>> be removed in Hadoop 0.21. Is that still the case? Also, what is the
>> expected delivery date of that release?
>>
>> Thanks.
>>
>> -- Andrew
>>
>>
>>
>
Re: HDFS name node HA roadmap
Posted by Jie Qiu <qi...@cn.ibm.com>.
Hi, Dhruba,
Yes, we plan to open a JIRA to let community to review our implementation
and give comments. Recently, we are going with IBM internal process before
submitting to the community.
I also want to discuss with shv whether we need to another separate JIRA
or work on the same integrated JIRA.
Do you any suggestion on this?
==================================================
True insight comes from within!
Qiu Jie
Autonomic Middleware and Service Delivery
IBM China Research Lab.
Tel: 8610-58748086 Tieline: 905-8086
FAX: 8610-58748330
Add.: Building 19 Zhongguancun Software Park, 8 Dongbeiwang WestRoad,
Haidian District, Beijing,P.R.C.100094
Email: qiujie@cn.ibm.com
==================================================
Dhruba Borthakur <dh...@gmail.com>
2009-06-20 01:19
Please respond to
general@hadoop.apache.org
To
general@hadoop.apache.org
cc
shv@yahoo-inc.com, andrew.wharton@nokia.com
Subject
Re: HDFS name node HA roadmap
Hi Hie,
If you are working on this one, it woudl be nice if you can open a JIRA
and
let the community comment/review your design. This process usually takes a
while and it is good to get a an early start on it.
thanks,
dhruba
On Fri, Jun 19, 2009 at 9:23 AM, Jie Qiu <qi...@cn.ibm.com> wrote:
> Hi Andrew,
>
> Yes, our IBM is implementing HA for Hadoop which will be open sourced
> soon. I think shv from Yahoo also post JIRA to describe this issue.
>
> However, our approach may be different with shv. Let me discuss with shv
> about how to combine our thoughts and implementation together.
>
> ==================================================
> True insight comes from within!
>
> Qiu Jie
>
> Autonomic Middleware and Service Delivery
> IBM China Research Lab.
> Tel: 8610-58748086 Tieline: 905-8086
> FAX: 8610-58748330
> Add.: Building 19 Zhongguancun Software Park, 8 Dongbeiwang WestRoad,
> Haidian District, Beijing,P.R.C.100094
> Email: qiujie@cn.ibm.com
> ==================================================
>
>
>
> Andrew Wharton <an...@nokia.com>
> 2009-06-20 00:10
> Please respond to
> general@hadoop.apache.org
>
>
> To
> general@hadoop.apache.org
> cc
>
> Subject
> HDFS name node HA roadmap
>
>
>
>
>
>
> Hello,
>
> My understanding is that the HDFS name node single point of failure will
> be removed in Hadoop 0.21. Is that still the case? Also, what is the
> expected delivery date of that release?
>
> Thanks.
>
> -- Andrew
>
>
>
Re: HDFS name node HA roadmap
Posted by Dhruba Borthakur <dh...@gmail.com>.
Hi Hie,
If you are working on this one, it woudl be nice if you can open a JIRA and
let the community comment/review your design. This process usually takes a
while and it is good to get a an early start on it.
thanks,
dhruba
On Fri, Jun 19, 2009 at 9:23 AM, Jie Qiu <qi...@cn.ibm.com> wrote:
> Hi Andrew,
>
> Yes, our IBM is implementing HA for Hadoop which will be open sourced
> soon. I think shv from Yahoo also post JIRA to describe this issue.
>
> However, our approach may be different with shv. Let me discuss with shv
> about how to combine our thoughts and implementation together.
>
> ==================================================
> True insight comes from within!
>
> Qiu Jie
>
> Autonomic Middleware and Service Delivery
> IBM China Research Lab.
> Tel: 8610-58748086 Tieline: 905-8086
> FAX: 8610-58748330
> Add.: Building 19 Zhongguancun Software Park, 8 Dongbeiwang WestRoad,
> Haidian District, Beijing,P.R.C.100094
> Email: qiujie@cn.ibm.com
> ==================================================
>
>
>
> Andrew Wharton <an...@nokia.com>
> 2009-06-20 00:10
> Please respond to
> general@hadoop.apache.org
>
>
> To
> general@hadoop.apache.org
> cc
>
> Subject
> HDFS name node HA roadmap
>
>
>
>
>
>
> Hello,
>
> My understanding is that the HDFS name node single point of failure will
> be removed in Hadoop 0.21. Is that still the case? Also, what is the
> expected delivery date of that release?
>
> Thanks.
>
> -- Andrew
>
>
>
Re: HDFS name node HA roadmap
Posted by Jie Qiu <qi...@cn.ibm.com>.
Hi Andrew,
Yes, our IBM is implementing HA for Hadoop which will be open sourced
soon. I think shv from Yahoo also post JIRA to describe this issue.
However, our approach may be different with shv. Let me discuss with shv
about how to combine our thoughts and implementation together.
==================================================
True insight comes from within!
Qiu Jie
Autonomic Middleware and Service Delivery
IBM China Research Lab.
Tel: 8610-58748086 Tieline: 905-8086
FAX: 8610-58748330
Add.: Building 19 Zhongguancun Software Park, 8 Dongbeiwang WestRoad,
Haidian District, Beijing,P.R.C.100094
Email: qiujie@cn.ibm.com
==================================================
Andrew Wharton <an...@nokia.com>
2009-06-20 00:10
Please respond to
general@hadoop.apache.org
To
general@hadoop.apache.org
cc
Subject
HDFS name node HA roadmap
Hello,
My understanding is that the HDFS name node single point of failure will
be removed in Hadoop 0.21. Is that still the case? Also, what is the
expected delivery date of that release?
Thanks.
-- Andrew