You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by George Kousiouris <gk...@mail.ntua.gr> on 2011/08/25 11:43:55 UTC
Any related paper on how to resolve hadoop SPOF issue?
Hi guys,
We are currently in the process of writing a paper regarding hadoop and
we would like to reference any attempt to remove the single point of
failure of the Namenode. We have found in various presentations some
efforts (like dividing the namespace between more than one namenodes if
i remember correctly) but the search for a concrete paper on the issue
came to nothing.
Is anyone aware or has participated in such an effort?
Thanks,
George
--
---------------------------
George Kousiouris
Electrical and Computer Engineer
Division of Communications,
Electronics and Information Engineering
School of Electrical and Computer Engineering
Tel: +30 210 772 2546
Mobile: +30 6939354121
Fax: +30 210 772 2569
Email: gkousiou@mail.ntua.gr
Site: http://users.ntua.gr/gkousiou/
National Technical University of Athens
9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
Re: Any related paper on how to resolve hadoop SPOF issue?
Posted by "M. C. Srivas" <mc...@gmail.com>.
Are there any performance benchmarks available for Ceph? (with Hadoop,
without, both?)
On Thu, Aug 25, 2011 at 11:44 AM, Alex Nelson <aj...@cs.ucsc.edu> wrote:
> Hi George,
>
> UC Santa Cruz contributed a ;login: article describing replacing HDFS with
> Ceph. (I was one of the authors.) One of the key architectural advantages
> of Ceph over HDFS is that Ceph distributes its metadata service over
> multiple metadata servers.
>
> I hope that helps.
>
> --Alex
>
>
> On Aug 25, 2011, at 02:54 , George Kousiouris wrote:
>
> >
> > Hi,
> >
> > many thanks!!
> >
> > I also found this one, which has some related work also with other
> efforts, in case it is helpful for someone else:
> >
> https://ritdml.rit.edu/bitstream/handle/1850/13321/ATalwalkarThesis1-2011.pdf?sequence=1
> >
> >
> > BR,
> > George
> >
> > On 8/25/2011 12:46 PM, Nan Zhu wrote:
> >> Hope it helps
> >>
> >> http://www.springerlink.com/content/h17r882710314147/
> >>
> >> Best,
> >>
> >> Nan
> >>
> >> On Thu, Aug 25, 2011 at 5:43 PM, George Kousiouris<
> gkousiou@mail.ntua.gr>wrote:
> >>
> >>> Hi guys,
> >>>
> >>> We are currently in the process of writing a paper regarding hadoop and
> we
> >>> would like to reference any attempt to remove the single point of
> failure of
> >>> the Namenode. We have found in various presentations some efforts (like
> >>> dividing the namespace between more than one namenodes if i remember
> >>> correctly) but the search for a concrete paper on the issue came to
> nothing.
> >>>
> >>> Is anyone aware or has participated in such an effort?
> >>>
> >>> Thanks,
> >>> George
> >>>
> >>> --
> >>>
> >>> ---------------------------
> >>>
> >>> George Kousiouris
> >>> Electrical and Computer Engineer
> >>> Division of Communications,
> >>> Electronics and Information Engineering
> >>> School of Electrical and Computer Engineering
> >>> Tel: +30 210 772 2546
> >>> Mobile: +30 6939354121
> >>> Fax: +30 210 772 2569
> >>> Email: gkousiou@mail.ntua.gr
> >>> Site: http://users.ntua.gr/gkousiou/
> >>>
> >>> National Technical University of Athens
> >>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
> >>>
> >>>
> >>
> >
> >
> > --
> >
> > ---------------------------
> >
> > George Kousiouris
> > Electrical and Computer Engineer
> > Division of Communications,
> > Electronics and Information Engineering
> > School of Electrical and Computer Engineering
> > Tel: +30 210 772 2546
> > Mobile: +30 6939354121
> > Fax: +30 210 772 2569
> > Email: gkousiou@mail.ntua.gr
> > Site: http://users.ntua.gr/gkousiou/
> >
> > National Technical University of Athens
> > 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
> >
>
>
Re: Any related paper on how to resolve hadoop SPOF issue?
Posted by Amandeep Khurana <am...@gmail.com>.
Andy,
We did run HBase on Ceph but were not able to get a stable performance for
sustained periods. It would give good throughput for a while and the client
would crash soon thereafter. However, this was more than a year ago and a
re-evaluation should be done before commenting further.
-Amandeep
On Mon, Aug 29, 2011 at 8:09 PM, Alex Nelson <aj...@cs.ucsc.edu> wrote:
> Hi Andy,
>
> When I ran Hadoop on Ceph, I was able to complete a run of Terasort. We
> didn't run HBase.
>
> After I submitted a patch to the Ceph mailing list, Noah Watkins (CC'd)
> took over developing the file system class in Hadoop's code base. I don't
> know what's become of the work since then. The JIRA is here:
> https://issues.apache.org/jira/browse/HADOOP-6779
>
> The last I heard, which was a while ago, there was a snag with the URI
> scheme (ceph:// vs. file:// or something similar) which had a subtle but
> challenging complexity. Noah is more able to answer questions at this
> point. He has posted Java RADOS bindings to the Ceph mailing list recently,
> but I'm not sure what he has been up to aside from that.
>
> --Alex
>
>
> On Aug 27, 2011, at 02:07 , Andrew Purtell wrote:
>
> > Hi Alex,
> >
> >> UC Santa Cruz contributed a ;login: article describing replacing HDFS
> with
> >> Ceph. (I was one of the authors.) One of the key architectural
> advantages of
> >> Ceph over HDFS is that Ceph distributes its metadata service over
> multiple
> >> metadata servers.
> >
> > Were you able to maintain stable operation with Ceph at a high load? Did
> you try running HBase?
> >
> > Best regards,
> >
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
> >
> >
> > ----- Original Message -----
> >> From: Alex Nelson <aj...@cs.ucsc.edu>
> >> To: George Kousiouris <gk...@mail.ntua.gr>
> >> Cc: common-dev@hadoop.apache.org; common-user@hadoop.apache.org; Carlos
> Maltzahn <ca...@soe.ucsc.edu>
> >> Sent: Friday, August 26, 2011 2:44 AM
> >> Subject: Re: Any related paper on how to resolve hadoop SPOF issue?
> >>
> >> Hi George,
> >>
> >> UC Santa Cruz contributed a ;login: article describing replacing HDFS
> with
> >> Ceph. (I was one of the authors.) One of the key architectural
> advantages of
> >> Ceph over HDFS is that Ceph distributes its metadata service over
> multiple
> >> metadata servers.
> >>
> >> I hope that helps.
> >>
> >> --Alex
> >>
> >>
> >> On Aug 25, 2011, at 02:54 , George Kousiouris wrote:
> >>
> >>>
> >>> Hi,
> >>>
> >>> many thanks!!
> >>>
> >>> I also found this one, which has some related work also with other
> efforts,
> >> in case it is helpful for someone else:
> >>>
> >>
> https://ritdml.rit.edu/bitstream/handle/1850/13321/ATalwalkarThesis1-2011.pdf?sequence=1
> >>>
> >>>
> >>> BR,
> >>> George
> >>>
> >>> On 8/25/2011 12:46 PM, Nan Zhu wrote:
> >>>> Hope it helps
> >>>>
> >>>> http://www.springerlink.com/content/h17r882710314147/
> >>>>
> >>>> Best,
> >>>>
> >>>> Nan
> >>>>
> >>>> On Thu, Aug 25, 2011 at 5:43 PM, George
> >> Kousiouris<gk...@mail.ntua.gr>wrote:
> >>>>
> >>>>> Hi guys,
> >>>>>
> >>>>> We are currently in the process of writing a paper regarding hadoop
> >> and we
> >>>>> would like to reference any attempt to remove the single point of
> >> failure of
> >>>>> the Namenode. We have found in various presentations some efforts
> >> (like
> >>>>> dividing the namespace between more than one namenodes if i
> >> remember
> >>>>> correctly) but the search for a concrete paper on the issue came to
> >> nothing.
> >>>>>
> >>>>> Is anyone aware or has participated in such an effort?
> >>>>>
> >>>>> Thanks,
> >>>>> George
> >>>>>
> >>>>> --
> >>>>>
> >>>>> ---------------------------
> >>>>>
> >>>>> George Kousiouris
> >>>>> Electrical and Computer Engineer
> >>>>> Division of Communications,
> >>>>> Electronics and Information Engineering
> >>>>> School of Electrical and Computer Engineering
> >>>>> Tel: +30 210 772 2546
> >>>>> Mobile: +30 6939354121
> >>>>> Fax: +30 210 772 2569
> >>>>> Email: gkousiou@mail.ntua.gr
> >>>>> Site: http://users.ntua.gr/gkousiou/
> >>>>>
> >>>>> National Technical University of Athens
> >>>>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>> --
> >>>
> >>> ---------------------------
> >>>
> >>> George Kousiouris
> >>> Electrical and Computer Engineer
> >>> Division of Communications,
> >>> Electronics and Information Engineering
> >>> School of Electrical and Computer Engineering
> >>> Tel: +30 210 772 2546
> >>> Mobile: +30 6939354121
> >>> Fax: +30 210 772 2569
> >>> Email: gkousiou@mail.ntua.gr
> >>> Site: http://users.ntua.gr/gkousiou/
> >>>
> >>> National Technical University of Athens
> >>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
> >>>
> >>
>
>
Re: Any related paper on how to resolve hadoop SPOF issue?
Posted by Alex Nelson <aj...@cs.ucsc.edu>.
Hi Andy,
When I ran Hadoop on Ceph, I was able to complete a run of Terasort. We didn't run HBase.
After I submitted a patch to the Ceph mailing list, Noah Watkins (CC'd) took over developing the file system class in Hadoop's code base. I don't know what's become of the work since then. The JIRA is here:
https://issues.apache.org/jira/browse/HADOOP-6779
The last I heard, which was a while ago, there was a snag with the URI scheme (ceph:// vs. file:// or something similar) which had a subtle but challenging complexity. Noah is more able to answer questions at this point. He has posted Java RADOS bindings to the Ceph mailing list recently, but I'm not sure what he has been up to aside from that.
--Alex
On Aug 27, 2011, at 02:07 , Andrew Purtell wrote:
> Hi Alex,
>
>> UC Santa Cruz contributed a ;login: article describing replacing HDFS with
>> Ceph. (I was one of the authors.) One of the key architectural advantages of
>> Ceph over HDFS is that Ceph distributes its metadata service over multiple
>> metadata servers.
>
> Were you able to maintain stable operation with Ceph at a high load? Did you try running HBase?
>
> Best regards,
>
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
>
>
> ----- Original Message -----
>> From: Alex Nelson <aj...@cs.ucsc.edu>
>> To: George Kousiouris <gk...@mail.ntua.gr>
>> Cc: common-dev@hadoop.apache.org; common-user@hadoop.apache.org; Carlos Maltzahn <ca...@soe.ucsc.edu>
>> Sent: Friday, August 26, 2011 2:44 AM
>> Subject: Re: Any related paper on how to resolve hadoop SPOF issue?
>>
>> Hi George,
>>
>> UC Santa Cruz contributed a ;login: article describing replacing HDFS with
>> Ceph. (I was one of the authors.) One of the key architectural advantages of
>> Ceph over HDFS is that Ceph distributes its metadata service over multiple
>> metadata servers.
>>
>> I hope that helps.
>>
>> --Alex
>>
>>
>> On Aug 25, 2011, at 02:54 , George Kousiouris wrote:
>>
>>>
>>> Hi,
>>>
>>> many thanks!!
>>>
>>> I also found this one, which has some related work also with other efforts,
>> in case it is helpful for someone else:
>>>
>> https://ritdml.rit.edu/bitstream/handle/1850/13321/ATalwalkarThesis1-2011.pdf?sequence=1
>>>
>>>
>>> BR,
>>> George
>>>
>>> On 8/25/2011 12:46 PM, Nan Zhu wrote:
>>>> Hope it helps
>>>>
>>>> http://www.springerlink.com/content/h17r882710314147/
>>>>
>>>> Best,
>>>>
>>>> Nan
>>>>
>>>> On Thu, Aug 25, 2011 at 5:43 PM, George
>> Kousiouris<gk...@mail.ntua.gr>wrote:
>>>>
>>>>> Hi guys,
>>>>>
>>>>> We are currently in the process of writing a paper regarding hadoop
>> and we
>>>>> would like to reference any attempt to remove the single point of
>> failure of
>>>>> the Namenode. We have found in various presentations some efforts
>> (like
>>>>> dividing the namespace between more than one namenodes if i
>> remember
>>>>> correctly) but the search for a concrete paper on the issue came to
>> nothing.
>>>>>
>>>>> Is anyone aware or has participated in such an effort?
>>>>>
>>>>> Thanks,
>>>>> George
>>>>>
>>>>> --
>>>>>
>>>>> ---------------------------
>>>>>
>>>>> George Kousiouris
>>>>> Electrical and Computer Engineer
>>>>> Division of Communications,
>>>>> Electronics and Information Engineering
>>>>> School of Electrical and Computer Engineering
>>>>> Tel: +30 210 772 2546
>>>>> Mobile: +30 6939354121
>>>>> Fax: +30 210 772 2569
>>>>> Email: gkousiou@mail.ntua.gr
>>>>> Site: http://users.ntua.gr/gkousiou/
>>>>>
>>>>> National Technical University of Athens
>>>>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> ---------------------------
>>>
>>> George Kousiouris
>>> Electrical and Computer Engineer
>>> Division of Communications,
>>> Electronics and Information Engineering
>>> School of Electrical and Computer Engineering
>>> Tel: +30 210 772 2546
>>> Mobile: +30 6939354121
>>> Fax: +30 210 772 2569
>>> Email: gkousiou@mail.ntua.gr
>>> Site: http://users.ntua.gr/gkousiou/
>>>
>>> National Technical University of Athens
>>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>>>
>>
Re: Any related paper on how to resolve hadoop SPOF issue?
Posted by Andrew Purtell <ap...@apache.org>.
Hi Alex,
> UC Santa Cruz contributed a ;login: article describing replacing HDFS with
> Ceph. (I was one of the authors.) One of the key architectural advantages of
> Ceph over HDFS is that Ceph distributes its metadata service over multiple
> metadata servers.
Were you able to maintain stable operation with Ceph at a high load? Did you try running HBase?
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
----- Original Message -----
> From: Alex Nelson <aj...@cs.ucsc.edu>
> To: George Kousiouris <gk...@mail.ntua.gr>
> Cc: common-dev@hadoop.apache.org; common-user@hadoop.apache.org; Carlos Maltzahn <ca...@soe.ucsc.edu>
> Sent: Friday, August 26, 2011 2:44 AM
> Subject: Re: Any related paper on how to resolve hadoop SPOF issue?
>
> Hi George,
>
> UC Santa Cruz contributed a ;login: article describing replacing HDFS with
> Ceph. (I was one of the authors.) One of the key architectural advantages of
> Ceph over HDFS is that Ceph distributes its metadata service over multiple
> metadata servers.
>
> I hope that helps.
>
> --Alex
>
>
> On Aug 25, 2011, at 02:54 , George Kousiouris wrote:
>
>>
>> Hi,
>>
>> many thanks!!
>>
>> I also found this one, which has some related work also with other efforts,
> in case it is helpful for someone else:
>>
> https://ritdml.rit.edu/bitstream/handle/1850/13321/ATalwalkarThesis1-2011.pdf?sequence=1
>>
>>
>> BR,
>> George
>>
>> On 8/25/2011 12:46 PM, Nan Zhu wrote:
>>> Hope it helps
>>>
>>> http://www.springerlink.com/content/h17r882710314147/
>>>
>>> Best,
>>>
>>> Nan
>>>
>>> On Thu, Aug 25, 2011 at 5:43 PM, George
> Kousiouris<gk...@mail.ntua.gr>wrote:
>>>
>>>> Hi guys,
>>>>
>>>> We are currently in the process of writing a paper regarding hadoop
> and we
>>>> would like to reference any attempt to remove the single point of
> failure of
>>>> the Namenode. We have found in various presentations some efforts
> (like
>>>> dividing the namespace between more than one namenodes if i
> remember
>>>> correctly) but the search for a concrete paper on the issue came to
> nothing.
>>>>
>>>> Is anyone aware or has participated in such an effort?
>>>>
>>>> Thanks,
>>>> George
>>>>
>>>> --
>>>>
>>>> ---------------------------
>>>>
>>>> George Kousiouris
>>>> Electrical and Computer Engineer
>>>> Division of Communications,
>>>> Electronics and Information Engineering
>>>> School of Electrical and Computer Engineering
>>>> Tel: +30 210 772 2546
>>>> Mobile: +30 6939354121
>>>> Fax: +30 210 772 2569
>>>> Email: gkousiou@mail.ntua.gr
>>>> Site: http://users.ntua.gr/gkousiou/
>>>>
>>>> National Technical University of Athens
>>>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>>>>
>>>>
>>>
>>
>>
>> --
>>
>> ---------------------------
>>
>> George Kousiouris
>> Electrical and Computer Engineer
>> Division of Communications,
>> Electronics and Information Engineering
>> School of Electrical and Computer Engineering
>> Tel: +30 210 772 2546
>> Mobile: +30 6939354121
>> Fax: +30 210 772 2569
>> Email: gkousiou@mail.ntua.gr
>> Site: http://users.ntua.gr/gkousiou/
>>
>> National Technical University of Athens
>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>>
>
Re: Any related paper on how to resolve hadoop SPOF issue?
Posted by "M. C. Srivas" <mc...@gmail.com>.
Are there any performance benchmarks available for Ceph? (with Hadoop,
without, both?)
On Thu, Aug 25, 2011 at 11:44 AM, Alex Nelson <aj...@cs.ucsc.edu> wrote:
> Hi George,
>
> UC Santa Cruz contributed a ;login: article describing replacing HDFS with
> Ceph. (I was one of the authors.) One of the key architectural advantages
> of Ceph over HDFS is that Ceph distributes its metadata service over
> multiple metadata servers.
>
> I hope that helps.
>
> --Alex
>
>
> On Aug 25, 2011, at 02:54 , George Kousiouris wrote:
>
> >
> > Hi,
> >
> > many thanks!!
> >
> > I also found this one, which has some related work also with other
> efforts, in case it is helpful for someone else:
> >
> https://ritdml.rit.edu/bitstream/handle/1850/13321/ATalwalkarThesis1-2011.pdf?sequence=1
> >
> >
> > BR,
> > George
> >
> > On 8/25/2011 12:46 PM, Nan Zhu wrote:
> >> Hope it helps
> >>
> >> http://www.springerlink.com/content/h17r882710314147/
> >>
> >> Best,
> >>
> >> Nan
> >>
> >> On Thu, Aug 25, 2011 at 5:43 PM, George Kousiouris<
> gkousiou@mail.ntua.gr>wrote:
> >>
> >>> Hi guys,
> >>>
> >>> We are currently in the process of writing a paper regarding hadoop and
> we
> >>> would like to reference any attempt to remove the single point of
> failure of
> >>> the Namenode. We have found in various presentations some efforts (like
> >>> dividing the namespace between more than one namenodes if i remember
> >>> correctly) but the search for a concrete paper on the issue came to
> nothing.
> >>>
> >>> Is anyone aware or has participated in such an effort?
> >>>
> >>> Thanks,
> >>> George
> >>>
> >>> --
> >>>
> >>> ---------------------------
> >>>
> >>> George Kousiouris
> >>> Electrical and Computer Engineer
> >>> Division of Communications,
> >>> Electronics and Information Engineering
> >>> School of Electrical and Computer Engineering
> >>> Tel: +30 210 772 2546
> >>> Mobile: +30 6939354121
> >>> Fax: +30 210 772 2569
> >>> Email: gkousiou@mail.ntua.gr
> >>> Site: http://users.ntua.gr/gkousiou/
> >>>
> >>> National Technical University of Athens
> >>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
> >>>
> >>>
> >>
> >
> >
> > --
> >
> > ---------------------------
> >
> > George Kousiouris
> > Electrical and Computer Engineer
> > Division of Communications,
> > Electronics and Information Engineering
> > School of Electrical and Computer Engineering
> > Tel: +30 210 772 2546
> > Mobile: +30 6939354121
> > Fax: +30 210 772 2569
> > Email: gkousiou@mail.ntua.gr
> > Site: http://users.ntua.gr/gkousiou/
> >
> > National Technical University of Athens
> > 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
> >
>
>
Re: Any related paper on how to resolve hadoop SPOF issue?
Posted by Alex Nelson <aj...@cs.ucsc.edu>.
Hi George,
UC Santa Cruz contributed a ;login: article describing replacing HDFS with Ceph. (I was one of the authors.) One of the key architectural advantages of Ceph over HDFS is that Ceph distributes its metadata service over multiple metadata servers.
I hope that helps.
--Alex
On Aug 25, 2011, at 02:54 , George Kousiouris wrote:
>
> Hi,
>
> many thanks!!
>
> I also found this one, which has some related work also with other efforts, in case it is helpful for someone else:
> https://ritdml.rit.edu/bitstream/handle/1850/13321/ATalwalkarThesis1-2011.pdf?sequence=1
>
>
> BR,
> George
>
> On 8/25/2011 12:46 PM, Nan Zhu wrote:
>> Hope it helps
>>
>> http://www.springerlink.com/content/h17r882710314147/
>>
>> Best,
>>
>> Nan
>>
>> On Thu, Aug 25, 2011 at 5:43 PM, George Kousiouris<gk...@mail.ntua.gr>wrote:
>>
>>> Hi guys,
>>>
>>> We are currently in the process of writing a paper regarding hadoop and we
>>> would like to reference any attempt to remove the single point of failure of
>>> the Namenode. We have found in various presentations some efforts (like
>>> dividing the namespace between more than one namenodes if i remember
>>> correctly) but the search for a concrete paper on the issue came to nothing.
>>>
>>> Is anyone aware or has participated in such an effort?
>>>
>>> Thanks,
>>> George
>>>
>>> --
>>>
>>> ---------------------------
>>>
>>> George Kousiouris
>>> Electrical and Computer Engineer
>>> Division of Communications,
>>> Electronics and Information Engineering
>>> School of Electrical and Computer Engineering
>>> Tel: +30 210 772 2546
>>> Mobile: +30 6939354121
>>> Fax: +30 210 772 2569
>>> Email: gkousiou@mail.ntua.gr
>>> Site: http://users.ntua.gr/gkousiou/
>>>
>>> National Technical University of Athens
>>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>>>
>>>
>>
>
>
> --
>
> ---------------------------
>
> George Kousiouris
> Electrical and Computer Engineer
> Division of Communications,
> Electronics and Information Engineering
> School of Electrical and Computer Engineering
> Tel: +30 210 772 2546
> Mobile: +30 6939354121
> Fax: +30 210 772 2569
> Email: gkousiou@mail.ntua.gr
> Site: http://users.ntua.gr/gkousiou/
>
> National Technical University of Athens
> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>
Re: Any related paper on how to resolve hadoop SPOF issue?
Posted by Alex Nelson <aj...@cs.ucsc.edu>.
Hi George,
UC Santa Cruz contributed a ;login: article describing replacing HDFS with Ceph. (I was one of the authors.) One of the key architectural advantages of Ceph over HDFS is that Ceph distributes its metadata service over multiple metadata servers.
I hope that helps.
--Alex
On Aug 25, 2011, at 02:54 , George Kousiouris wrote:
>
> Hi,
>
> many thanks!!
>
> I also found this one, which has some related work also with other efforts, in case it is helpful for someone else:
> https://ritdml.rit.edu/bitstream/handle/1850/13321/ATalwalkarThesis1-2011.pdf?sequence=1
>
>
> BR,
> George
>
> On 8/25/2011 12:46 PM, Nan Zhu wrote:
>> Hope it helps
>>
>> http://www.springerlink.com/content/h17r882710314147/
>>
>> Best,
>>
>> Nan
>>
>> On Thu, Aug 25, 2011 at 5:43 PM, George Kousiouris<gk...@mail.ntua.gr>wrote:
>>
>>> Hi guys,
>>>
>>> We are currently in the process of writing a paper regarding hadoop and we
>>> would like to reference any attempt to remove the single point of failure of
>>> the Namenode. We have found in various presentations some efforts (like
>>> dividing the namespace between more than one namenodes if i remember
>>> correctly) but the search for a concrete paper on the issue came to nothing.
>>>
>>> Is anyone aware or has participated in such an effort?
>>>
>>> Thanks,
>>> George
>>>
>>> --
>>>
>>> ---------------------------
>>>
>>> George Kousiouris
>>> Electrical and Computer Engineer
>>> Division of Communications,
>>> Electronics and Information Engineering
>>> School of Electrical and Computer Engineering
>>> Tel: +30 210 772 2546
>>> Mobile: +30 6939354121
>>> Fax: +30 210 772 2569
>>> Email: gkousiou@mail.ntua.gr
>>> Site: http://users.ntua.gr/gkousiou/
>>>
>>> National Technical University of Athens
>>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>>>
>>>
>>
>
>
> --
>
> ---------------------------
>
> George Kousiouris
> Electrical and Computer Engineer
> Division of Communications,
> Electronics and Information Engineering
> School of Electrical and Computer Engineering
> Tel: +30 210 772 2546
> Mobile: +30 6939354121
> Fax: +30 210 772 2569
> Email: gkousiou@mail.ntua.gr
> Site: http://users.ntua.gr/gkousiou/
>
> National Technical University of Athens
> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>
Re: Any related paper on how to resolve hadoop SPOF issue?
Posted by George Kousiouris <gk...@mail.ntua.gr>.
Hi,
many thanks!!
I also found this one, which has some related work also with other
efforts, in case it is helpful for someone else:
https://ritdml.rit.edu/bitstream/handle/1850/13321/ATalwalkarThesis1-2011.pdf?sequence=1
BR,
George
On 8/25/2011 12:46 PM, Nan Zhu wrote:
> Hope it helps
>
> http://www.springerlink.com/content/h17r882710314147/
>
> Best,
>
> Nan
>
> On Thu, Aug 25, 2011 at 5:43 PM, George Kousiouris<gk...@mail.ntua.gr>wrote:
>
>> Hi guys,
>>
>> We are currently in the process of writing a paper regarding hadoop and we
>> would like to reference any attempt to remove the single point of failure of
>> the Namenode. We have found in various presentations some efforts (like
>> dividing the namespace between more than one namenodes if i remember
>> correctly) but the search for a concrete paper on the issue came to nothing.
>>
>> Is anyone aware or has participated in such an effort?
>>
>> Thanks,
>> George
>>
>> --
>>
>> ---------------------------
>>
>> George Kousiouris
>> Electrical and Computer Engineer
>> Division of Communications,
>> Electronics and Information Engineering
>> School of Electrical and Computer Engineering
>> Tel: +30 210 772 2546
>> Mobile: +30 6939354121
>> Fax: +30 210 772 2569
>> Email: gkousiou@mail.ntua.gr
>> Site: http://users.ntua.gr/gkousiou/
>>
>> National Technical University of Athens
>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>>
>>
>
--
---------------------------
George Kousiouris
Electrical and Computer Engineer
Division of Communications,
Electronics and Information Engineering
School of Electrical and Computer Engineering
Tel: +30 210 772 2546
Mobile: +30 6939354121
Fax: +30 210 772 2569
Email: gkousiou@mail.ntua.gr
Site: http://users.ntua.gr/gkousiou/
National Technical University of Athens
9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
Re: Any related paper on how to resolve hadoop SPOF issue?
Posted by George Kousiouris <gk...@mail.ntua.gr>.
Hi,
many thanks!!
I also found this one, which has some related work also with other
efforts, in case it is helpful for someone else:
https://ritdml.rit.edu/bitstream/handle/1850/13321/ATalwalkarThesis1-2011.pdf?sequence=1
BR,
George
On 8/25/2011 12:46 PM, Nan Zhu wrote:
> Hope it helps
>
> http://www.springerlink.com/content/h17r882710314147/
>
> Best,
>
> Nan
>
> On Thu, Aug 25, 2011 at 5:43 PM, George Kousiouris<gk...@mail.ntua.gr>wrote:
>
>> Hi guys,
>>
>> We are currently in the process of writing a paper regarding hadoop and we
>> would like to reference any attempt to remove the single point of failure of
>> the Namenode. We have found in various presentations some efforts (like
>> dividing the namespace between more than one namenodes if i remember
>> correctly) but the search for a concrete paper on the issue came to nothing.
>>
>> Is anyone aware or has participated in such an effort?
>>
>> Thanks,
>> George
>>
>> --
>>
>> ---------------------------
>>
>> George Kousiouris
>> Electrical and Computer Engineer
>> Division of Communications,
>> Electronics and Information Engineering
>> School of Electrical and Computer Engineering
>> Tel: +30 210 772 2546
>> Mobile: +30 6939354121
>> Fax: +30 210 772 2569
>> Email: gkousiou@mail.ntua.gr
>> Site: http://users.ntua.gr/gkousiou/
>>
>> National Technical University of Athens
>> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>>
>>
>
--
---------------------------
George Kousiouris
Electrical and Computer Engineer
Division of Communications,
Electronics and Information Engineering
School of Electrical and Computer Engineering
Tel: +30 210 772 2546
Mobile: +30 6939354121
Fax: +30 210 772 2569
Email: gkousiou@mail.ntua.gr
Site: http://users.ntua.gr/gkousiou/
National Technical University of Athens
9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
Re: Any related paper on how to resolve hadoop SPOF issue?
Posted by Nan Zhu <zh...@gmail.com>.
Hope it helps
http://www.springerlink.com/content/h17r882710314147/
Best,
Nan
On Thu, Aug 25, 2011 at 5:43 PM, George Kousiouris <gk...@mail.ntua.gr>wrote:
>
> Hi guys,
>
> We are currently in the process of writing a paper regarding hadoop and we
> would like to reference any attempt to remove the single point of failure of
> the Namenode. We have found in various presentations some efforts (like
> dividing the namespace between more than one namenodes if i remember
> correctly) but the search for a concrete paper on the issue came to nothing.
>
> Is anyone aware or has participated in such an effort?
>
> Thanks,
> George
>
> --
>
> ---------------------------
>
> George Kousiouris
> Electrical and Computer Engineer
> Division of Communications,
> Electronics and Information Engineering
> School of Electrical and Computer Engineering
> Tel: +30 210 772 2546
> Mobile: +30 6939354121
> Fax: +30 210 772 2569
> Email: gkousiou@mail.ntua.gr
> Site: http://users.ntua.gr/gkousiou/
>
> National Technical University of Athens
> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>
>
--
Nan Zhu
School of Electronic, Information and Electrical Engineering,229
Shanghai Jiao Tong University
800,Dongchuan Road,Shanghai,China
E-Mail: zhunansjtu@gmail.com
Re: Any related paper on how to resolve hadoop SPOF issue?
Posted by Uma Maheswara Rao G 72686 <ma...@huawei.com>.
Hi,
Community started working on NameNode High availabily.
you can refer : https://issues.apache.org/jira/browse/HDFS-1623
regards,
Uma
----- Original Message -----
From: George Kousiouris <gk...@mail.ntua.gr>
Date: Thursday, August 25, 2011 3:15 pm
Subject: Any related paper on how to resolve hadoop SPOF issue?
To: "common-user@hadoop.apache.org" <co...@hadoop.apache.org>, common-dev@hadoop.apache.org
>
> Hi guys,
>
> We are currently in the process of writing a paper regarding
> hadoop and
> we would like to reference any attempt to remove the single point
> of
> failure of the Namenode. We have found in various presentations
> some
> efforts (like dividing the namespace between more than one
> namenodes if
> i remember correctly) but the search for a concrete paper on the
> issue
> came to nothing.
>
> Is anyone aware or has participated in such an effort?
>
> Thanks,
> George
>
> --
>
> ---------------------------
>
> George Kousiouris
> Electrical and Computer Engineer
> Division of Communications,
> Electronics and Information Engineering
> School of Electrical and Computer Engineering
> Tel: +30 210 772 2546
> Mobile: +30 6939354121
> Fax: +30 210 772 2569
> Email: gkousiou@mail.ntua.gr
> Site: http://users.ntua.gr/gkousiou/
>
> National Technical University of Athens
> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>
>
Re: Any related paper on how to resolve hadoop SPOF issue?
Posted by Nan Zhu <zh...@gmail.com>.
Hope it helps
http://www.springerlink.com/content/h17r882710314147/
Best,
Nan
On Thu, Aug 25, 2011 at 5:43 PM, George Kousiouris <gk...@mail.ntua.gr>wrote:
>
> Hi guys,
>
> We are currently in the process of writing a paper regarding hadoop and we
> would like to reference any attempt to remove the single point of failure of
> the Namenode. We have found in various presentations some efforts (like
> dividing the namespace between more than one namenodes if i remember
> correctly) but the search for a concrete paper on the issue came to nothing.
>
> Is anyone aware or has participated in such an effort?
>
> Thanks,
> George
>
> --
>
> ---------------------------
>
> George Kousiouris
> Electrical and Computer Engineer
> Division of Communications,
> Electronics and Information Engineering
> School of Electrical and Computer Engineering
> Tel: +30 210 772 2546
> Mobile: +30 6939354121
> Fax: +30 210 772 2569
> Email: gkousiou@mail.ntua.gr
> Site: http://users.ntua.gr/gkousiou/
>
> National Technical University of Athens
> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>
>
--
Nan Zhu
School of Electronic, Information and Electrical Engineering,229
Shanghai Jiao Tong University
800,Dongchuan Road,Shanghai,China
E-Mail: zhunansjtu@gmail.com
Re: Any related paper on how to resolve hadoop SPOF issue?
Posted by Uma Maheswara Rao G 72686 <ma...@huawei.com>.
Hi,
Community started working on NameNode High availabily.
you can refer : https://issues.apache.org/jira/browse/HDFS-1623
regards,
Uma
----- Original Message -----
From: George Kousiouris <gk...@mail.ntua.gr>
Date: Thursday, August 25, 2011 3:15 pm
Subject: Any related paper on how to resolve hadoop SPOF issue?
To: "common-user@hadoop.apache.org" <co...@hadoop.apache.org>, common-dev@hadoop.apache.org
>
> Hi guys,
>
> We are currently in the process of writing a paper regarding
> hadoop and
> we would like to reference any attempt to remove the single point
> of
> failure of the Namenode. We have found in various presentations
> some
> efforts (like dividing the namespace between more than one
> namenodes if
> i remember correctly) but the search for a concrete paper on the
> issue
> came to nothing.
>
> Is anyone aware or has participated in such an effort?
>
> Thanks,
> George
>
> --
>
> ---------------------------
>
> George Kousiouris
> Electrical and Computer Engineer
> Division of Communications,
> Electronics and Information Engineering
> School of Electrical and Computer Engineering
> Tel: +30 210 772 2546
> Mobile: +30 6939354121
> Fax: +30 210 772 2569
> Email: gkousiou@mail.ntua.gr
> Site: http://users.ntua.gr/gkousiou/
>
> National Technical University of Athens
> 9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>
>