You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Ted Yu <yu...@gmail.com> on 2011/12/25 00:13:01 UTC
hdfs-1623 Was: ceph and hbase.
Andrey:
A quick search led me to https://github.com/toddlipcon where you would be
able to find:
https://github.com/toddlipcon/hadoop-common
Cheers
On Sat, Dec 24, 2011 at 12:39 PM, Andrey Stepachev <oc...@gmail.com> wrote:
> 23 декабря 2011 г. 22:48 пользователь Todd Lipcon <todd@cloudera.com
> >написал:
>
> > On Tue, Dec 20, 2011 at 12:56 AM, Andrey Stepachev <oc...@gmail.com>
> > wrote:
> > > I see, that most issues addressed to HA branch.
> > > What version this branch is applied to? 0.22 or
> > > i should build hadoop from sources?
> >
> > It's currently only available in the HDFS-1623 branch, which is in
> > active development and definitely shouldn't be used unless you plan to
> > help us finish debugging it :) There are still some known bugs and
> > deficiencies to address before we merge.
> >
>
> Looks like this is one of possible scenarios :). We need
> some solution for HA, but no one (except MapR) still exists.
> So we must use MapR (costs, closed) or debug some
> not-ready-yet solution.
>
>
> >
> > The plan is to merge it into 0.23 but not 0.22 - it will hopefully be
> > merged in January. We'll also make it available in CDH4 beta releases
> > early next year.
> >
>
> Sounds good. But still can't find such branch in cloudera (or any other)
> github
> repositories.
> Is it publicly available right now, or it in some private repository?
>
>
> >
> > -Todd
> >
> > >
> > > 20 декабря 2011 г. 11:46 пользователь Todd Lipcon <todd@cloudera.com
> > >написал:
> > >
> > >> Hi Andrey,
> > >>
> > >> I don't think anyone has ever run Ceph in a production use case, much
> > >> less run it in a production use case with HBase.
> > >>
> > >> The SPOF problem is currently being addressed and will be available
> > >> for production use by the middle of next calendar year. It will be
> > >> available for preview/beta usage in Q1 (it's already working in
> > >> development setups - feel free to follow HDFS-1623 for details)
> > >>
> > >> I think you're way better off waiting for the HA NameNode support in
> > >> HDFS than going with an unproven FS like Ceph. Ceph certainly has
> > >> promise and is a nice design, but it hasn't had the years of pounding
> > >> that HDFS has.
> > >>
> > >> -Todd
> > >>
> > >> On Mon, Dec 19, 2011 at 10:23 PM, Andrey Stepachev <oc...@gmail.com>
> > >> wrote:
> > >> > Hi all.
> > >> >
> > >> > I have requirements to use hbase in several datacenters.
> > >> > But HDFS has as SPOF, so we can't use it.
> > >> > I plan to use ceph as the file system for hbase.
> > >> >
> > >> > In general, interested in issues of:
> > >> > a) use the hadoop + сeph in the production environment
> > >> > b) using hbase + ceph in the production environment
> > >> >
> > >> > In particular:
> > >> > Ceph Wiki states that the integration of hadoop/ceph has the problem
> > in
> > >> the
> > >> > form of modification dates of the files.
> > >> > Does hbase uses modification times for some critical tasks?
> > >> >
> > >> > --
> > >> > Andrey.
> > >>
> > >>
> > >>
> > >> --
> > >> Todd Lipcon
> > >> Software Engineer, Cloudera
> > >>
> > >
> > >
> > >
> > > --
> > > Андрей.
> >
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>
>
>
> --
> Андрей.
>
Re: hdfs-1623 Was: ceph and hbase.
Posted by Andrey Stepachev <oc...@gmail.com>.
Thanks Todd. I'll check svn.
27 декабря 2011 г. 5:00 пользователь Todd Lipcon <to...@cloudera.com>написал:
> That's my github but I don't regularly push there -- so it's a bit out of
> date. Working from the apache svn is probably the best bet
>
> Todd
>
> On Sunday, December 25, 2011, Andrey Stepachev <oc...@gmail.com> wrote:
> > Thanks!
> >
> > 25 декабря 2011 г. 3:13 пользователь Ted Yu <yu...@gmail.com>
> написал:
> >
> >> Andrey:
> >> A quick search led me to https://github.com/toddlipcon where you would
> be
> >> able to find:
> >> https://github.com/toddlipcon/hadoop-common
> >>
> >> Cheers
> >>
> >> On Sat, Dec 24, 2011 at 12:39 PM, Andrey Stepachev <oc...@gmail.com>
> >> wrote:
> >>
> >> > 23 декабря 2011 г. 22:48 пользователь Todd Lipcon <todd@cloudera.com
> >> > >написал:
> >> >
> >> > > On Tue, Dec 20, 2011 at 12:56 AM, Andrey Stepachev <
> octo47@gmail.com>
> >> > > wrote:
> >> > > > I see, that most issues addressed to HA branch.
> >> > > > What version this branch is applied to? 0.22 or
> >> > > > i should build hadoop from sources?
> >> > >
> >> > > It's currently only available in the HDFS-1623 branch, which is in
> >> > > active development and definitely shouldn't be used unless you plan
> to
> >> > > help us finish debugging it :) There are still some known bugs and
> >> > > deficiencies to address before we merge.
> >> > >
> >> >
> >> > Looks like this is one of possible scenarios :). We need
> >> > some solution for HA, but no one (except MapR) still exists.
> >> > So we must use MapR (costs, closed) or debug some
> >> > not-ready-yet solution.
> >> >
> >> >
> >> > >
> >> > > The plan is to merge it into 0.23 but not 0.22 - it will hopefully
> be
> >> > > merged in January. We'll also make it available in CDH4 beta
> releases
> >> > > early next year.
> >> > >
> >> >
> >> > Sounds good. But still can't find such branch in cloudera (or any
> other)
> >> > github
> >> > repositories.
> >> > Is it publicly available right now, or it in some private repository?
> >> >
> >> >
> >> > >
> >> > > -Todd
> >> > >
> >> > > >
> >> > > > 20 декабря 2011 г. 11:46 пользователь Todd Lipcon <
> todd@cloudera.com
> >> > > >написал:
> >> > > >
> >> > > >> Hi Andrey,
> >> > > >>
> >> > > >> I don't think anyone has ever run Ceph in a production use case,
> >> much
> >> > > >> less run it in a production use case with HBase.
> >> > > >>
> >> > > >> The SPOF problem is currently being addressed and will be
> available
> >> > > >> for production use by the middle of next calendar year. It will
> be
> >> > > >> available for preview/beta usage in Q1 (it's already working in
> >> > > >> development setups - feel free to follow HDFS-1623 for details)
> >> > > >>
> >> > > >> I think you're way better off waiting for the HA NameNode support
> in
> >> > > >> HDFS than going with an unproven FS like Ceph. Ceph certainly has
> >> > > >> promise and is a nice design, but it hasn't had the years of
> >> pounding
> >> > > >> that HDFS has.
> >> > > >>
> >> > > >> -Todd
> >> > > >>
> >> > > >> On Mon, Dec 19, 2011 at 10:23 PM, Andrey Stepachev <
> >> octo47@gmail.com>
> >> > > >> wrote:
> >> > > >> > Hi all.
> >> > > >> >
> >> > > >> > I have requirements to use hbase in several datacenters.
> >> > > >> > But HDFS has as SPOF, so we can't use it.
> >> > > >> > I plan to use ceph as the file system for hbase.
> >> > > >> >
> >> > > >> > In general, interested in issues of:
> >> > > >> > a) use the hadoop + сeph in the production environment
> >> > > >> > b) using hbase + ceph in the production environment
> >> > > >> >
> >> > > >> > In particular:
> >> > > >> > Ceph Wiki states that the integration of hadoop/ceph has the
> >> problem
> >> > > in
> >> > > >> the
> >> > > >> > form of modification dates of the files.
> >> > > >> > Does hbase uses modification times f--
> > Андрей.
> >
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
--
Андрей.
Re: hdfs-1623 Was: ceph and hbase.
Posted by Todd Lipcon <to...@cloudera.com>.
That's my github but I don't regularly push there -- so it's a bit out of
date. Working from the apache svn is probably the best bet
Todd
On Sunday, December 25, 2011, Andrey Stepachev <oc...@gmail.com> wrote:
> Thanks!
>
> 25 декабря 2011 г. 3:13 пользователь Ted Yu <yu...@gmail.com> написал:
>
>> Andrey:
>> A quick search led me to https://github.com/toddlipcon where you would be
>> able to find:
>> https://github.com/toddlipcon/hadoop-common
>>
>> Cheers
>>
>> On Sat, Dec 24, 2011 at 12:39 PM, Andrey Stepachev <oc...@gmail.com>
>> wrote:
>>
>> > 23 декабря 2011 г. 22:48 пользователь Todd Lipcon <todd@cloudera.com
>> > >написал:
>> >
>> > > On Tue, Dec 20, 2011 at 12:56 AM, Andrey Stepachev <oc...@gmail.com>
>> > > wrote:
>> > > > I see, that most issues addressed to HA branch.
>> > > > What version this branch is applied to? 0.22 or
>> > > > i should build hadoop from sources?
>> > >
>> > > It's currently only available in the HDFS-1623 branch, which is in
>> > > active development and definitely shouldn't be used unless you plan
to
>> > > help us finish debugging it :) There are still some known bugs and
>> > > deficiencies to address before we merge.
>> > >
>> >
>> > Looks like this is one of possible scenarios :). We need
>> > some solution for HA, but no one (except MapR) still exists.
>> > So we must use MapR (costs, closed) or debug some
>> > not-ready-yet solution.
>> >
>> >
>> > >
>> > > The plan is to merge it into 0.23 but not 0.22 - it will hopefully be
>> > > merged in January. We'll also make it available in CDH4 beta releases
>> > > early next year.
>> > >
>> >
>> > Sounds good. But still can't find such branch in cloudera (or any
other)
>> > github
>> > repositories.
>> > Is it publicly available right now, or it in some private repository?
>> >
>> >
>> > >
>> > > -Todd
>> > >
>> > > >
>> > > > 20 декабря 2011 г. 11:46 пользователь Todd Lipcon <
todd@cloudera.com
>> > > >написал:
>> > > >
>> > > >> Hi Andrey,
>> > > >>
>> > > >> I don't think anyone has ever run Ceph in a production use case,
>> much
>> > > >> less run it in a production use case with HBase.
>> > > >>
>> > > >> The SPOF problem is currently being addressed and will be
available
>> > > >> for production use by the middle of next calendar year. It will be
>> > > >> available for preview/beta usage in Q1 (it's already working in
>> > > >> development setups - feel free to follow HDFS-1623 for details)
>> > > >>
>> > > >> I think you're way better off waiting for the HA NameNode support
in
>> > > >> HDFS than going with an unproven FS like Ceph. Ceph certainly has
>> > > >> promise and is a nice design, but it hasn't had the years of
>> pounding
>> > > >> that HDFS has.
>> > > >>
>> > > >> -Todd
>> > > >>
>> > > >> On Mon, Dec 19, 2011 at 10:23 PM, Andrey Stepachev <
>> octo47@gmail.com>
>> > > >> wrote:
>> > > >> > Hi all.
>> > > >> >
>> > > >> > I have requirements to use hbase in several datacenters.
>> > > >> > But HDFS has as SPOF, so we can't use it.
>> > > >> > I plan to use ceph as the file system for hbase.
>> > > >> >
>> > > >> > In general, interested in issues of:
>> > > >> > a) use the hadoop + сeph in the production environment
>> > > >> > b) using hbase + ceph in the production environment
>> > > >> >
>> > > >> > In particular:
>> > > >> > Ceph Wiki states that the integration of hadoop/ceph has the
>> problem
>> > > in
>> > > >> the
>> > > >> > form of modification dates of the files.
>> > > >> > Does hbase uses modification times f--
> Андрей.
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: hdfs-1623 Was: ceph and hbase.
Posted by Andrey Stepachev <oc...@gmail.com>.
Thanks!
25 декабря 2011 г. 3:13 пользователь Ted Yu <yu...@gmail.com> написал:
> Andrey:
> A quick search led me to https://github.com/toddlipcon where you would be
> able to find:
> https://github.com/toddlipcon/hadoop-common
>
> Cheers
>
> On Sat, Dec 24, 2011 at 12:39 PM, Andrey Stepachev <oc...@gmail.com>
> wrote:
>
> > 23 декабря 2011 г. 22:48 пользователь Todd Lipcon <todd@cloudera.com
> > >написал:
> >
> > > On Tue, Dec 20, 2011 at 12:56 AM, Andrey Stepachev <oc...@gmail.com>
> > > wrote:
> > > > I see, that most issues addressed to HA branch.
> > > > What version this branch is applied to? 0.22 or
> > > > i should build hadoop from sources?
> > >
> > > It's currently only available in the HDFS-1623 branch, which is in
> > > active development and definitely shouldn't be used unless you plan to
> > > help us finish debugging it :) There are still some known bugs and
> > > deficiencies to address before we merge.
> > >
> >
> > Looks like this is one of possible scenarios :). We need
> > some solution for HA, but no one (except MapR) still exists.
> > So we must use MapR (costs, closed) or debug some
> > not-ready-yet solution.
> >
> >
> > >
> > > The plan is to merge it into 0.23 but not 0.22 - it will hopefully be
> > > merged in January. We'll also make it available in CDH4 beta releases
> > > early next year.
> > >
> >
> > Sounds good. But still can't find such branch in cloudera (or any other)
> > github
> > repositories.
> > Is it publicly available right now, or it in some private repository?
> >
> >
> > >
> > > -Todd
> > >
> > > >
> > > > 20 декабря 2011 г. 11:46 пользователь Todd Lipcon <todd@cloudera.com
> > > >написал:
> > > >
> > > >> Hi Andrey,
> > > >>
> > > >> I don't think anyone has ever run Ceph in a production use case,
> much
> > > >> less run it in a production use case with HBase.
> > > >>
> > > >> The SPOF problem is currently being addressed and will be available
> > > >> for production use by the middle of next calendar year. It will be
> > > >> available for preview/beta usage in Q1 (it's already working in
> > > >> development setups - feel free to follow HDFS-1623 for details)
> > > >>
> > > >> I think you're way better off waiting for the HA NameNode support in
> > > >> HDFS than going with an unproven FS like Ceph. Ceph certainly has
> > > >> promise and is a nice design, but it hasn't had the years of
> pounding
> > > >> that HDFS has.
> > > >>
> > > >> -Todd
> > > >>
> > > >> On Mon, Dec 19, 2011 at 10:23 PM, Andrey Stepachev <
> octo47@gmail.com>
> > > >> wrote:
> > > >> > Hi all.
> > > >> >
> > > >> > I have requirements to use hbase in several datacenters.
> > > >> > But HDFS has as SPOF, so we can't use it.
> > > >> > I plan to use ceph as the file system for hbase.
> > > >> >
> > > >> > In general, interested in issues of:
> > > >> > a) use the hadoop + сeph in the production environment
> > > >> > b) using hbase + ceph in the production environment
> > > >> >
> > > >> > In particular:
> > > >> > Ceph Wiki states that the integration of hadoop/ceph has the
> problem
> > > in
> > > >> the
> > > >> > form of modification dates of the files.
> > > >> > Does hbase uses modification times for some critical tasks?
> > > >> >
> > > >> > --
> > > >> > Andrey.
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Todd Lipcon
> > > >> Software Engineer, Cloudera
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Андрей.
> > >
> > >
> > >
> > > --
> > > Todd Lipcon
> > > Software Engineer, Cloudera
> > >
> >
> >
> >
> > --
> > Андрей.
> >
>
--
Андрей.