You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Sean Busbey <bu...@apache.org> on 2019/04/26 14:33:53 UTC
[DISCUSS] lazy consensus on "hbase-filesystem" repository
Hi folks!
HBASE-22149 "HBOSS: A FileSystem implementation to provide HBase's
required semantics" has an approach to reliably running HBase on top
of non-HDFS Filesystem implementations.
Copying from my comment there:
> Given the combination of this a) needing hadoop 3 only and b) being an experimental approach that we're not sure on sustainability in production I'd much prefer a different repository.
>
> Is anyone opposed to landing this in a new repository, i.e. `hbase-filesystem`? Provided it includes instructions for installation / set up we wouldn't even need to add the artifacts from that repository as a dependency for the main repo's binary artifacts.
Re: [DISCUSS] lazy consensus on "hbase-filesystem" repository
Posted by Josh Elser <el...@apache.org>.
Thanks all. Let me create this repo so we can keep this HBOSS train
moving as I think Busbey is on vacation today.
On HBASE-20952, that work, while helpful to the bigger cause, doesn't
impact anything that folks are working on with HBOSS. We could certainly
use hbase-filesystem as a place to iterate on both (the repo we use to
store the implementation we're plugging into the new WAL API). I'm sure
Duo intended this, just wanted to clarify for posterity :)
On 4/27/19 9:11 AM, 张铎(Duo Zhang) wrote:
> Oh let me take a look at this interesting issue, haven't noticed it yet...
>
> But ideally, I think HBASE-20952 is the correct way to decouple with the
> hadoop filesystem, that we first decouple WAL system with the hadoop
> filesystem, then we can abstract a 'FileSystem' for storing HFile, which
> could be very simple, as in HBase we only rely on a very small set of file
> system APIs for storing HFile, and then maybe we can just implement it
> based on the S3 API directly...
>
> And +1 on landing it to hbase-filesystem repo first.
>
> Zach York <zy...@gmail.com> 于2019年4月27日周六 上午6:05写道：
>
>> +1
>>
>> On Fri, Apr 26, 2019 at 2:48 PM Andrew Purtell <ap...@apache.org>
>> wrote:
>>
>>> +1
>>>
>>> On Fri, Apr 26, 2019 at 7:33 AM Sean Busbey <bu...@apache.org> wrote:
>>>
>>>> Hi folks!
>>>>
>>>> HBASE-22149 "HBOSS: A FileSystem implementation to provide HBase's
>>>> required semantics" has an approach to reliably running HBase on top
>>>> of non-HDFS Filesystem implementations.
>>>>
>>>> Copying from my comment there:
>>>>
>>>>> Given the combination of this a) needing hadoop 3 only and b) being
>> an
>>>> experimental approach that we're not sure on sustainability in
>> production
>>>> I'd much prefer a different repository.
>>>>>
>>>>> Is anyone opposed to landing this in a new repository, i.e.
>>>> `hbase-filesystem`? Provided it includes instructions for installation
>> /
>>>> set up we wouldn't even need to add the artifacts from that repository
>>> as a
>>>> dependency for the main repo's binary artifacts.
>>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Andrew
>>>
>>> Words like orphans lost among the crosstalk, meaning torn from truth's
>>> decrepit hands
>>> - A23, Crosstalk
>>>
>>
>
Re: [DISCUSS] lazy consensus on "hbase-filesystem" repository
Posted by "张铎 (Duo Zhang)" <pa...@gmail.com>.
Oh let me take a look at this interesting issue, haven't noticed it yet...
But ideally, I think HBASE-20952 is the correct way to decouple with the
hadoop filesystem, that we first decouple WAL system with the hadoop
filesystem, then we can abstract a 'FileSystem' for storing HFile, which
could be very simple, as in HBase we only rely on a very small set of file
system APIs for storing HFile, and then maybe we can just implement it
based on the S3 API directly...
And +1 on landing it to hbase-filesystem repo first.
Zach York <zy...@gmail.com> 于2019年4月27日周六 上午6:05写道：
> +1
>
> On Fri, Apr 26, 2019 at 2:48 PM Andrew Purtell <ap...@apache.org>
> wrote:
>
> > +1
> >
> > On Fri, Apr 26, 2019 at 7:33 AM Sean Busbey <bu...@apache.org> wrote:
> >
> > > Hi folks!
> > >
> > > HBASE-22149 "HBOSS: A FileSystem implementation to provide HBase's
> > > required semantics" has an approach to reliably running HBase on top
> > > of non-HDFS Filesystem implementations.
> > >
> > > Copying from my comment there:
> > >
> > > > Given the combination of this a) needing hadoop 3 only and b) being
> an
> > > experimental approach that we're not sure on sustainability in
> production
> > > I'd much prefer a different repository.
> > > >
> > > > Is anyone opposed to landing this in a new repository, i.e.
> > > `hbase-filesystem`? Provided it includes instructions for installation
> /
> > > set up we wouldn't even need to add the artifacts from that repository
> > as a
> > > dependency for the main repo's binary artifacts.
> > >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> > - A23, Crosstalk
> >
>
Re: [DISCUSS] lazy consensus on "hbase-filesystem" repository
Posted by Zach York <zy...@gmail.com>.
+1
On Fri, Apr 26, 2019 at 2:48 PM Andrew Purtell <ap...@apache.org> wrote:
> +1
>
> On Fri, Apr 26, 2019 at 7:33 AM Sean Busbey <bu...@apache.org> wrote:
>
> > Hi folks!
> >
> > HBASE-22149 "HBOSS: A FileSystem implementation to provide HBase's
> > required semantics" has an approach to reliably running HBase on top
> > of non-HDFS Filesystem implementations.
> >
> > Copying from my comment there:
> >
> > > Given the combination of this a) needing hadoop 3 only and b) being an
> > experimental approach that we're not sure on sustainability in production
> > I'd much prefer a different repository.
> > >
> > > Is anyone opposed to landing this in a new repository, i.e.
> > `hbase-filesystem`? Provided it includes instructions for installation /
> > set up we wouldn't even need to add the artifacts from that repository
> as a
> > dependency for the main repo's binary artifacts.
> >
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
> - A23, Crosstalk
>
Re: [DISCUSS] lazy consensus on "hbase-filesystem" repository
Posted by Andrew Purtell <ap...@apache.org>.
+1
On Fri, Apr 26, 2019 at 7:33 AM Sean Busbey <bu...@apache.org> wrote:
> Hi folks!
>
> HBASE-22149 "HBOSS: A FileSystem implementation to provide HBase's
> required semantics" has an approach to reliably running HBase on top
> of non-HDFS Filesystem implementations.
>
> Copying from my comment there:
>
> > Given the combination of this a) needing hadoop 3 only and b) being an
> experimental approach that we're not sure on sustainability in production
> I'd much prefer a different repository.
> >
> > Is anyone opposed to landing this in a new repository, i.e.
> `hbase-filesystem`? Provided it includes instructions for installation /
> set up we wouldn't even need to add the artifacts from that repository as a
> dependency for the main repo's binary artifacts.
>
--
Best regards,
Andrew
Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
- A23, Crosstalk