Re: [DISCUSS] Expanded "work items" for HBase-in-the-Cloud doc

[Josh Elser <el...@apache.org>]

Yup, we're on the same page :)

On 7/26/18 1:28 PM, Zach York wrote:
> I would REALLY hope that the WAL interface/API changes would go into master
> even if the feature work for Ratis is going in a feature branch. Not only
> would this enable other backends to be developed in parallel with the Ratis
> solution if there are other good fits for a non-HDFS WAL, but also it would
> save the burden of having to rebase these core changes onto the latest
> master to maintain compatibility. I'm assuming the Ratis portion of the
> code would be mostly new files so these would be less of a concern.
> 
> On Thu, Jul 26, 2018 at 9:24 AM, Josh Elser <el...@apache.org> wrote:
> 
>> On 7/26/18 1:00 AM, Stack wrote:
>>
>>> All this said, I'd like to start moving toward the point where we start
>>>> breaking out this work into a feature-branch off of master and start
>>>> building code. My hope is that this is amenable to everyone, with the
>>>> acknowledge that the Ratis work is considered "experimental" and not an
>>>> attempt to make all of HBase use Ratis-backed WALs.
>>>>
>>>>
>>>> Go for it.
>>>
>>> The branch would have WAL API changes only or would it include Ratis WAL
>>> dev? (If the latter, would that be better done over on Ratis project?).
>>>
>>
>> I think we would start with WAL API changes, get those "blessed", and then
>> continue Ratis WAL dev after that.
>>
> 

Re: [DISCUSS] Expanded "work items" for HBase-in-the-Cloud doc

[Josh Elser <el...@apache.org>]

Thanks for your interest, Jack.

Taking a read through the "design doc" and related discussion is a good 
starting point. Re-linked here for your convenience [1]

https://issues.apache.org/jira/browse/HBASE-20951 has a break-down of 
work items. The first thing we need to work on is the WAL API. I know 
that our Ted and Ankit have been digging through the current codebase to 
get a starting point. I'd expect some initial "pseudocode" this week, 
but this will be a long pole to get some agreed upon, I imagine.

You can also look at https://issues.apache.org/jira/browse/RATIS-271 
which will likely be quicker to have code getting committed. A few of us 
have been playing around with the code there already, and doing a 
similar API "deep-dive" exercise.

And, if it doesn't go without saying, feel free to ask questions on the 
mailing list here or in Slack. I'll do my best to answer them in a 
timely fashion :)

- Josh

[1] 
https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit?disco=AAAACBm3RLM

On 8/3/18 2:42 PM, Jack Bearden wrote:
> Great work! I'm excited about this feature and want to help with
> development. What do you guys suggest are the best topics to ramp up in
> preparation for these upcoming sprints?
> 
> On Fri, Aug 3, 2018 at 10:18 AM, Josh Elser <el...@apache.org> wrote:
> 
>> Yup, we're on the same page :)
>>
>>
>> On 7/26/18 1:28 PM, Zach York wrote:
>>
>>> I would REALLY hope that the WAL interface/API changes would go into
>>> master
>>> even if the feature work for Ratis is going in a feature branch. Not only
>>> would this enable other backends to be developed in parallel with the
>>> Ratis
>>> solution if there are other good fits for a non-HDFS WAL, but also it
>>> would
>>> save the burden of having to rebase these core changes onto the latest
>>> master to maintain compatibility. I'm assuming the Ratis portion of the
>>> code would be mostly new files so these would be less of a concern.
>>>
>>> On Thu, Jul 26, 2018 at 9:24 AM, Josh Elser <el...@apache.org> wrote:
>>>
>>> On 7/26/18 1:00 AM, Stack wrote:
>>>>
>>>> All this said, I'd like to start moving toward the point where we start
>>>>>
>>>>>> breaking out this work into a feature-branch off of master and start
>>>>>> building code. My hope is that this is amenable to everyone, with the
>>>>>> acknowledge that the Ratis work is considered "experimental" and not an
>>>>>> attempt to make all of HBase use Ratis-backed WALs.
>>>>>>
>>>>>>
>>>>>> Go for it.
>>>>>>
>>>>>
>>>>> The branch would have WAL API changes only or would it include Ratis WAL
>>>>> dev? (If the latter, would that be better done over on Ratis project?).
>>>>>
>>>>>
>>>> I think we would start with WAL API changes, get those "blessed", and
>>>> then
>>>> continue Ratis WAL dev after that.
>>>>
>>>>
>>>
> 

Re: [DISCUSS] Expanded "work items" for HBase-in-the-Cloud doc

[Jack Bearden <ja...@altiscale.com>]

Great work! I'm excited about this feature and want to help with
development. What do you guys suggest are the best topics to ramp up in
preparation for these upcoming sprints?

On Fri, Aug 3, 2018 at 10:18 AM, Josh Elser <el...@apache.org> wrote:

> Yup, we're on the same page :)
>
>
> On 7/26/18 1:28 PM, Zach York wrote:
>
>> I would REALLY hope that the WAL interface/API changes would go into
>> master
>> even if the feature work for Ratis is going in a feature branch. Not only
>> would this enable other backends to be developed in parallel with the
>> Ratis
>> solution if there are other good fits for a non-HDFS WAL, but also it
>> would
>> save the burden of having to rebase these core changes onto the latest
>> master to maintain compatibility. I'm assuming the Ratis portion of the
>> code would be mostly new files so these would be less of a concern.
>>
>> On Thu, Jul 26, 2018 at 9:24 AM, Josh Elser <el...@apache.org> wrote:
>>
>> On 7/26/18 1:00 AM, Stack wrote:
>>>
>>> All this said, I'd like to start moving toward the point where we start
>>>>
>>>>> breaking out this work into a feature-branch off of master and start
>>>>> building code. My hope is that this is amenable to everyone, with the
>>>>> acknowledge that the Ratis work is considered "experimental" and not an
>>>>> attempt to make all of HBase use Ratis-backed WALs.
>>>>>
>>>>>
>>>>> Go for it.
>>>>>
>>>>
>>>> The branch would have WAL API changes only or would it include Ratis WAL
>>>> dev? (If the latter, would that be better done over on Ratis project?).
>>>>
>>>>
>>> I think we would start with WAL API changes, get those "blessed", and
>>> then
>>> continue Ratis WAL dev after that.
>>>
>>>
>>