You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by Istvan Toth <st...@apache.org> on 2020/02/06 13:40:04 UTC
[DISCUSS] Unifying the 4.x branches
Hello!
Now that we have a working solution in master for handling different HBase
minor versions, I think that we should think about applying the same
template to 4.x., and unifying the 4.x-HBase-1.3, 1.4, and 1.5 branches.
Are there any intentional differences between the branches, apart from
having to conform to the slightly different APIs ?
If there are, what are they, and are they considered blockers?
Any other reasons not do this ?
I expect that based on my experience with the master branch, I can do this
in a few days, but I don't want to put in the effort if there is no
interest in it.
My plan is to take the 1.5 branch as a base.
best regards
Istvan
Re: [DISCUSS] Unifying the 4.x branches
Posted by Josh Elser <el...@apache.org>.
Left a couple of minor requests to fix (no consequence changes --
comments, whitespace), which you can just fix on commit. Great work, Istvan.
Any of y'all want to look before Istvan merges it?
On 2/27/20 4:58 AM, Istvan Toth wrote:
> Thanks Andrew, I like that solution.
>
> I've added the "final" PRs and patches to
> https://issues.apache.org/jira/browse/PHOENIX-5721 ,
> they are ready for review.
>
> On Wed, Feb 19, 2020 at 9:01 PM Andrew Purtell <an...@gmail.com>
> wrote:
>
>> Since you have already done this great work, and 1.3 isn’t dead yet, and
>> won’t be “this year”, and it serves as an example of how to bring in entire
>> new features on later code lines, perhaps it should go in. Just my 0.02.
>>
>>> On Feb 19, 2020, at 10:39 AM, Istvan Toth <st...@apache.org> wrote:
>>>
>>> Geoffrey,
>>>
>>> Absolutely.
>>> 80% of this patch is dealing with 1.3.
>>> 1.4 vs 1.5 affects two or three java files.
>>>
>>> My game plan is to submit two different patches, a small one for 1.4 and
>>> 1.5 support, and a bigger one other that adds 1.3, so that it can be
>>> reverted easily after 1.3 is dropped.
>>>
>>> I think that maintaining a separate 1.3 branch is the least desirable
>>> outcome, as we'd still have two 4.x branches to maintain, with different
>>> jenkins jobs, and slightly different jar names and maven artifact
>> structure.
>>>
>>> Of course the easiest route is to just drop 1.3 support ASAP, and then
>>> unify 1.4 and 1.5 .
>>>
>>> If we are not ready to drop 1.3 very soon, I'd vote for unifying with
>>> 1.3 included (I've finished the hard part), and then reverting a lot of
>>> the compatibility cruft when we drop 1.3 support.
>>>
>>> As for the 4.15 maintenance releases, those could stay on the current
>>> branches, as we probably want point releases to be drop-in compatible,
>>> and the unified version changes the maven artifact versioning, and some
>>> jar filenames. This would be one way to extend some support for 1.3
>>>
>>> regards
>>> István
>>>
>>>
>>>
>>>> On 2020. 02. 19. 19:11, Geoffrey Jacoby wrote:
>>>> Istvan,
>>>>
>>>> The HBase community has been on the verge of EOLing 1.3 for some time
>>>> now -- are there significant gains or simplification if we either end
>>>> 1.3 support in Phoenix before PHOENIX-5721 goes in, or alternately,
>>>> don't include it in the unified profile since it will be EOLed in the
>>>> not-too-distant-future even if we don't do it now?
>>>>
>>>> Geoffrey
>>>>
>>>> On Wed, Feb 19, 2020 at 5:36 AM Istvan Toth <stoty@apache.org
>>>> <ma...@apache.org>> wrote:
>>>>
>>>> Now I have an unpolished version of the unified 4.x branch at
>>>> https://github.com/stoty/phoenix/tree/PHOENIX-5721
>>>>
>>>> It takes the same approach as the master branch, though there are a
>> bit
>>>> more differences to hide in the versions.
>>>>
>>>> I need to finish the assembly stuff, go over once more the changes,
>>>> and run
>>>> full ITs on each profile, but I expect that
>>>> the Java code will mostly see whitespace changes, so you can check
>> the
>>>> logic behind the changes.
>>>>
>>>> The HBase metrics stuff in particular is interesting, because the
>> whole
>>>> feature is missing from 1.3, so it can be used as an example on how
>>>> we can
>>>> adopt new HBase features later.
>>>>
>>>> On Mon, Feb 10, 2020 at 9:01 AM Istvan Toth <stoty@apache.org
>>>> <ma...@apache.org>> wrote:
>>>>
>>>>> Created PHOENIX-5721
>>>> <https://issues.apache.org/jira/browse/PHOENIX-5721> to
>>>>> track this.
>>>>>
>>>>> On Mon, Feb 10, 2020 at 8:21 AM István Tóth <stoty@cloudera.com
>>>> <ma...@cloudera.com>> wrote:
>>>>>
>>>>>> Thanks for the feedback, Geoffrey.
>>>>>>
>>>>>> I took the lazy option of just creating compatibility methods to
>>>> paper
>>>>>> over the HBase API changes (emulate the latest version) when we
>>>> are calling
>>>>>> into HBase.
>>>>>>
>>>>>> For the APIs implemented by Phoenix, I added compatibility
>>>> superclasses.
>>>>>> So I expect that we will be able to add a dummy
>>>> RegionObserver.preWALAppend
>>>>>> to the compatibility coprocessor superclass(es), so that the same
>>>> code
>>>>>> compiles for older versions as well.
>>>>>>
>>>>>> Of course if other code paths depend on having that
>>>> functionality, those
>>>>>> should also be gated by similar compatibility shims/version checks.
>>>>>>
>>>>>> My current approach is a quick fix, and does not preclude (in fact it
>>>>>> enables) later efforts at refactoring/cleanup.
>>>>>>
>>>>>> On Fri, Feb 7, 2020 at 7:31 PM Geoffrey Jacoby
>>>> <gjacoby@apache.org <ma...@apache.org>>
>>>>>> wrote:
>>>>>>
>>>>>>> If unification could be done, that would be great. (I apologize
>>>> that I
>>>>>>> haven't had the bandwidth over the past week or two to take a
>>>> close look
>>>>>>> at
>>>>>>> the work Istvan has been doing to unify the 5.x branches -- as
>>>> one who
>>>>>>> spends too much time cherry-picking I very much appreciate this
>>>> effort!
>>>>>>> :-)
>>>>>>> )
>>>>>>>
>>>>>>> I still think the hardest part, as I think I mentioned in some
>>>> previous
>>>>>>> thread, is what to do when the HBase coprocs themselves diverge
>>>> between
>>>>>>> minor versions, as they can do. How do you handle the fact that
>>>>>>> RegionObserver.preWALAppend exists in 1.5 but not 1.3 and 1.4,
>>>> and will
>>>>>>> exist in 2.3 but not 2.2 and 2.1? And that therefore any
>>>> features that
>>>>>>> depend on preWALAppend existing (none yet, but they're coming
>>>> later this
>>>>>>> year) will work in a 1.5 or 2.3 environment, but not a 1.4 or
>>>> 2.2 one?
>>>>>>>
>>>>>>> Since our coprocessors tend to be giant monoliths, trying to create
>>>>>>> release-based versions of them selectable by maven profile would
>>>>>>> either require lots of developer copy/paste for each change, or a
>>>>>>> significant (probably long overdue) refactor to make the coprocs
>>>> small
>>>>>>> shims that call out to smaller, fine-grained classes that can
>>>>>>> occasionally
>>>>>>> be release-specific.
>>>>>>>
>>>>>>> Geoffrey
>>>>>>>
>>>>>>> On Fri, Feb 7, 2020 at 9:31 AM Josh Elser <elserj@apache.org
>>>> <ma...@apache.org>> wrote:
>>>>>>>
>>>>>>>> Sounds like a good idea to me.
>>>>>>>>
>>>>>>>> On 2/6/20 8:40 AM, Istvan Toth wrote:
>>>>>>>>> Hello!
>>>>>>>>>
>>>>>>>>> Now that we have a working solution in master for handling
>>>> different
>>>>>>>> HBase
>>>>>>>>> minor versions, I think that we should think about applying
>>>> the same
>>>>>>>>> template to 4.x., and unifying the 4.x-HBase-1.3, 1.4, and 1.5
>>>>>>> branches.
>>>>>>>>>
>>>>>>>>> Are there any intentional differences between the branches,
>>>> apart
>>>>>>> from
>>>>>>>>> having to conform to the slightly different APIs ?
>>>>>>>>> If there are, what are they, and are they considered blockers?
>>>>>>>>>
>>>>>>>>> Any other reasons not do this ?
>>>>>>>>>
>>>>>>>>> I expect that based on my experience with the master branch,
>>>> I can do
>>>>>>>> this
>>>>>>>>> in a few days, but I don't want to put in the effort if
>>>> there is no
>>>>>>>>> interest in it.
>>>>>>>>>
>>>>>>>>> My plan is to take the 1.5 branch as a base.
>>>>>>>>>
>>>>>>>>> best regards
>>>>>>>>> Istvan
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *István Tóth* | Sr. Software Engineer
>>>>>> t. (36) 70 283-1788
>>>>>> stoty@cloudera.com <ma...@cloudera.com>
>>>> <https://www.cloudera.com>
>>>>>> [image: Cloudera] <https://www.cloudera.com/>
>>>>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>>>>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>>>>>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>>>>>> <https://www.cloudera.com/>
>>>>>> ------------------------------
>>>>>>
>>>>>
>>>>
>>
>
Re: [DISCUSS] Unifying the 4.x branches
Posted by Istvan Toth <st...@apache.org>.
Thanks Andrew, I like that solution.
I've added the "final" PRs and patches to
https://issues.apache.org/jira/browse/PHOENIX-5721 ,
they are ready for review.
On Wed, Feb 19, 2020 at 9:01 PM Andrew Purtell <an...@gmail.com>
wrote:
> Since you have already done this great work, and 1.3 isn’t dead yet, and
> won’t be “this year”, and it serves as an example of how to bring in entire
> new features on later code lines, perhaps it should go in. Just my 0.02.
>
> > On Feb 19, 2020, at 10:39 AM, Istvan Toth <st...@apache.org> wrote:
> >
> > Geoffrey,
> >
> > Absolutely.
> > 80% of this patch is dealing with 1.3.
> > 1.4 vs 1.5 affects two or three java files.
> >
> > My game plan is to submit two different patches, a small one for 1.4 and
> > 1.5 support, and a bigger one other that adds 1.3, so that it can be
> > reverted easily after 1.3 is dropped.
> >
> > I think that maintaining a separate 1.3 branch is the least desirable
> > outcome, as we'd still have two 4.x branches to maintain, with different
> > jenkins jobs, and slightly different jar names and maven artifact
> structure.
> >
> > Of course the easiest route is to just drop 1.3 support ASAP, and then
> > unify 1.4 and 1.5 .
> >
> > If we are not ready to drop 1.3 very soon, I'd vote for unifying with
> > 1.3 included (I've finished the hard part), and then reverting a lot of
> > the compatibility cruft when we drop 1.3 support.
> >
> > As for the 4.15 maintenance releases, those could stay on the current
> > branches, as we probably want point releases to be drop-in compatible,
> > and the unified version changes the maven artifact versioning, and some
> > jar filenames. This would be one way to extend some support for 1.3
> >
> > regards
> > István
> >
> >
> >
> >> On 2020. 02. 19. 19:11, Geoffrey Jacoby wrote:
> >> Istvan,
> >>
> >> The HBase community has been on the verge of EOLing 1.3 for some time
> >> now -- are there significant gains or simplification if we either end
> >> 1.3 support in Phoenix before PHOENIX-5721 goes in, or alternately,
> >> don't include it in the unified profile since it will be EOLed in the
> >> not-too-distant-future even if we don't do it now?
> >>
> >> Geoffrey
> >>
> >> On Wed, Feb 19, 2020 at 5:36 AM Istvan Toth <stoty@apache.org
> >> <ma...@apache.org>> wrote:
> >>
> >> Now I have an unpolished version of the unified 4.x branch at
> >> https://github.com/stoty/phoenix/tree/PHOENIX-5721
> >>
> >> It takes the same approach as the master branch, though there are a
> bit
> >> more differences to hide in the versions.
> >>
> >> I need to finish the assembly stuff, go over once more the changes,
> >> and run
> >> full ITs on each profile, but I expect that
> >> the Java code will mostly see whitespace changes, so you can check
> the
> >> logic behind the changes.
> >>
> >> The HBase metrics stuff in particular is interesting, because the
> whole
> >> feature is missing from 1.3, so it can be used as an example on how
> >> we can
> >> adopt new HBase features later.
> >>
> >> On Mon, Feb 10, 2020 at 9:01 AM Istvan Toth <stoty@apache.org
> >> <ma...@apache.org>> wrote:
> >>
> >>> Created PHOENIX-5721
> >> <https://issues.apache.org/jira/browse/PHOENIX-5721> to
> >>> track this.
> >>>
> >>> On Mon, Feb 10, 2020 at 8:21 AM István Tóth <stoty@cloudera.com
> >> <ma...@cloudera.com>> wrote:
> >>>
> >>>> Thanks for the feedback, Geoffrey.
> >>>>
> >>>> I took the lazy option of just creating compatibility methods to
> >> paper
> >>>> over the HBase API changes (emulate the latest version) when we
> >> are calling
> >>>> into HBase.
> >>>>
> >>>> For the APIs implemented by Phoenix, I added compatibility
> >> superclasses.
> >>>> So I expect that we will be able to add a dummy
> >> RegionObserver.preWALAppend
> >>>> to the compatibility coprocessor superclass(es), so that the same
> >> code
> >>>> compiles for older versions as well.
> >>>>
> >>>> Of course if other code paths depend on having that
> >> functionality, those
> >>>> should also be gated by similar compatibility shims/version checks.
> >>>>
> >>>> My current approach is a quick fix, and does not preclude (in fact it
> >>>> enables) later efforts at refactoring/cleanup.
> >>>>
> >>>> On Fri, Feb 7, 2020 at 7:31 PM Geoffrey Jacoby
> >> <gjacoby@apache.org <ma...@apache.org>>
> >>>> wrote:
> >>>>
> >>>>> If unification could be done, that would be great. (I apologize
> >> that I
> >>>>> haven't had the bandwidth over the past week or two to take a
> >> close look
> >>>>> at
> >>>>> the work Istvan has been doing to unify the 5.x branches -- as
> >> one who
> >>>>> spends too much time cherry-picking I very much appreciate this
> >> effort!
> >>>>> :-)
> >>>>> )
> >>>>>
> >>>>> I still think the hardest part, as I think I mentioned in some
> >> previous
> >>>>> thread, is what to do when the HBase coprocs themselves diverge
> >> between
> >>>>> minor versions, as they can do. How do you handle the fact that
> >>>>> RegionObserver.preWALAppend exists in 1.5 but not 1.3 and 1.4,
> >> and will
> >>>>> exist in 2.3 but not 2.2 and 2.1? And that therefore any
> >> features that
> >>>>> depend on preWALAppend existing (none yet, but they're coming
> >> later this
> >>>>> year) will work in a 1.5 or 2.3 environment, but not a 1.4 or
> >> 2.2 one?
> >>>>>
> >>>>> Since our coprocessors tend to be giant monoliths, trying to create
> >>>>> release-based versions of them selectable by maven profile would
> >>>>> either require lots of developer copy/paste for each change, or a
> >>>>> significant (probably long overdue) refactor to make the coprocs
> >> small
> >>>>> shims that call out to smaller, fine-grained classes that can
> >>>>> occasionally
> >>>>> be release-specific.
> >>>>>
> >>>>> Geoffrey
> >>>>>
> >>>>> On Fri, Feb 7, 2020 at 9:31 AM Josh Elser <elserj@apache.org
> >> <ma...@apache.org>> wrote:
> >>>>>
> >>>>>> Sounds like a good idea to me.
> >>>>>>
> >>>>>> On 2/6/20 8:40 AM, Istvan Toth wrote:
> >>>>>>> Hello!
> >>>>>>>
> >>>>>>> Now that we have a working solution in master for handling
> >> different
> >>>>>> HBase
> >>>>>>> minor versions, I think that we should think about applying
> >> the same
> >>>>>>> template to 4.x., and unifying the 4.x-HBase-1.3, 1.4, and 1.5
> >>>>> branches.
> >>>>>>>
> >>>>>>> Are there any intentional differences between the branches,
> >> apart
> >>>>> from
> >>>>>>> having to conform to the slightly different APIs ?
> >>>>>>> If there are, what are they, and are they considered blockers?
> >>>>>>>
> >>>>>>> Any other reasons not do this ?
> >>>>>>>
> >>>>>>> I expect that based on my experience with the master branch,
> >> I can do
> >>>>>> this
> >>>>>>> in a few days, but I don't want to put in the effort if
> >> there is no
> >>>>>>> interest in it.
> >>>>>>>
> >>>>>>> My plan is to take the 1.5 branch as a base.
> >>>>>>>
> >>>>>>> best regards
> >>>>>>> Istvan
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> *István Tóth* | Sr. Software Engineer
> >>>> t. (36) 70 283-1788
> >>>> stoty@cloudera.com <ma...@cloudera.com>
> >> <https://www.cloudera.com>
> >>>> [image: Cloudera] <https://www.cloudera.com/>
> >>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> >>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> >>>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> >>>> <https://www.cloudera.com/>
> >>>> ------------------------------
> >>>>
> >>>
> >>
>
Re: [DISCUSS] Unifying the 4.x branches
Posted by Andrew Purtell <an...@gmail.com>.
Since you have already done this great work, and 1.3 isn’t dead yet, and won’t be “this year”, and it serves as an example of how to bring in entire new features on later code lines, perhaps it should go in. Just my 0.02.
> On Feb 19, 2020, at 10:39 AM, Istvan Toth <st...@apache.org> wrote:
>
> Geoffrey,
>
> Absolutely.
> 80% of this patch is dealing with 1.3.
> 1.4 vs 1.5 affects two or three java files.
>
> My game plan is to submit two different patches, a small one for 1.4 and
> 1.5 support, and a bigger one other that adds 1.3, so that it can be
> reverted easily after 1.3 is dropped.
>
> I think that maintaining a separate 1.3 branch is the least desirable
> outcome, as we'd still have two 4.x branches to maintain, with different
> jenkins jobs, and slightly different jar names and maven artifact structure.
>
> Of course the easiest route is to just drop 1.3 support ASAP, and then
> unify 1.4 and 1.5 .
>
> If we are not ready to drop 1.3 very soon, I'd vote for unifying with
> 1.3 included (I've finished the hard part), and then reverting a lot of
> the compatibility cruft when we drop 1.3 support.
>
> As for the 4.15 maintenance releases, those could stay on the current
> branches, as we probably want point releases to be drop-in compatible,
> and the unified version changes the maven artifact versioning, and some
> jar filenames. This would be one way to extend some support for 1.3
>
> regards
> István
>
>
>
>> On 2020. 02. 19. 19:11, Geoffrey Jacoby wrote:
>> Istvan,
>>
>> The HBase community has been on the verge of EOLing 1.3 for some time
>> now -- are there significant gains or simplification if we either end
>> 1.3 support in Phoenix before PHOENIX-5721 goes in, or alternately,
>> don't include it in the unified profile since it will be EOLed in the
>> not-too-distant-future even if we don't do it now?
>>
>> Geoffrey
>>
>> On Wed, Feb 19, 2020 at 5:36 AM Istvan Toth <stoty@apache.org
>> <ma...@apache.org>> wrote:
>>
>> Now I have an unpolished version of the unified 4.x branch at
>> https://github.com/stoty/phoenix/tree/PHOENIX-5721
>>
>> It takes the same approach as the master branch, though there are a bit
>> more differences to hide in the versions.
>>
>> I need to finish the assembly stuff, go over once more the changes,
>> and run
>> full ITs on each profile, but I expect that
>> the Java code will mostly see whitespace changes, so you can check the
>> logic behind the changes.
>>
>> The HBase metrics stuff in particular is interesting, because the whole
>> feature is missing from 1.3, so it can be used as an example on how
>> we can
>> adopt new HBase features later.
>>
>> On Mon, Feb 10, 2020 at 9:01 AM Istvan Toth <stoty@apache.org
>> <ma...@apache.org>> wrote:
>>
>>> Created PHOENIX-5721
>> <https://issues.apache.org/jira/browse/PHOENIX-5721> to
>>> track this.
>>>
>>> On Mon, Feb 10, 2020 at 8:21 AM István Tóth <stoty@cloudera.com
>> <ma...@cloudera.com>> wrote:
>>>
>>>> Thanks for the feedback, Geoffrey.
>>>>
>>>> I took the lazy option of just creating compatibility methods to
>> paper
>>>> over the HBase API changes (emulate the latest version) when we
>> are calling
>>>> into HBase.
>>>>
>>>> For the APIs implemented by Phoenix, I added compatibility
>> superclasses.
>>>> So I expect that we will be able to add a dummy
>> RegionObserver.preWALAppend
>>>> to the compatibility coprocessor superclass(es), so that the same
>> code
>>>> compiles for older versions as well.
>>>>
>>>> Of course if other code paths depend on having that
>> functionality, those
>>>> should also be gated by similar compatibility shims/version checks.
>>>>
>>>> My current approach is a quick fix, and does not preclude (in fact it
>>>> enables) later efforts at refactoring/cleanup.
>>>>
>>>> On Fri, Feb 7, 2020 at 7:31 PM Geoffrey Jacoby
>> <gjacoby@apache.org <ma...@apache.org>>
>>>> wrote:
>>>>
>>>>> If unification could be done, that would be great. (I apologize
>> that I
>>>>> haven't had the bandwidth over the past week or two to take a
>> close look
>>>>> at
>>>>> the work Istvan has been doing to unify the 5.x branches -- as
>> one who
>>>>> spends too much time cherry-picking I very much appreciate this
>> effort!
>>>>> :-)
>>>>> )
>>>>>
>>>>> I still think the hardest part, as I think I mentioned in some
>> previous
>>>>> thread, is what to do when the HBase coprocs themselves diverge
>> between
>>>>> minor versions, as they can do. How do you handle the fact that
>>>>> RegionObserver.preWALAppend exists in 1.5 but not 1.3 and 1.4,
>> and will
>>>>> exist in 2.3 but not 2.2 and 2.1? And that therefore any
>> features that
>>>>> depend on preWALAppend existing (none yet, but they're coming
>> later this
>>>>> year) will work in a 1.5 or 2.3 environment, but not a 1.4 or
>> 2.2 one?
>>>>>
>>>>> Since our coprocessors tend to be giant monoliths, trying to create
>>>>> release-based versions of them selectable by maven profile would
>>>>> either require lots of developer copy/paste for each change, or a
>>>>> significant (probably long overdue) refactor to make the coprocs
>> small
>>>>> shims that call out to smaller, fine-grained classes that can
>>>>> occasionally
>>>>> be release-specific.
>>>>>
>>>>> Geoffrey
>>>>>
>>>>> On Fri, Feb 7, 2020 at 9:31 AM Josh Elser <elserj@apache.org
>> <ma...@apache.org>> wrote:
>>>>>
>>>>>> Sounds like a good idea to me.
>>>>>>
>>>>>> On 2/6/20 8:40 AM, Istvan Toth wrote:
>>>>>>> Hello!
>>>>>>>
>>>>>>> Now that we have a working solution in master for handling
>> different
>>>>>> HBase
>>>>>>> minor versions, I think that we should think about applying
>> the same
>>>>>>> template to 4.x., and unifying the 4.x-HBase-1.3, 1.4, and 1.5
>>>>> branches.
>>>>>>>
>>>>>>> Are there any intentional differences between the branches,
>> apart
>>>>> from
>>>>>>> having to conform to the slightly different APIs ?
>>>>>>> If there are, what are they, and are they considered blockers?
>>>>>>>
>>>>>>> Any other reasons not do this ?
>>>>>>>
>>>>>>> I expect that based on my experience with the master branch,
>> I can do
>>>>>> this
>>>>>>> in a few days, but I don't want to put in the effort if
>> there is no
>>>>>>> interest in it.
>>>>>>>
>>>>>>> My plan is to take the 1.5 branch as a base.
>>>>>>>
>>>>>>> best regards
>>>>>>> Istvan
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *István Tóth* | Sr. Software Engineer
>>>> t. (36) 70 283-1788
>>>> stoty@cloudera.com <ma...@cloudera.com>
>> <https://www.cloudera.com>
>>>> [image: Cloudera] <https://www.cloudera.com/>
>>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>>>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>>>> <https://www.cloudera.com/>
>>>> ------------------------------
>>>>
>>>
>>
Re: [DISCUSS] Unifying the 4.x branches
Posted by Istvan Toth <st...@apache.org>.
Geoffrey,
Absolutely.
80% of this patch is dealing with 1.3.
1.4 vs 1.5 affects two or three java files.
My game plan is to submit two different patches, a small one for 1.4 and
1.5 support, and a bigger one other that adds 1.3, so that it can be
reverted easily after 1.3 is dropped.
I think that maintaining a separate 1.3 branch is the least desirable
outcome, as we'd still have two 4.x branches to maintain, with different
jenkins jobs, and slightly different jar names and maven artifact structure.
Of course the easiest route is to just drop 1.3 support ASAP, and then
unify 1.4 and 1.5 .
If we are not ready to drop 1.3 very soon, I'd vote for unifying with
1.3 included (I've finished the hard part), and then reverting a lot of
the compatibility cruft when we drop 1.3 support.
As for the 4.15 maintenance releases, those could stay on the current
branches, as we probably want point releases to be drop-in compatible,
and the unified version changes the maven artifact versioning, and some
jar filenames. This would be one way to extend some support for 1.3
regards
István
On 2020. 02. 19. 19:11, Geoffrey Jacoby wrote:
> Istvan,
>
> The HBase community has been on the verge of EOLing 1.3 for some time
> now -- are there significant gains or simplification if we either end
> 1.3 support in Phoenix before PHOENIX-5721 goes in, or alternately,
> don't include it in the unified profile since it will be EOLed in the
> not-too-distant-future even if we don't do it now?
>
> Geoffrey
>
> On Wed, Feb 19, 2020 at 5:36 AM Istvan Toth <stoty@apache.org
> <ma...@apache.org>> wrote:
>
> Now I have an unpolished version of the unified 4.x branch at
> https://github.com/stoty/phoenix/tree/PHOENIX-5721
>
> It takes the same approach as the master branch, though there are a bit
> more differences to hide in the versions.
>
> I need to finish the assembly stuff, go over once more the changes,
> and run
> full ITs on each profile, but I expect that
> the Java code will mostly see whitespace changes, so you can check the
> logic behind the changes.
>
> The HBase metrics stuff in particular is interesting, because the whole
> feature is missing from 1.3, so it can be used as an example on how
> we can
> adopt new HBase features later.
>
> On Mon, Feb 10, 2020 at 9:01 AM Istvan Toth <stoty@apache.org
> <ma...@apache.org>> wrote:
>
> > Created PHOENIX-5721
> <https://issues.apache.org/jira/browse/PHOENIX-5721> to
> > track this.
> >
> > On Mon, Feb 10, 2020 at 8:21 AM István Tóth <stoty@cloudera.com
> <ma...@cloudera.com>> wrote:
> >
> >> Thanks for the feedback, Geoffrey.
> >>
> >> I took the lazy option of just creating compatibility methods to
> paper
> >> over the HBase API changes (emulate the latest version) when we
> are calling
> >> into HBase.
> >>
> >> For the APIs implemented by Phoenix, I added compatibility
> superclasses.
> >> So I expect that we will be able to add a dummy
> RegionObserver.preWALAppend
> >> to the compatibility coprocessor superclass(es), so that the same
> code
> >> compiles for older versions as well.
> >>
> >> Of course if other code paths depend on having that
> functionality, those
> >> should also be gated by similar compatibility shims/version checks.
> >>
> >> My current approach is a quick fix, and does not preclude (in fact it
> >> enables) later efforts at refactoring/cleanup.
> >>
> >> On Fri, Feb 7, 2020 at 7:31 PM Geoffrey Jacoby
> <gjacoby@apache.org <ma...@apache.org>>
> >> wrote:
> >>
> >>> If unification could be done, that would be great. (I apologize
> that I
> >>> haven't had the bandwidth over the past week or two to take a
> close look
> >>> at
> >>> the work Istvan has been doing to unify the 5.x branches -- as
> one who
> >>> spends too much time cherry-picking I very much appreciate this
> effort!
> >>> :-)
> >>> )
> >>>
> >>> I still think the hardest part, as I think I mentioned in some
> previous
> >>> thread, is what to do when the HBase coprocs themselves diverge
> between
> >>> minor versions, as they can do. How do you handle the fact that
> >>> RegionObserver.preWALAppend exists in 1.5 but not 1.3 and 1.4,
> and will
> >>> exist in 2.3 but not 2.2 and 2.1? And that therefore any
> features that
> >>> depend on preWALAppend existing (none yet, but they're coming
> later this
> >>> year) will work in a 1.5 or 2.3 environment, but not a 1.4 or
> 2.2 one?
> >>>
> >>> Since our coprocessors tend to be giant monoliths, trying to create
> >>> release-based versions of them selectable by maven profile would
> >>> either require lots of developer copy/paste for each change, or a
> >>> significant (probably long overdue) refactor to make the coprocs
> small
> >>> shims that call out to smaller, fine-grained classes that can
> >>> occasionally
> >>> be release-specific.
> >>>
> >>> Geoffrey
> >>>
> >>> On Fri, Feb 7, 2020 at 9:31 AM Josh Elser <elserj@apache.org
> <ma...@apache.org>> wrote:
> >>>
> >>> > Sounds like a good idea to me.
> >>> >
> >>> > On 2/6/20 8:40 AM, Istvan Toth wrote:
> >>> > > Hello!
> >>> > >
> >>> > > Now that we have a working solution in master for handling
> different
> >>> > HBase
> >>> > > minor versions, I think that we should think about applying
> the same
> >>> > > template to 4.x., and unifying the 4.x-HBase-1.3, 1.4, and 1.5
> >>> branches.
> >>> > >
> >>> > > Are there any intentional differences between the branches,
> apart
> >>> from
> >>> > > having to conform to the slightly different APIs ?
> >>> > > If there are, what are they, and are they considered blockers?
> >>> > >
> >>> > > Any other reasons not do this ?
> >>> > >
> >>> > > I expect that based on my experience with the master branch,
> I can do
> >>> > this
> >>> > > in a few days, but I don't want to put in the effort if
> there is no
> >>> > > interest in it.
> >>> > >
> >>> > > My plan is to take the 1.5 branch as a base.
> >>> > >
> >>> > > best regards
> >>> > > Istvan
> >>> > >
> >>> >
> >>>
> >>
> >>
> >> --
> >> *István Tóth* | Sr. Software Engineer
> >> t. (36) 70 283-1788
> >> stoty@cloudera.com <ma...@cloudera.com>
> <https://www.cloudera.com>
> >> [image: Cloudera] <https://www.cloudera.com/>
> >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> >> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> >> <https://www.cloudera.com/>
> >> ------------------------------
> >>
> >
>
Re: [DISCUSS] Unifying the 4.x branches
Posted by Geoffrey Jacoby <gj...@apache.org>.
Istvan,
The HBase community has been on the verge of EOLing 1.3 for some time now
-- are there significant gains or simplification if we either end 1.3
support in Phoenix before PHOENIX-5721 goes in, or alternately, don't
include it in the unified profile since it will be EOLed in the
not-too-distant-future even if we don't do it now?
Geoffrey
On Wed, Feb 19, 2020 at 5:36 AM Istvan Toth <st...@apache.org> wrote:
> Now I have an unpolished version of the unified 4.x branch at
> https://github.com/stoty/phoenix/tree/PHOENIX-5721
>
> It takes the same approach as the master branch, though there are a bit
> more differences to hide in the versions.
>
> I need to finish the assembly stuff, go over once more the changes, and run
> full ITs on each profile, but I expect that
> the Java code will mostly see whitespace changes, so you can check the
> logic behind the changes.
>
> The HBase metrics stuff in particular is interesting, because the whole
> feature is missing from 1.3, so it can be used as an example on how we can
> adopt new HBase features later.
>
> On Mon, Feb 10, 2020 at 9:01 AM Istvan Toth <st...@apache.org> wrote:
>
> > Created PHOENIX-5721 <https://issues.apache.org/jira/browse/PHOENIX-5721>
> to
> > track this.
> >
> > On Mon, Feb 10, 2020 at 8:21 AM István Tóth <st...@cloudera.com> wrote:
> >
> >> Thanks for the feedback, Geoffrey.
> >>
> >> I took the lazy option of just creating compatibility methods to paper
> >> over the HBase API changes (emulate the latest version) when we are
> calling
> >> into HBase.
> >>
> >> For the APIs implemented by Phoenix, I added compatibility superclasses.
> >> So I expect that we will be able to add a dummy
> RegionObserver.preWALAppend
> >> to the compatibility coprocessor superclass(es), so that the same code
> >> compiles for older versions as well.
> >>
> >> Of course if other code paths depend on having that functionality, those
> >> should also be gated by similar compatibility shims/version checks.
> >>
> >> My current approach is a quick fix, and does not preclude (in fact it
> >> enables) later efforts at refactoring/cleanup.
> >>
> >> On Fri, Feb 7, 2020 at 7:31 PM Geoffrey Jacoby <gj...@apache.org>
> >> wrote:
> >>
> >>> If unification could be done, that would be great. (I apologize that I
> >>> haven't had the bandwidth over the past week or two to take a close
> look
> >>> at
> >>> the work Istvan has been doing to unify the 5.x branches -- as one who
> >>> spends too much time cherry-picking I very much appreciate this effort!
> >>> :-)
> >>> )
> >>>
> >>> I still think the hardest part, as I think I mentioned in some previous
> >>> thread, is what to do when the HBase coprocs themselves diverge between
> >>> minor versions, as they can do. How do you handle the fact that
> >>> RegionObserver.preWALAppend exists in 1.5 but not 1.3 and 1.4, and will
> >>> exist in 2.3 but not 2.2 and 2.1? And that therefore any features that
> >>> depend on preWALAppend existing (none yet, but they're coming later
> this
> >>> year) will work in a 1.5 or 2.3 environment, but not a 1.4 or 2.2 one?
> >>>
> >>> Since our coprocessors tend to be giant monoliths, trying to create
> >>> release-based versions of them selectable by maven profile would
> >>> either require lots of developer copy/paste for each change, or a
> >>> significant (probably long overdue) refactor to make the coprocs small
> >>> shims that call out to smaller, fine-grained classes that can
> >>> occasionally
> >>> be release-specific.
> >>>
> >>> Geoffrey
> >>>
> >>> On Fri, Feb 7, 2020 at 9:31 AM Josh Elser <el...@apache.org> wrote:
> >>>
> >>> > Sounds like a good idea to me.
> >>> >
> >>> > On 2/6/20 8:40 AM, Istvan Toth wrote:
> >>> > > Hello!
> >>> > >
> >>> > > Now that we have a working solution in master for handling
> different
> >>> > HBase
> >>> > > minor versions, I think that we should think about applying the
> same
> >>> > > template to 4.x., and unifying the 4.x-HBase-1.3, 1.4, and 1.5
> >>> branches.
> >>> > >
> >>> > > Are there any intentional differences between the branches, apart
> >>> from
> >>> > > having to conform to the slightly different APIs ?
> >>> > > If there are, what are they, and are they considered blockers?
> >>> > >
> >>> > > Any other reasons not do this ?
> >>> > >
> >>> > > I expect that based on my experience with the master branch, I can
> do
> >>> > this
> >>> > > in a few days, but I don't want to put in the effort if there is no
> >>> > > interest in it.
> >>> > >
> >>> > > My plan is to take the 1.5 branch as a base.
> >>> > >
> >>> > > best regards
> >>> > > Istvan
> >>> > >
> >>> >
> >>>
> >>
> >>
> >> --
> >> *István Tóth* | Sr. Software Engineer
> >> t. (36) 70 283-1788
> >> stoty@cloudera.com <https://www.cloudera.com>
> >> [image: Cloudera] <https://www.cloudera.com/>
> >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> >> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> >> <https://www.cloudera.com/>
> >> ------------------------------
> >>
> >
>
Re: [DISCUSS] Unifying the 4.x branches
Posted by Istvan Toth <st...@apache.org>.
Now I have an unpolished version of the unified 4.x branch at
https://github.com/stoty/phoenix/tree/PHOENIX-5721
It takes the same approach as the master branch, though there are a bit
more differences to hide in the versions.
I need to finish the assembly stuff, go over once more the changes, and run
full ITs on each profile, but I expect that
the Java code will mostly see whitespace changes, so you can check the
logic behind the changes.
The HBase metrics stuff in particular is interesting, because the whole
feature is missing from 1.3, so it can be used as an example on how we can
adopt new HBase features later.
On Mon, Feb 10, 2020 at 9:01 AM Istvan Toth <st...@apache.org> wrote:
> Created PHOENIX-5721 <https://issues.apache.org/jira/browse/PHOENIX-5721> to
> track this.
>
> On Mon, Feb 10, 2020 at 8:21 AM István Tóth <st...@cloudera.com> wrote:
>
>> Thanks for the feedback, Geoffrey.
>>
>> I took the lazy option of just creating compatibility methods to paper
>> over the HBase API changes (emulate the latest version) when we are calling
>> into HBase.
>>
>> For the APIs implemented by Phoenix, I added compatibility superclasses.
>> So I expect that we will be able to add a dummy RegionObserver.preWALAppend
>> to the compatibility coprocessor superclass(es), so that the same code
>> compiles for older versions as well.
>>
>> Of course if other code paths depend on having that functionality, those
>> should also be gated by similar compatibility shims/version checks.
>>
>> My current approach is a quick fix, and does not preclude (in fact it
>> enables) later efforts at refactoring/cleanup.
>>
>> On Fri, Feb 7, 2020 at 7:31 PM Geoffrey Jacoby <gj...@apache.org>
>> wrote:
>>
>>> If unification could be done, that would be great. (I apologize that I
>>> haven't had the bandwidth over the past week or two to take a close look
>>> at
>>> the work Istvan has been doing to unify the 5.x branches -- as one who
>>> spends too much time cherry-picking I very much appreciate this effort!
>>> :-)
>>> )
>>>
>>> I still think the hardest part, as I think I mentioned in some previous
>>> thread, is what to do when the HBase coprocs themselves diverge between
>>> minor versions, as they can do. How do you handle the fact that
>>> RegionObserver.preWALAppend exists in 1.5 but not 1.3 and 1.4, and will
>>> exist in 2.3 but not 2.2 and 2.1? And that therefore any features that
>>> depend on preWALAppend existing (none yet, but they're coming later this
>>> year) will work in a 1.5 or 2.3 environment, but not a 1.4 or 2.2 one?
>>>
>>> Since our coprocessors tend to be giant monoliths, trying to create
>>> release-based versions of them selectable by maven profile would
>>> either require lots of developer copy/paste for each change, or a
>>> significant (probably long overdue) refactor to make the coprocs small
>>> shims that call out to smaller, fine-grained classes that can
>>> occasionally
>>> be release-specific.
>>>
>>> Geoffrey
>>>
>>> On Fri, Feb 7, 2020 at 9:31 AM Josh Elser <el...@apache.org> wrote:
>>>
>>> > Sounds like a good idea to me.
>>> >
>>> > On 2/6/20 8:40 AM, Istvan Toth wrote:
>>> > > Hello!
>>> > >
>>> > > Now that we have a working solution in master for handling different
>>> > HBase
>>> > > minor versions, I think that we should think about applying the same
>>> > > template to 4.x., and unifying the 4.x-HBase-1.3, 1.4, and 1.5
>>> branches.
>>> > >
>>> > > Are there any intentional differences between the branches, apart
>>> from
>>> > > having to conform to the slightly different APIs ?
>>> > > If there are, what are they, and are they considered blockers?
>>> > >
>>> > > Any other reasons not do this ?
>>> > >
>>> > > I expect that based on my experience with the master branch, I can do
>>> > this
>>> > > in a few days, but I don't want to put in the effort if there is no
>>> > > interest in it.
>>> > >
>>> > > My plan is to take the 1.5 branch as a base.
>>> > >
>>> > > best regards
>>> > > Istvan
>>> > >
>>> >
>>>
>>
>>
>> --
>> *István Tóth* | Sr. Software Engineer
>> t. (36) 70 283-1788
>> stoty@cloudera.com <https://www.cloudera.com>
>> [image: Cloudera] <https://www.cloudera.com/>
>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>> <https://www.cloudera.com/>
>> ------------------------------
>>
>
Re: [DISCUSS] Unifying the 4.x branches
Posted by Istvan Toth <st...@apache.org>.
Created PHOENIX-5721 <https://issues.apache.org/jira/browse/PHOENIX-5721> to
track this.
On Mon, Feb 10, 2020 at 8:21 AM István Tóth <st...@cloudera.com> wrote:
> Thanks for the feedback, Geoffrey.
>
> I took the lazy option of just creating compatibility methods to paper
> over the HBase API changes (emulate the latest version) when we are calling
> into HBase.
>
> For the APIs implemented by Phoenix, I added compatibility superclasses.
> So I expect that we will be able to add a dummy RegionObserver.preWALAppend
> to the compatibility coprocessor superclass(es), so that the same code
> compiles for older versions as well.
>
> Of course if other code paths depend on having that functionality, those
> should also be gated by similar compatibility shims/version checks.
>
> My current approach is a quick fix, and does not preclude (in fact it
> enables) later efforts at refactoring/cleanup.
>
> On Fri, Feb 7, 2020 at 7:31 PM Geoffrey Jacoby <gj...@apache.org> wrote:
>
>> If unification could be done, that would be great. (I apologize that I
>> haven't had the bandwidth over the past week or two to take a close look
>> at
>> the work Istvan has been doing to unify the 5.x branches -- as one who
>> spends too much time cherry-picking I very much appreciate this effort!
>> :-)
>> )
>>
>> I still think the hardest part, as I think I mentioned in some previous
>> thread, is what to do when the HBase coprocs themselves diverge between
>> minor versions, as they can do. How do you handle the fact that
>> RegionObserver.preWALAppend exists in 1.5 but not 1.3 and 1.4, and will
>> exist in 2.3 but not 2.2 and 2.1? And that therefore any features that
>> depend on preWALAppend existing (none yet, but they're coming later this
>> year) will work in a 1.5 or 2.3 environment, but not a 1.4 or 2.2 one?
>>
>> Since our coprocessors tend to be giant monoliths, trying to create
>> release-based versions of them selectable by maven profile would
>> either require lots of developer copy/paste for each change, or a
>> significant (probably long overdue) refactor to make the coprocs small
>> shims that call out to smaller, fine-grained classes that can occasionally
>> be release-specific.
>>
>> Geoffrey
>>
>> On Fri, Feb 7, 2020 at 9:31 AM Josh Elser <el...@apache.org> wrote:
>>
>> > Sounds like a good idea to me.
>> >
>> > On 2/6/20 8:40 AM, Istvan Toth wrote:
>> > > Hello!
>> > >
>> > > Now that we have a working solution in master for handling different
>> > HBase
>> > > minor versions, I think that we should think about applying the same
>> > > template to 4.x., and unifying the 4.x-HBase-1.3, 1.4, and 1.5
>> branches.
>> > >
>> > > Are there any intentional differences between the branches, apart from
>> > > having to conform to the slightly different APIs ?
>> > > If there are, what are they, and are they considered blockers?
>> > >
>> > > Any other reasons not do this ?
>> > >
>> > > I expect that based on my experience with the master branch, I can do
>> > this
>> > > in a few days, but I don't want to put in the effort if there is no
>> > > interest in it.
>> > >
>> > > My plan is to take the 1.5 branch as a base.
>> > >
>> > > best regards
>> > > Istvan
>> > >
>> >
>>
>
>
> --
> *István Tóth* | Sr. Software Engineer
> t. (36) 70 283-1788
> stoty@cloudera.com <https://www.cloudera.com>
> [image: Cloudera] <https://www.cloudera.com/>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> <https://www.cloudera.com/>
> ------------------------------
>
Re: [DISCUSS] Unifying the 4.x branches
Posted by István Tóth <st...@cloudera.com.INVALID>.
Thanks for the feedback, Geoffrey.
I took the lazy option of just creating compatibility methods to paper over
the HBase API changes (emulate the latest version) when we are calling into
HBase.
For the APIs implemented by Phoenix, I added compatibility superclasses. So
I expect that we will be able to add a dummy RegionObserver.preWALAppend to
the compatibility coprocessor superclass(es), so that the same code
compiles for older versions as well.
Of course if other code paths depend on having that functionality, those
should also be gated by similar compatibility shims/version checks.
My current approach is a quick fix, and does not preclude (in fact it
enables) later efforts at refactoring/cleanup.
On Fri, Feb 7, 2020 at 7:31 PM Geoffrey Jacoby <gj...@apache.org> wrote:
> If unification could be done, that would be great. (I apologize that I
> haven't had the bandwidth over the past week or two to take a close look at
> the work Istvan has been doing to unify the 5.x branches -- as one who
> spends too much time cherry-picking I very much appreciate this effort! :-)
> )
>
> I still think the hardest part, as I think I mentioned in some previous
> thread, is what to do when the HBase coprocs themselves diverge between
> minor versions, as they can do. How do you handle the fact that
> RegionObserver.preWALAppend exists in 1.5 but not 1.3 and 1.4, and will
> exist in 2.3 but not 2.2 and 2.1? And that therefore any features that
> depend on preWALAppend existing (none yet, but they're coming later this
> year) will work in a 1.5 or 2.3 environment, but not a 1.4 or 2.2 one?
>
> Since our coprocessors tend to be giant monoliths, trying to create
> release-based versions of them selectable by maven profile would
> either require lots of developer copy/paste for each change, or a
> significant (probably long overdue) refactor to make the coprocs small
> shims that call out to smaller, fine-grained classes that can occasionally
> be release-specific.
>
> Geoffrey
>
> On Fri, Feb 7, 2020 at 9:31 AM Josh Elser <el...@apache.org> wrote:
>
> > Sounds like a good idea to me.
> >
> > On 2/6/20 8:40 AM, Istvan Toth wrote:
> > > Hello!
> > >
> > > Now that we have a working solution in master for handling different
> > HBase
> > > minor versions, I think that we should think about applying the same
> > > template to 4.x., and unifying the 4.x-HBase-1.3, 1.4, and 1.5
> branches.
> > >
> > > Are there any intentional differences between the branches, apart from
> > > having to conform to the slightly different APIs ?
> > > If there are, what are they, and are they considered blockers?
> > >
> > > Any other reasons not do this ?
> > >
> > > I expect that based on my experience with the master branch, I can do
> > this
> > > in a few days, but I don't want to put in the effort if there is no
> > > interest in it.
> > >
> > > My plan is to take the 1.5 branch as a base.
> > >
> > > best regards
> > > Istvan
> > >
> >
>
--
*István Tóth* | Sr. Software Engineer
t. (36) 70 283-1788
stoty@cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
<https://www.cloudera.com/>
------------------------------
Re: [DISCUSS] Unifying the 4.x branches
Posted by Geoffrey Jacoby <gj...@apache.org>.
If unification could be done, that would be great. (I apologize that I
haven't had the bandwidth over the past week or two to take a close look at
the work Istvan has been doing to unify the 5.x branches -- as one who
spends too much time cherry-picking I very much appreciate this effort! :-)
)
I still think the hardest part, as I think I mentioned in some previous
thread, is what to do when the HBase coprocs themselves diverge between
minor versions, as they can do. How do you handle the fact that
RegionObserver.preWALAppend exists in 1.5 but not 1.3 and 1.4, and will
exist in 2.3 but not 2.2 and 2.1? And that therefore any features that
depend on preWALAppend existing (none yet, but they're coming later this
year) will work in a 1.5 or 2.3 environment, but not a 1.4 or 2.2 one?
Since our coprocessors tend to be giant monoliths, trying to create
release-based versions of them selectable by maven profile would
either require lots of developer copy/paste for each change, or a
significant (probably long overdue) refactor to make the coprocs small
shims that call out to smaller, fine-grained classes that can occasionally
be release-specific.
Geoffrey
On Fri, Feb 7, 2020 at 9:31 AM Josh Elser <el...@apache.org> wrote:
> Sounds like a good idea to me.
>
> On 2/6/20 8:40 AM, Istvan Toth wrote:
> > Hello!
> >
> > Now that we have a working solution in master for handling different
> HBase
> > minor versions, I think that we should think about applying the same
> > template to 4.x., and unifying the 4.x-HBase-1.3, 1.4, and 1.5 branches.
> >
> > Are there any intentional differences between the branches, apart from
> > having to conform to the slightly different APIs ?
> > If there are, what are they, and are they considered blockers?
> >
> > Any other reasons not do this ?
> >
> > I expect that based on my experience with the master branch, I can do
> this
> > in a few days, but I don't want to put in the effort if there is no
> > interest in it.
> >
> > My plan is to take the 1.5 branch as a base.
> >
> > best regards
> > Istvan
> >
>
Re: [DISCUSS] Unifying the 4.x branches
Posted by Josh Elser <el...@apache.org>.
Sounds like a good idea to me.
On 2/6/20 8:40 AM, Istvan Toth wrote:
> Hello!
>
> Now that we have a working solution in master for handling different HBase
> minor versions, I think that we should think about applying the same
> template to 4.x., and unifying the 4.x-HBase-1.3, 1.4, and 1.5 branches.
>
> Are there any intentional differences between the branches, apart from
> having to conform to the slightly different APIs ?
> If there are, what are they, and are they considered blockers?
>
> Any other reasons not do this ?
>
> I expect that based on my experience with the master branch, I can do this
> in a few days, but I don't want to put in the effort if there is no
> interest in it.
>
> My plan is to take the 1.5 branch as a base.
>
> best regards
> Istvan
>