You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by Josh Elser <el...@apache.org> on 2020/03/02 23:25:11 UTC
Re: [DISCUSS] Unifying the 4.x branches
Left a couple of minor requests to fix (no consequence changes --
comments, whitespace), which you can just fix on commit. Great work, Istvan.
Any of y'all want to look before Istvan merges it?
On 2/27/20 4:58 AM, Istvan Toth wrote:
> Thanks Andrew, I like that solution.
>
> I've added the "final" PRs and patches to
> https://issues.apache.org/jira/browse/PHOENIX-5721 ,
> they are ready for review.
>
> On Wed, Feb 19, 2020 at 9:01 PM Andrew Purtell <an...@gmail.com>
> wrote:
>
>> Since you have already done this great work, and 1.3 isn’t dead yet, and
>> won’t be “this year”, and it serves as an example of how to bring in entire
>> new features on later code lines, perhaps it should go in. Just my 0.02.
>>
>>> On Feb 19, 2020, at 10:39 AM, Istvan Toth <st...@apache.org> wrote:
>>>
>>> Geoffrey,
>>>
>>> Absolutely.
>>> 80% of this patch is dealing with 1.3.
>>> 1.4 vs 1.5 affects two or three java files.
>>>
>>> My game plan is to submit two different patches, a small one for 1.4 and
>>> 1.5 support, and a bigger one other that adds 1.3, so that it can be
>>> reverted easily after 1.3 is dropped.
>>>
>>> I think that maintaining a separate 1.3 branch is the least desirable
>>> outcome, as we'd still have two 4.x branches to maintain, with different
>>> jenkins jobs, and slightly different jar names and maven artifact
>> structure.
>>>
>>> Of course the easiest route is to just drop 1.3 support ASAP, and then
>>> unify 1.4 and 1.5 .
>>>
>>> If we are not ready to drop 1.3 very soon, I'd vote for unifying with
>>> 1.3 included (I've finished the hard part), and then reverting a lot of
>>> the compatibility cruft when we drop 1.3 support.
>>>
>>> As for the 4.15 maintenance releases, those could stay on the current
>>> branches, as we probably want point releases to be drop-in compatible,
>>> and the unified version changes the maven artifact versioning, and some
>>> jar filenames. This would be one way to extend some support for 1.3
>>>
>>> regards
>>> István
>>>
>>>
>>>
>>>> On 2020. 02. 19. 19:11, Geoffrey Jacoby wrote:
>>>> Istvan,
>>>>
>>>> The HBase community has been on the verge of EOLing 1.3 for some time
>>>> now -- are there significant gains or simplification if we either end
>>>> 1.3 support in Phoenix before PHOENIX-5721 goes in, or alternately,
>>>> don't include it in the unified profile since it will be EOLed in the
>>>> not-too-distant-future even if we don't do it now?
>>>>
>>>> Geoffrey
>>>>
>>>> On Wed, Feb 19, 2020 at 5:36 AM Istvan Toth <stoty@apache.org
>>>> <ma...@apache.org>> wrote:
>>>>
>>>> Now I have an unpolished version of the unified 4.x branch at
>>>> https://github.com/stoty/phoenix/tree/PHOENIX-5721
>>>>
>>>> It takes the same approach as the master branch, though there are a
>> bit
>>>> more differences to hide in the versions.
>>>>
>>>> I need to finish the assembly stuff, go over once more the changes,
>>>> and run
>>>> full ITs on each profile, but I expect that
>>>> the Java code will mostly see whitespace changes, so you can check
>> the
>>>> logic behind the changes.
>>>>
>>>> The HBase metrics stuff in particular is interesting, because the
>> whole
>>>> feature is missing from 1.3, so it can be used as an example on how
>>>> we can
>>>> adopt new HBase features later.
>>>>
>>>> On Mon, Feb 10, 2020 at 9:01 AM Istvan Toth <stoty@apache.org
>>>> <ma...@apache.org>> wrote:
>>>>
>>>>> Created PHOENIX-5721
>>>> <https://issues.apache.org/jira/browse/PHOENIX-5721> to
>>>>> track this.
>>>>>
>>>>> On Mon, Feb 10, 2020 at 8:21 AM István Tóth <stoty@cloudera.com
>>>> <ma...@cloudera.com>> wrote:
>>>>>
>>>>>> Thanks for the feedback, Geoffrey.
>>>>>>
>>>>>> I took the lazy option of just creating compatibility methods to
>>>> paper
>>>>>> over the HBase API changes (emulate the latest version) when we
>>>> are calling
>>>>>> into HBase.
>>>>>>
>>>>>> For the APIs implemented by Phoenix, I added compatibility
>>>> superclasses.
>>>>>> So I expect that we will be able to add a dummy
>>>> RegionObserver.preWALAppend
>>>>>> to the compatibility coprocessor superclass(es), so that the same
>>>> code
>>>>>> compiles for older versions as well.
>>>>>>
>>>>>> Of course if other code paths depend on having that
>>>> functionality, those
>>>>>> should also be gated by similar compatibility shims/version checks.
>>>>>>
>>>>>> My current approach is a quick fix, and does not preclude (in fact it
>>>>>> enables) later efforts at refactoring/cleanup.
>>>>>>
>>>>>> On Fri, Feb 7, 2020 at 7:31 PM Geoffrey Jacoby
>>>> <gjacoby@apache.org <ma...@apache.org>>
>>>>>> wrote:
>>>>>>
>>>>>>> If unification could be done, that would be great. (I apologize
>>>> that I
>>>>>>> haven't had the bandwidth over the past week or two to take a
>>>> close look
>>>>>>> at
>>>>>>> the work Istvan has been doing to unify the 5.x branches -- as
>>>> one who
>>>>>>> spends too much time cherry-picking I very much appreciate this
>>>> effort!
>>>>>>> :-)
>>>>>>> )
>>>>>>>
>>>>>>> I still think the hardest part, as I think I mentioned in some
>>>> previous
>>>>>>> thread, is what to do when the HBase coprocs themselves diverge
>>>> between
>>>>>>> minor versions, as they can do. How do you handle the fact that
>>>>>>> RegionObserver.preWALAppend exists in 1.5 but not 1.3 and 1.4,
>>>> and will
>>>>>>> exist in 2.3 but not 2.2 and 2.1? And that therefore any
>>>> features that
>>>>>>> depend on preWALAppend existing (none yet, but they're coming
>>>> later this
>>>>>>> year) will work in a 1.5 or 2.3 environment, but not a 1.4 or
>>>> 2.2 one?
>>>>>>>
>>>>>>> Since our coprocessors tend to be giant monoliths, trying to create
>>>>>>> release-based versions of them selectable by maven profile would
>>>>>>> either require lots of developer copy/paste for each change, or a
>>>>>>> significant (probably long overdue) refactor to make the coprocs
>>>> small
>>>>>>> shims that call out to smaller, fine-grained classes that can
>>>>>>> occasionally
>>>>>>> be release-specific.
>>>>>>>
>>>>>>> Geoffrey
>>>>>>>
>>>>>>> On Fri, Feb 7, 2020 at 9:31 AM Josh Elser <elserj@apache.org
>>>> <ma...@apache.org>> wrote:
>>>>>>>
>>>>>>>> Sounds like a good idea to me.
>>>>>>>>
>>>>>>>> On 2/6/20 8:40 AM, Istvan Toth wrote:
>>>>>>>>> Hello!
>>>>>>>>>
>>>>>>>>> Now that we have a working solution in master for handling
>>>> different
>>>>>>>> HBase
>>>>>>>>> minor versions, I think that we should think about applying
>>>> the same
>>>>>>>>> template to 4.x., and unifying the 4.x-HBase-1.3, 1.4, and 1.5
>>>>>>> branches.
>>>>>>>>>
>>>>>>>>> Are there any intentional differences between the branches,
>>>> apart
>>>>>>> from
>>>>>>>>> having to conform to the slightly different APIs ?
>>>>>>>>> If there are, what are they, and are they considered blockers?
>>>>>>>>>
>>>>>>>>> Any other reasons not do this ?
>>>>>>>>>
>>>>>>>>> I expect that based on my experience with the master branch,
>>>> I can do
>>>>>>>> this
>>>>>>>>> in a few days, but I don't want to put in the effort if
>>>> there is no
>>>>>>>>> interest in it.
>>>>>>>>>
>>>>>>>>> My plan is to take the 1.5 branch as a base.
>>>>>>>>>
>>>>>>>>> best regards
>>>>>>>>> Istvan
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *István Tóth* | Sr. Software Engineer
>>>>>> t. (36) 70 283-1788
>>>>>> stoty@cloudera.com <ma...@cloudera.com>
>>>> <https://www.cloudera.com>
>>>>>> [image: Cloudera] <https://www.cloudera.com/>
>>>>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>>>>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>>>>>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>>>>>> <https://www.cloudera.com/>
>>>>>> ------------------------------
>>>>>>
>>>>>
>>>>
>>
>