You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by Xinyi Yan <ya...@apache.org> on 2020/12/01 03:30:24 UTC
[Discuss] Releasing Phoenix 4.16
Hi all,
It's time to discuss the Phoenix 4.16 release. After many people's
contributions on the bug fixes, new features, and other works in the past
few months, we are kind of close to the point to have a RC (still need to
fix test flappers). Please let me know if you think any JIRA must be part
of the Phoenix 4.16 release other than major blocker PHOENIX-5712.
If no surprise comes up, I will not wait for any new major features and
focus on the RC as soon as possible.
Sincerely,
Xinyi
Re: [Discuss] Releasing Phoenix 4.16
Posted by Istvan Toth <st...@cloudera.com.INVALID>.
The new scripts also automatically generate the CHANGES and README files.
(Of course it's also possible to run those Yetus scripts as part of the
make-rc release process)
On Sat, Jan 16, 2021 at 6:38 AM Istvan Toth <st...@cloudera.com> wrote:
> Hi!
>
> I can see that Xinyi has filed PHOENX-6319 and PHOENIX-6320 for the "old"
> make_rc script.
> While of course every release manager is free to choose to the tools they
> use, I ask you to consider using
> the HBase-derived scripts at dev/create-release in the master branch. They
> are not copied to 4.x,
> because they operate directly on the ASF git repository, but they are
> capable of handling a 4.x release,
>
> Using them has several advantages:
> * They handle the profiles in a single step, meaning less manual work and
> place for errors
> * They handle more step of the release process automatically, again less
> work and more consistency
>
> * I also intend to use them for 5.0, and as they use the HBase convention
> for the files names and git tags,
> using them would mean consistency for the 4.16 and 5.1 release artifacts.
>
> They are documented in the README file, and in the patch attached to
> PHOENIX-6315.
> There is a wall of text in the README about using the scripts with GKE,
> but don't let that put you off,
> that is completely optional. (I have never even tested that)
> I am also happy to assist if you run into trouble with them.
>
> Istvan
>
> On Wed, Jan 13, 2021 at 4:12 AM Xinyi Yan <ya...@apache.org> wrote:
>
>> Thank you for the update and great work, Istvan. I feel the same way, this
>> is the best approach so far for 4.16 and 5.1.
>>
>> On Tue, Jan 12, 2021 at 7:01 PM Josh Elser <el...@apache.org> wrote:
>>
>> > IMO, a slow build is better than forcing users to run their own dev
>> > build of a release.
>> >
>> > Very proud of all of the work y'all have been doing on 4.16. Keep
>> > pushing on the release :)
>> >
>> > On 1/12/21 12:35 PM, Istvan Toth wrote:
>> > > I've been working this, and came up with the following:
>> > >
>> > > * We no longer generate phoenix-client.jar and phoenix-server.jar, we
>> > call
>> > > them phoenix-client-hbase-X.Y.jar and phoenix-server-hbase-X.Y.jar
>> > instead.
>> > > * These file names are used in the binary assemblies, as well as as
>> maven
>> > > coordinates for consistency.
>> > > * We generate four release binaries, for each HBase profile
>> > > * We build and publish to maven the phoenix-client-hbase-X.Y and
>> > > phoenix-server-hbase-X.Y artifacts for each profile (Actually, we
>> deploy
>> > > four times, but the rest of the artifacts are identical)
>> > >
>> > > * On the downside, running the release script on a Macbook takes
>> several
>> > > hours, as we build HBase 4 and Phoenix 8 times on the slow and kludgy
>> Mac
>> > > Docker filesystem.
>> > >
>> > > The almost finished patch to the build system is linked to
>> > > https://issues.apache.org/jira/browse/PHOENIX-6307
>> > >
>> > > 4.x would have the same build system changes once this is approved.
>> > >
>> > > Please check it out, and let me know if this solution is
>> satisfactory, or
>> > > if you have a better idea.
>> > >
>> > > regards
>> > > Istvan
>> > >
>> > >
>> > > On Fri, Jan 8, 2021 at 4:22 PM Istvan Toth <st...@cloudera.com>
>> wrote:
>> > >
>> > >> As I started to work on this I realized that while providing binary
>> > >> tarballs for each HBase profile is fine,
>> > >> this does not solve the maven artifact issue.
>> > >>
>> > >> Are we OK with publishing a single phoenix-client maven artifact (for
>> > the
>> > >> oldest HBase),
>> > >> or do we want to publish a separate one for each HBase version ?
>> > >>
>> > >> I looked at publishing multiple client versions, but none of them are
>> > >> particularly easy or attractive.
>> > >>
>> > >> The best I could come up with is adding a separate maven module for
>> > (each
>> > >> version x embedded) (i.e 6 for 4.x, 8 for master),
>> > >> and activating them according to hbase.profile.
>> > >>
>> > >> This would also mean that we need to add the hbase version to the
>> > artifact
>> > >> id. i.e.: phoenix-client-hbase-2.1
>> > >>
>> > >> Once we publish separate binaries for the HBase profiles, we can undo
>> > the
>> > >> change that excludes compat-module from phoenix-server,
>> > >> and shade it in again for the binary assemblies.
>> > >>
>> > >> In this case we'd also have to do something about the phoenix-server
>> > maven
>> > >> artifacts. Either they get the same treatment as phoenix-client,
>> > >> or we simply skip publishing them. I personally do not see anyone
>> > getting
>> > >> phoenix-server.jar from maven.
>> > >>
>> > >> The easiest version is
>> > >> * publish the oldest profile phoenix-client to maven
>> > >> * do not publish phoenix-server to maven
>> > >> * add compat-module to phoenix-server for the binary artifact
>> > >>
>> > >>
>> > >> regards
>> > >> Istvan
>> > >>
>> > >> On Fri, Jan 8, 2021 at 6:56 AM Istvan Toth <st...@cloudera.com>
>> wrote:
>> > >>
>> > >>> On the release binaries:
>> > >>>
>> > >>> The current solution (which the default profile change has broken)
>> was
>> > >>> based on Lars's idea at
>> > >>>
>> > >>>
>> >
>> https://issues.apache.org/jira/browse/PHOENIX-5902?focusedCommentId=17125122&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17125122
>> > >>> However, I agree that providing separate assemblies for each HBase
>> > >>> profile is better for our users, as they won't have to rebuild
>> Phoenix
>> > to
>> > >>> take advantage of any new features, and to get the general
>> > improvements in
>> > >>> later HBase releases.
>> > >>> I have opened https://issues.apache.org/jira/browse/PHOENIX-6307 to
>> > >>> track this.
>> > >>>
>> > >>> On the 5.1 release:
>> > >>>
>> > >>> Yes, I do want to release 5.1 shortly. In fact $dayjob desperately
>> > needs
>> > >>> it ASAP.
>> > >>> I have a few older PRs waiting for review that fell between the
>> cracks,
>> > >>> but as soon as those are merged, I want to cut the first RC.
>> > >>> It would be nice to have 4.16 and 5.1 as close as possible, and
>> > >>> PHOENIX-6211 seems to be ready, so I hope to include it in 5.1 too.
>> > >>> I will start an official 5.1 release thread and volunteer to be the
>> > >>> release manager soon. (unless you want to take that up too, Xinyi).
>> > >>>
>> > >>>
>> > >>>
>> > >>> On Fri, Jan 8, 2021 at 1:08 AM Xinyi Yan <ya...@apache.org>
>> wrote:
>> > >>>
>> > >>>> If we can modify the dev/create-release scripts and make them work
>> for
>> > >>>> the
>> > >>>> 4.16 release with this hbase.profile option, it would make our life
>> > much
>> > >>>> easier to release multiple HBase profiles from the single branch in
>> > the
>> > >>>> future too(the master branch will have a release shortly right?).
>> > >>>> Geoffrey
>> > >>>> and Istvan, what do you think?
>> > >>>>
>> > >>>> Thanks,
>> > >>>> Xinyi
>> > >>>>
>> > >>>> On Thu, Jan 7, 2021 at 11:28 AM Geoffrey Jacoby <
>> gjacoby@apache.org>
>> > >>>> wrote:
>> > >>>>
>> > >>>>> Thanks for bringing up the default branch issue, Istvan, I've been
>> > >>>> meaning
>> > >>>>> to start a conversation about it on this list.
>> > >>>>>
>> > >>>>> As part of PHOENIX-5435, I changed the default 4.x HBase release
>> to
>> > >>>> 1.5 and
>> > >>>>> the default 5.x HBase to 2.3 because the WAL annotation feature
>> > >>>> introduced
>> > >>>>> by 5435 only works with HBase 1.5+ or 2.3+. (It depends on a
>> coproc
>> > >>>> hook
>> > >>>>> introduced in HBASE-22623). That means that all tests of that
>> feature
>> > >>>> must
>> > >>>>> no-op when run against an earlier HBase, which means that it would
>> > >>>> never be
>> > >>>>> tested in our CI pipelines if the default was 1.3 or 2.1.
>> > >>>>>
>> > >>>>> This has come up quite a few times recently. In particular, the
>> new
>> > >>>>> indexing framework runs in a degraded state against HBase 2.1 and
>> 2.2
>> > >>>>> (still better than the old indexing framework though!), because we
>> > lack
>> > >>>>> support in 2.1 for Filters on raw scans and can't use the "max
>> > lookback
>> > >>>>> age" feature with HBase 2.1 or 2.2, which keeps major compactions
>> > from
>> > >>>>> messing with index rebuilds. That's why lots of indexing tests
>> no-op
>> > or
>> > >>>>> have to make different assertions when run against 2.1 or 2.2, so
>> we
>> > >>>> only
>> > >>>>> properly test the indexing framework if CI and local developer
>> > >>>>> tests run against HBase 2.3 or up.
>> > >>>>>
>> > >>>>> As for the release, don't we usually release separate binaries
>> that
>> > are
>> > >>>>> suitable for all the HBase versions we support? Back before we
>> had a
>> > >>>>> unified 4.x branch, I believe we used to release from 3 different
>> > >>>>> 4.x-HBase-* branches each time. I've been assuming that part of
>> the
>> > >>>> release
>> > >>>>> process would be generating binaries for 4.x-HBase-1.3,
>> > 4.x-HBase-1.4,
>> > >>>>> 4.x-HBase-1.5, and 4.x-HBase-1.6 (if that was different from 1.5).
>> > >>>>>
>> > >>>>> Geoffrey
>> > >>>>>
>> > >>>>> On Thu, Jan 7, 2021 at 11:19 AM Istvan Toth <st...@apache.org>
>> > wrote:
>> > >>>>>
>> > >>>>>> Two more issues came to my mind:
>> > >>>>>>
>> > >>>>>> In PHOENIX-5435 the default profile was changed to HBase 1.5.0
>> > >>>>>> This causes two conflicts:
>> > >>>>>> - Our documentation says that the default profile is the lowest
>> > >>>>> supported,
>> > >>>>>> which is no longer true
>> > >>>>>> - Unless we change our binary release process, the published
>> maven
>> > >>>>>> artifacts and binaries will also be built with HBase 1.5, thus
>> they
>> > >>>> will
>> > >>>>> be
>> > >>>>>> incompatible with Hbase 1.3 and 1.4
>> > >>>>>>
>> > >>>>>> This should be addressed before release.
>> > >>>>>> One possible solution:
>> > >>>>>> * Add an "oldest" hbase.profile option which always chooses the
>> > >>>> oldest
>> > >>>>>> supported version
>> > >>>>>> * Update the release scripts to specify this profile
>> > >>>>>> * Update the docs.
>> > >>>>>>
>> > >>>>>> I have also adopted the hbase release scripts to Phoenix, they
>> are
>> > >>>>> present
>> > >>>>>> in dev/create-release in the master branch, but
>> > >>>>>> should be able to perform 4.16 release as well. I have used this
>> for
>> > >>>> the
>> > >>>>>> phoenix-thirdparty, omid, and tephra releases, but no live
>> phoenix
>> > >>>>> releases
>> > >>>>>> have been done with them yet, obviously.
>> > >>>>>>
>> > >>>>>> regards
>> > >>>>>> Istvan
>> > >>>>>>
>> > >>>>>> On Thu, Jan 7, 2021 at 8:19 AM Istvan Toth <st...@apache.org>
>> > wrote:
>> > >>>>>>
>> > >>>>>>> Hi!
>> > >>>>>>>
>> > >>>>>>> A question of process: Should we backport fixes to the 4.16
>> branch
>> > >>>> at
>> > >>>>> our
>> > >>>>>>> discretion, or is backporting those handled by the release
>> manager
>> > >>>>>> (Xinyi) ?
>> > >>>>>>>
>> > >>>>>>> On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth <st...@cloudera.com>
>> > >>>> wrote:
>> > >>>>>>>
>> > >>>>>>>> Hi!
>> > >>>>>>>>
>> > >>>>>>>> Thanks for everyone's effort on fixing the flakey tests.
>> > >>>>>>>> However, our ASF Jenkins runs are still very unstable, and I am
>> > >>>> afraid
>> > >>>>>>>> that the single clean 4.x run was down more to luck than to
>> > >>>> fixing the
>> > >>>>>>>> underlying problem.
>> > >>>>>>>> While I do not consider this a blocker, fixing this would make
>> the
>> > >>>>> pre-
>> > >>>>>>>> and post commit tests far more useful, and let us start with a
>> > >>>> clean
>> > >>>>>> slate
>> > >>>>>>>> for the next release.
>> > >>>>>>>> I have created
>> https://issues.apache.org/jira/browse/PHOENIX-6288
>> > >>>> to
>> > >>>>>>>> track the generic cluster setup issue, but my attempts to fix
>> > >>>> this, or
>> > >>>>>> at
>> > >>>>>>>> least figure out the root cause have been unsuccessful so far.
>> > >>>>>>>>
>> > >>>>>>>> I ask for your help in fixing this issue. I have added some
>> > >>>>> inconclusive
>> > >>>>>>>> analysis to the ticket, and the full failsafe output directory
>> is
>> > >>>>>>>> downloadable as artifacts from the failed multibranch runs
>> (stored
>> > >>>>> for a
>> > >>>>>>>> few days), which should hold the answer to this riddle.
>> > >>>>>>>>
>> > >>>>>>>> regards,
>> > >>>>>>>> Istvan
>> > >>>>>>>>
>> > >>>>>>>> On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan <ya...@apache.org>
>> > >>>> wrote:
>> > >>>>>>>>
>> > >>>>>>>>> Hi,
>> > >>>>>>>>>
>> > >>>>>>>>> We finally have a stable 4.x branch build after many flapper
>> test
>> > >>>>>> fixing
>> > >>>>>>>>> contributions from the community. I'm going to fork a new
>> > >>>>> branch(4.16)
>> > >>>>>>>>> from
>> > >>>>>>>>> the current 4.x branch for the 4.16 release. As a cutoff, I
>> will
>> > >>>> not
>> > >>>>>>>>> include any new features other than PHOENIX-6211 and bug fix.
>> > >>>> Please
>> > >>>>>> let
>> > >>>>>>>>> me
>> > >>>>>>>>> know if you have any questions or concerns.
>> > >>>>>>>>>
>> > >>>>>>>>> Thanks,
>> > >>>>>>>>> Xinyi
>> > >>>>>>>>>
>> > >>>>>>>>> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani <
>> vjasani@apache.org
>> > >>>>>
>> > >>>>>> wrote:
>> > >>>>>>>>>
>> > >>>>>>>>>> An update on test stability:
>> > >>>>>>>>>>
>> > >>>>>>>>>> As per recent 4.x build results, we are left with very few
>> > >>>> flappers
>> > >>>>>> and
>> > >>>>>>>>>> specifically
>> > >>>>>>>>>> with HBase profile 1.6, I can see recent 7 build (multibranch
>> > >>>> + PR
>> > >>>>>>>>>> precommit results)
>> > >>>>>>>>>> results without any test failure.
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>> On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <
>> yanxinyi@apache.org
>> > >>>>>
>> > >>>>>> wrote:
>> > >>>>>>>>>>
>> > >>>>>>>>>>> Yes, we are currently working on fixing the 4.x branch test
>> > >>>>>> flappers
>> > >>>>>>>>>> while
>> > >>>>>>>>>>> waiting for the PHOENIX-5435. After that, I will try to have
>> > >>>> an
>> > >>>>> RC
>> > >>>>>>>>> ASAP.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <
>> > >>>>>>>>> ankitsinghal59@gmail.com>
>> > >>>>>>>>>>> wrote:
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>> I see that both the blockers listed here PHOENIX-5712 and
>> > >>>>>>>>> PHOENIX-6241
>> > >>>>>>>>>>>> have been resolved(Thanks to Xinyi), and also as per the
>> > >>>> JIRA
>> > >>>>>> query
>> > >>>>>>>>>> there
>> > >>>>>>>>>>>> is no Jira marked as a blocker for 4.16 except the one
>> > >>>> related
>> > >>>>> to
>> > >>>>>>>>>>>> documentation
>> > >>>>>>>>>>>> for "splittable catalog table".
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Xinyi, so are we good to start the release process now?
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <
>> > >>>> yanxinyi@apache.org>
>> > >>>>>>>>> wrote:
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>> Thanks for replying and providing suggestions. I looked
>> > >>>> at
>> > >>>>> the
>> > >>>>>>>>> wrong
>> > >>>>>>>>>>>> result
>> > >>>>>>>>>>>>> Jira list that Daniel provided and did some local
>> > >>>> testing,
>> > >>>>> and
>> > >>>>>>>>> here
>> > >>>>>>>>>> is
>> > >>>>>>>>>>>> the
>> > >>>>>>>>>>>>> result:
>> > >>>>>>>>>>>>> [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642:
>> > >>>>> cannot
>> > >>>>>>>>>>> reproduce
>> > >>>>>>>>>>>>> it.
>> > >>>>>>>>>>>>> [More information is required] PHOENIX-4504 cannot
>> > >>>> reproduce
>> > >>>>> it
>> > >>>>>>>>> but
>> > >>>>>>>>>>>> someone
>> > >>>>>>>>>>>>> claimed that he had a similar issue.
>> > >>>>>>>>>>>>> [unusual query] PHOENIX-4540 and PHOENIX-6217
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Based on my finding, I think it's better to have more
>> > >>>>> frequent
>> > >>>>>>>>>>>> housekeeping
>> > >>>>>>>>>>>>> and resolve unreproducible bugs especially since many of
>> > >>>> them
>> > >>>>>> are
>> > >>>>>>>>>>>>> considering out of date (phoenix-4.11 or even
>> > >>>> phoenix-4.6).
>> > >>>>>>>>> Since I
>> > >>>>>>>>>>> still
>> > >>>>>>>>>>>>> need time to work on the blocker Jira(PHOENIX-5712,
>> > >>>>>>>>> PHOENIX-6241) and
>> > >>>>>>>>>>> fix
>> > >>>>>>>>>>>>> test flappers, if you want to fix "unusual query" bugs,
>> > >>>> feel
>> > >>>>>>>>> free to
>> > >>>>>>>>>> do
>> > >>>>>>>>>>>> so.
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Sincerely,
>> > >>>>>>>>>>>>> Xinyi
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <
>> > >>>>>> ankit@apache.org>
>> > >>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Thanks Daniel and appreciate the effort you put in
>> > >>>> getting
>> > >>>>>> the
>> > >>>>>>>>> list
>> > >>>>>>>>>>>> ready
>> > >>>>>>>>>>>>>> for bugs producing wrong results
>> > >>>>>>>>>>>>>> but none of them seems to be a blocker to me for 4.16
>> > >>>> as
>> > >>>>> they
>> > >>>>>>>>> are
>> > >>>>>>>>>> not
>> > >>>>>>>>>>>> the
>> > >>>>>>>>>>>>>> regression and doesn't break the general functionality
>> > >>>>>>>>>>>>>> except for specific features, RVC/desc as Chenglei also
>> > >>>>>>>>> pointed out
>> > >>>>>>>>>>>>> (though
>> > >>>>>>>>>>>>>> I'll defer the assessment to RM "Xinyi").
>> > >>>>>>>>>>>>>> Probably these can be a part of 4.16.1 or we can do
>> > >>>> 4.17.0
>> > >>>>>> soon
>> > >>>>>>>>>> maybe
>> > >>>>>>>>>>>>> after
>> > >>>>>>>>>>>>>> a few weeks/month?
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Considering that we have already fixed 137 bugs and
>> > >>>> done
>> > >>>>> 85+
>> > >>>>>>>>>>>>>> improvements/features in 4.16,
>> > >>>>>>>>>>>>>> it will not be a good idea to deprive the user from
>> > >>>> such
>> > >>>>>> fixes.
>> > >>>>>>>>>>>>>> It's been a year since our last 4.15 release, having no
>> > >>>>>> release
>> > >>>>>>>>>>> brings
>> > >>>>>>>>>>>>> more
>> > >>>>>>>>>>>>>> questions on the project
>> > >>>>>>>>>>>>>> rather than the bugs which affect a certain % of
>> > >>>>>> feature/users,
>> > >>>>>>>>>> would
>> > >>>>>>>>>>>> the
>> > >>>>>>>>>>>>>> release notes
>> > >>>>>>>>>>>>>> explaining the stability of certain features set the
>> > >>>> right
>> > >>>>>>>>>>> expectation
>> > >>>>>>>>>>>>> for
>> > >>>>>>>>>>>>>> those users who rely on these features to wait for a
>> > >>>> future
>> > >>>>>>>>>> release?
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Regards,
>> > >>>>>>>>>>>>>> Ankit Singhal
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
>> > >>>>>>>>>>>> chenglei@apache.org>
>> > >>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> In my opinion, we should keep releases light and
>> > >>>>> frequent,
>> > >>>>>>>>> and
>> > >>>>>>>>>> for
>> > >>>>>>>>>>>>> some
>> > >>>>>>>>>>>>>>> unusual query bugs like RVC and DESC
>> > >>>>>>>>>>>>>>> we could delay fix to next release . I think we
>> > >>>> should
>> > >>>>>>>>> release
>> > >>>>>>>>>>> 4.16.0
>> > >>>>>>>>>>>>> and
>> > >>>>>>>>>>>>>>> 5.1.0 as quickly as possible. In China, many users
>> > >>>>>>>>>>>>>>> in HBase&Phoenix User Group thought that Phoenix was
>> > >>>>> dead
>> > >>>>>>>>>> because
>> > >>>>>>>>>>>> our
>> > >>>>>>>>>>>>>> too
>> > >>>>>>>>>>>>>>> long interval release and stopped using
>> > >>>>>>>>>>>>>>> Phoenix.
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
>> > >>>>>>>>>>>> chinmayskulkarni@gmail.com
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>> I agree. These are all major bugs and we should aim
>> > >>>> at
>> > >>>>>>>>> solving
>> > >>>>>>>>>>> them
>> > >>>>>>>>>>>>>> after
>> > >>>>>>>>>>>>>>>> checking that they are still issues. I am +1 on 5833
>> > >>>>> and I
>> > >>>>>>>>> think
>> > >>>>>>>>>>>> 5484
>> > >>>>>>>>>>>>>>> would
>> > >>>>>>>>>>>>>>>> be a great addition to 4.16 as well. We should aim
>> > >>>> at
>> > >>>>>>>>> resolving
>> > >>>>>>>>>>> high
>> > >>>>>>>>>>>>>>>> priority bugs like this in every release.
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> Sometimes we let these bugs slip without a
>> > >>>> resolution
>> > >>>>>>>>> before a
>> > >>>>>>>>>>>>> release,
>> > >>>>>>>>>>>>>>>> citing that these are "known issues" or "not
>> > >>>> regressions
>> > >>>>>>>>> from
>> > >>>>>>>>>> the
>> > >>>>>>>>>>>> last
>> > >>>>>>>>>>>>>>>> release". In some cases this may be fine since we
>> > >>>> want
>> > >>>>> to
>> > >>>>>>>>> keep
>> > >>>>>>>>>>>>> releases
>> > >>>>>>>>>>>>>>>> light and frequent, but perhaps we can track such
>> > >>>> issues
>> > >>>>>>>>> and aim
>> > >>>>>>>>>>> at
>> > >>>>>>>>>>>>>>>> reducing the number of bugs by x% in each release?
>> > >>>> This
>> > >>>>>> will
>> > >>>>>>>>>> also
>> > >>>>>>>>>>>> keep
>> > >>>>>>>>>>>>>> old
>> > >>>>>>>>>>>>>>>> Jiras alive since we will potentially periodically
>> > >>>>> review
>> > >>>>>>>>> them.
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <
>> > >>>>>>>>>>> gjacoby@apache.org>
>> > >>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> I've got PHOENIX-5435 in review right now, and
>> > >>>> would
>> > >>>>>> like
>> > >>>>>>>>> to
>> > >>>>>>>>>> get
>> > >>>>>>>>>>>> it
>> > >>>>>>>>>>>>> in
>> > >>>>>>>>>>>>>>> 4.16
>> > >>>>>>>>>>>>>>>>> / 5.1.
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> It's allowing the annotation of Phoenix metadata
>> > >>>> into
>> > >>>>>>>>> HBase
>> > >>>>>>>>>> WALs
>> > >>>>>>>>>>>> as
>> > >>>>>>>>>>>>> a
>> > >>>>>>>>>>>>>>>>> pre-req for the Phoenix Change Detection Capture
>> > >>>>>> framework
>> > >>>>>>>>>>>>>>> (PHOENIX-5442).
>> > >>>>>>>>>>>>>>>>> Since it has both client/server logic, and adds a
>> > >>>>> field
>> > >>>>>> to
>> > >>>>>>>>>>>>>>> System.Catalog,
>> > >>>>>>>>>>>>>>>>> it can't go in a patch release.
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Depending on timing, I'd _like_ to get
>> > >>>> PHOENIX-6227,
>> > >>>>>>>>> which is
>> > >>>>>>>>>>> the
>> > >>>>>>>>>>>>> last
>> > >>>>>>>>>>>>>>> part
>> > >>>>>>>>>>>>>>>>> of CDC that will go into core Phoenix, into 4.16,
>> > >>>> but
>> > >>>>>>>>> since
>> > >>>>>>>>>> that
>> > >>>>>>>>>>>>> _can_
>> > >>>>>>>>>>>>>>> go
>> > >>>>>>>>>>>>>>>>> in a patch release and I haven't started it yet,
>> > >>>> if
>> > >>>>> the
>> > >>>>>>>>>> release
>> > >>>>>>>>>>>> gets
>> > >>>>>>>>>>>>>> cut
>> > >>>>>>>>>>>>>>>>> before it's ready, no big deal. (The rest of CDC
>> > >>>> will
>> > >>>>> go
>> > >>>>>>>>> into
>> > >>>>>>>>>>>>>>>>> phoenix-connectors for a future release of that
>> > >>>>>> project.)
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> As for the correctness problems that Daniel points
>> > >>>>> out,
>> > >>>>>> I
>> > >>>>>>>>>> think
>> > >>>>>>>>>>> we
>> > >>>>>>>>>>>>>>> should
>> > >>>>>>>>>>>>>>>>> fix the ones that were detected with a recent
>> > >>>> version
>> > >>>>>>>>> (4.14 or
>> > >>>>>>>>>>>>> 4.15?),
>> > >>>>>>>>>>>>>>> and
>> > >>>>>>>>>>>>>>>>> test to see which of the older ones can still be
>> > >>>>>>>>> reproduced.
>> > >>>>>>>>>>> Once
>> > >>>>>>>>>>>> we
>> > >>>>>>>>>>>>>>> know
>> > >>>>>>>>>>>>>>>>> which bugs are real and which are just
>> > >>>> historical, we
>> > >>>>>> can
>> > >>>>>>>>>> better
>> > >>>>>>>>>>>>> judge
>> > >>>>>>>>>>>>>>>>> scope. And hopefully close a bunch of obsolete
>> > >>>> bugs.
>> > >>>>>>>>> (Thanks,
>> > >>>>>>>>>>>>> Daniel,
>> > >>>>>>>>>>>>>>> for
>> > >>>>>>>>>>>>>>>>> collecting that list!)
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Geoffrey
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
>> > >>>>>>>>>>>>>>>>> <da...@salesforce.com.invalid> wrote:
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> Hi, I wanted to bring up wrong results in
>> > >>>> Phoenix
>> > >>>>> and
>> > >>>>>>>>> some
>> > >>>>>>>>>>> JIRAs
>> > >>>>>>>>>>>>>>> around
>> > >>>>>>>>>>>>>>>>>> them that I think we should fix as the wrong
>> > >>>> result
>> > >>>>>>>>> lessens
>> > >>>>>>>>>>> the
>> > >>>>>>>>>>>>> end
>> > >>>>>>>>>>>>>>>>> user's
>> > >>>>>>>>>>>>>>>>>> trust in Phoenix. Releasing a new version
>> > >>>> without
>> > >>>>>>>>>> addressing
>> > >>>>>>>>>>>>> these
>> > >>>>>>>>>>>>>>> in a
>> > >>>>>>>>>>>>>>>>>> minor release hurts our visibility in that these
>> > >>>>>>>>> critical
>> > >>>>>>>>>>> issues
>> > >>>>>>>>>>>>> are
>> > >>>>>>>>>>>>>>> not
>> > >>>>>>>>>>>>>>>>>> addressed.
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> Jira's that I'm involved with for example: I've
>> > >>>>>> already
>> > >>>>>>>>>> given
>> > >>>>>>>>>>> a
>> > >>>>>>>>>>>>>> patch
>> > >>>>>>>>>>>>>>>>>> several months ago for 5833 and there is a
>> > >>>> chance it
>> > >>>>>>>>> may fix
>> > >>>>>>>>>>>> 5484.
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-5833
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-5484
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> In addition, inspecting apache JIRA i see
>> > >>>> several
>> > >>>>>> other
>> > >>>>>>>>>> wrong
>> > >>>>>>>>>>>>> result
>> > >>>>>>>>>>>>>>>>> JIRAs
>> > >>>>>>>>>>>>>>>>>> from the community. Some of these certainly are
>> > >>>>>>>>> probably
>> > >>>>>>>>>> old
>> > >>>>>>>>>>>>> issues
>> > >>>>>>>>>>>>>>> or
>> > >>>>>>>>>>>>>>>>>> incorrect understanding but some of these are
>> > >>>> opened
>> > >>>>>> by
>> > >>>>>>>>> our
>> > >>>>>>>>>>> own
>> > >>>>>>>>>>>>> dev
>> > >>>>>>>>>>>>>>>>>> community and are likely real problems.
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-6217
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-5571
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-4642
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-4540
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-4504
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-4419
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-4116
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> What is our stance on this type of issue? Are
>> > >>>> we
>> > >>>>>> going
>> > >>>>>>>>> to
>> > >>>>>>>>>> say
>> > >>>>>>>>>>>>> these
>> > >>>>>>>>>>>>>>> were
>> > >>>>>>>>>>>>>>>>>> issues prior to 4.15 and not address them?
>> > >>>> Should
>> > >>>>> we
>> > >>>>>>>>> have
>> > >>>>>>>>>>>>>>> requirements
>> > >>>>>>>>>>>>>>>>> for
>> > >>>>>>>>>>>>>>>>>> our releases to fix wrong results?
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> Daniel Wong
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <
>> > >>>>>>>>>>> yanxinyi@apache.org>
>> > >>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> Hi all,
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> It's time to discuss the Phoenix 4.16 release.
>> > >>>>> After
>> > >>>>>>>>> many
>> > >>>>>>>>>>>>> people's
>> > >>>>>>>>>>>>>>>>>>> contributions on the bug fixes, new features,
>> > >>>> and
>> > >>>>>>>>> other
>> > >>>>>>>>>>> works
>> > >>>>>>>>>>>> in
>> > >>>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>>> past
>> > >>>>>>>>>>>>>>>>>>> few months, we are kind of close to the point
>> > >>>> to
>> > >>>>>> have
>> > >>>>>>>>> a RC
>> > >>>>>>>>>>>>> (still
>> > >>>>>>>>>>>>>>> need
>> > >>>>>>>>>>>>>>>>> to
>> > >>>>>>>>>>>>>>>>>>> fix test flappers). Please let me know if you
>> > >>>>> think
>> > >>>>>>>>> any
>> > >>>>>>>>>> JIRA
>> > >>>>>>>>>>>>> must
>> > >>>>>>>>>>>>>> be
>> > >>>>>>>>>>>>>>>>> part
>> > >>>>>>>>>>>>>>>>>>> of the Phoenix 4.16 release other than major
>> > >>>>> blocker
>> > >>>>>>>>>>>>> PHOENIX-5712.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> If no surprise comes up, I will not wait for
>> > >>>> any
>> > >>>>> new
>> > >>>>>>>>> major
>> > >>>>>>>>>>>>>> features
>> > >>>>>>>>>>>>>>> and
>> > >>>>>>>>>>>>>>>>>>> focus on the RC as soon as possible.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> Sincerely,
>> > >>>>>>>>>>>>>>>>>>> Xinyi
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>>>>>> Daniel Wong
>> > >>>>>>>>>>>>>>>>>> Salesforce
>> > >>>>>>>>>>>>>>>>>> Mobile: 628.217.1808
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>>>> Chinmay Kulkarni
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>> --
>> > >>>>>>>> *István Tóth* | Staff Software Engineer
>> > >>>>>>>> stoty@cloudera.com <https://www.cloudera.com>
>> > >>>>>>>> [image: Cloudera] <https://www.cloudera.com/>
>> > >>>>>>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera>
>> > >>>> [image:
>> > >>>>>>>> Cloudera on Facebook] <https://www.facebook.com/cloudera>
>> [image:
>> > >>>>>>>> Cloudera on LinkedIn] <
>> https://www.linkedin.com/company/cloudera>
>> > >>>>>>>> <https://www.cloudera.com/>
>> > >>>>>>>> ------------------------------
>> > >>>>>>>>
>> > >>>>>>>
>> > >>>>>>
>> > >>>>>
>> > >>>>
>> > >>>
>> > >>>
>> > >>> --
>> > >>> *István Tóth* | Staff Software Engineer
>> > >>> stoty@cloudera.com <https://www.cloudera.com>
>> > >>> [image: Cloudera] <https://www.cloudera.com/>
>> > >>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>> > >>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>> > >>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>> > >>> <https://www.cloudera.com/>
>> > >>> ------------------------------
>> > >>>
>> > >>
>> > >>
>> > >> --
>> > >> *István Tóth* | Staff Software Engineer
>> > >> stoty@cloudera.com <https://www.cloudera.com>
>> > >> [image: Cloudera] <https://www.cloudera.com/>
>> > >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>> > >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>> > >> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>> > >> <https://www.cloudera.com/>
>> > >> ------------------------------
>> > >>
>> > >
>> >
>>
>
>
> --
> *István Tóth* | Staff Software Engineer
> stoty@cloudera.com <https://www.cloudera.com>
> [image: Cloudera] <https://www.cloudera.com/>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> <https://www.cloudera.com/>
> ------------------------------
>
--
*István Tóth* | Staff Software Engineer
stoty@cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
<https://www.cloudera.com/>
------------------------------
Re: [Discuss] Releasing Phoenix 4.16
Posted by Istvan Toth <st...@cloudera.com.INVALID>.
Hi!
I can see that Xinyi has filed PHOENX-6319 and PHOENIX-6320 for the "old"
make_rc script.
While of course every release manager is free to choose to the tools they
use, I ask you to consider using
the HBase-derived scripts at dev/create-release in the master branch. They
are not copied to 4.x,
because they operate directly on the ASF git repository, but they are
capable of handling a 4.x release,
Using them has several advantages:
* They handle the profiles in a single step, meaning less manual work and
place for errors
* They handle more step of the release process automatically, again less
work and more consistency
* I also intend to use them for 5.0, and as they use the HBase convention
for the files names and git tags,
using them would mean consistency for the 4.16 and 5.1 release artifacts.
They are documented in the README file, and in the patch attached to
PHOENIX-6315.
There is a wall of text in the README about using the scripts with GKE, but
don't let that put you off,
that is completely optional. (I have never even tested that)
I am also happy to assist if you run into trouble with them.
Istvan
On Wed, Jan 13, 2021 at 4:12 AM Xinyi Yan <ya...@apache.org> wrote:
> Thank you for the update and great work, Istvan. I feel the same way, this
> is the best approach so far for 4.16 and 5.1.
>
> On Tue, Jan 12, 2021 at 7:01 PM Josh Elser <el...@apache.org> wrote:
>
> > IMO, a slow build is better than forcing users to run their own dev
> > build of a release.
> >
> > Very proud of all of the work y'all have been doing on 4.16. Keep
> > pushing on the release :)
> >
> > On 1/12/21 12:35 PM, Istvan Toth wrote:
> > > I've been working this, and came up with the following:
> > >
> > > * We no longer generate phoenix-client.jar and phoenix-server.jar, we
> > call
> > > them phoenix-client-hbase-X.Y.jar and phoenix-server-hbase-X.Y.jar
> > instead.
> > > * These file names are used in the binary assemblies, as well as as
> maven
> > > coordinates for consistency.
> > > * We generate four release binaries, for each HBase profile
> > > * We build and publish to maven the phoenix-client-hbase-X.Y and
> > > phoenix-server-hbase-X.Y artifacts for each profile (Actually, we
> deploy
> > > four times, but the rest of the artifacts are identical)
> > >
> > > * On the downside, running the release script on a Macbook takes
> several
> > > hours, as we build HBase 4 and Phoenix 8 times on the slow and kludgy
> Mac
> > > Docker filesystem.
> > >
> > > The almost finished patch to the build system is linked to
> > > https://issues.apache.org/jira/browse/PHOENIX-6307
> > >
> > > 4.x would have the same build system changes once this is approved.
> > >
> > > Please check it out, and let me know if this solution is satisfactory,
> or
> > > if you have a better idea.
> > >
> > > regards
> > > Istvan
> > >
> > >
> > > On Fri, Jan 8, 2021 at 4:22 PM Istvan Toth <st...@cloudera.com> wrote:
> > >
> > >> As I started to work on this I realized that while providing binary
> > >> tarballs for each HBase profile is fine,
> > >> this does not solve the maven artifact issue.
> > >>
> > >> Are we OK with publishing a single phoenix-client maven artifact (for
> > the
> > >> oldest HBase),
> > >> or do we want to publish a separate one for each HBase version ?
> > >>
> > >> I looked at publishing multiple client versions, but none of them are
> > >> particularly easy or attractive.
> > >>
> > >> The best I could come up with is adding a separate maven module for
> > (each
> > >> version x embedded) (i.e 6 for 4.x, 8 for master),
> > >> and activating them according to hbase.profile.
> > >>
> > >> This would also mean that we need to add the hbase version to the
> > artifact
> > >> id. i.e.: phoenix-client-hbase-2.1
> > >>
> > >> Once we publish separate binaries for the HBase profiles, we can undo
> > the
> > >> change that excludes compat-module from phoenix-server,
> > >> and shade it in again for the binary assemblies.
> > >>
> > >> In this case we'd also have to do something about the phoenix-server
> > maven
> > >> artifacts. Either they get the same treatment as phoenix-client,
> > >> or we simply skip publishing them. I personally do not see anyone
> > getting
> > >> phoenix-server.jar from maven.
> > >>
> > >> The easiest version is
> > >> * publish the oldest profile phoenix-client to maven
> > >> * do not publish phoenix-server to maven
> > >> * add compat-module to phoenix-server for the binary artifact
> > >>
> > >>
> > >> regards
> > >> Istvan
> > >>
> > >> On Fri, Jan 8, 2021 at 6:56 AM Istvan Toth <st...@cloudera.com>
> wrote:
> > >>
> > >>> On the release binaries:
> > >>>
> > >>> The current solution (which the default profile change has broken)
> was
> > >>> based on Lars's idea at
> > >>>
> > >>>
> >
> https://issues.apache.org/jira/browse/PHOENIX-5902?focusedCommentId=17125122&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17125122
> > >>> However, I agree that providing separate assemblies for each HBase
> > >>> profile is better for our users, as they won't have to rebuild
> Phoenix
> > to
> > >>> take advantage of any new features, and to get the general
> > improvements in
> > >>> later HBase releases.
> > >>> I have opened https://issues.apache.org/jira/browse/PHOENIX-6307 to
> > >>> track this.
> > >>>
> > >>> On the 5.1 release:
> > >>>
> > >>> Yes, I do want to release 5.1 shortly. In fact $dayjob desperately
> > needs
> > >>> it ASAP.
> > >>> I have a few older PRs waiting for review that fell between the
> cracks,
> > >>> but as soon as those are merged, I want to cut the first RC.
> > >>> It would be nice to have 4.16 and 5.1 as close as possible, and
> > >>> PHOENIX-6211 seems to be ready, so I hope to include it in 5.1 too.
> > >>> I will start an official 5.1 release thread and volunteer to be the
> > >>> release manager soon. (unless you want to take that up too, Xinyi).
> > >>>
> > >>>
> > >>>
> > >>> On Fri, Jan 8, 2021 at 1:08 AM Xinyi Yan <ya...@apache.org>
> wrote:
> > >>>
> > >>>> If we can modify the dev/create-release scripts and make them work
> for
> > >>>> the
> > >>>> 4.16 release with this hbase.profile option, it would make our life
> > much
> > >>>> easier to release multiple HBase profiles from the single branch in
> > the
> > >>>> future too(the master branch will have a release shortly right?).
> > >>>> Geoffrey
> > >>>> and Istvan, what do you think?
> > >>>>
> > >>>> Thanks,
> > >>>> Xinyi
> > >>>>
> > >>>> On Thu, Jan 7, 2021 at 11:28 AM Geoffrey Jacoby <gjacoby@apache.org
> >
> > >>>> wrote:
> > >>>>
> > >>>>> Thanks for bringing up the default branch issue, Istvan, I've been
> > >>>> meaning
> > >>>>> to start a conversation about it on this list.
> > >>>>>
> > >>>>> As part of PHOENIX-5435, I changed the default 4.x HBase release to
> > >>>> 1.5 and
> > >>>>> the default 5.x HBase to 2.3 because the WAL annotation feature
> > >>>> introduced
> > >>>>> by 5435 only works with HBase 1.5+ or 2.3+. (It depends on a coproc
> > >>>> hook
> > >>>>> introduced in HBASE-22623). That means that all tests of that
> feature
> > >>>> must
> > >>>>> no-op when run against an earlier HBase, which means that it would
> > >>>> never be
> > >>>>> tested in our CI pipelines if the default was 1.3 or 2.1.
> > >>>>>
> > >>>>> This has come up quite a few times recently. In particular, the new
> > >>>>> indexing framework runs in a degraded state against HBase 2.1 and
> 2.2
> > >>>>> (still better than the old indexing framework though!), because we
> > lack
> > >>>>> support in 2.1 for Filters on raw scans and can't use the "max
> > lookback
> > >>>>> age" feature with HBase 2.1 or 2.2, which keeps major compactions
> > from
> > >>>>> messing with index rebuilds. That's why lots of indexing tests
> no-op
> > or
> > >>>>> have to make different assertions when run against 2.1 or 2.2, so
> we
> > >>>> only
> > >>>>> properly test the indexing framework if CI and local developer
> > >>>>> tests run against HBase 2.3 or up.
> > >>>>>
> > >>>>> As for the release, don't we usually release separate binaries that
> > are
> > >>>>> suitable for all the HBase versions we support? Back before we had
> a
> > >>>>> unified 4.x branch, I believe we used to release from 3 different
> > >>>>> 4.x-HBase-* branches each time. I've been assuming that part of the
> > >>>> release
> > >>>>> process would be generating binaries for 4.x-HBase-1.3,
> > 4.x-HBase-1.4,
> > >>>>> 4.x-HBase-1.5, and 4.x-HBase-1.6 (if that was different from 1.5).
> > >>>>>
> > >>>>> Geoffrey
> > >>>>>
> > >>>>> On Thu, Jan 7, 2021 at 11:19 AM Istvan Toth <st...@apache.org>
> > wrote:
> > >>>>>
> > >>>>>> Two more issues came to my mind:
> > >>>>>>
> > >>>>>> In PHOENIX-5435 the default profile was changed to HBase 1.5.0
> > >>>>>> This causes two conflicts:
> > >>>>>> - Our documentation says that the default profile is the lowest
> > >>>>> supported,
> > >>>>>> which is no longer true
> > >>>>>> - Unless we change our binary release process, the published maven
> > >>>>>> artifacts and binaries will also be built with HBase 1.5, thus
> they
> > >>>> will
> > >>>>> be
> > >>>>>> incompatible with Hbase 1.3 and 1.4
> > >>>>>>
> > >>>>>> This should be addressed before release.
> > >>>>>> One possible solution:
> > >>>>>> * Add an "oldest" hbase.profile option which always chooses the
> > >>>> oldest
> > >>>>>> supported version
> > >>>>>> * Update the release scripts to specify this profile
> > >>>>>> * Update the docs.
> > >>>>>>
> > >>>>>> I have also adopted the hbase release scripts to Phoenix, they are
> > >>>>> present
> > >>>>>> in dev/create-release in the master branch, but
> > >>>>>> should be able to perform 4.16 release as well. I have used this
> for
> > >>>> the
> > >>>>>> phoenix-thirdparty, omid, and tephra releases, but no live phoenix
> > >>>>> releases
> > >>>>>> have been done with them yet, obviously.
> > >>>>>>
> > >>>>>> regards
> > >>>>>> Istvan
> > >>>>>>
> > >>>>>> On Thu, Jan 7, 2021 at 8:19 AM Istvan Toth <st...@apache.org>
> > wrote:
> > >>>>>>
> > >>>>>>> Hi!
> > >>>>>>>
> > >>>>>>> A question of process: Should we backport fixes to the 4.16
> branch
> > >>>> at
> > >>>>> our
> > >>>>>>> discretion, or is backporting those handled by the release
> manager
> > >>>>>> (Xinyi) ?
> > >>>>>>>
> > >>>>>>> On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth <st...@cloudera.com>
> > >>>> wrote:
> > >>>>>>>
> > >>>>>>>> Hi!
> > >>>>>>>>
> > >>>>>>>> Thanks for everyone's effort on fixing the flakey tests.
> > >>>>>>>> However, our ASF Jenkins runs are still very unstable, and I am
> > >>>> afraid
> > >>>>>>>> that the single clean 4.x run was down more to luck than to
> > >>>> fixing the
> > >>>>>>>> underlying problem.
> > >>>>>>>> While I do not consider this a blocker, fixing this would make
> the
> > >>>>> pre-
> > >>>>>>>> and post commit tests far more useful, and let us start with a
> > >>>> clean
> > >>>>>> slate
> > >>>>>>>> for the next release.
> > >>>>>>>> I have created
> https://issues.apache.org/jira/browse/PHOENIX-6288
> > >>>> to
> > >>>>>>>> track the generic cluster setup issue, but my attempts to fix
> > >>>> this, or
> > >>>>>> at
> > >>>>>>>> least figure out the root cause have been unsuccessful so far.
> > >>>>>>>>
> > >>>>>>>> I ask for your help in fixing this issue. I have added some
> > >>>>> inconclusive
> > >>>>>>>> analysis to the ticket, and the full failsafe output directory
> is
> > >>>>>>>> downloadable as artifacts from the failed multibranch runs
> (stored
> > >>>>> for a
> > >>>>>>>> few days), which should hold the answer to this riddle.
> > >>>>>>>>
> > >>>>>>>> regards,
> > >>>>>>>> Istvan
> > >>>>>>>>
> > >>>>>>>> On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan <ya...@apache.org>
> > >>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hi,
> > >>>>>>>>>
> > >>>>>>>>> We finally have a stable 4.x branch build after many flapper
> test
> > >>>>>> fixing
> > >>>>>>>>> contributions from the community. I'm going to fork a new
> > >>>>> branch(4.16)
> > >>>>>>>>> from
> > >>>>>>>>> the current 4.x branch for the 4.16 release. As a cutoff, I
> will
> > >>>> not
> > >>>>>>>>> include any new features other than PHOENIX-6211 and bug fix.
> > >>>> Please
> > >>>>>> let
> > >>>>>>>>> me
> > >>>>>>>>> know if you have any questions or concerns.
> > >>>>>>>>>
> > >>>>>>>>> Thanks,
> > >>>>>>>>> Xinyi
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani <
> vjasani@apache.org
> > >>>>>
> > >>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> An update on test stability:
> > >>>>>>>>>>
> > >>>>>>>>>> As per recent 4.x build results, we are left with very few
> > >>>> flappers
> > >>>>>> and
> > >>>>>>>>>> specifically
> > >>>>>>>>>> with HBase profile 1.6, I can see recent 7 build (multibranch
> > >>>> + PR
> > >>>>>>>>>> precommit results)
> > >>>>>>>>>> results without any test failure.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <
> yanxinyi@apache.org
> > >>>>>
> > >>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Yes, we are currently working on fixing the 4.x branch test
> > >>>>>> flappers
> > >>>>>>>>>> while
> > >>>>>>>>>>> waiting for the PHOENIX-5435. After that, I will try to have
> > >>>> an
> > >>>>> RC
> > >>>>>>>>> ASAP.
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <
> > >>>>>>>>> ankitsinghal59@gmail.com>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> I see that both the blockers listed here PHOENIX-5712 and
> > >>>>>>>>> PHOENIX-6241
> > >>>>>>>>>>>> have been resolved(Thanks to Xinyi), and also as per the
> > >>>> JIRA
> > >>>>>> query
> > >>>>>>>>>> there
> > >>>>>>>>>>>> is no Jira marked as a blocker for 4.16 except the one
> > >>>> related
> > >>>>> to
> > >>>>>>>>>>>> documentation
> > >>>>>>>>>>>> for "splittable catalog table".
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Xinyi, so are we good to start the release process now?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <
> > >>>> yanxinyi@apache.org>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks for replying and providing suggestions. I looked
> > >>>> at
> > >>>>> the
> > >>>>>>>>> wrong
> > >>>>>>>>>>>> result
> > >>>>>>>>>>>>> Jira list that Daniel provided and did some local
> > >>>> testing,
> > >>>>> and
> > >>>>>>>>> here
> > >>>>>>>>>> is
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>> result:
> > >>>>>>>>>>>>> [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642:
> > >>>>> cannot
> > >>>>>>>>>>> reproduce
> > >>>>>>>>>>>>> it.
> > >>>>>>>>>>>>> [More information is required] PHOENIX-4504 cannot
> > >>>> reproduce
> > >>>>> it
> > >>>>>>>>> but
> > >>>>>>>>>>>> someone
> > >>>>>>>>>>>>> claimed that he had a similar issue.
> > >>>>>>>>>>>>> [unusual query] PHOENIX-4540 and PHOENIX-6217
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Based on my finding, I think it's better to have more
> > >>>>> frequent
> > >>>>>>>>>>>> housekeeping
> > >>>>>>>>>>>>> and resolve unreproducible bugs especially since many of
> > >>>> them
> > >>>>>> are
> > >>>>>>>>>>>>> considering out of date (phoenix-4.11 or even
> > >>>> phoenix-4.6).
> > >>>>>>>>> Since I
> > >>>>>>>>>>> still
> > >>>>>>>>>>>>> need time to work on the blocker Jira(PHOENIX-5712,
> > >>>>>>>>> PHOENIX-6241) and
> > >>>>>>>>>>> fix
> > >>>>>>>>>>>>> test flappers, if you want to fix "unusual query" bugs,
> > >>>> feel
> > >>>>>>>>> free to
> > >>>>>>>>>> do
> > >>>>>>>>>>>> so.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Sincerely,
> > >>>>>>>>>>>>> Xinyi
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <
> > >>>>>> ankit@apache.org>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks Daniel and appreciate the effort you put in
> > >>>> getting
> > >>>>>> the
> > >>>>>>>>> list
> > >>>>>>>>>>>> ready
> > >>>>>>>>>>>>>> for bugs producing wrong results
> > >>>>>>>>>>>>>> but none of them seems to be a blocker to me for 4.16
> > >>>> as
> > >>>>> they
> > >>>>>>>>> are
> > >>>>>>>>>> not
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>> regression and doesn't break the general functionality
> > >>>>>>>>>>>>>> except for specific features, RVC/desc as Chenglei also
> > >>>>>>>>> pointed out
> > >>>>>>>>>>>>> (though
> > >>>>>>>>>>>>>> I'll defer the assessment to RM "Xinyi").
> > >>>>>>>>>>>>>> Probably these can be a part of 4.16.1 or we can do
> > >>>> 4.17.0
> > >>>>>> soon
> > >>>>>>>>>> maybe
> > >>>>>>>>>>>>> after
> > >>>>>>>>>>>>>> a few weeks/month?
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Considering that we have already fixed 137 bugs and
> > >>>> done
> > >>>>> 85+
> > >>>>>>>>>>>>>> improvements/features in 4.16,
> > >>>>>>>>>>>>>> it will not be a good idea to deprive the user from
> > >>>> such
> > >>>>>> fixes.
> > >>>>>>>>>>>>>> It's been a year since our last 4.15 release, having no
> > >>>>>> release
> > >>>>>>>>>>> brings
> > >>>>>>>>>>>>> more
> > >>>>>>>>>>>>>> questions on the project
> > >>>>>>>>>>>>>> rather than the bugs which affect a certain % of
> > >>>>>> feature/users,
> > >>>>>>>>>> would
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>> release notes
> > >>>>>>>>>>>>>> explaining the stability of certain features set the
> > >>>> right
> > >>>>>>>>>>> expectation
> > >>>>>>>>>>>>> for
> > >>>>>>>>>>>>>> those users who rely on these features to wait for a
> > >>>> future
> > >>>>>>>>>> release?
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Regards,
> > >>>>>>>>>>>>>> Ankit Singhal
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
> > >>>>>>>>>>>> chenglei@apache.org>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> In my opinion, we should keep releases light and
> > >>>>> frequent,
> > >>>>>>>>> and
> > >>>>>>>>>> for
> > >>>>>>>>>>>>> some
> > >>>>>>>>>>>>>>> unusual query bugs like RVC and DESC
> > >>>>>>>>>>>>>>> we could delay fix to next release . I think we
> > >>>> should
> > >>>>>>>>> release
> > >>>>>>>>>>> 4.16.0
> > >>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>> 5.1.0 as quickly as possible. In China, many users
> > >>>>>>>>>>>>>>> in HBase&Phoenix User Group thought that Phoenix was
> > >>>>> dead
> > >>>>>>>>>> because
> > >>>>>>>>>>>> our
> > >>>>>>>>>>>>>> too
> > >>>>>>>>>>>>>>> long interval release and stopped using
> > >>>>>>>>>>>>>>> Phoenix.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
> > >>>>>>>>>>>> chinmayskulkarni@gmail.com
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>> I agree. These are all major bugs and we should aim
> > >>>> at
> > >>>>>>>>> solving
> > >>>>>>>>>>> them
> > >>>>>>>>>>>>>> after
> > >>>>>>>>>>>>>>>> checking that they are still issues. I am +1 on 5833
> > >>>>> and I
> > >>>>>>>>> think
> > >>>>>>>>>>>> 5484
> > >>>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>> be a great addition to 4.16 as well. We should aim
> > >>>> at
> > >>>>>>>>> resolving
> > >>>>>>>>>>> high
> > >>>>>>>>>>>>>>>> priority bugs like this in every release.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Sometimes we let these bugs slip without a
> > >>>> resolution
> > >>>>>>>>> before a
> > >>>>>>>>>>>>> release,
> > >>>>>>>>>>>>>>>> citing that these are "known issues" or "not
> > >>>> regressions
> > >>>>>>>>> from
> > >>>>>>>>>> the
> > >>>>>>>>>>>> last
> > >>>>>>>>>>>>>>>> release". In some cases this may be fine since we
> > >>>> want
> > >>>>> to
> > >>>>>>>>> keep
> > >>>>>>>>>>>>> releases
> > >>>>>>>>>>>>>>>> light and frequent, but perhaps we can track such
> > >>>> issues
> > >>>>>>>>> and aim
> > >>>>>>>>>>> at
> > >>>>>>>>>>>>>>>> reducing the number of bugs by x% in each release?
> > >>>> This
> > >>>>>> will
> > >>>>>>>>>> also
> > >>>>>>>>>>>> keep
> > >>>>>>>>>>>>>> old
> > >>>>>>>>>>>>>>>> Jiras alive since we will potentially periodically
> > >>>>> review
> > >>>>>>>>> them.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <
> > >>>>>>>>>>> gjacoby@apache.org>
> > >>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I've got PHOENIX-5435 in review right now, and
> > >>>> would
> > >>>>>> like
> > >>>>>>>>> to
> > >>>>>>>>>> get
> > >>>>>>>>>>>> it
> > >>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>> 4.16
> > >>>>>>>>>>>>>>>>> / 5.1.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> It's allowing the annotation of Phoenix metadata
> > >>>> into
> > >>>>>>>>> HBase
> > >>>>>>>>>> WALs
> > >>>>>>>>>>>> as
> > >>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>> pre-req for the Phoenix Change Detection Capture
> > >>>>>> framework
> > >>>>>>>>>>>>>>> (PHOENIX-5442).
> > >>>>>>>>>>>>>>>>> Since it has both client/server logic, and adds a
> > >>>>> field
> > >>>>>> to
> > >>>>>>>>>>>>>>> System.Catalog,
> > >>>>>>>>>>>>>>>>> it can't go in a patch release.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Depending on timing, I'd _like_ to get
> > >>>> PHOENIX-6227,
> > >>>>>>>>> which is
> > >>>>>>>>>>> the
> > >>>>>>>>>>>>> last
> > >>>>>>>>>>>>>>> part
> > >>>>>>>>>>>>>>>>> of CDC that will go into core Phoenix, into 4.16,
> > >>>> but
> > >>>>>>>>> since
> > >>>>>>>>>> that
> > >>>>>>>>>>>>> _can_
> > >>>>>>>>>>>>>>> go
> > >>>>>>>>>>>>>>>>> in a patch release and I haven't started it yet,
> > >>>> if
> > >>>>> the
> > >>>>>>>>>> release
> > >>>>>>>>>>>> gets
> > >>>>>>>>>>>>>> cut
> > >>>>>>>>>>>>>>>>> before it's ready, no big deal. (The rest of CDC
> > >>>> will
> > >>>>> go
> > >>>>>>>>> into
> > >>>>>>>>>>>>>>>>> phoenix-connectors for a future release of that
> > >>>>>> project.)
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> As for the correctness problems that Daniel points
> > >>>>> out,
> > >>>>>> I
> > >>>>>>>>>> think
> > >>>>>>>>>>> we
> > >>>>>>>>>>>>>>> should
> > >>>>>>>>>>>>>>>>> fix the ones that were detected with a recent
> > >>>> version
> > >>>>>>>>> (4.14 or
> > >>>>>>>>>>>>> 4.15?),
> > >>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>> test to see which of the older ones can still be
> > >>>>>>>>> reproduced.
> > >>>>>>>>>>> Once
> > >>>>>>>>>>>> we
> > >>>>>>>>>>>>>>> know
> > >>>>>>>>>>>>>>>>> which bugs are real and which are just
> > >>>> historical, we
> > >>>>>> can
> > >>>>>>>>>> better
> > >>>>>>>>>>>>> judge
> > >>>>>>>>>>>>>>>>> scope. And hopefully close a bunch of obsolete
> > >>>> bugs.
> > >>>>>>>>> (Thanks,
> > >>>>>>>>>>>>> Daniel,
> > >>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>> collecting that list!)
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Geoffrey
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
> > >>>>>>>>>>>>>>>>> <da...@salesforce.com.invalid> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Hi, I wanted to bring up wrong results in
> > >>>> Phoenix
> > >>>>> and
> > >>>>>>>>> some
> > >>>>>>>>>>> JIRAs
> > >>>>>>>>>>>>>>> around
> > >>>>>>>>>>>>>>>>>> them that I think we should fix as the wrong
> > >>>> result
> > >>>>>>>>> lessens
> > >>>>>>>>>>> the
> > >>>>>>>>>>>>> end
> > >>>>>>>>>>>>>>>>> user's
> > >>>>>>>>>>>>>>>>>> trust in Phoenix. Releasing a new version
> > >>>> without
> > >>>>>>>>>> addressing
> > >>>>>>>>>>>>> these
> > >>>>>>>>>>>>>>> in a
> > >>>>>>>>>>>>>>>>>> minor release hurts our visibility in that these
> > >>>>>>>>> critical
> > >>>>>>>>>>> issues
> > >>>>>>>>>>>>> are
> > >>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>>> addressed.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Jira's that I'm involved with for example: I've
> > >>>>>> already
> > >>>>>>>>>> given
> > >>>>>>>>>>> a
> > >>>>>>>>>>>>>> patch
> > >>>>>>>>>>>>>>>>>> several months ago for 5833 and there is a
> > >>>> chance it
> > >>>>>>>>> may fix
> > >>>>>>>>>>>> 5484.
> > >>>>>>>>>>>>>>>>>>
> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-5833
> > >>>>>>>>>>>>>>>>>>
> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-5484
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> In addition, inspecting apache JIRA i see
> > >>>> several
> > >>>>>> other
> > >>>>>>>>>> wrong
> > >>>>>>>>>>>>> result
> > >>>>>>>>>>>>>>>>> JIRAs
> > >>>>>>>>>>>>>>>>>> from the community. Some of these certainly are
> > >>>>>>>>> probably
> > >>>>>>>>>> old
> > >>>>>>>>>>>>> issues
> > >>>>>>>>>>>>>>> or
> > >>>>>>>>>>>>>>>>>> incorrect understanding but some of these are
> > >>>> opened
> > >>>>>> by
> > >>>>>>>>> our
> > >>>>>>>>>>> own
> > >>>>>>>>>>>>> dev
> > >>>>>>>>>>>>>>>>>> community and are likely real problems.
> > >>>>>>>>>>>>>>>>>>
> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-6217
> > >>>>>>>>>>>>>>>>>>
> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-5571
> > >>>>>>>>>>>>>>>>>>
> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-4642
> > >>>>>>>>>>>>>>>>>>
> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-4540
> > >>>>>>>>>>>>>>>>>>
> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-4504
> > >>>>>>>>>>>>>>>>>>
> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-4419
> > >>>>>>>>>>>>>>>>>>
> > >>>>> https://issues.apache.org/jira/browse/PHOENIX-4116
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> What is our stance on this type of issue? Are
> > >>>> we
> > >>>>>> going
> > >>>>>>>>> to
> > >>>>>>>>>> say
> > >>>>>>>>>>>>> these
> > >>>>>>>>>>>>>>> were
> > >>>>>>>>>>>>>>>>>> issues prior to 4.15 and not address them?
> > >>>> Should
> > >>>>> we
> > >>>>>>>>> have
> > >>>>>>>>>>>>>>> requirements
> > >>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>> our releases to fix wrong results?
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Daniel Wong
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <
> > >>>>>>>>>>> yanxinyi@apache.org>
> > >>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Hi all,
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> It's time to discuss the Phoenix 4.16 release.
> > >>>>> After
> > >>>>>>>>> many
> > >>>>>>>>>>>>> people's
> > >>>>>>>>>>>>>>>>>>> contributions on the bug fixes, new features,
> > >>>> and
> > >>>>>>>>> other
> > >>>>>>>>>>> works
> > >>>>>>>>>>>> in
> > >>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>> past
> > >>>>>>>>>>>>>>>>>>> few months, we are kind of close to the point
> > >>>> to
> > >>>>>> have
> > >>>>>>>>> a RC
> > >>>>>>>>>>>>> (still
> > >>>>>>>>>>>>>>> need
> > >>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>> fix test flappers). Please let me know if you
> > >>>>> think
> > >>>>>>>>> any
> > >>>>>>>>>> JIRA
> > >>>>>>>>>>>>> must
> > >>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>> part
> > >>>>>>>>>>>>>>>>>>> of the Phoenix 4.16 release other than major
> > >>>>> blocker
> > >>>>>>>>>>>>> PHOENIX-5712.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> If no surprise comes up, I will not wait for
> > >>>> any
> > >>>>> new
> > >>>>>>>>> major
> > >>>>>>>>>>>>>> features
> > >>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>> focus on the RC as soon as possible.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Sincerely,
> > >>>>>>>>>>>>>>>>>>> Xinyi
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>> Daniel Wong
> > >>>>>>>>>>>>>>>>>> Salesforce
> > >>>>>>>>>>>>>>>>>> Mobile: 628.217.1808
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>> Chinmay Kulkarni
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> *István Tóth* | Staff Software Engineer
> > >>>>>>>> stoty@cloudera.com <https://www.cloudera.com>
> > >>>>>>>> [image: Cloudera] <https://www.cloudera.com/>
> > >>>>>>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera>
> > >>>> [image:
> > >>>>>>>> Cloudera on Facebook] <https://www.facebook.com/cloudera>
> [image:
> > >>>>>>>> Cloudera on LinkedIn] <
> https://www.linkedin.com/company/cloudera>
> > >>>>>>>> <https://www.cloudera.com/>
> > >>>>>>>> ------------------------------
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>> --
> > >>> *István Tóth* | Staff Software Engineer
> > >>> stoty@cloudera.com <https://www.cloudera.com>
> > >>> [image: Cloudera] <https://www.cloudera.com/>
> > >>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > >>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > >>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > >>> <https://www.cloudera.com/>
> > >>> ------------------------------
> > >>>
> > >>
> > >>
> > >> --
> > >> *István Tóth* | Staff Software Engineer
> > >> stoty@cloudera.com <https://www.cloudera.com>
> > >> [image: Cloudera] <https://www.cloudera.com/>
> > >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > >> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > >> <https://www.cloudera.com/>
> > >> ------------------------------
> > >>
> > >
> >
>
--
*István Tóth* | Staff Software Engineer
stoty@cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
<https://www.cloudera.com/>
------------------------------
Re: [Discuss] Releasing Phoenix 4.16
Posted by Xinyi Yan <ya...@apache.org>.
Thank you for the update and great work, Istvan. I feel the same way, this
is the best approach so far for 4.16 and 5.1.
On Tue, Jan 12, 2021 at 7:01 PM Josh Elser <el...@apache.org> wrote:
> IMO, a slow build is better than forcing users to run their own dev
> build of a release.
>
> Very proud of all of the work y'all have been doing on 4.16. Keep
> pushing on the release :)
>
> On 1/12/21 12:35 PM, Istvan Toth wrote:
> > I've been working this, and came up with the following:
> >
> > * We no longer generate phoenix-client.jar and phoenix-server.jar, we
> call
> > them phoenix-client-hbase-X.Y.jar and phoenix-server-hbase-X.Y.jar
> instead.
> > * These file names are used in the binary assemblies, as well as as maven
> > coordinates for consistency.
> > * We generate four release binaries, for each HBase profile
> > * We build and publish to maven the phoenix-client-hbase-X.Y and
> > phoenix-server-hbase-X.Y artifacts for each profile (Actually, we deploy
> > four times, but the rest of the artifacts are identical)
> >
> > * On the downside, running the release script on a Macbook takes several
> > hours, as we build HBase 4 and Phoenix 8 times on the slow and kludgy Mac
> > Docker filesystem.
> >
> > The almost finished patch to the build system is linked to
> > https://issues.apache.org/jira/browse/PHOENIX-6307
> >
> > 4.x would have the same build system changes once this is approved.
> >
> > Please check it out, and let me know if this solution is satisfactory, or
> > if you have a better idea.
> >
> > regards
> > Istvan
> >
> >
> > On Fri, Jan 8, 2021 at 4:22 PM Istvan Toth <st...@cloudera.com> wrote:
> >
> >> As I started to work on this I realized that while providing binary
> >> tarballs for each HBase profile is fine,
> >> this does not solve the maven artifact issue.
> >>
> >> Are we OK with publishing a single phoenix-client maven artifact (for
> the
> >> oldest HBase),
> >> or do we want to publish a separate one for each HBase version ?
> >>
> >> I looked at publishing multiple client versions, but none of them are
> >> particularly easy or attractive.
> >>
> >> The best I could come up with is adding a separate maven module for
> (each
> >> version x embedded) (i.e 6 for 4.x, 8 for master),
> >> and activating them according to hbase.profile.
> >>
> >> This would also mean that we need to add the hbase version to the
> artifact
> >> id. i.e.: phoenix-client-hbase-2.1
> >>
> >> Once we publish separate binaries for the HBase profiles, we can undo
> the
> >> change that excludes compat-module from phoenix-server,
> >> and shade it in again for the binary assemblies.
> >>
> >> In this case we'd also have to do something about the phoenix-server
> maven
> >> artifacts. Either they get the same treatment as phoenix-client,
> >> or we simply skip publishing them. I personally do not see anyone
> getting
> >> phoenix-server.jar from maven.
> >>
> >> The easiest version is
> >> * publish the oldest profile phoenix-client to maven
> >> * do not publish phoenix-server to maven
> >> * add compat-module to phoenix-server for the binary artifact
> >>
> >>
> >> regards
> >> Istvan
> >>
> >> On Fri, Jan 8, 2021 at 6:56 AM Istvan Toth <st...@cloudera.com> wrote:
> >>
> >>> On the release binaries:
> >>>
> >>> The current solution (which the default profile change has broken) was
> >>> based on Lars's idea at
> >>>
> >>>
> https://issues.apache.org/jira/browse/PHOENIX-5902?focusedCommentId=17125122&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17125122
> >>> However, I agree that providing separate assemblies for each HBase
> >>> profile is better for our users, as they won't have to rebuild Phoenix
> to
> >>> take advantage of any new features, and to get the general
> improvements in
> >>> later HBase releases.
> >>> I have opened https://issues.apache.org/jira/browse/PHOENIX-6307 to
> >>> track this.
> >>>
> >>> On the 5.1 release:
> >>>
> >>> Yes, I do want to release 5.1 shortly. In fact $dayjob desperately
> needs
> >>> it ASAP.
> >>> I have a few older PRs waiting for review that fell between the cracks,
> >>> but as soon as those are merged, I want to cut the first RC.
> >>> It would be nice to have 4.16 and 5.1 as close as possible, and
> >>> PHOENIX-6211 seems to be ready, so I hope to include it in 5.1 too.
> >>> I will start an official 5.1 release thread and volunteer to be the
> >>> release manager soon. (unless you want to take that up too, Xinyi).
> >>>
> >>>
> >>>
> >>> On Fri, Jan 8, 2021 at 1:08 AM Xinyi Yan <ya...@apache.org> wrote:
> >>>
> >>>> If we can modify the dev/create-release scripts and make them work for
> >>>> the
> >>>> 4.16 release with this hbase.profile option, it would make our life
> much
> >>>> easier to release multiple HBase profiles from the single branch in
> the
> >>>> future too(the master branch will have a release shortly right?).
> >>>> Geoffrey
> >>>> and Istvan, what do you think?
> >>>>
> >>>> Thanks,
> >>>> Xinyi
> >>>>
> >>>> On Thu, Jan 7, 2021 at 11:28 AM Geoffrey Jacoby <gj...@apache.org>
> >>>> wrote:
> >>>>
> >>>>> Thanks for bringing up the default branch issue, Istvan, I've been
> >>>> meaning
> >>>>> to start a conversation about it on this list.
> >>>>>
> >>>>> As part of PHOENIX-5435, I changed the default 4.x HBase release to
> >>>> 1.5 and
> >>>>> the default 5.x HBase to 2.3 because the WAL annotation feature
> >>>> introduced
> >>>>> by 5435 only works with HBase 1.5+ or 2.3+. (It depends on a coproc
> >>>> hook
> >>>>> introduced in HBASE-22623). That means that all tests of that feature
> >>>> must
> >>>>> no-op when run against an earlier HBase, which means that it would
> >>>> never be
> >>>>> tested in our CI pipelines if the default was 1.3 or 2.1.
> >>>>>
> >>>>> This has come up quite a few times recently. In particular, the new
> >>>>> indexing framework runs in a degraded state against HBase 2.1 and 2.2
> >>>>> (still better than the old indexing framework though!), because we
> lack
> >>>>> support in 2.1 for Filters on raw scans and can't use the "max
> lookback
> >>>>> age" feature with HBase 2.1 or 2.2, which keeps major compactions
> from
> >>>>> messing with index rebuilds. That's why lots of indexing tests no-op
> or
> >>>>> have to make different assertions when run against 2.1 or 2.2, so we
> >>>> only
> >>>>> properly test the indexing framework if CI and local developer
> >>>>> tests run against HBase 2.3 or up.
> >>>>>
> >>>>> As for the release, don't we usually release separate binaries that
> are
> >>>>> suitable for all the HBase versions we support? Back before we had a
> >>>>> unified 4.x branch, I believe we used to release from 3 different
> >>>>> 4.x-HBase-* branches each time. I've been assuming that part of the
> >>>> release
> >>>>> process would be generating binaries for 4.x-HBase-1.3,
> 4.x-HBase-1.4,
> >>>>> 4.x-HBase-1.5, and 4.x-HBase-1.6 (if that was different from 1.5).
> >>>>>
> >>>>> Geoffrey
> >>>>>
> >>>>> On Thu, Jan 7, 2021 at 11:19 AM Istvan Toth <st...@apache.org>
> wrote:
> >>>>>
> >>>>>> Two more issues came to my mind:
> >>>>>>
> >>>>>> In PHOENIX-5435 the default profile was changed to HBase 1.5.0
> >>>>>> This causes two conflicts:
> >>>>>> - Our documentation says that the default profile is the lowest
> >>>>> supported,
> >>>>>> which is no longer true
> >>>>>> - Unless we change our binary release process, the published maven
> >>>>>> artifacts and binaries will also be built with HBase 1.5, thus they
> >>>> will
> >>>>> be
> >>>>>> incompatible with Hbase 1.3 and 1.4
> >>>>>>
> >>>>>> This should be addressed before release.
> >>>>>> One possible solution:
> >>>>>> * Add an "oldest" hbase.profile option which always chooses the
> >>>> oldest
> >>>>>> supported version
> >>>>>> * Update the release scripts to specify this profile
> >>>>>> * Update the docs.
> >>>>>>
> >>>>>> I have also adopted the hbase release scripts to Phoenix, they are
> >>>>> present
> >>>>>> in dev/create-release in the master branch, but
> >>>>>> should be able to perform 4.16 release as well. I have used this for
> >>>> the
> >>>>>> phoenix-thirdparty, omid, and tephra releases, but no live phoenix
> >>>>> releases
> >>>>>> have been done with them yet, obviously.
> >>>>>>
> >>>>>> regards
> >>>>>> Istvan
> >>>>>>
> >>>>>> On Thu, Jan 7, 2021 at 8:19 AM Istvan Toth <st...@apache.org>
> wrote:
> >>>>>>
> >>>>>>> Hi!
> >>>>>>>
> >>>>>>> A question of process: Should we backport fixes to the 4.16 branch
> >>>> at
> >>>>> our
> >>>>>>> discretion, or is backporting those handled by the release manager
> >>>>>> (Xinyi) ?
> >>>>>>>
> >>>>>>> On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth <st...@cloudera.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>>> Hi!
> >>>>>>>>
> >>>>>>>> Thanks for everyone's effort on fixing the flakey tests.
> >>>>>>>> However, our ASF Jenkins runs are still very unstable, and I am
> >>>> afraid
> >>>>>>>> that the single clean 4.x run was down more to luck than to
> >>>> fixing the
> >>>>>>>> underlying problem.
> >>>>>>>> While I do not consider this a blocker, fixing this would make the
> >>>>> pre-
> >>>>>>>> and post commit tests far more useful, and let us start with a
> >>>> clean
> >>>>>> slate
> >>>>>>>> for the next release.
> >>>>>>>> I have created https://issues.apache.org/jira/browse/PHOENIX-6288
> >>>> to
> >>>>>>>> track the generic cluster setup issue, but my attempts to fix
> >>>> this, or
> >>>>>> at
> >>>>>>>> least figure out the root cause have been unsuccessful so far.
> >>>>>>>>
> >>>>>>>> I ask for your help in fixing this issue. I have added some
> >>>>> inconclusive
> >>>>>>>> analysis to the ticket, and the full failsafe output directory is
> >>>>>>>> downloadable as artifacts from the failed multibranch runs (stored
> >>>>> for a
> >>>>>>>> few days), which should hold the answer to this riddle.
> >>>>>>>>
> >>>>>>>> regards,
> >>>>>>>> Istvan
> >>>>>>>>
> >>>>>>>> On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan <ya...@apache.org>
> >>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> We finally have a stable 4.x branch build after many flapper test
> >>>>>> fixing
> >>>>>>>>> contributions from the community. I'm going to fork a new
> >>>>> branch(4.16)
> >>>>>>>>> from
> >>>>>>>>> the current 4.x branch for the 4.16 release. As a cutoff, I will
> >>>> not
> >>>>>>>>> include any new features other than PHOENIX-6211 and bug fix.
> >>>> Please
> >>>>>> let
> >>>>>>>>> me
> >>>>>>>>> know if you have any questions or concerns.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Xinyi
> >>>>>>>>>
> >>>>>>>>> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani <vjasani@apache.org
> >>>>>
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> An update on test stability:
> >>>>>>>>>>
> >>>>>>>>>> As per recent 4.x build results, we are left with very few
> >>>> flappers
> >>>>>> and
> >>>>>>>>>> specifically
> >>>>>>>>>> with HBase profile 1.6, I can see recent 7 build (multibranch
> >>>> + PR
> >>>>>>>>>> precommit results)
> >>>>>>>>>> results without any test failure.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <yanxinyi@apache.org
> >>>>>
> >>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Yes, we are currently working on fixing the 4.x branch test
> >>>>>> flappers
> >>>>>>>>>> while
> >>>>>>>>>>> waiting for the PHOENIX-5435. After that, I will try to have
> >>>> an
> >>>>> RC
> >>>>>>>>> ASAP.
> >>>>>>>>>>>
> >>>>>>>>>>> On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <
> >>>>>>>>> ankitsinghal59@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> I see that both the blockers listed here PHOENIX-5712 and
> >>>>>>>>> PHOENIX-6241
> >>>>>>>>>>>> have been resolved(Thanks to Xinyi), and also as per the
> >>>> JIRA
> >>>>>> query
> >>>>>>>>>> there
> >>>>>>>>>>>> is no Jira marked as a blocker for 4.16 except the one
> >>>> related
> >>>>> to
> >>>>>>>>>>>> documentation
> >>>>>>>>>>>> for "splittable catalog table".
> >>>>>>>>>>>>
> >>>>>>>>>>>> Xinyi, so are we good to start the release process now?
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <
> >>>> yanxinyi@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for replying and providing suggestions. I looked
> >>>> at
> >>>>> the
> >>>>>>>>> wrong
> >>>>>>>>>>>> result
> >>>>>>>>>>>>> Jira list that Daniel provided and did some local
> >>>> testing,
> >>>>> and
> >>>>>>>>> here
> >>>>>>>>>> is
> >>>>>>>>>>>> the
> >>>>>>>>>>>>> result:
> >>>>>>>>>>>>> [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642:
> >>>>> cannot
> >>>>>>>>>>> reproduce
> >>>>>>>>>>>>> it.
> >>>>>>>>>>>>> [More information is required] PHOENIX-4504 cannot
> >>>> reproduce
> >>>>> it
> >>>>>>>>> but
> >>>>>>>>>>>> someone
> >>>>>>>>>>>>> claimed that he had a similar issue.
> >>>>>>>>>>>>> [unusual query] PHOENIX-4540 and PHOENIX-6217
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Based on my finding, I think it's better to have more
> >>>>> frequent
> >>>>>>>>>>>> housekeeping
> >>>>>>>>>>>>> and resolve unreproducible bugs especially since many of
> >>>> them
> >>>>>> are
> >>>>>>>>>>>>> considering out of date (phoenix-4.11 or even
> >>>> phoenix-4.6).
> >>>>>>>>> Since I
> >>>>>>>>>>> still
> >>>>>>>>>>>>> need time to work on the blocker Jira(PHOENIX-5712,
> >>>>>>>>> PHOENIX-6241) and
> >>>>>>>>>>> fix
> >>>>>>>>>>>>> test flappers, if you want to fix "unusual query" bugs,
> >>>> feel
> >>>>>>>>> free to
> >>>>>>>>>> do
> >>>>>>>>>>>> so.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Sincerely,
> >>>>>>>>>>>>> Xinyi
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <
> >>>>>> ankit@apache.org>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks Daniel and appreciate the effort you put in
> >>>> getting
> >>>>>> the
> >>>>>>>>> list
> >>>>>>>>>>>> ready
> >>>>>>>>>>>>>> for bugs producing wrong results
> >>>>>>>>>>>>>> but none of them seems to be a blocker to me for 4.16
> >>>> as
> >>>>> they
> >>>>>>>>> are
> >>>>>>>>>> not
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>> regression and doesn't break the general functionality
> >>>>>>>>>>>>>> except for specific features, RVC/desc as Chenglei also
> >>>>>>>>> pointed out
> >>>>>>>>>>>>> (though
> >>>>>>>>>>>>>> I'll defer the assessment to RM "Xinyi").
> >>>>>>>>>>>>>> Probably these can be a part of 4.16.1 or we can do
> >>>> 4.17.0
> >>>>>> soon
> >>>>>>>>>> maybe
> >>>>>>>>>>>>> after
> >>>>>>>>>>>>>> a few weeks/month?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Considering that we have already fixed 137 bugs and
> >>>> done
> >>>>> 85+
> >>>>>>>>>>>>>> improvements/features in 4.16,
> >>>>>>>>>>>>>> it will not be a good idea to deprive the user from
> >>>> such
> >>>>>> fixes.
> >>>>>>>>>>>>>> It's been a year since our last 4.15 release, having no
> >>>>>> release
> >>>>>>>>>>> brings
> >>>>>>>>>>>>> more
> >>>>>>>>>>>>>> questions on the project
> >>>>>>>>>>>>>> rather than the bugs which affect a certain % of
> >>>>>> feature/users,
> >>>>>>>>>> would
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>> release notes
> >>>>>>>>>>>>>> explaining the stability of certain features set the
> >>>> right
> >>>>>>>>>>> expectation
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>> those users who rely on these features to wait for a
> >>>> future
> >>>>>>>>>> release?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>> Ankit Singhal
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
> >>>>>>>>>>>> chenglei@apache.org>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> In my opinion, we should keep releases light and
> >>>>> frequent,
> >>>>>>>>> and
> >>>>>>>>>> for
> >>>>>>>>>>>>> some
> >>>>>>>>>>>>>>> unusual query bugs like RVC and DESC
> >>>>>>>>>>>>>>> we could delay fix to next release . I think we
> >>>> should
> >>>>>>>>> release
> >>>>>>>>>>> 4.16.0
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>> 5.1.0 as quickly as possible. In China, many users
> >>>>>>>>>>>>>>> in HBase&Phoenix User Group thought that Phoenix was
> >>>>> dead
> >>>>>>>>>> because
> >>>>>>>>>>>> our
> >>>>>>>>>>>>>> too
> >>>>>>>>>>>>>>> long interval release and stopped using
> >>>>>>>>>>>>>>> Phoenix.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
> >>>>>>>>>>>> chinmayskulkarni@gmail.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>> I agree. These are all major bugs and we should aim
> >>>> at
> >>>>>>>>> solving
> >>>>>>>>>>> them
> >>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>> checking that they are still issues. I am +1 on 5833
> >>>>> and I
> >>>>>>>>> think
> >>>>>>>>>>>> 5484
> >>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>> be a great addition to 4.16 as well. We should aim
> >>>> at
> >>>>>>>>> resolving
> >>>>>>>>>>> high
> >>>>>>>>>>>>>>>> priority bugs like this in every release.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Sometimes we let these bugs slip without a
> >>>> resolution
> >>>>>>>>> before a
> >>>>>>>>>>>>> release,
> >>>>>>>>>>>>>>>> citing that these are "known issues" or "not
> >>>> regressions
> >>>>>>>>> from
> >>>>>>>>>> the
> >>>>>>>>>>>> last
> >>>>>>>>>>>>>>>> release". In some cases this may be fine since we
> >>>> want
> >>>>> to
> >>>>>>>>> keep
> >>>>>>>>>>>>> releases
> >>>>>>>>>>>>>>>> light and frequent, but perhaps we can track such
> >>>> issues
> >>>>>>>>> and aim
> >>>>>>>>>>> at
> >>>>>>>>>>>>>>>> reducing the number of bugs by x% in each release?
> >>>> This
> >>>>>> will
> >>>>>>>>>> also
> >>>>>>>>>>>> keep
> >>>>>>>>>>>>>> old
> >>>>>>>>>>>>>>>> Jiras alive since we will potentially periodically
> >>>>> review
> >>>>>>>>> them.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <
> >>>>>>>>>>> gjacoby@apache.org>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I've got PHOENIX-5435 in review right now, and
> >>>> would
> >>>>>> like
> >>>>>>>>> to
> >>>>>>>>>> get
> >>>>>>>>>>>> it
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>> 4.16
> >>>>>>>>>>>>>>>>> / 5.1.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> It's allowing the annotation of Phoenix metadata
> >>>> into
> >>>>>>>>> HBase
> >>>>>>>>>> WALs
> >>>>>>>>>>>> as
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>> pre-req for the Phoenix Change Detection Capture
> >>>>>> framework
> >>>>>>>>>>>>>>> (PHOENIX-5442).
> >>>>>>>>>>>>>>>>> Since it has both client/server logic, and adds a
> >>>>> field
> >>>>>> to
> >>>>>>>>>>>>>>> System.Catalog,
> >>>>>>>>>>>>>>>>> it can't go in a patch release.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Depending on timing, I'd _like_ to get
> >>>> PHOENIX-6227,
> >>>>>>>>> which is
> >>>>>>>>>>> the
> >>>>>>>>>>>>> last
> >>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>> of CDC that will go into core Phoenix, into 4.16,
> >>>> but
> >>>>>>>>> since
> >>>>>>>>>> that
> >>>>>>>>>>>>> _can_
> >>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>> in a patch release and I haven't started it yet,
> >>>> if
> >>>>> the
> >>>>>>>>>> release
> >>>>>>>>>>>> gets
> >>>>>>>>>>>>>> cut
> >>>>>>>>>>>>>>>>> before it's ready, no big deal. (The rest of CDC
> >>>> will
> >>>>> go
> >>>>>>>>> into
> >>>>>>>>>>>>>>>>> phoenix-connectors for a future release of that
> >>>>>> project.)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> As for the correctness problems that Daniel points
> >>>>> out,
> >>>>>> I
> >>>>>>>>>> think
> >>>>>>>>>>> we
> >>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>> fix the ones that were detected with a recent
> >>>> version
> >>>>>>>>> (4.14 or
> >>>>>>>>>>>>> 4.15?),
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> test to see which of the older ones can still be
> >>>>>>>>> reproduced.
> >>>>>>>>>>> Once
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>> which bugs are real and which are just
> >>>> historical, we
> >>>>>> can
> >>>>>>>>>> better
> >>>>>>>>>>>>> judge
> >>>>>>>>>>>>>>>>> scope. And hopefully close a bunch of obsolete
> >>>> bugs.
> >>>>>>>>> (Thanks,
> >>>>>>>>>>>>> Daniel,
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> collecting that list!)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Geoffrey
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
> >>>>>>>>>>>>>>>>> <da...@salesforce.com.invalid> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hi, I wanted to bring up wrong results in
> >>>> Phoenix
> >>>>> and
> >>>>>>>>> some
> >>>>>>>>>>> JIRAs
> >>>>>>>>>>>>>>> around
> >>>>>>>>>>>>>>>>>> them that I think we should fix as the wrong
> >>>> result
> >>>>>>>>> lessens
> >>>>>>>>>>> the
> >>>>>>>>>>>>> end
> >>>>>>>>>>>>>>>>> user's
> >>>>>>>>>>>>>>>>>> trust in Phoenix. Releasing a new version
> >>>> without
> >>>>>>>>>> addressing
> >>>>>>>>>>>>> these
> >>>>>>>>>>>>>>> in a
> >>>>>>>>>>>>>>>>>> minor release hurts our visibility in that these
> >>>>>>>>> critical
> >>>>>>>>>>> issues
> >>>>>>>>>>>>> are
> >>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>> addressed.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Jira's that I'm involved with for example: I've
> >>>>>> already
> >>>>>>>>>> given
> >>>>>>>>>>> a
> >>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>> several months ago for 5833 and there is a
> >>>> chance it
> >>>>>>>>> may fix
> >>>>>>>>>>>> 5484.
> >>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/jira/browse/PHOENIX-5833
> >>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/jira/browse/PHOENIX-5484
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> In addition, inspecting apache JIRA i see
> >>>> several
> >>>>>> other
> >>>>>>>>>> wrong
> >>>>>>>>>>>>> result
> >>>>>>>>>>>>>>>>> JIRAs
> >>>>>>>>>>>>>>>>>> from the community. Some of these certainly are
> >>>>>>>>> probably
> >>>>>>>>>> old
> >>>>>>>>>>>>> issues
> >>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>> incorrect understanding but some of these are
> >>>> opened
> >>>>>> by
> >>>>>>>>> our
> >>>>>>>>>>> own
> >>>>>>>>>>>>> dev
> >>>>>>>>>>>>>>>>>> community and are likely real problems.
> >>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/jira/browse/PHOENIX-6217
> >>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/jira/browse/PHOENIX-5571
> >>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/jira/browse/PHOENIX-4642
> >>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/jira/browse/PHOENIX-4540
> >>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/jira/browse/PHOENIX-4504
> >>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/jira/browse/PHOENIX-4419
> >>>>>>>>>>>>>>>>>>
> >>>>> https://issues.apache.org/jira/browse/PHOENIX-4116
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> What is our stance on this type of issue? Are
> >>>> we
> >>>>>> going
> >>>>>>>>> to
> >>>>>>>>>> say
> >>>>>>>>>>>>> these
> >>>>>>>>>>>>>>> were
> >>>>>>>>>>>>>>>>>> issues prior to 4.15 and not address them?
> >>>> Should
> >>>>> we
> >>>>>>>>> have
> >>>>>>>>>>>>>>> requirements
> >>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> our releases to fix wrong results?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Daniel Wong
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <
> >>>>>>>>>>> yanxinyi@apache.org>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> It's time to discuss the Phoenix 4.16 release.
> >>>>> After
> >>>>>>>>> many
> >>>>>>>>>>>>> people's
> >>>>>>>>>>>>>>>>>>> contributions on the bug fixes, new features,
> >>>> and
> >>>>>>>>> other
> >>>>>>>>>>> works
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> past
> >>>>>>>>>>>>>>>>>>> few months, we are kind of close to the point
> >>>> to
> >>>>>> have
> >>>>>>>>> a RC
> >>>>>>>>>>>>> (still
> >>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> fix test flappers). Please let me know if you
> >>>>> think
> >>>>>>>>> any
> >>>>>>>>>> JIRA
> >>>>>>>>>>>>> must
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>>>> of the Phoenix 4.16 release other than major
> >>>>> blocker
> >>>>>>>>>>>>> PHOENIX-5712.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> If no surprise comes up, I will not wait for
> >>>> any
> >>>>> new
> >>>>>>>>> major
> >>>>>>>>>>>>>> features
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> focus on the RC as soon as possible.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Sincerely,
> >>>>>>>>>>>>>>>>>>> Xinyi
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> Daniel Wong
> >>>>>>>>>>>>>>>>>> Salesforce
> >>>>>>>>>>>>>>>>>> Mobile: 628.217.1808
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> Chinmay Kulkarni
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *István Tóth* | Staff Software Engineer
> >>>>>>>> stoty@cloudera.com <https://www.cloudera.com>
> >>>>>>>> [image: Cloudera] <https://www.cloudera.com/>
> >>>>>>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera>
> >>>> [image:
> >>>>>>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> >>>>>>>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> >>>>>>>> <https://www.cloudera.com/>
> >>>>>>>> ------------------------------
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *István Tóth* | Staff Software Engineer
> >>> stoty@cloudera.com <https://www.cloudera.com>
> >>> [image: Cloudera] <https://www.cloudera.com/>
> >>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> >>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> >>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> >>> <https://www.cloudera.com/>
> >>> ------------------------------
> >>>
> >>
> >>
> >> --
> >> *István Tóth* | Staff Software Engineer
> >> stoty@cloudera.com <https://www.cloudera.com>
> >> [image: Cloudera] <https://www.cloudera.com/>
> >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> >> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> >> <https://www.cloudera.com/>
> >> ------------------------------
> >>
> >
>
Re: [Discuss] Releasing Phoenix 4.16
Posted by Josh Elser <el...@apache.org>.
IMO, a slow build is better than forcing users to run their own dev
build of a release.
Very proud of all of the work y'all have been doing on 4.16. Keep
pushing on the release :)
On 1/12/21 12:35 PM, Istvan Toth wrote:
> I've been working this, and came up with the following:
>
> * We no longer generate phoenix-client.jar and phoenix-server.jar, we call
> them phoenix-client-hbase-X.Y.jar and phoenix-server-hbase-X.Y.jar instead.
> * These file names are used in the binary assemblies, as well as as maven
> coordinates for consistency.
> * We generate four release binaries, for each HBase profile
> * We build and publish to maven the phoenix-client-hbase-X.Y and
> phoenix-server-hbase-X.Y artifacts for each profile (Actually, we deploy
> four times, but the rest of the artifacts are identical)
>
> * On the downside, running the release script on a Macbook takes several
> hours, as we build HBase 4 and Phoenix 8 times on the slow and kludgy Mac
> Docker filesystem.
>
> The almost finished patch to the build system is linked to
> https://issues.apache.org/jira/browse/PHOENIX-6307
>
> 4.x would have the same build system changes once this is approved.
>
> Please check it out, and let me know if this solution is satisfactory, or
> if you have a better idea.
>
> regards
> Istvan
>
>
> On Fri, Jan 8, 2021 at 4:22 PM Istvan Toth <st...@cloudera.com> wrote:
>
>> As I started to work on this I realized that while providing binary
>> tarballs for each HBase profile is fine,
>> this does not solve the maven artifact issue.
>>
>> Are we OK with publishing a single phoenix-client maven artifact (for the
>> oldest HBase),
>> or do we want to publish a separate one for each HBase version ?
>>
>> I looked at publishing multiple client versions, but none of them are
>> particularly easy or attractive.
>>
>> The best I could come up with is adding a separate maven module for (each
>> version x embedded) (i.e 6 for 4.x, 8 for master),
>> and activating them according to hbase.profile.
>>
>> This would also mean that we need to add the hbase version to the artifact
>> id. i.e.: phoenix-client-hbase-2.1
>>
>> Once we publish separate binaries for the HBase profiles, we can undo the
>> change that excludes compat-module from phoenix-server,
>> and shade it in again for the binary assemblies.
>>
>> In this case we'd also have to do something about the phoenix-server maven
>> artifacts. Either they get the same treatment as phoenix-client,
>> or we simply skip publishing them. I personally do not see anyone getting
>> phoenix-server.jar from maven.
>>
>> The easiest version is
>> * publish the oldest profile phoenix-client to maven
>> * do not publish phoenix-server to maven
>> * add compat-module to phoenix-server for the binary artifact
>>
>>
>> regards
>> Istvan
>>
>> On Fri, Jan 8, 2021 at 6:56 AM Istvan Toth <st...@cloudera.com> wrote:
>>
>>> On the release binaries:
>>>
>>> The current solution (which the default profile change has broken) was
>>> based on Lars's idea at
>>>
>>> https://issues.apache.org/jira/browse/PHOENIX-5902?focusedCommentId=17125122&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17125122
>>> However, I agree that providing separate assemblies for each HBase
>>> profile is better for our users, as they won't have to rebuild Phoenix to
>>> take advantage of any new features, and to get the general improvements in
>>> later HBase releases.
>>> I have opened https://issues.apache.org/jira/browse/PHOENIX-6307 to
>>> track this.
>>>
>>> On the 5.1 release:
>>>
>>> Yes, I do want to release 5.1 shortly. In fact $dayjob desperately needs
>>> it ASAP.
>>> I have a few older PRs waiting for review that fell between the cracks,
>>> but as soon as those are merged, I want to cut the first RC.
>>> It would be nice to have 4.16 and 5.1 as close as possible, and
>>> PHOENIX-6211 seems to be ready, so I hope to include it in 5.1 too.
>>> I will start an official 5.1 release thread and volunteer to be the
>>> release manager soon. (unless you want to take that up too, Xinyi).
>>>
>>>
>>>
>>> On Fri, Jan 8, 2021 at 1:08 AM Xinyi Yan <ya...@apache.org> wrote:
>>>
>>>> If we can modify the dev/create-release scripts and make them work for
>>>> the
>>>> 4.16 release with this hbase.profile option, it would make our life much
>>>> easier to release multiple HBase profiles from the single branch in the
>>>> future too(the master branch will have a release shortly right?).
>>>> Geoffrey
>>>> and Istvan, what do you think?
>>>>
>>>> Thanks,
>>>> Xinyi
>>>>
>>>> On Thu, Jan 7, 2021 at 11:28 AM Geoffrey Jacoby <gj...@apache.org>
>>>> wrote:
>>>>
>>>>> Thanks for bringing up the default branch issue, Istvan, I've been
>>>> meaning
>>>>> to start a conversation about it on this list.
>>>>>
>>>>> As part of PHOENIX-5435, I changed the default 4.x HBase release to
>>>> 1.5 and
>>>>> the default 5.x HBase to 2.3 because the WAL annotation feature
>>>> introduced
>>>>> by 5435 only works with HBase 1.5+ or 2.3+. (It depends on a coproc
>>>> hook
>>>>> introduced in HBASE-22623). That means that all tests of that feature
>>>> must
>>>>> no-op when run against an earlier HBase, which means that it would
>>>> never be
>>>>> tested in our CI pipelines if the default was 1.3 or 2.1.
>>>>>
>>>>> This has come up quite a few times recently. In particular, the new
>>>>> indexing framework runs in a degraded state against HBase 2.1 and 2.2
>>>>> (still better than the old indexing framework though!), because we lack
>>>>> support in 2.1 for Filters on raw scans and can't use the "max lookback
>>>>> age" feature with HBase 2.1 or 2.2, which keeps major compactions from
>>>>> messing with index rebuilds. That's why lots of indexing tests no-op or
>>>>> have to make different assertions when run against 2.1 or 2.2, so we
>>>> only
>>>>> properly test the indexing framework if CI and local developer
>>>>> tests run against HBase 2.3 or up.
>>>>>
>>>>> As for the release, don't we usually release separate binaries that are
>>>>> suitable for all the HBase versions we support? Back before we had a
>>>>> unified 4.x branch, I believe we used to release from 3 different
>>>>> 4.x-HBase-* branches each time. I've been assuming that part of the
>>>> release
>>>>> process would be generating binaries for 4.x-HBase-1.3, 4.x-HBase-1.4,
>>>>> 4.x-HBase-1.5, and 4.x-HBase-1.6 (if that was different from 1.5).
>>>>>
>>>>> Geoffrey
>>>>>
>>>>> On Thu, Jan 7, 2021 at 11:19 AM Istvan Toth <st...@apache.org> wrote:
>>>>>
>>>>>> Two more issues came to my mind:
>>>>>>
>>>>>> In PHOENIX-5435 the default profile was changed to HBase 1.5.0
>>>>>> This causes two conflicts:
>>>>>> - Our documentation says that the default profile is the lowest
>>>>> supported,
>>>>>> which is no longer true
>>>>>> - Unless we change our binary release process, the published maven
>>>>>> artifacts and binaries will also be built with HBase 1.5, thus they
>>>> will
>>>>> be
>>>>>> incompatible with Hbase 1.3 and 1.4
>>>>>>
>>>>>> This should be addressed before release.
>>>>>> One possible solution:
>>>>>> * Add an "oldest" hbase.profile option which always chooses the
>>>> oldest
>>>>>> supported version
>>>>>> * Update the release scripts to specify this profile
>>>>>> * Update the docs.
>>>>>>
>>>>>> I have also adopted the hbase release scripts to Phoenix, they are
>>>>> present
>>>>>> in dev/create-release in the master branch, but
>>>>>> should be able to perform 4.16 release as well. I have used this for
>>>> the
>>>>>> phoenix-thirdparty, omid, and tephra releases, but no live phoenix
>>>>> releases
>>>>>> have been done with them yet, obviously.
>>>>>>
>>>>>> regards
>>>>>> Istvan
>>>>>>
>>>>>> On Thu, Jan 7, 2021 at 8:19 AM Istvan Toth <st...@apache.org> wrote:
>>>>>>
>>>>>>> Hi!
>>>>>>>
>>>>>>> A question of process: Should we backport fixes to the 4.16 branch
>>>> at
>>>>> our
>>>>>>> discretion, or is backporting those handled by the release manager
>>>>>> (Xinyi) ?
>>>>>>>
>>>>>>> On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth <st...@cloudera.com>
>>>> wrote:
>>>>>>>
>>>>>>>> Hi!
>>>>>>>>
>>>>>>>> Thanks for everyone's effort on fixing the flakey tests.
>>>>>>>> However, our ASF Jenkins runs are still very unstable, and I am
>>>> afraid
>>>>>>>> that the single clean 4.x run was down more to luck than to
>>>> fixing the
>>>>>>>> underlying problem.
>>>>>>>> While I do not consider this a blocker, fixing this would make the
>>>>> pre-
>>>>>>>> and post commit tests far more useful, and let us start with a
>>>> clean
>>>>>> slate
>>>>>>>> for the next release.
>>>>>>>> I have created https://issues.apache.org/jira/browse/PHOENIX-6288
>>>> to
>>>>>>>> track the generic cluster setup issue, but my attempts to fix
>>>> this, or
>>>>>> at
>>>>>>>> least figure out the root cause have been unsuccessful so far.
>>>>>>>>
>>>>>>>> I ask for your help in fixing this issue. I have added some
>>>>> inconclusive
>>>>>>>> analysis to the ticket, and the full failsafe output directory is
>>>>>>>> downloadable as artifacts from the failed multibranch runs (stored
>>>>> for a
>>>>>>>> few days), which should hold the answer to this riddle.
>>>>>>>>
>>>>>>>> regards,
>>>>>>>> Istvan
>>>>>>>>
>>>>>>>> On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan <ya...@apache.org>
>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> We finally have a stable 4.x branch build after many flapper test
>>>>>> fixing
>>>>>>>>> contributions from the community. I'm going to fork a new
>>>>> branch(4.16)
>>>>>>>>> from
>>>>>>>>> the current 4.x branch for the 4.16 release. As a cutoff, I will
>>>> not
>>>>>>>>> include any new features other than PHOENIX-6211 and bug fix.
>>>> Please
>>>>>> let
>>>>>>>>> me
>>>>>>>>> know if you have any questions or concerns.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Xinyi
>>>>>>>>>
>>>>>>>>> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani <vjasani@apache.org
>>>>>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> An update on test stability:
>>>>>>>>>>
>>>>>>>>>> As per recent 4.x build results, we are left with very few
>>>> flappers
>>>>>> and
>>>>>>>>>> specifically
>>>>>>>>>> with HBase profile 1.6, I can see recent 7 build (multibranch
>>>> + PR
>>>>>>>>>> precommit results)
>>>>>>>>>> results without any test failure.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <yanxinyi@apache.org
>>>>>
>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Yes, we are currently working on fixing the 4.x branch test
>>>>>> flappers
>>>>>>>>>> while
>>>>>>>>>>> waiting for the PHOENIX-5435. After that, I will try to have
>>>> an
>>>>> RC
>>>>>>>>> ASAP.
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <
>>>>>>>>> ankitsinghal59@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I see that both the blockers listed here PHOENIX-5712 and
>>>>>>>>> PHOENIX-6241
>>>>>>>>>>>> have been resolved(Thanks to Xinyi), and also as per the
>>>> JIRA
>>>>>> query
>>>>>>>>>> there
>>>>>>>>>>>> is no Jira marked as a blocker for 4.16 except the one
>>>> related
>>>>> to
>>>>>>>>>>>> documentation
>>>>>>>>>>>> for "splittable catalog table".
>>>>>>>>>>>>
>>>>>>>>>>>> Xinyi, so are we good to start the release process now?
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <
>>>> yanxinyi@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for replying and providing suggestions. I looked
>>>> at
>>>>> the
>>>>>>>>> wrong
>>>>>>>>>>>> result
>>>>>>>>>>>>> Jira list that Daniel provided and did some local
>>>> testing,
>>>>> and
>>>>>>>>> here
>>>>>>>>>> is
>>>>>>>>>>>> the
>>>>>>>>>>>>> result:
>>>>>>>>>>>>> [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642:
>>>>> cannot
>>>>>>>>>>> reproduce
>>>>>>>>>>>>> it.
>>>>>>>>>>>>> [More information is required] PHOENIX-4504 cannot
>>>> reproduce
>>>>> it
>>>>>>>>> but
>>>>>>>>>>>> someone
>>>>>>>>>>>>> claimed that he had a similar issue.
>>>>>>>>>>>>> [unusual query] PHOENIX-4540 and PHOENIX-6217
>>>>>>>>>>>>>
>>>>>>>>>>>>> Based on my finding, I think it's better to have more
>>>>> frequent
>>>>>>>>>>>> housekeeping
>>>>>>>>>>>>> and resolve unreproducible bugs especially since many of
>>>> them
>>>>>> are
>>>>>>>>>>>>> considering out of date (phoenix-4.11 or even
>>>> phoenix-4.6).
>>>>>>>>> Since I
>>>>>>>>>>> still
>>>>>>>>>>>>> need time to work on the blocker Jira(PHOENIX-5712,
>>>>>>>>> PHOENIX-6241) and
>>>>>>>>>>> fix
>>>>>>>>>>>>> test flappers, if you want to fix "unusual query" bugs,
>>>> feel
>>>>>>>>> free to
>>>>>>>>>> do
>>>>>>>>>>>> so.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sincerely,
>>>>>>>>>>>>> Xinyi
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <
>>>>>> ankit@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks Daniel and appreciate the effort you put in
>>>> getting
>>>>>> the
>>>>>>>>> list
>>>>>>>>>>>> ready
>>>>>>>>>>>>>> for bugs producing wrong results
>>>>>>>>>>>>>> but none of them seems to be a blocker to me for 4.16
>>>> as
>>>>> they
>>>>>>>>> are
>>>>>>>>>> not
>>>>>>>>>>>> the
>>>>>>>>>>>>>> regression and doesn't break the general functionality
>>>>>>>>>>>>>> except for specific features, RVC/desc as Chenglei also
>>>>>>>>> pointed out
>>>>>>>>>>>>> (though
>>>>>>>>>>>>>> I'll defer the assessment to RM "Xinyi").
>>>>>>>>>>>>>> Probably these can be a part of 4.16.1 or we can do
>>>> 4.17.0
>>>>>> soon
>>>>>>>>>> maybe
>>>>>>>>>>>>> after
>>>>>>>>>>>>>> a few weeks/month?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Considering that we have already fixed 137 bugs and
>>>> done
>>>>> 85+
>>>>>>>>>>>>>> improvements/features in 4.16,
>>>>>>>>>>>>>> it will not be a good idea to deprive the user from
>>>> such
>>>>>> fixes.
>>>>>>>>>>>>>> It's been a year since our last 4.15 release, having no
>>>>>> release
>>>>>>>>>>> brings
>>>>>>>>>>>>> more
>>>>>>>>>>>>>> questions on the project
>>>>>>>>>>>>>> rather than the bugs which affect a certain % of
>>>>>> feature/users,
>>>>>>>>>> would
>>>>>>>>>>>> the
>>>>>>>>>>>>>> release notes
>>>>>>>>>>>>>> explaining the stability of certain features set the
>>>> right
>>>>>>>>>>> expectation
>>>>>>>>>>>>> for
>>>>>>>>>>>>>> those users who rely on these features to wait for a
>>>> future
>>>>>>>>>> release?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Ankit Singhal
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
>>>>>>>>>>>> chenglei@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In my opinion, we should keep releases light and
>>>>> frequent,
>>>>>>>>> and
>>>>>>>>>> for
>>>>>>>>>>>>> some
>>>>>>>>>>>>>>> unusual query bugs like RVC and DESC
>>>>>>>>>>>>>>> we could delay fix to next release . I think we
>>>> should
>>>>>>>>> release
>>>>>>>>>>> 4.16.0
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> 5.1.0 as quickly as possible. In China, many users
>>>>>>>>>>>>>>> in HBase&Phoenix User Group thought that Phoenix was
>>>>> dead
>>>>>>>>>> because
>>>>>>>>>>>> our
>>>>>>>>>>>>>> too
>>>>>>>>>>>>>>> long interval release and stopped using
>>>>>>>>>>>>>>> Phoenix.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
>>>>>>>>>>>> chinmayskulkarni@gmail.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> I agree. These are all major bugs and we should aim
>>>> at
>>>>>>>>> solving
>>>>>>>>>>> them
>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>> checking that they are still issues. I am +1 on 5833
>>>>> and I
>>>>>>>>> think
>>>>>>>>>>>> 5484
>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>> be a great addition to 4.16 as well. We should aim
>>>> at
>>>>>>>>> resolving
>>>>>>>>>>> high
>>>>>>>>>>>>>>>> priority bugs like this in every release.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Sometimes we let these bugs slip without a
>>>> resolution
>>>>>>>>> before a
>>>>>>>>>>>>> release,
>>>>>>>>>>>>>>>> citing that these are "known issues" or "not
>>>> regressions
>>>>>>>>> from
>>>>>>>>>> the
>>>>>>>>>>>> last
>>>>>>>>>>>>>>>> release". In some cases this may be fine since we
>>>> want
>>>>> to
>>>>>>>>> keep
>>>>>>>>>>>>> releases
>>>>>>>>>>>>>>>> light and frequent, but perhaps we can track such
>>>> issues
>>>>>>>>> and aim
>>>>>>>>>>> at
>>>>>>>>>>>>>>>> reducing the number of bugs by x% in each release?
>>>> This
>>>>>> will
>>>>>>>>>> also
>>>>>>>>>>>> keep
>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>> Jiras alive since we will potentially periodically
>>>>> review
>>>>>>>>> them.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <
>>>>>>>>>>> gjacoby@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I've got PHOENIX-5435 in review right now, and
>>>> would
>>>>>> like
>>>>>>>>> to
>>>>>>>>>> get
>>>>>>>>>>>> it
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> 4.16
>>>>>>>>>>>>>>>>> / 5.1.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It's allowing the annotation of Phoenix metadata
>>>> into
>>>>>>>>> HBase
>>>>>>>>>> WALs
>>>>>>>>>>>> as
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> pre-req for the Phoenix Change Detection Capture
>>>>>> framework
>>>>>>>>>>>>>>> (PHOENIX-5442).
>>>>>>>>>>>>>>>>> Since it has both client/server logic, and adds a
>>>>> field
>>>>>> to
>>>>>>>>>>>>>>> System.Catalog,
>>>>>>>>>>>>>>>>> it can't go in a patch release.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Depending on timing, I'd _like_ to get
>>>> PHOENIX-6227,
>>>>>>>>> which is
>>>>>>>>>>> the
>>>>>>>>>>>>> last
>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>> of CDC that will go into core Phoenix, into 4.16,
>>>> but
>>>>>>>>> since
>>>>>>>>>> that
>>>>>>>>>>>>> _can_
>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>> in a patch release and I haven't started it yet,
>>>> if
>>>>> the
>>>>>>>>>> release
>>>>>>>>>>>> gets
>>>>>>>>>>>>>> cut
>>>>>>>>>>>>>>>>> before it's ready, no big deal. (The rest of CDC
>>>> will
>>>>> go
>>>>>>>>> into
>>>>>>>>>>>>>>>>> phoenix-connectors for a future release of that
>>>>>> project.)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> As for the correctness problems that Daniel points
>>>>> out,
>>>>>> I
>>>>>>>>>> think
>>>>>>>>>>> we
>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>> fix the ones that were detected with a recent
>>>> version
>>>>>>>>> (4.14 or
>>>>>>>>>>>>> 4.15?),
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> test to see which of the older ones can still be
>>>>>>>>> reproduced.
>>>>>>>>>>> Once
>>>>>>>>>>>> we
>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>> which bugs are real and which are just
>>>> historical, we
>>>>>> can
>>>>>>>>>> better
>>>>>>>>>>>>> judge
>>>>>>>>>>>>>>>>> scope. And hopefully close a bunch of obsolete
>>>> bugs.
>>>>>>>>> (Thanks,
>>>>>>>>>>>>> Daniel,
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> collecting that list!)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Geoffrey
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
>>>>>>>>>>>>>>>>> <da...@salesforce.com.invalid> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi, I wanted to bring up wrong results in
>>>> Phoenix
>>>>> and
>>>>>>>>> some
>>>>>>>>>>> JIRAs
>>>>>>>>>>>>>>> around
>>>>>>>>>>>>>>>>>> them that I think we should fix as the wrong
>>>> result
>>>>>>>>> lessens
>>>>>>>>>>> the
>>>>>>>>>>>>> end
>>>>>>>>>>>>>>>>> user's
>>>>>>>>>>>>>>>>>> trust in Phoenix. Releasing a new version
>>>> without
>>>>>>>>>> addressing
>>>>>>>>>>>>> these
>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>> minor release hurts our visibility in that these
>>>>>>>>> critical
>>>>>>>>>>> issues
>>>>>>>>>>>>> are
>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>> addressed.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Jira's that I'm involved with for example: I've
>>>>>> already
>>>>>>>>>> given
>>>>>>>>>>> a
>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>> several months ago for 5833 and there is a
>>>> chance it
>>>>>>>>> may fix
>>>>>>>>>>>> 5484.
>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/jira/browse/PHOENIX-5833
>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/jira/browse/PHOENIX-5484
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> In addition, inspecting apache JIRA i see
>>>> several
>>>>>> other
>>>>>>>>>> wrong
>>>>>>>>>>>>> result
>>>>>>>>>>>>>>>>> JIRAs
>>>>>>>>>>>>>>>>>> from the community. Some of these certainly are
>>>>>>>>> probably
>>>>>>>>>> old
>>>>>>>>>>>>> issues
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>> incorrect understanding but some of these are
>>>> opened
>>>>>> by
>>>>>>>>> our
>>>>>>>>>>> own
>>>>>>>>>>>>> dev
>>>>>>>>>>>>>>>>>> community and are likely real problems.
>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/jira/browse/PHOENIX-6217
>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/jira/browse/PHOENIX-5571
>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/jira/browse/PHOENIX-4642
>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/jira/browse/PHOENIX-4540
>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/jira/browse/PHOENIX-4504
>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/jira/browse/PHOENIX-4419
>>>>>>>>>>>>>>>>>>
>>>>> https://issues.apache.org/jira/browse/PHOENIX-4116
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> What is our stance on this type of issue? Are
>>>> we
>>>>>> going
>>>>>>>>> to
>>>>>>>>>> say
>>>>>>>>>>>>> these
>>>>>>>>>>>>>>> were
>>>>>>>>>>>>>>>>>> issues prior to 4.15 and not address them?
>>>> Should
>>>>> we
>>>>>>>>> have
>>>>>>>>>>>>>>> requirements
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> our releases to fix wrong results?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Daniel Wong
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <
>>>>>>>>>>> yanxinyi@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> It's time to discuss the Phoenix 4.16 release.
>>>>> After
>>>>>>>>> many
>>>>>>>>>>>>> people's
>>>>>>>>>>>>>>>>>>> contributions on the bug fixes, new features,
>>>> and
>>>>>>>>> other
>>>>>>>>>>> works
>>>>>>>>>>>> in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> past
>>>>>>>>>>>>>>>>>>> few months, we are kind of close to the point
>>>> to
>>>>>> have
>>>>>>>>> a RC
>>>>>>>>>>>>> (still
>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> fix test flappers). Please let me know if you
>>>>> think
>>>>>>>>> any
>>>>>>>>>> JIRA
>>>>>>>>>>>>> must
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>>>> of the Phoenix 4.16 release other than major
>>>>> blocker
>>>>>>>>>>>>> PHOENIX-5712.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If no surprise comes up, I will not wait for
>>>> any
>>>>> new
>>>>>>>>> major
>>>>>>>>>>>>>> features
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> focus on the RC as soon as possible.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Sincerely,
>>>>>>>>>>>>>>>>>>> Xinyi
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Daniel Wong
>>>>>>>>>>>>>>>>>> Salesforce
>>>>>>>>>>>>>>>>>> Mobile: 628.217.1808
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Chinmay Kulkarni
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *István Tóth* | Staff Software Engineer
>>>>>>>> stoty@cloudera.com <https://www.cloudera.com>
>>>>>>>> [image: Cloudera] <https://www.cloudera.com/>
>>>>>>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera>
>>>> [image:
>>>>>>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>>>>>>>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>>>>>>>> <https://www.cloudera.com/>
>>>>>>>> ------------------------------
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> *István Tóth* | Staff Software Engineer
>>> stoty@cloudera.com <https://www.cloudera.com>
>>> [image: Cloudera] <https://www.cloudera.com/>
>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>>> <https://www.cloudera.com/>
>>> ------------------------------
>>>
>>
>>
>> --
>> *István Tóth* | Staff Software Engineer
>> stoty@cloudera.com <https://www.cloudera.com>
>> [image: Cloudera] <https://www.cloudera.com/>
>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>> <https://www.cloudera.com/>
>> ------------------------------
>>
>
Re: [Discuss] Releasing Phoenix 4.16
Posted by Istvan Toth <st...@apache.org>.
I've been working this, and came up with the following:
* We no longer generate phoenix-client.jar and phoenix-server.jar, we call
them phoenix-client-hbase-X.Y.jar and phoenix-server-hbase-X.Y.jar instead.
* These file names are used in the binary assemblies, as well as as maven
coordinates for consistency.
* We generate four release binaries, for each HBase profile
* We build and publish to maven the phoenix-client-hbase-X.Y and
phoenix-server-hbase-X.Y artifacts for each profile (Actually, we deploy
four times, but the rest of the artifacts are identical)
* On the downside, running the release script on a Macbook takes several
hours, as we build HBase 4 and Phoenix 8 times on the slow and kludgy Mac
Docker filesystem.
The almost finished patch to the build system is linked to
https://issues.apache.org/jira/browse/PHOENIX-6307
4.x would have the same build system changes once this is approved.
Please check it out, and let me know if this solution is satisfactory, or
if you have a better idea.
regards
Istvan
On Fri, Jan 8, 2021 at 4:22 PM Istvan Toth <st...@cloudera.com> wrote:
> As I started to work on this I realized that while providing binary
> tarballs for each HBase profile is fine,
> this does not solve the maven artifact issue.
>
> Are we OK with publishing a single phoenix-client maven artifact (for the
> oldest HBase),
> or do we want to publish a separate one for each HBase version ?
>
> I looked at publishing multiple client versions, but none of them are
> particularly easy or attractive.
>
> The best I could come up with is adding a separate maven module for (each
> version x embedded) (i.e 6 for 4.x, 8 for master),
> and activating them according to hbase.profile.
>
> This would also mean that we need to add the hbase version to the artifact
> id. i.e.: phoenix-client-hbase-2.1
>
> Once we publish separate binaries for the HBase profiles, we can undo the
> change that excludes compat-module from phoenix-server,
> and shade it in again for the binary assemblies.
>
> In this case we'd also have to do something about the phoenix-server maven
> artifacts. Either they get the same treatment as phoenix-client,
> or we simply skip publishing them. I personally do not see anyone getting
> phoenix-server.jar from maven.
>
> The easiest version is
> * publish the oldest profile phoenix-client to maven
> * do not publish phoenix-server to maven
> * add compat-module to phoenix-server for the binary artifact
>
>
> regards
> Istvan
>
> On Fri, Jan 8, 2021 at 6:56 AM Istvan Toth <st...@cloudera.com> wrote:
>
>> On the release binaries:
>>
>> The current solution (which the default profile change has broken) was
>> based on Lars's idea at
>>
>> https://issues.apache.org/jira/browse/PHOENIX-5902?focusedCommentId=17125122&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17125122
>> However, I agree that providing separate assemblies for each HBase
>> profile is better for our users, as they won't have to rebuild Phoenix to
>> take advantage of any new features, and to get the general improvements in
>> later HBase releases.
>> I have opened https://issues.apache.org/jira/browse/PHOENIX-6307 to
>> track this.
>>
>> On the 5.1 release:
>>
>> Yes, I do want to release 5.1 shortly. In fact $dayjob desperately needs
>> it ASAP.
>> I have a few older PRs waiting for review that fell between the cracks,
>> but as soon as those are merged, I want to cut the first RC.
>> It would be nice to have 4.16 and 5.1 as close as possible, and
>> PHOENIX-6211 seems to be ready, so I hope to include it in 5.1 too.
>> I will start an official 5.1 release thread and volunteer to be the
>> release manager soon. (unless you want to take that up too, Xinyi).
>>
>>
>>
>> On Fri, Jan 8, 2021 at 1:08 AM Xinyi Yan <ya...@apache.org> wrote:
>>
>>> If we can modify the dev/create-release scripts and make them work for
>>> the
>>> 4.16 release with this hbase.profile option, it would make our life much
>>> easier to release multiple HBase profiles from the single branch in the
>>> future too(the master branch will have a release shortly right?).
>>> Geoffrey
>>> and Istvan, what do you think?
>>>
>>> Thanks,
>>> Xinyi
>>>
>>> On Thu, Jan 7, 2021 at 11:28 AM Geoffrey Jacoby <gj...@apache.org>
>>> wrote:
>>>
>>> > Thanks for bringing up the default branch issue, Istvan, I've been
>>> meaning
>>> > to start a conversation about it on this list.
>>> >
>>> > As part of PHOENIX-5435, I changed the default 4.x HBase release to
>>> 1.5 and
>>> > the default 5.x HBase to 2.3 because the WAL annotation feature
>>> introduced
>>> > by 5435 only works with HBase 1.5+ or 2.3+. (It depends on a coproc
>>> hook
>>> > introduced in HBASE-22623). That means that all tests of that feature
>>> must
>>> > no-op when run against an earlier HBase, which means that it would
>>> never be
>>> > tested in our CI pipelines if the default was 1.3 or 2.1.
>>> >
>>> > This has come up quite a few times recently. In particular, the new
>>> > indexing framework runs in a degraded state against HBase 2.1 and 2.2
>>> > (still better than the old indexing framework though!), because we lack
>>> > support in 2.1 for Filters on raw scans and can't use the "max lookback
>>> > age" feature with HBase 2.1 or 2.2, which keeps major compactions from
>>> > messing with index rebuilds. That's why lots of indexing tests no-op or
>>> > have to make different assertions when run against 2.1 or 2.2, so we
>>> only
>>> > properly test the indexing framework if CI and local developer
>>> > tests run against HBase 2.3 or up.
>>> >
>>> > As for the release, don't we usually release separate binaries that are
>>> > suitable for all the HBase versions we support? Back before we had a
>>> > unified 4.x branch, I believe we used to release from 3 different
>>> > 4.x-HBase-* branches each time. I've been assuming that part of the
>>> release
>>> > process would be generating binaries for 4.x-HBase-1.3, 4.x-HBase-1.4,
>>> > 4.x-HBase-1.5, and 4.x-HBase-1.6 (if that was different from 1.5).
>>> >
>>> > Geoffrey
>>> >
>>> > On Thu, Jan 7, 2021 at 11:19 AM Istvan Toth <st...@apache.org> wrote:
>>> >
>>> > > Two more issues came to my mind:
>>> > >
>>> > > In PHOENIX-5435 the default profile was changed to HBase 1.5.0
>>> > > This causes two conflicts:
>>> > > - Our documentation says that the default profile is the lowest
>>> > supported,
>>> > > which is no longer true
>>> > > - Unless we change our binary release process, the published maven
>>> > > artifacts and binaries will also be built with HBase 1.5, thus they
>>> will
>>> > be
>>> > > incompatible with Hbase 1.3 and 1.4
>>> > >
>>> > > This should be addressed before release.
>>> > > One possible solution:
>>> > > * Add an "oldest" hbase.profile option which always chooses the
>>> oldest
>>> > > supported version
>>> > > * Update the release scripts to specify this profile
>>> > > * Update the docs.
>>> > >
>>> > > I have also adopted the hbase release scripts to Phoenix, they are
>>> > present
>>> > > in dev/create-release in the master branch, but
>>> > > should be able to perform 4.16 release as well. I have used this for
>>> the
>>> > > phoenix-thirdparty, omid, and tephra releases, but no live phoenix
>>> > releases
>>> > > have been done with them yet, obviously.
>>> > >
>>> > > regards
>>> > > Istvan
>>> > >
>>> > > On Thu, Jan 7, 2021 at 8:19 AM Istvan Toth <st...@apache.org> wrote:
>>> > >
>>> > > > Hi!
>>> > > >
>>> > > > A question of process: Should we backport fixes to the 4.16 branch
>>> at
>>> > our
>>> > > > discretion, or is backporting those handled by the release manager
>>> > > (Xinyi) ?
>>> > > >
>>> > > > On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth <st...@cloudera.com>
>>> wrote:
>>> > > >
>>> > > >> Hi!
>>> > > >>
>>> > > >> Thanks for everyone's effort on fixing the flakey tests.
>>> > > >> However, our ASF Jenkins runs are still very unstable, and I am
>>> afraid
>>> > > >> that the single clean 4.x run was down more to luck than to
>>> fixing the
>>> > > >> underlying problem.
>>> > > >> While I do not consider this a blocker, fixing this would make the
>>> > pre-
>>> > > >> and post commit tests far more useful, and let us start with a
>>> clean
>>> > > slate
>>> > > >> for the next release.
>>> > > >> I have created https://issues.apache.org/jira/browse/PHOENIX-6288
>>> to
>>> > > >> track the generic cluster setup issue, but my attempts to fix
>>> this, or
>>> > > at
>>> > > >> least figure out the root cause have been unsuccessful so far.
>>> > > >>
>>> > > >> I ask for your help in fixing this issue. I have added some
>>> > inconclusive
>>> > > >> analysis to the ticket, and the full failsafe output directory is
>>> > > >> downloadable as artifacts from the failed multibranch runs (stored
>>> > for a
>>> > > >> few days), which should hold the answer to this riddle.
>>> > > >>
>>> > > >> regards,
>>> > > >> Istvan
>>> > > >>
>>> > > >> On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan <ya...@apache.org>
>>> wrote:
>>> > > >>
>>> > > >>> Hi,
>>> > > >>>
>>> > > >>> We finally have a stable 4.x branch build after many flapper test
>>> > > fixing
>>> > > >>> contributions from the community. I'm going to fork a new
>>> > branch(4.16)
>>> > > >>> from
>>> > > >>> the current 4.x branch for the 4.16 release. As a cutoff, I will
>>> not
>>> > > >>> include any new features other than PHOENIX-6211 and bug fix.
>>> Please
>>> > > let
>>> > > >>> me
>>> > > >>> know if you have any questions or concerns.
>>> > > >>>
>>> > > >>> Thanks,
>>> > > >>> Xinyi
>>> > > >>>
>>> > > >>> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani <vjasani@apache.org
>>> >
>>> > > wrote:
>>> > > >>>
>>> > > >>> > An update on test stability:
>>> > > >>> >
>>> > > >>> > As per recent 4.x build results, we are left with very few
>>> flappers
>>> > > and
>>> > > >>> > specifically
>>> > > >>> > with HBase profile 1.6, I can see recent 7 build (multibranch
>>> + PR
>>> > > >>> > precommit results)
>>> > > >>> > results without any test failure.
>>> > > >>> >
>>> > > >>> >
>>> > > >>> > On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <yanxinyi@apache.org
>>> >
>>> > > wrote:
>>> > > >>> >
>>> > > >>> > > Yes, we are currently working on fixing the 4.x branch test
>>> > > flappers
>>> > > >>> > while
>>> > > >>> > > waiting for the PHOENIX-5435. After that, I will try to have
>>> an
>>> > RC
>>> > > >>> ASAP.
>>> > > >>> > >
>>> > > >>> > > On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <
>>> > > >>> ankitsinghal59@gmail.com>
>>> > > >>> > > wrote:
>>> > > >>> > >
>>> > > >>> > > > I see that both the blockers listed here PHOENIX-5712 and
>>> > > >>> PHOENIX-6241
>>> > > >>> > > > have been resolved(Thanks to Xinyi), and also as per the
>>> JIRA
>>> > > query
>>> > > >>> > there
>>> > > >>> > > > is no Jira marked as a blocker for 4.16 except the one
>>> related
>>> > to
>>> > > >>> > > > documentation
>>> > > >>> > > > for "splittable catalog table".
>>> > > >>> > > >
>>> > > >>> > > > Xinyi, so are we good to start the release process now?
>>> > > >>> > > >
>>> > > >>> > > > On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <
>>> yanxinyi@apache.org>
>>> > > >>> wrote:
>>> > > >>> > > >
>>> > > >>> > > > > Thanks for replying and providing suggestions. I looked
>>> at
>>> > the
>>> > > >>> wrong
>>> > > >>> > > > result
>>> > > >>> > > > > Jira list that Daniel provided and did some local
>>> testing,
>>> > and
>>> > > >>> here
>>> > > >>> > is
>>> > > >>> > > > the
>>> > > >>> > > > > result:
>>> > > >>> > > > > [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642:
>>> > cannot
>>> > > >>> > > reproduce
>>> > > >>> > > > > it.
>>> > > >>> > > > > [More information is required] PHOENIX-4504 cannot
>>> reproduce
>>> > it
>>> > > >>> but
>>> > > >>> > > > someone
>>> > > >>> > > > > claimed that he had a similar issue.
>>> > > >>> > > > > [unusual query] PHOENIX-4540 and PHOENIX-6217
>>> > > >>> > > > >
>>> > > >>> > > > > Based on my finding, I think it's better to have more
>>> > frequent
>>> > > >>> > > > housekeeping
>>> > > >>> > > > > and resolve unreproducible bugs especially since many of
>>> them
>>> > > are
>>> > > >>> > > > > considering out of date (phoenix-4.11 or even
>>> phoenix-4.6).
>>> > > >>> Since I
>>> > > >>> > > still
>>> > > >>> > > > > need time to work on the blocker Jira(PHOENIX-5712,
>>> > > >>> PHOENIX-6241) and
>>> > > >>> > > fix
>>> > > >>> > > > > test flappers, if you want to fix "unusual query" bugs,
>>> feel
>>> > > >>> free to
>>> > > >>> > do
>>> > > >>> > > > so.
>>> > > >>> > > > >
>>> > > >>> > > > >
>>> > > >>> > > > > Sincerely,
>>> > > >>> > > > > Xinyi
>>> > > >>> > > > >
>>> > > >>> > > > > On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <
>>> > > ankit@apache.org>
>>> > > >>> > > wrote:
>>> > > >>> > > > >
>>> > > >>> > > > > > Thanks Daniel and appreciate the effort you put in
>>> getting
>>> > > the
>>> > > >>> list
>>> > > >>> > > > ready
>>> > > >>> > > > > > for bugs producing wrong results
>>> > > >>> > > > > > but none of them seems to be a blocker to me for 4.16
>>> as
>>> > they
>>> > > >>> are
>>> > > >>> > not
>>> > > >>> > > > the
>>> > > >>> > > > > > regression and doesn't break the general functionality
>>> > > >>> > > > > > except for specific features, RVC/desc as Chenglei also
>>> > > >>> pointed out
>>> > > >>> > > > > (though
>>> > > >>> > > > > > I'll defer the assessment to RM "Xinyi").
>>> > > >>> > > > > > Probably these can be a part of 4.16.1 or we can do
>>> 4.17.0
>>> > > soon
>>> > > >>> > maybe
>>> > > >>> > > > > after
>>> > > >>> > > > > > a few weeks/month?
>>> > > >>> > > > > >
>>> > > >>> > > > > > Considering that we have already fixed 137 bugs and
>>> done
>>> > 85+
>>> > > >>> > > > > > improvements/features in 4.16,
>>> > > >>> > > > > > it will not be a good idea to deprive the user from
>>> such
>>> > > fixes.
>>> > > >>> > > > > > It's been a year since our last 4.15 release, having no
>>> > > release
>>> > > >>> > > brings
>>> > > >>> > > > > more
>>> > > >>> > > > > > questions on the project
>>> > > >>> > > > > > rather than the bugs which affect a certain % of
>>> > > feature/users,
>>> > > >>> > would
>>> > > >>> > > > the
>>> > > >>> > > > > > release notes
>>> > > >>> > > > > > explaining the stability of certain features set the
>>> right
>>> > > >>> > > expectation
>>> > > >>> > > > > for
>>> > > >>> > > > > > those users who rely on these features to wait for a
>>> future
>>> > > >>> > release?
>>> > > >>> > > > > >
>>> > > >>> > > > > > Regards,
>>> > > >>> > > > > > Ankit Singhal
>>> > > >>> > > > > >
>>> > > >>> > > > > > On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
>>> > > >>> > > > chenglei@apache.org>
>>> > > >>> > > > > > wrote:
>>> > > >>> > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > > In my opinion, we should keep releases light and
>>> > frequent,
>>> > > >>> and
>>> > > >>> > for
>>> > > >>> > > > > some
>>> > > >>> > > > > > > unusual query bugs like RVC and DESC
>>> > > >>> > > > > > > we could delay fix to next release . I think we
>>> should
>>> > > >>> release
>>> > > >>> > > 4.16.0
>>> > > >>> > > > > and
>>> > > >>> > > > > > > 5.1.0 as quickly as possible. In China, many users
>>> > > >>> > > > > > > in HBase&Phoenix User Group thought that Phoenix was
>>> > dead
>>> > > >>> > because
>>> > > >>> > > > our
>>> > > >>> > > > > > too
>>> > > >>> > > > > > > long interval release and stopped using
>>> > > >>> > > > > > > Phoenix.
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > >
>>> > > >>> > > > > > > At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
>>> > > >>> > > > chinmayskulkarni@gmail.com
>>> > > >>> > > > > >
>>> > > >>> > > > > > > wrote:
>>> > > >>> > > > > > > >I agree. These are all major bugs and we should aim
>>> at
>>> > > >>> solving
>>> > > >>> > > them
>>> > > >>> > > > > > after
>>> > > >>> > > > > > > >checking that they are still issues. I am +1 on 5833
>>> > and I
>>> > > >>> think
>>> > > >>> > > > 5484
>>> > > >>> > > > > > > would
>>> > > >>> > > > > > > >be a great addition to 4.16 as well. We should aim
>>> at
>>> > > >>> resolving
>>> > > >>> > > high
>>> > > >>> > > > > > > >priority bugs like this in every release.
>>> > > >>> > > > > > > >
>>> > > >>> > > > > > > >Sometimes we let these bugs slip without a
>>> resolution
>>> > > >>> before a
>>> > > >>> > > > > release,
>>> > > >>> > > > > > > >citing that these are "known issues" or "not
>>> regressions
>>> > > >>> from
>>> > > >>> > the
>>> > > >>> > > > last
>>> > > >>> > > > > > > >release". In some cases this may be fine since we
>>> want
>>> > to
>>> > > >>> keep
>>> > > >>> > > > > releases
>>> > > >>> > > > > > > >light and frequent, but perhaps we can track such
>>> issues
>>> > > >>> and aim
>>> > > >>> > > at
>>> > > >>> > > > > > > >reducing the number of bugs by x% in each release?
>>> This
>>> > > will
>>> > > >>> > also
>>> > > >>> > > > keep
>>> > > >>> > > > > > old
>>> > > >>> > > > > > > >Jiras alive since we will potentially periodically
>>> > review
>>> > > >>> them.
>>> > > >>> > > > > > > >
>>> > > >>> > > > > > > >
>>> > > >>> > > > > > > >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <
>>> > > >>> > > gjacoby@apache.org>
>>> > > >>> > > > > > > wrote:
>>> > > >>> > > > > > > >
>>> > > >>> > > > > > > >> I've got PHOENIX-5435 in review right now, and
>>> would
>>> > > like
>>> > > >>> to
>>> > > >>> > get
>>> > > >>> > > > it
>>> > > >>> > > > > in
>>> > > >>> > > > > > > 4.16
>>> > > >>> > > > > > > >> / 5.1.
>>> > > >>> > > > > > > >>
>>> > > >>> > > > > > > >> It's allowing the annotation of Phoenix metadata
>>> into
>>> > > >>> HBase
>>> > > >>> > WALs
>>> > > >>> > > > as
>>> > > >>> > > > > a
>>> > > >>> > > > > > > >> pre-req for the Phoenix Change Detection Capture
>>> > > framework
>>> > > >>> > > > > > > (PHOENIX-5442).
>>> > > >>> > > > > > > >> Since it has both client/server logic, and adds a
>>> > field
>>> > > to
>>> > > >>> > > > > > > System.Catalog,
>>> > > >>> > > > > > > >> it can't go in a patch release.
>>> > > >>> > > > > > > >>
>>> > > >>> > > > > > > >> Depending on timing, I'd _like_ to get
>>> PHOENIX-6227,
>>> > > >>> which is
>>> > > >>> > > the
>>> > > >>> > > > > last
>>> > > >>> > > > > > > part
>>> > > >>> > > > > > > >> of CDC that will go into core Phoenix, into 4.16,
>>> but
>>> > > >>> since
>>> > > >>> > that
>>> > > >>> > > > > _can_
>>> > > >>> > > > > > > go
>>> > > >>> > > > > > > >> in a patch release and I haven't started it yet,
>>> if
>>> > the
>>> > > >>> > release
>>> > > >>> > > > gets
>>> > > >>> > > > > > cut
>>> > > >>> > > > > > > >> before it's ready, no big deal. (The rest of CDC
>>> will
>>> > go
>>> > > >>> into
>>> > > >>> > > > > > > >> phoenix-connectors for a future release of that
>>> > > project.)
>>> > > >>> > > > > > > >>
>>> > > >>> > > > > > > >> As for the correctness problems that Daniel points
>>> > out,
>>> > > I
>>> > > >>> > think
>>> > > >>> > > we
>>> > > >>> > > > > > > should
>>> > > >>> > > > > > > >> fix the ones that were detected with a recent
>>> version
>>> > > >>> (4.14 or
>>> > > >>> > > > > 4.15?),
>>> > > >>> > > > > > > and
>>> > > >>> > > > > > > >> test to see which of the older ones can still be
>>> > > >>> reproduced.
>>> > > >>> > > Once
>>> > > >>> > > > we
>>> > > >>> > > > > > > know
>>> > > >>> > > > > > > >> which bugs are real and which are just
>>> historical, we
>>> > > can
>>> > > >>> > better
>>> > > >>> > > > > judge
>>> > > >>> > > > > > > >> scope. And hopefully close a bunch of obsolete
>>> bugs.
>>> > > >>> (Thanks,
>>> > > >>> > > > > Daniel,
>>> > > >>> > > > > > > for
>>> > > >>> > > > > > > >> collecting that list!)
>>> > > >>> > > > > > > >>
>>> > > >>> > > > > > > >> Geoffrey
>>> > > >>> > > > > > > >>
>>> > > >>> > > > > > > >>
>>> > > >>> > > > > > > >>
>>> > > >>> > > > > > > >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
>>> > > >>> > > > > > > >> <da...@salesforce.com.invalid> wrote:
>>> > > >>> > > > > > > >>
>>> > > >>> > > > > > > >> > Hi, I wanted to bring up wrong results in
>>> Phoenix
>>> > and
>>> > > >>> some
>>> > > >>> > > JIRAs
>>> > > >>> > > > > > > around
>>> > > >>> > > > > > > >> > them that I think we should fix as the wrong
>>> result
>>> > > >>> lessens
>>> > > >>> > > the
>>> > > >>> > > > > end
>>> > > >>> > > > > > > >> user's
>>> > > >>> > > > > > > >> > trust in Phoenix. Releasing a new version
>>> without
>>> > > >>> > addressing
>>> > > >>> > > > > these
>>> > > >>> > > > > > > in a
>>> > > >>> > > > > > > >> > minor release hurts our visibility in that these
>>> > > >>> critical
>>> > > >>> > > issues
>>> > > >>> > > > > are
>>> > > >>> > > > > > > not
>>> > > >>> > > > > > > >> > addressed.
>>> > > >>> > > > > > > >> >
>>> > > >>> > > > > > > >> > Jira's that I'm involved with for example: I've
>>> > > already
>>> > > >>> > given
>>> > > >>> > > a
>>> > > >>> > > > > > patch
>>> > > >>> > > > > > > >> > several months ago for 5833 and there is a
>>> chance it
>>> > > >>> may fix
>>> > > >>> > > > 5484.
>>> > > >>> > > > > > > >> >
>>> > https://issues.apache.org/jira/browse/PHOENIX-5833
>>> > > >>> > > > > > > >> >
>>> > https://issues.apache.org/jira/browse/PHOENIX-5484
>>> > > >>> > > > > > > >> >
>>> > > >>> > > > > > > >> > In addition, inspecting apache JIRA i see
>>> several
>>> > > other
>>> > > >>> > wrong
>>> > > >>> > > > > result
>>> > > >>> > > > > > > >> JIRAs
>>> > > >>> > > > > > > >> > from the community. Some of these certainly are
>>> > > >>> probably
>>> > > >>> > old
>>> > > >>> > > > > issues
>>> > > >>> > > > > > > or
>>> > > >>> > > > > > > >> > incorrect understanding but some of these are
>>> opened
>>> > > by
>>> > > >>> our
>>> > > >>> > > own
>>> > > >>> > > > > dev
>>> > > >>> > > > > > > >> > community and are likely real problems.
>>> > > >>> > > > > > > >> >
>>> > https://issues.apache.org/jira/browse/PHOENIX-6217
>>> > > >>> > > > > > > >> >
>>> > https://issues.apache.org/jira/browse/PHOENIX-5571
>>> > > >>> > > > > > > >> >
>>> > https://issues.apache.org/jira/browse/PHOENIX-4642
>>> > > >>> > > > > > > >> >
>>> > https://issues.apache.org/jira/browse/PHOENIX-4540
>>> > > >>> > > > > > > >> >
>>> > https://issues.apache.org/jira/browse/PHOENIX-4504
>>> > > >>> > > > > > > >> >
>>> > https://issues.apache.org/jira/browse/PHOENIX-4419
>>> > > >>> > > > > > > >> >
>>> > https://issues.apache.org/jira/browse/PHOENIX-4116
>>> > > >>> > > > > > > >> >
>>> > > >>> > > > > > > >> > What is our stance on this type of issue? Are
>>> we
>>> > > going
>>> > > >>> to
>>> > > >>> > say
>>> > > >>> > > > > these
>>> > > >>> > > > > > > were
>>> > > >>> > > > > > > >> > issues prior to 4.15 and not address them?
>>> Should
>>> > we
>>> > > >>> have
>>> > > >>> > > > > > > requirements
>>> > > >>> > > > > > > >> for
>>> > > >>> > > > > > > >> > our releases to fix wrong results?
>>> > > >>> > > > > > > >> >
>>> > > >>> > > > > > > >> > Daniel Wong
>>> > > >>> > > > > > > >> >
>>> > > >>> > > > > > > >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <
>>> > > >>> > > yanxinyi@apache.org>
>>> > > >>> > > > > > > wrote:
>>> > > >>> > > > > > > >> >
>>> > > >>> > > > > > > >> > > Hi all,
>>> > > >>> > > > > > > >> > >
>>> > > >>> > > > > > > >> > > It's time to discuss the Phoenix 4.16 release.
>>> > After
>>> > > >>> many
>>> > > >>> > > > > people's
>>> > > >>> > > > > > > >> > > contributions on the bug fixes, new features,
>>> and
>>> > > >>> other
>>> > > >>> > > works
>>> > > >>> > > > in
>>> > > >>> > > > > > the
>>> > > >>> > > > > > > >> past
>>> > > >>> > > > > > > >> > > few months, we are kind of close to the point
>>> to
>>> > > have
>>> > > >>> a RC
>>> > > >>> > > > > (still
>>> > > >>> > > > > > > need
>>> > > >>> > > > > > > >> to
>>> > > >>> > > > > > > >> > > fix test flappers). Please let me know if you
>>> > think
>>> > > >>> any
>>> > > >>> > JIRA
>>> > > >>> > > > > must
>>> > > >>> > > > > > be
>>> > > >>> > > > > > > >> part
>>> > > >>> > > > > > > >> > > of the Phoenix 4.16 release other than major
>>> > blocker
>>> > > >>> > > > > PHOENIX-5712.
>>> > > >>> > > > > > > >> > >
>>> > > >>> > > > > > > >> > > If no surprise comes up, I will not wait for
>>> any
>>> > new
>>> > > >>> major
>>> > > >>> > > > > > features
>>> > > >>> > > > > > > and
>>> > > >>> > > > > > > >> > > focus on the RC as soon as possible.
>>> > > >>> > > > > > > >> > >
>>> > > >>> > > > > > > >> > > Sincerely,
>>> > > >>> > > > > > > >> > > Xinyi
>>> > > >>> > > > > > > >> > >
>>> > > >>> > > > > > > >> >
>>> > > >>> > > > > > > >> >
>>> > > >>> > > > > > > >> > --
>>> > > >>> > > > > > > >> > Daniel Wong
>>> > > >>> > > > > > > >> > Salesforce
>>> > > >>> > > > > > > >> > Mobile: 628.217.1808
>>> > > >>> > > > > > > >> >
>>> > > >>> > > > > > > >>
>>> > > >>> > > > > > > >
>>> > > >>> > > > > > > >
>>> > > >>> > > > > > > >--
>>> > > >>> > > > > > > >Chinmay Kulkarni
>>> > > >>> > > > > > >
>>> > > >>> > > > > >
>>> > > >>> > > > >
>>> > > >>> > > >
>>> > > >>> > >
>>> > > >>> >
>>> > > >>>
>>> > > >>
>>> > > >>
>>> > > >> --
>>> > > >> *István Tóth* | Staff Software Engineer
>>> > > >> stoty@cloudera.com <https://www.cloudera.com>
>>> > > >> [image: Cloudera] <https://www.cloudera.com/>
>>> > > >> [image: Cloudera on Twitter] <https://twitter.com/cloudera>
>>> [image:
>>> > > >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>>> > > >> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>>> > > >> <https://www.cloudera.com/>
>>> > > >> ------------------------------
>>> > > >>
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>> --
>> *István Tóth* | Staff Software Engineer
>> stoty@cloudera.com <https://www.cloudera.com>
>> [image: Cloudera] <https://www.cloudera.com/>
>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>> <https://www.cloudera.com/>
>> ------------------------------
>>
>
>
> --
> *István Tóth* | Staff Software Engineer
> stoty@cloudera.com <https://www.cloudera.com>
> [image: Cloudera] <https://www.cloudera.com/>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> <https://www.cloudera.com/>
> ------------------------------
>
Re: [Discuss] Releasing Phoenix 4.16
Posted by Istvan Toth <st...@cloudera.com.INVALID>.
As I started to work on this I realized that while providing binary
tarballs for each HBase profile is fine,
this does not solve the maven artifact issue.
Are we OK with publishing a single phoenix-client maven artifact (for the
oldest HBase),
or do we want to publish a separate one for each HBase version ?
I looked at publishing multiple client versions, but none of them are
particularly easy or attractive.
The best I could come up with is adding a separate maven module for (each
version x embedded) (i.e 6 for 4.x, 8 for master),
and activating them according to hbase.profile.
This would also mean that we need to add the hbase version to the artifact
id. i.e.: phoenix-client-hbase-2.1
Once we publish separate binaries for the HBase profiles, we can undo the
change that excludes compat-module from phoenix-server,
and shade it in again for the binary assemblies.
In this case we'd also have to do something about the phoenix-server maven
artifacts. Either they get the same treatment as phoenix-client,
or we simply skip publishing them. I personally do not see anyone getting
phoenix-server.jar from maven.
The easiest version is
* publish the oldest profile phoenix-client to maven
* do not publish phoenix-server to maven
* add compat-module to phoenix-server for the binary artifact
regards
Istvan
On Fri, Jan 8, 2021 at 6:56 AM Istvan Toth <st...@cloudera.com> wrote:
> On the release binaries:
>
> The current solution (which the default profile change has broken) was
> based on Lars's idea at
>
> https://issues.apache.org/jira/browse/PHOENIX-5902?focusedCommentId=17125122&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17125122
> However, I agree that providing separate assemblies for each HBase profile
> is better for our users, as they won't have to rebuild Phoenix to take
> advantage of any new features, and to get the general improvements in
> later HBase releases.
> I have opened https://issues.apache.org/jira/browse/PHOENIX-6307 to track
> this.
>
> On the 5.1 release:
>
> Yes, I do want to release 5.1 shortly. In fact $dayjob desperately needs
> it ASAP.
> I have a few older PRs waiting for review that fell between the cracks,
> but as soon as those are merged, I want to cut the first RC.
> It would be nice to have 4.16 and 5.1 as close as possible, and
> PHOENIX-6211 seems to be ready, so I hope to include it in 5.1 too.
> I will start an official 5.1 release thread and volunteer to be the
> release manager soon. (unless you want to take that up too, Xinyi).
>
>
>
> On Fri, Jan 8, 2021 at 1:08 AM Xinyi Yan <ya...@apache.org> wrote:
>
>> If we can modify the dev/create-release scripts and make them work for the
>> 4.16 release with this hbase.profile option, it would make our life much
>> easier to release multiple HBase profiles from the single branch in the
>> future too(the master branch will have a release shortly right?). Geoffrey
>> and Istvan, what do you think?
>>
>> Thanks,
>> Xinyi
>>
>> On Thu, Jan 7, 2021 at 11:28 AM Geoffrey Jacoby <gj...@apache.org>
>> wrote:
>>
>> > Thanks for bringing up the default branch issue, Istvan, I've been
>> meaning
>> > to start a conversation about it on this list.
>> >
>> > As part of PHOENIX-5435, I changed the default 4.x HBase release to 1.5
>> and
>> > the default 5.x HBase to 2.3 because the WAL annotation feature
>> introduced
>> > by 5435 only works with HBase 1.5+ or 2.3+. (It depends on a coproc hook
>> > introduced in HBASE-22623). That means that all tests of that feature
>> must
>> > no-op when run against an earlier HBase, which means that it would
>> never be
>> > tested in our CI pipelines if the default was 1.3 or 2.1.
>> >
>> > This has come up quite a few times recently. In particular, the new
>> > indexing framework runs in a degraded state against HBase 2.1 and 2.2
>> > (still better than the old indexing framework though!), because we lack
>> > support in 2.1 for Filters on raw scans and can't use the "max lookback
>> > age" feature with HBase 2.1 or 2.2, which keeps major compactions from
>> > messing with index rebuilds. That's why lots of indexing tests no-op or
>> > have to make different assertions when run against 2.1 or 2.2, so we
>> only
>> > properly test the indexing framework if CI and local developer
>> > tests run against HBase 2.3 or up.
>> >
>> > As for the release, don't we usually release separate binaries that are
>> > suitable for all the HBase versions we support? Back before we had a
>> > unified 4.x branch, I believe we used to release from 3 different
>> > 4.x-HBase-* branches each time. I've been assuming that part of the
>> release
>> > process would be generating binaries for 4.x-HBase-1.3, 4.x-HBase-1.4,
>> > 4.x-HBase-1.5, and 4.x-HBase-1.6 (if that was different from 1.5).
>> >
>> > Geoffrey
>> >
>> > On Thu, Jan 7, 2021 at 11:19 AM Istvan Toth <st...@apache.org> wrote:
>> >
>> > > Two more issues came to my mind:
>> > >
>> > > In PHOENIX-5435 the default profile was changed to HBase 1.5.0
>> > > This causes two conflicts:
>> > > - Our documentation says that the default profile is the lowest
>> > supported,
>> > > which is no longer true
>> > > - Unless we change our binary release process, the published maven
>> > > artifacts and binaries will also be built with HBase 1.5, thus they
>> will
>> > be
>> > > incompatible with Hbase 1.3 and 1.4
>> > >
>> > > This should be addressed before release.
>> > > One possible solution:
>> > > * Add an "oldest" hbase.profile option which always chooses the oldest
>> > > supported version
>> > > * Update the release scripts to specify this profile
>> > > * Update the docs.
>> > >
>> > > I have also adopted the hbase release scripts to Phoenix, they are
>> > present
>> > > in dev/create-release in the master branch, but
>> > > should be able to perform 4.16 release as well. I have used this for
>> the
>> > > phoenix-thirdparty, omid, and tephra releases, but no live phoenix
>> > releases
>> > > have been done with them yet, obviously.
>> > >
>> > > regards
>> > > Istvan
>> > >
>> > > On Thu, Jan 7, 2021 at 8:19 AM Istvan Toth <st...@apache.org> wrote:
>> > >
>> > > > Hi!
>> > > >
>> > > > A question of process: Should we backport fixes to the 4.16 branch
>> at
>> > our
>> > > > discretion, or is backporting those handled by the release manager
>> > > (Xinyi) ?
>> > > >
>> > > > On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth <st...@cloudera.com>
>> wrote:
>> > > >
>> > > >> Hi!
>> > > >>
>> > > >> Thanks for everyone's effort on fixing the flakey tests.
>> > > >> However, our ASF Jenkins runs are still very unstable, and I am
>> afraid
>> > > >> that the single clean 4.x run was down more to luck than to fixing
>> the
>> > > >> underlying problem.
>> > > >> While I do not consider this a blocker, fixing this would make the
>> > pre-
>> > > >> and post commit tests far more useful, and let us start with a
>> clean
>> > > slate
>> > > >> for the next release.
>> > > >> I have created https://issues.apache.org/jira/browse/PHOENIX-6288
>> to
>> > > >> track the generic cluster setup issue, but my attempts to fix
>> this, or
>> > > at
>> > > >> least figure out the root cause have been unsuccessful so far.
>> > > >>
>> > > >> I ask for your help in fixing this issue. I have added some
>> > inconclusive
>> > > >> analysis to the ticket, and the full failsafe output directory is
>> > > >> downloadable as artifacts from the failed multibranch runs (stored
>> > for a
>> > > >> few days), which should hold the answer to this riddle.
>> > > >>
>> > > >> regards,
>> > > >> Istvan
>> > > >>
>> > > >> On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan <ya...@apache.org>
>> wrote:
>> > > >>
>> > > >>> Hi,
>> > > >>>
>> > > >>> We finally have a stable 4.x branch build after many flapper test
>> > > fixing
>> > > >>> contributions from the community. I'm going to fork a new
>> > branch(4.16)
>> > > >>> from
>> > > >>> the current 4.x branch for the 4.16 release. As a cutoff, I will
>> not
>> > > >>> include any new features other than PHOENIX-6211 and bug fix.
>> Please
>> > > let
>> > > >>> me
>> > > >>> know if you have any questions or concerns.
>> > > >>>
>> > > >>> Thanks,
>> > > >>> Xinyi
>> > > >>>
>> > > >>> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani <vj...@apache.org>
>> > > wrote:
>> > > >>>
>> > > >>> > An update on test stability:
>> > > >>> >
>> > > >>> > As per recent 4.x build results, we are left with very few
>> flappers
>> > > and
>> > > >>> > specifically
>> > > >>> > with HBase profile 1.6, I can see recent 7 build (multibranch +
>> PR
>> > > >>> > precommit results)
>> > > >>> > results without any test failure.
>> > > >>> >
>> > > >>> >
>> > > >>> > On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <ya...@apache.org>
>> > > wrote:
>> > > >>> >
>> > > >>> > > Yes, we are currently working on fixing the 4.x branch test
>> > > flappers
>> > > >>> > while
>> > > >>> > > waiting for the PHOENIX-5435. After that, I will try to have
>> an
>> > RC
>> > > >>> ASAP.
>> > > >>> > >
>> > > >>> > > On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <
>> > > >>> ankitsinghal59@gmail.com>
>> > > >>> > > wrote:
>> > > >>> > >
>> > > >>> > > > I see that both the blockers listed here PHOENIX-5712 and
>> > > >>> PHOENIX-6241
>> > > >>> > > > have been resolved(Thanks to Xinyi), and also as per the
>> JIRA
>> > > query
>> > > >>> > there
>> > > >>> > > > is no Jira marked as a blocker for 4.16 except the one
>> related
>> > to
>> > > >>> > > > documentation
>> > > >>> > > > for "splittable catalog table".
>> > > >>> > > >
>> > > >>> > > > Xinyi, so are we good to start the release process now?
>> > > >>> > > >
>> > > >>> > > > On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <
>> yanxinyi@apache.org>
>> > > >>> wrote:
>> > > >>> > > >
>> > > >>> > > > > Thanks for replying and providing suggestions. I looked at
>> > the
>> > > >>> wrong
>> > > >>> > > > result
>> > > >>> > > > > Jira list that Daniel provided and did some local testing,
>> > and
>> > > >>> here
>> > > >>> > is
>> > > >>> > > > the
>> > > >>> > > > > result:
>> > > >>> > > > > [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642:
>> > cannot
>> > > >>> > > reproduce
>> > > >>> > > > > it.
>> > > >>> > > > > [More information is required] PHOENIX-4504 cannot
>> reproduce
>> > it
>> > > >>> but
>> > > >>> > > > someone
>> > > >>> > > > > claimed that he had a similar issue.
>> > > >>> > > > > [unusual query] PHOENIX-4540 and PHOENIX-6217
>> > > >>> > > > >
>> > > >>> > > > > Based on my finding, I think it's better to have more
>> > frequent
>> > > >>> > > > housekeeping
>> > > >>> > > > > and resolve unreproducible bugs especially since many of
>> them
>> > > are
>> > > >>> > > > > considering out of date (phoenix-4.11 or even
>> phoenix-4.6).
>> > > >>> Since I
>> > > >>> > > still
>> > > >>> > > > > need time to work on the blocker Jira(PHOENIX-5712,
>> > > >>> PHOENIX-6241) and
>> > > >>> > > fix
>> > > >>> > > > > test flappers, if you want to fix "unusual query" bugs,
>> feel
>> > > >>> free to
>> > > >>> > do
>> > > >>> > > > so.
>> > > >>> > > > >
>> > > >>> > > > >
>> > > >>> > > > > Sincerely,
>> > > >>> > > > > Xinyi
>> > > >>> > > > >
>> > > >>> > > > > On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <
>> > > ankit@apache.org>
>> > > >>> > > wrote:
>> > > >>> > > > >
>> > > >>> > > > > > Thanks Daniel and appreciate the effort you put in
>> getting
>> > > the
>> > > >>> list
>> > > >>> > > > ready
>> > > >>> > > > > > for bugs producing wrong results
>> > > >>> > > > > > but none of them seems to be a blocker to me for 4.16 as
>> > they
>> > > >>> are
>> > > >>> > not
>> > > >>> > > > the
>> > > >>> > > > > > regression and doesn't break the general functionality
>> > > >>> > > > > > except for specific features, RVC/desc as Chenglei also
>> > > >>> pointed out
>> > > >>> > > > > (though
>> > > >>> > > > > > I'll defer the assessment to RM "Xinyi").
>> > > >>> > > > > > Probably these can be a part of 4.16.1 or we can do
>> 4.17.0
>> > > soon
>> > > >>> > maybe
>> > > >>> > > > > after
>> > > >>> > > > > > a few weeks/month?
>> > > >>> > > > > >
>> > > >>> > > > > > Considering that we have already fixed 137 bugs and done
>> > 85+
>> > > >>> > > > > > improvements/features in 4.16,
>> > > >>> > > > > > it will not be a good idea to deprive the user from such
>> > > fixes.
>> > > >>> > > > > > It's been a year since our last 4.15 release, having no
>> > > release
>> > > >>> > > brings
>> > > >>> > > > > more
>> > > >>> > > > > > questions on the project
>> > > >>> > > > > > rather than the bugs which affect a certain % of
>> > > feature/users,
>> > > >>> > would
>> > > >>> > > > the
>> > > >>> > > > > > release notes
>> > > >>> > > > > > explaining the stability of certain features set the
>> right
>> > > >>> > > expectation
>> > > >>> > > > > for
>> > > >>> > > > > > those users who rely on these features to wait for a
>> future
>> > > >>> > release?
>> > > >>> > > > > >
>> > > >>> > > > > > Regards,
>> > > >>> > > > > > Ankit Singhal
>> > > >>> > > > > >
>> > > >>> > > > > > On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
>> > > >>> > > > chenglei@apache.org>
>> > > >>> > > > > > wrote:
>> > > >>> > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > > In my opinion, we should keep releases light and
>> > frequent,
>> > > >>> and
>> > > >>> > for
>> > > >>> > > > > some
>> > > >>> > > > > > > unusual query bugs like RVC and DESC
>> > > >>> > > > > > > we could delay fix to next release . I think we should
>> > > >>> release
>> > > >>> > > 4.16.0
>> > > >>> > > > > and
>> > > >>> > > > > > > 5.1.0 as quickly as possible. In China, many users
>> > > >>> > > > > > > in HBase&Phoenix User Group thought that Phoenix was
>> > dead
>> > > >>> > because
>> > > >>> > > > our
>> > > >>> > > > > > too
>> > > >>> > > > > > > long interval release and stopped using
>> > > >>> > > > > > > Phoenix.
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > > At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
>> > > >>> > > > chinmayskulkarni@gmail.com
>> > > >>> > > > > >
>> > > >>> > > > > > > wrote:
>> > > >>> > > > > > > >I agree. These are all major bugs and we should aim
>> at
>> > > >>> solving
>> > > >>> > > them
>> > > >>> > > > > > after
>> > > >>> > > > > > > >checking that they are still issues. I am +1 on 5833
>> > and I
>> > > >>> think
>> > > >>> > > > 5484
>> > > >>> > > > > > > would
>> > > >>> > > > > > > >be a great addition to 4.16 as well. We should aim at
>> > > >>> resolving
>> > > >>> > > high
>> > > >>> > > > > > > >priority bugs like this in every release.
>> > > >>> > > > > > > >
>> > > >>> > > > > > > >Sometimes we let these bugs slip without a resolution
>> > > >>> before a
>> > > >>> > > > > release,
>> > > >>> > > > > > > >citing that these are "known issues" or "not
>> regressions
>> > > >>> from
>> > > >>> > the
>> > > >>> > > > last
>> > > >>> > > > > > > >release". In some cases this may be fine since we
>> want
>> > to
>> > > >>> keep
>> > > >>> > > > > releases
>> > > >>> > > > > > > >light and frequent, but perhaps we can track such
>> issues
>> > > >>> and aim
>> > > >>> > > at
>> > > >>> > > > > > > >reducing the number of bugs by x% in each release?
>> This
>> > > will
>> > > >>> > also
>> > > >>> > > > keep
>> > > >>> > > > > > old
>> > > >>> > > > > > > >Jiras alive since we will potentially periodically
>> > review
>> > > >>> them.
>> > > >>> > > > > > > >
>> > > >>> > > > > > > >
>> > > >>> > > > > > > >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <
>> > > >>> > > gjacoby@apache.org>
>> > > >>> > > > > > > wrote:
>> > > >>> > > > > > > >
>> > > >>> > > > > > > >> I've got PHOENIX-5435 in review right now, and
>> would
>> > > like
>> > > >>> to
>> > > >>> > get
>> > > >>> > > > it
>> > > >>> > > > > in
>> > > >>> > > > > > > 4.16
>> > > >>> > > > > > > >> / 5.1.
>> > > >>> > > > > > > >>
>> > > >>> > > > > > > >> It's allowing the annotation of Phoenix metadata
>> into
>> > > >>> HBase
>> > > >>> > WALs
>> > > >>> > > > as
>> > > >>> > > > > a
>> > > >>> > > > > > > >> pre-req for the Phoenix Change Detection Capture
>> > > framework
>> > > >>> > > > > > > (PHOENIX-5442).
>> > > >>> > > > > > > >> Since it has both client/server logic, and adds a
>> > field
>> > > to
>> > > >>> > > > > > > System.Catalog,
>> > > >>> > > > > > > >> it can't go in a patch release.
>> > > >>> > > > > > > >>
>> > > >>> > > > > > > >> Depending on timing, I'd _like_ to get
>> PHOENIX-6227,
>> > > >>> which is
>> > > >>> > > the
>> > > >>> > > > > last
>> > > >>> > > > > > > part
>> > > >>> > > > > > > >> of CDC that will go into core Phoenix, into 4.16,
>> but
>> > > >>> since
>> > > >>> > that
>> > > >>> > > > > _can_
>> > > >>> > > > > > > go
>> > > >>> > > > > > > >> in a patch release and I haven't started it yet, if
>> > the
>> > > >>> > release
>> > > >>> > > > gets
>> > > >>> > > > > > cut
>> > > >>> > > > > > > >> before it's ready, no big deal. (The rest of CDC
>> will
>> > go
>> > > >>> into
>> > > >>> > > > > > > >> phoenix-connectors for a future release of that
>> > > project.)
>> > > >>> > > > > > > >>
>> > > >>> > > > > > > >> As for the correctness problems that Daniel points
>> > out,
>> > > I
>> > > >>> > think
>> > > >>> > > we
>> > > >>> > > > > > > should
>> > > >>> > > > > > > >> fix the ones that were detected with a recent
>> version
>> > > >>> (4.14 or
>> > > >>> > > > > 4.15?),
>> > > >>> > > > > > > and
>> > > >>> > > > > > > >> test to see which of the older ones can still be
>> > > >>> reproduced.
>> > > >>> > > Once
>> > > >>> > > > we
>> > > >>> > > > > > > know
>> > > >>> > > > > > > >> which bugs are real and which are just historical,
>> we
>> > > can
>> > > >>> > better
>> > > >>> > > > > judge
>> > > >>> > > > > > > >> scope. And hopefully close a bunch of obsolete
>> bugs.
>> > > >>> (Thanks,
>> > > >>> > > > > Daniel,
>> > > >>> > > > > > > for
>> > > >>> > > > > > > >> collecting that list!)
>> > > >>> > > > > > > >>
>> > > >>> > > > > > > >> Geoffrey
>> > > >>> > > > > > > >>
>> > > >>> > > > > > > >>
>> > > >>> > > > > > > >>
>> > > >>> > > > > > > >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
>> > > >>> > > > > > > >> <da...@salesforce.com.invalid> wrote:
>> > > >>> > > > > > > >>
>> > > >>> > > > > > > >> > Hi, I wanted to bring up wrong results in Phoenix
>> > and
>> > > >>> some
>> > > >>> > > JIRAs
>> > > >>> > > > > > > around
>> > > >>> > > > > > > >> > them that I think we should fix as the wrong
>> result
>> > > >>> lessens
>> > > >>> > > the
>> > > >>> > > > > end
>> > > >>> > > > > > > >> user's
>> > > >>> > > > > > > >> > trust in Phoenix. Releasing a new version
>> without
>> > > >>> > addressing
>> > > >>> > > > > these
>> > > >>> > > > > > > in a
>> > > >>> > > > > > > >> > minor release hurts our visibility in that these
>> > > >>> critical
>> > > >>> > > issues
>> > > >>> > > > > are
>> > > >>> > > > > > > not
>> > > >>> > > > > > > >> > addressed.
>> > > >>> > > > > > > >> >
>> > > >>> > > > > > > >> > Jira's that I'm involved with for example: I've
>> > > already
>> > > >>> > given
>> > > >>> > > a
>> > > >>> > > > > > patch
>> > > >>> > > > > > > >> > several months ago for 5833 and there is a
>> chance it
>> > > >>> may fix
>> > > >>> > > > 5484.
>> > > >>> > > > > > > >> >
>> > https://issues.apache.org/jira/browse/PHOENIX-5833
>> > > >>> > > > > > > >> >
>> > https://issues.apache.org/jira/browse/PHOENIX-5484
>> > > >>> > > > > > > >> >
>> > > >>> > > > > > > >> > In addition, inspecting apache JIRA i see several
>> > > other
>> > > >>> > wrong
>> > > >>> > > > > result
>> > > >>> > > > > > > >> JIRAs
>> > > >>> > > > > > > >> > from the community. Some of these certainly are
>> > > >>> probably
>> > > >>> > old
>> > > >>> > > > > issues
>> > > >>> > > > > > > or
>> > > >>> > > > > > > >> > incorrect understanding but some of these are
>> opened
>> > > by
>> > > >>> our
>> > > >>> > > own
>> > > >>> > > > > dev
>> > > >>> > > > > > > >> > community and are likely real problems.
>> > > >>> > > > > > > >> >
>> > https://issues.apache.org/jira/browse/PHOENIX-6217
>> > > >>> > > > > > > >> >
>> > https://issues.apache.org/jira/browse/PHOENIX-5571
>> > > >>> > > > > > > >> >
>> > https://issues.apache.org/jira/browse/PHOENIX-4642
>> > > >>> > > > > > > >> >
>> > https://issues.apache.org/jira/browse/PHOENIX-4540
>> > > >>> > > > > > > >> >
>> > https://issues.apache.org/jira/browse/PHOENIX-4504
>> > > >>> > > > > > > >> >
>> > https://issues.apache.org/jira/browse/PHOENIX-4419
>> > > >>> > > > > > > >> >
>> > https://issues.apache.org/jira/browse/PHOENIX-4116
>> > > >>> > > > > > > >> >
>> > > >>> > > > > > > >> > What is our stance on this type of issue? Are we
>> > > going
>> > > >>> to
>> > > >>> > say
>> > > >>> > > > > these
>> > > >>> > > > > > > were
>> > > >>> > > > > > > >> > issues prior to 4.15 and not address them?
>> Should
>> > we
>> > > >>> have
>> > > >>> > > > > > > requirements
>> > > >>> > > > > > > >> for
>> > > >>> > > > > > > >> > our releases to fix wrong results?
>> > > >>> > > > > > > >> >
>> > > >>> > > > > > > >> > Daniel Wong
>> > > >>> > > > > > > >> >
>> > > >>> > > > > > > >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <
>> > > >>> > > yanxinyi@apache.org>
>> > > >>> > > > > > > wrote:
>> > > >>> > > > > > > >> >
>> > > >>> > > > > > > >> > > Hi all,
>> > > >>> > > > > > > >> > >
>> > > >>> > > > > > > >> > > It's time to discuss the Phoenix 4.16 release.
>> > After
>> > > >>> many
>> > > >>> > > > > people's
>> > > >>> > > > > > > >> > > contributions on the bug fixes, new features,
>> and
>> > > >>> other
>> > > >>> > > works
>> > > >>> > > > in
>> > > >>> > > > > > the
>> > > >>> > > > > > > >> past
>> > > >>> > > > > > > >> > > few months, we are kind of close to the point
>> to
>> > > have
>> > > >>> a RC
>> > > >>> > > > > (still
>> > > >>> > > > > > > need
>> > > >>> > > > > > > >> to
>> > > >>> > > > > > > >> > > fix test flappers). Please let me know if you
>> > think
>> > > >>> any
>> > > >>> > JIRA
>> > > >>> > > > > must
>> > > >>> > > > > > be
>> > > >>> > > > > > > >> part
>> > > >>> > > > > > > >> > > of the Phoenix 4.16 release other than major
>> > blocker
>> > > >>> > > > > PHOENIX-5712.
>> > > >>> > > > > > > >> > >
>> > > >>> > > > > > > >> > > If no surprise comes up, I will not wait for
>> any
>> > new
>> > > >>> major
>> > > >>> > > > > > features
>> > > >>> > > > > > > and
>> > > >>> > > > > > > >> > > focus on the RC as soon as possible.
>> > > >>> > > > > > > >> > >
>> > > >>> > > > > > > >> > > Sincerely,
>> > > >>> > > > > > > >> > > Xinyi
>> > > >>> > > > > > > >> > >
>> > > >>> > > > > > > >> >
>> > > >>> > > > > > > >> >
>> > > >>> > > > > > > >> > --
>> > > >>> > > > > > > >> > Daniel Wong
>> > > >>> > > > > > > >> > Salesforce
>> > > >>> > > > > > > >> > Mobile: 628.217.1808
>> > > >>> > > > > > > >> >
>> > > >>> > > > > > > >>
>> > > >>> > > > > > > >
>> > > >>> > > > > > > >
>> > > >>> > > > > > > >--
>> > > >>> > > > > > > >Chinmay Kulkarni
>> > > >>> > > > > > >
>> > > >>> > > > > >
>> > > >>> > > > >
>> > > >>> > > >
>> > > >>> > >
>> > > >>> >
>> > > >>>
>> > > >>
>> > > >>
>> > > >> --
>> > > >> *István Tóth* | Staff Software Engineer
>> > > >> stoty@cloudera.com <https://www.cloudera.com>
>> > > >> [image: Cloudera] <https://www.cloudera.com/>
>> > > >> [image: Cloudera on Twitter] <https://twitter.com/cloudera>
>> [image:
>> > > >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>> > > >> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>> > > >> <https://www.cloudera.com/>
>> > > >> ------------------------------
>> > > >>
>> > > >
>> > >
>> >
>>
>
>
> --
> *István Tóth* | Staff Software Engineer
> stoty@cloudera.com <https://www.cloudera.com>
> [image: Cloudera] <https://www.cloudera.com/>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> <https://www.cloudera.com/>
> ------------------------------
>
--
*István Tóth* | Staff Software Engineer
stoty@cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
<https://www.cloudera.com/>
------------------------------
Re: [Discuss] Releasing Phoenix 4.16
Posted by Istvan Toth <st...@cloudera.com.INVALID>.
On the release binaries:
The current solution (which the default profile change has broken) was
based on Lars's idea at
https://issues.apache.org/jira/browse/PHOENIX-5902?focusedCommentId=17125122&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17125122
However, I agree that providing separate assemblies for each HBase profile
is better for our users, as they won't have to rebuild Phoenix to take
advantage of any new features, and to get the general improvements in
later HBase releases.
I have opened https://issues.apache.org/jira/browse/PHOENIX-6307 to track
this.
On the 5.1 release:
Yes, I do want to release 5.1 shortly. In fact $dayjob desperately needs it
ASAP.
I have a few older PRs waiting for review that fell between the cracks, but
as soon as those are merged, I want to cut the first RC.
It would be nice to have 4.16 and 5.1 as close as possible, and
PHOENIX-6211 seems to be ready, so I hope to include it in 5.1 too.
I will start an official 5.1 release thread and volunteer to be the release
manager soon. (unless you want to take that up too, Xinyi).
On Fri, Jan 8, 2021 at 1:08 AM Xinyi Yan <ya...@apache.org> wrote:
> If we can modify the dev/create-release scripts and make them work for the
> 4.16 release with this hbase.profile option, it would make our life much
> easier to release multiple HBase profiles from the single branch in the
> future too(the master branch will have a release shortly right?). Geoffrey
> and Istvan, what do you think?
>
> Thanks,
> Xinyi
>
> On Thu, Jan 7, 2021 at 11:28 AM Geoffrey Jacoby <gj...@apache.org>
> wrote:
>
> > Thanks for bringing up the default branch issue, Istvan, I've been
> meaning
> > to start a conversation about it on this list.
> >
> > As part of PHOENIX-5435, I changed the default 4.x HBase release to 1.5
> and
> > the default 5.x HBase to 2.3 because the WAL annotation feature
> introduced
> > by 5435 only works with HBase 1.5+ or 2.3+. (It depends on a coproc hook
> > introduced in HBASE-22623). That means that all tests of that feature
> must
> > no-op when run against an earlier HBase, which means that it would never
> be
> > tested in our CI pipelines if the default was 1.3 or 2.1.
> >
> > This has come up quite a few times recently. In particular, the new
> > indexing framework runs in a degraded state against HBase 2.1 and 2.2
> > (still better than the old indexing framework though!), because we lack
> > support in 2.1 for Filters on raw scans and can't use the "max lookback
> > age" feature with HBase 2.1 or 2.2, which keeps major compactions from
> > messing with index rebuilds. That's why lots of indexing tests no-op or
> > have to make different assertions when run against 2.1 or 2.2, so we only
> > properly test the indexing framework if CI and local developer
> > tests run against HBase 2.3 or up.
> >
> > As for the release, don't we usually release separate binaries that are
> > suitable for all the HBase versions we support? Back before we had a
> > unified 4.x branch, I believe we used to release from 3 different
> > 4.x-HBase-* branches each time. I've been assuming that part of the
> release
> > process would be generating binaries for 4.x-HBase-1.3, 4.x-HBase-1.4,
> > 4.x-HBase-1.5, and 4.x-HBase-1.6 (if that was different from 1.5).
> >
> > Geoffrey
> >
> > On Thu, Jan 7, 2021 at 11:19 AM Istvan Toth <st...@apache.org> wrote:
> >
> > > Two more issues came to my mind:
> > >
> > > In PHOENIX-5435 the default profile was changed to HBase 1.5.0
> > > This causes two conflicts:
> > > - Our documentation says that the default profile is the lowest
> > supported,
> > > which is no longer true
> > > - Unless we change our binary release process, the published maven
> > > artifacts and binaries will also be built with HBase 1.5, thus they
> will
> > be
> > > incompatible with Hbase 1.3 and 1.4
> > >
> > > This should be addressed before release.
> > > One possible solution:
> > > * Add an "oldest" hbase.profile option which always chooses the oldest
> > > supported version
> > > * Update the release scripts to specify this profile
> > > * Update the docs.
> > >
> > > I have also adopted the hbase release scripts to Phoenix, they are
> > present
> > > in dev/create-release in the master branch, but
> > > should be able to perform 4.16 release as well. I have used this for
> the
> > > phoenix-thirdparty, omid, and tephra releases, but no live phoenix
> > releases
> > > have been done with them yet, obviously.
> > >
> > > regards
> > > Istvan
> > >
> > > On Thu, Jan 7, 2021 at 8:19 AM Istvan Toth <st...@apache.org> wrote:
> > >
> > > > Hi!
> > > >
> > > > A question of process: Should we backport fixes to the 4.16 branch at
> > our
> > > > discretion, or is backporting those handled by the release manager
> > > (Xinyi) ?
> > > >
> > > > On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth <st...@cloudera.com>
> wrote:
> > > >
> > > >> Hi!
> > > >>
> > > >> Thanks for everyone's effort on fixing the flakey tests.
> > > >> However, our ASF Jenkins runs are still very unstable, and I am
> afraid
> > > >> that the single clean 4.x run was down more to luck than to fixing
> the
> > > >> underlying problem.
> > > >> While I do not consider this a blocker, fixing this would make the
> > pre-
> > > >> and post commit tests far more useful, and let us start with a clean
> > > slate
> > > >> for the next release.
> > > >> I have created https://issues.apache.org/jira/browse/PHOENIX-6288
> to
> > > >> track the generic cluster setup issue, but my attempts to fix this,
> or
> > > at
> > > >> least figure out the root cause have been unsuccessful so far.
> > > >>
> > > >> I ask for your help in fixing this issue. I have added some
> > inconclusive
> > > >> analysis to the ticket, and the full failsafe output directory is
> > > >> downloadable as artifacts from the failed multibranch runs (stored
> > for a
> > > >> few days), which should hold the answer to this riddle.
> > > >>
> > > >> regards,
> > > >> Istvan
> > > >>
> > > >> On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan <ya...@apache.org>
> wrote:
> > > >>
> > > >>> Hi,
> > > >>>
> > > >>> We finally have a stable 4.x branch build after many flapper test
> > > fixing
> > > >>> contributions from the community. I'm going to fork a new
> > branch(4.16)
> > > >>> from
> > > >>> the current 4.x branch for the 4.16 release. As a cutoff, I will
> not
> > > >>> include any new features other than PHOENIX-6211 and bug fix.
> Please
> > > let
> > > >>> me
> > > >>> know if you have any questions or concerns.
> > > >>>
> > > >>> Thanks,
> > > >>> Xinyi
> > > >>>
> > > >>> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani <vj...@apache.org>
> > > wrote:
> > > >>>
> > > >>> > An update on test stability:
> > > >>> >
> > > >>> > As per recent 4.x build results, we are left with very few
> flappers
> > > and
> > > >>> > specifically
> > > >>> > with HBase profile 1.6, I can see recent 7 build (multibranch +
> PR
> > > >>> > precommit results)
> > > >>> > results without any test failure.
> > > >>> >
> > > >>> >
> > > >>> > On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <ya...@apache.org>
> > > wrote:
> > > >>> >
> > > >>> > > Yes, we are currently working on fixing the 4.x branch test
> > > flappers
> > > >>> > while
> > > >>> > > waiting for the PHOENIX-5435. After that, I will try to have an
> > RC
> > > >>> ASAP.
> > > >>> > >
> > > >>> > > On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <
> > > >>> ankitsinghal59@gmail.com>
> > > >>> > > wrote:
> > > >>> > >
> > > >>> > > > I see that both the blockers listed here PHOENIX-5712 and
> > > >>> PHOENIX-6241
> > > >>> > > > have been resolved(Thanks to Xinyi), and also as per the JIRA
> > > query
> > > >>> > there
> > > >>> > > > is no Jira marked as a blocker for 4.16 except the one
> related
> > to
> > > >>> > > > documentation
> > > >>> > > > for "splittable catalog table".
> > > >>> > > >
> > > >>> > > > Xinyi, so are we good to start the release process now?
> > > >>> > > >
> > > >>> > > > On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <
> yanxinyi@apache.org>
> > > >>> wrote:
> > > >>> > > >
> > > >>> > > > > Thanks for replying and providing suggestions. I looked at
> > the
> > > >>> wrong
> > > >>> > > > result
> > > >>> > > > > Jira list that Daniel provided and did some local testing,
> > and
> > > >>> here
> > > >>> > is
> > > >>> > > > the
> > > >>> > > > > result:
> > > >>> > > > > [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642:
> > cannot
> > > >>> > > reproduce
> > > >>> > > > > it.
> > > >>> > > > > [More information is required] PHOENIX-4504 cannot
> reproduce
> > it
> > > >>> but
> > > >>> > > > someone
> > > >>> > > > > claimed that he had a similar issue.
> > > >>> > > > > [unusual query] PHOENIX-4540 and PHOENIX-6217
> > > >>> > > > >
> > > >>> > > > > Based on my finding, I think it's better to have more
> > frequent
> > > >>> > > > housekeeping
> > > >>> > > > > and resolve unreproducible bugs especially since many of
> them
> > > are
> > > >>> > > > > considering out of date (phoenix-4.11 or even phoenix-4.6).
> > > >>> Since I
> > > >>> > > still
> > > >>> > > > > need time to work on the blocker Jira(PHOENIX-5712,
> > > >>> PHOENIX-6241) and
> > > >>> > > fix
> > > >>> > > > > test flappers, if you want to fix "unusual query" bugs,
> feel
> > > >>> free to
> > > >>> > do
> > > >>> > > > so.
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > > Sincerely,
> > > >>> > > > > Xinyi
> > > >>> > > > >
> > > >>> > > > > On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <
> > > ankit@apache.org>
> > > >>> > > wrote:
> > > >>> > > > >
> > > >>> > > > > > Thanks Daniel and appreciate the effort you put in
> getting
> > > the
> > > >>> list
> > > >>> > > > ready
> > > >>> > > > > > for bugs producing wrong results
> > > >>> > > > > > but none of them seems to be a blocker to me for 4.16 as
> > they
> > > >>> are
> > > >>> > not
> > > >>> > > > the
> > > >>> > > > > > regression and doesn't break the general functionality
> > > >>> > > > > > except for specific features, RVC/desc as Chenglei also
> > > >>> pointed out
> > > >>> > > > > (though
> > > >>> > > > > > I'll defer the assessment to RM "Xinyi").
> > > >>> > > > > > Probably these can be a part of 4.16.1 or we can do
> 4.17.0
> > > soon
> > > >>> > maybe
> > > >>> > > > > after
> > > >>> > > > > > a few weeks/month?
> > > >>> > > > > >
> > > >>> > > > > > Considering that we have already fixed 137 bugs and done
> > 85+
> > > >>> > > > > > improvements/features in 4.16,
> > > >>> > > > > > it will not be a good idea to deprive the user from such
> > > fixes.
> > > >>> > > > > > It's been a year since our last 4.15 release, having no
> > > release
> > > >>> > > brings
> > > >>> > > > > more
> > > >>> > > > > > questions on the project
> > > >>> > > > > > rather than the bugs which affect a certain % of
> > > feature/users,
> > > >>> > would
> > > >>> > > > the
> > > >>> > > > > > release notes
> > > >>> > > > > > explaining the stability of certain features set the
> right
> > > >>> > > expectation
> > > >>> > > > > for
> > > >>> > > > > > those users who rely on these features to wait for a
> future
> > > >>> > release?
> > > >>> > > > > >
> > > >>> > > > > > Regards,
> > > >>> > > > > > Ankit Singhal
> > > >>> > > > > >
> > > >>> > > > > > On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
> > > >>> > > > chenglei@apache.org>
> > > >>> > > > > > wrote:
> > > >>> > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > > In my opinion, we should keep releases light and
> > frequent,
> > > >>> and
> > > >>> > for
> > > >>> > > > > some
> > > >>> > > > > > > unusual query bugs like RVC and DESC
> > > >>> > > > > > > we could delay fix to next release . I think we should
> > > >>> release
> > > >>> > > 4.16.0
> > > >>> > > > > and
> > > >>> > > > > > > 5.1.0 as quickly as possible. In China, many users
> > > >>> > > > > > > in HBase&Phoenix User Group thought that Phoenix was
> > dead
> > > >>> > because
> > > >>> > > > our
> > > >>> > > > > > too
> > > >>> > > > > > > long interval release and stopped using
> > > >>> > > > > > > Phoenix.
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > > At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
> > > >>> > > > chinmayskulkarni@gmail.com
> > > >>> > > > > >
> > > >>> > > > > > > wrote:
> > > >>> > > > > > > >I agree. These are all major bugs and we should aim at
> > > >>> solving
> > > >>> > > them
> > > >>> > > > > > after
> > > >>> > > > > > > >checking that they are still issues. I am +1 on 5833
> > and I
> > > >>> think
> > > >>> > > > 5484
> > > >>> > > > > > > would
> > > >>> > > > > > > >be a great addition to 4.16 as well. We should aim at
> > > >>> resolving
> > > >>> > > high
> > > >>> > > > > > > >priority bugs like this in every release.
> > > >>> > > > > > > >
> > > >>> > > > > > > >Sometimes we let these bugs slip without a resolution
> > > >>> before a
> > > >>> > > > > release,
> > > >>> > > > > > > >citing that these are "known issues" or "not
> regressions
> > > >>> from
> > > >>> > the
> > > >>> > > > last
> > > >>> > > > > > > >release". In some cases this may be fine since we want
> > to
> > > >>> keep
> > > >>> > > > > releases
> > > >>> > > > > > > >light and frequent, but perhaps we can track such
> issues
> > > >>> and aim
> > > >>> > > at
> > > >>> > > > > > > >reducing the number of bugs by x% in each release?
> This
> > > will
> > > >>> > also
> > > >>> > > > keep
> > > >>> > > > > > old
> > > >>> > > > > > > >Jiras alive since we will potentially periodically
> > review
> > > >>> them.
> > > >>> > > > > > > >
> > > >>> > > > > > > >
> > > >>> > > > > > > >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <
> > > >>> > > gjacoby@apache.org>
> > > >>> > > > > > > wrote:
> > > >>> > > > > > > >
> > > >>> > > > > > > >> I've got PHOENIX-5435 in review right now, and would
> > > like
> > > >>> to
> > > >>> > get
> > > >>> > > > it
> > > >>> > > > > in
> > > >>> > > > > > > 4.16
> > > >>> > > > > > > >> / 5.1.
> > > >>> > > > > > > >>
> > > >>> > > > > > > >> It's allowing the annotation of Phoenix metadata
> into
> > > >>> HBase
> > > >>> > WALs
> > > >>> > > > as
> > > >>> > > > > a
> > > >>> > > > > > > >> pre-req for the Phoenix Change Detection Capture
> > > framework
> > > >>> > > > > > > (PHOENIX-5442).
> > > >>> > > > > > > >> Since it has both client/server logic, and adds a
> > field
> > > to
> > > >>> > > > > > > System.Catalog,
> > > >>> > > > > > > >> it can't go in a patch release.
> > > >>> > > > > > > >>
> > > >>> > > > > > > >> Depending on timing, I'd _like_ to get PHOENIX-6227,
> > > >>> which is
> > > >>> > > the
> > > >>> > > > > last
> > > >>> > > > > > > part
> > > >>> > > > > > > >> of CDC that will go into core Phoenix, into 4.16,
> but
> > > >>> since
> > > >>> > that
> > > >>> > > > > _can_
> > > >>> > > > > > > go
> > > >>> > > > > > > >> in a patch release and I haven't started it yet, if
> > the
> > > >>> > release
> > > >>> > > > gets
> > > >>> > > > > > cut
> > > >>> > > > > > > >> before it's ready, no big deal. (The rest of CDC
> will
> > go
> > > >>> into
> > > >>> > > > > > > >> phoenix-connectors for a future release of that
> > > project.)
> > > >>> > > > > > > >>
> > > >>> > > > > > > >> As for the correctness problems that Daniel points
> > out,
> > > I
> > > >>> > think
> > > >>> > > we
> > > >>> > > > > > > should
> > > >>> > > > > > > >> fix the ones that were detected with a recent
> version
> > > >>> (4.14 or
> > > >>> > > > > 4.15?),
> > > >>> > > > > > > and
> > > >>> > > > > > > >> test to see which of the older ones can still be
> > > >>> reproduced.
> > > >>> > > Once
> > > >>> > > > we
> > > >>> > > > > > > know
> > > >>> > > > > > > >> which bugs are real and which are just historical,
> we
> > > can
> > > >>> > better
> > > >>> > > > > judge
> > > >>> > > > > > > >> scope. And hopefully close a bunch of obsolete bugs.
> > > >>> (Thanks,
> > > >>> > > > > Daniel,
> > > >>> > > > > > > for
> > > >>> > > > > > > >> collecting that list!)
> > > >>> > > > > > > >>
> > > >>> > > > > > > >> Geoffrey
> > > >>> > > > > > > >>
> > > >>> > > > > > > >>
> > > >>> > > > > > > >>
> > > >>> > > > > > > >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
> > > >>> > > > > > > >> <da...@salesforce.com.invalid> wrote:
> > > >>> > > > > > > >>
> > > >>> > > > > > > >> > Hi, I wanted to bring up wrong results in Phoenix
> > and
> > > >>> some
> > > >>> > > JIRAs
> > > >>> > > > > > > around
> > > >>> > > > > > > >> > them that I think we should fix as the wrong
> result
> > > >>> lessens
> > > >>> > > the
> > > >>> > > > > end
> > > >>> > > > > > > >> user's
> > > >>> > > > > > > >> > trust in Phoenix. Releasing a new version without
> > > >>> > addressing
> > > >>> > > > > these
> > > >>> > > > > > > in a
> > > >>> > > > > > > >> > minor release hurts our visibility in that these
> > > >>> critical
> > > >>> > > issues
> > > >>> > > > > are
> > > >>> > > > > > > not
> > > >>> > > > > > > >> > addressed.
> > > >>> > > > > > > >> >
> > > >>> > > > > > > >> > Jira's that I'm involved with for example: I've
> > > already
> > > >>> > given
> > > >>> > > a
> > > >>> > > > > > patch
> > > >>> > > > > > > >> > several months ago for 5833 and there is a chance
> it
> > > >>> may fix
> > > >>> > > > 5484.
> > > >>> > > > > > > >> >
> > https://issues.apache.org/jira/browse/PHOENIX-5833
> > > >>> > > > > > > >> >
> > https://issues.apache.org/jira/browse/PHOENIX-5484
> > > >>> > > > > > > >> >
> > > >>> > > > > > > >> > In addition, inspecting apache JIRA i see several
> > > other
> > > >>> > wrong
> > > >>> > > > > result
> > > >>> > > > > > > >> JIRAs
> > > >>> > > > > > > >> > from the community. Some of these certainly are
> > > >>> probably
> > > >>> > old
> > > >>> > > > > issues
> > > >>> > > > > > > or
> > > >>> > > > > > > >> > incorrect understanding but some of these are
> opened
> > > by
> > > >>> our
> > > >>> > > own
> > > >>> > > > > dev
> > > >>> > > > > > > >> > community and are likely real problems.
> > > >>> > > > > > > >> >
> > https://issues.apache.org/jira/browse/PHOENIX-6217
> > > >>> > > > > > > >> >
> > https://issues.apache.org/jira/browse/PHOENIX-5571
> > > >>> > > > > > > >> >
> > https://issues.apache.org/jira/browse/PHOENIX-4642
> > > >>> > > > > > > >> >
> > https://issues.apache.org/jira/browse/PHOENIX-4540
> > > >>> > > > > > > >> >
> > https://issues.apache.org/jira/browse/PHOENIX-4504
> > > >>> > > > > > > >> >
> > https://issues.apache.org/jira/browse/PHOENIX-4419
> > > >>> > > > > > > >> >
> > https://issues.apache.org/jira/browse/PHOENIX-4116
> > > >>> > > > > > > >> >
> > > >>> > > > > > > >> > What is our stance on this type of issue? Are we
> > > going
> > > >>> to
> > > >>> > say
> > > >>> > > > > these
> > > >>> > > > > > > were
> > > >>> > > > > > > >> > issues prior to 4.15 and not address them? Should
> > we
> > > >>> have
> > > >>> > > > > > > requirements
> > > >>> > > > > > > >> for
> > > >>> > > > > > > >> > our releases to fix wrong results?
> > > >>> > > > > > > >> >
> > > >>> > > > > > > >> > Daniel Wong
> > > >>> > > > > > > >> >
> > > >>> > > > > > > >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <
> > > >>> > > yanxinyi@apache.org>
> > > >>> > > > > > > wrote:
> > > >>> > > > > > > >> >
> > > >>> > > > > > > >> > > Hi all,
> > > >>> > > > > > > >> > >
> > > >>> > > > > > > >> > > It's time to discuss the Phoenix 4.16 release.
> > After
> > > >>> many
> > > >>> > > > > people's
> > > >>> > > > > > > >> > > contributions on the bug fixes, new features,
> and
> > > >>> other
> > > >>> > > works
> > > >>> > > > in
> > > >>> > > > > > the
> > > >>> > > > > > > >> past
> > > >>> > > > > > > >> > > few months, we are kind of close to the point to
> > > have
> > > >>> a RC
> > > >>> > > > > (still
> > > >>> > > > > > > need
> > > >>> > > > > > > >> to
> > > >>> > > > > > > >> > > fix test flappers). Please let me know if you
> > think
> > > >>> any
> > > >>> > JIRA
> > > >>> > > > > must
> > > >>> > > > > > be
> > > >>> > > > > > > >> part
> > > >>> > > > > > > >> > > of the Phoenix 4.16 release other than major
> > blocker
> > > >>> > > > > PHOENIX-5712.
> > > >>> > > > > > > >> > >
> > > >>> > > > > > > >> > > If no surprise comes up, I will not wait for any
> > new
> > > >>> major
> > > >>> > > > > > features
> > > >>> > > > > > > and
> > > >>> > > > > > > >> > > focus on the RC as soon as possible.
> > > >>> > > > > > > >> > >
> > > >>> > > > > > > >> > > Sincerely,
> > > >>> > > > > > > >> > > Xinyi
> > > >>> > > > > > > >> > >
> > > >>> > > > > > > >> >
> > > >>> > > > > > > >> >
> > > >>> > > > > > > >> > --
> > > >>> > > > > > > >> > Daniel Wong
> > > >>> > > > > > > >> > Salesforce
> > > >>> > > > > > > >> > Mobile: 628.217.1808
> > > >>> > > > > > > >> >
> > > >>> > > > > > > >>
> > > >>> > > > > > > >
> > > >>> > > > > > > >
> > > >>> > > > > > > >--
> > > >>> > > > > > > >Chinmay Kulkarni
> > > >>> > > > > > >
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >> --
> > > >> *István Tóth* | Staff Software Engineer
> > > >> stoty@cloudera.com <https://www.cloudera.com>
> > > >> [image: Cloudera] <https://www.cloudera.com/>
> > > >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > > >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > > >> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > >> <https://www.cloudera.com/>
> > > >> ------------------------------
> > > >>
> > > >
> > >
> >
>
--
*István Tóth* | Staff Software Engineer
stoty@cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
<https://www.cloudera.com/>
------------------------------
Re: [Discuss] Releasing Phoenix 4.16
Posted by Xinyi Yan <ya...@apache.org>.
If we can modify the dev/create-release scripts and make them work for the
4.16 release with this hbase.profile option, it would make our life much
easier to release multiple HBase profiles from the single branch in the
future too(the master branch will have a release shortly right?). Geoffrey
and Istvan, what do you think?
Thanks,
Xinyi
On Thu, Jan 7, 2021 at 11:28 AM Geoffrey Jacoby <gj...@apache.org> wrote:
> Thanks for bringing up the default branch issue, Istvan, I've been meaning
> to start a conversation about it on this list.
>
> As part of PHOENIX-5435, I changed the default 4.x HBase release to 1.5 and
> the default 5.x HBase to 2.3 because the WAL annotation feature introduced
> by 5435 only works with HBase 1.5+ or 2.3+. (It depends on a coproc hook
> introduced in HBASE-22623). That means that all tests of that feature must
> no-op when run against an earlier HBase, which means that it would never be
> tested in our CI pipelines if the default was 1.3 or 2.1.
>
> This has come up quite a few times recently. In particular, the new
> indexing framework runs in a degraded state against HBase 2.1 and 2.2
> (still better than the old indexing framework though!), because we lack
> support in 2.1 for Filters on raw scans and can't use the "max lookback
> age" feature with HBase 2.1 or 2.2, which keeps major compactions from
> messing with index rebuilds. That's why lots of indexing tests no-op or
> have to make different assertions when run against 2.1 or 2.2, so we only
> properly test the indexing framework if CI and local developer
> tests run against HBase 2.3 or up.
>
> As for the release, don't we usually release separate binaries that are
> suitable for all the HBase versions we support? Back before we had a
> unified 4.x branch, I believe we used to release from 3 different
> 4.x-HBase-* branches each time. I've been assuming that part of the release
> process would be generating binaries for 4.x-HBase-1.3, 4.x-HBase-1.4,
> 4.x-HBase-1.5, and 4.x-HBase-1.6 (if that was different from 1.5).
>
> Geoffrey
>
> On Thu, Jan 7, 2021 at 11:19 AM Istvan Toth <st...@apache.org> wrote:
>
> > Two more issues came to my mind:
> >
> > In PHOENIX-5435 the default profile was changed to HBase 1.5.0
> > This causes two conflicts:
> > - Our documentation says that the default profile is the lowest
> supported,
> > which is no longer true
> > - Unless we change our binary release process, the published maven
> > artifacts and binaries will also be built with HBase 1.5, thus they will
> be
> > incompatible with Hbase 1.3 and 1.4
> >
> > This should be addressed before release.
> > One possible solution:
> > * Add an "oldest" hbase.profile option which always chooses the oldest
> > supported version
> > * Update the release scripts to specify this profile
> > * Update the docs.
> >
> > I have also adopted the hbase release scripts to Phoenix, they are
> present
> > in dev/create-release in the master branch, but
> > should be able to perform 4.16 release as well. I have used this for the
> > phoenix-thirdparty, omid, and tephra releases, but no live phoenix
> releases
> > have been done with them yet, obviously.
> >
> > regards
> > Istvan
> >
> > On Thu, Jan 7, 2021 at 8:19 AM Istvan Toth <st...@apache.org> wrote:
> >
> > > Hi!
> > >
> > > A question of process: Should we backport fixes to the 4.16 branch at
> our
> > > discretion, or is backporting those handled by the release manager
> > (Xinyi) ?
> > >
> > > On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth <st...@cloudera.com> wrote:
> > >
> > >> Hi!
> > >>
> > >> Thanks for everyone's effort on fixing the flakey tests.
> > >> However, our ASF Jenkins runs are still very unstable, and I am afraid
> > >> that the single clean 4.x run was down more to luck than to fixing the
> > >> underlying problem.
> > >> While I do not consider this a blocker, fixing this would make the
> pre-
> > >> and post commit tests far more useful, and let us start with a clean
> > slate
> > >> for the next release.
> > >> I have created https://issues.apache.org/jira/browse/PHOENIX-6288 to
> > >> track the generic cluster setup issue, but my attempts to fix this, or
> > at
> > >> least figure out the root cause have been unsuccessful so far.
> > >>
> > >> I ask for your help in fixing this issue. I have added some
> inconclusive
> > >> analysis to the ticket, and the full failsafe output directory is
> > >> downloadable as artifacts from the failed multibranch runs (stored
> for a
> > >> few days), which should hold the answer to this riddle.
> > >>
> > >> regards,
> > >> Istvan
> > >>
> > >> On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan <ya...@apache.org> wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> We finally have a stable 4.x branch build after many flapper test
> > fixing
> > >>> contributions from the community. I'm going to fork a new
> branch(4.16)
> > >>> from
> > >>> the current 4.x branch for the 4.16 release. As a cutoff, I will not
> > >>> include any new features other than PHOENIX-6211 and bug fix. Please
> > let
> > >>> me
> > >>> know if you have any questions or concerns.
> > >>>
> > >>> Thanks,
> > >>> Xinyi
> > >>>
> > >>> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani <vj...@apache.org>
> > wrote:
> > >>>
> > >>> > An update on test stability:
> > >>> >
> > >>> > As per recent 4.x build results, we are left with very few flappers
> > and
> > >>> > specifically
> > >>> > with HBase profile 1.6, I can see recent 7 build (multibranch + PR
> > >>> > precommit results)
> > >>> > results without any test failure.
> > >>> >
> > >>> >
> > >>> > On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <ya...@apache.org>
> > wrote:
> > >>> >
> > >>> > > Yes, we are currently working on fixing the 4.x branch test
> > flappers
> > >>> > while
> > >>> > > waiting for the PHOENIX-5435. After that, I will try to have an
> RC
> > >>> ASAP.
> > >>> > >
> > >>> > > On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <
> > >>> ankitsinghal59@gmail.com>
> > >>> > > wrote:
> > >>> > >
> > >>> > > > I see that both the blockers listed here PHOENIX-5712 and
> > >>> PHOENIX-6241
> > >>> > > > have been resolved(Thanks to Xinyi), and also as per the JIRA
> > query
> > >>> > there
> > >>> > > > is no Jira marked as a blocker for 4.16 except the one related
> to
> > >>> > > > documentation
> > >>> > > > for "splittable catalog table".
> > >>> > > >
> > >>> > > > Xinyi, so are we good to start the release process now?
> > >>> > > >
> > >>> > > > On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <ya...@apache.org>
> > >>> wrote:
> > >>> > > >
> > >>> > > > > Thanks for replying and providing suggestions. I looked at
> the
> > >>> wrong
> > >>> > > > result
> > >>> > > > > Jira list that Daniel provided and did some local testing,
> and
> > >>> here
> > >>> > is
> > >>> > > > the
> > >>> > > > > result:
> > >>> > > > > [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642:
> cannot
> > >>> > > reproduce
> > >>> > > > > it.
> > >>> > > > > [More information is required] PHOENIX-4504 cannot reproduce
> it
> > >>> but
> > >>> > > > someone
> > >>> > > > > claimed that he had a similar issue.
> > >>> > > > > [unusual query] PHOENIX-4540 and PHOENIX-6217
> > >>> > > > >
> > >>> > > > > Based on my finding, I think it's better to have more
> frequent
> > >>> > > > housekeeping
> > >>> > > > > and resolve unreproducible bugs especially since many of them
> > are
> > >>> > > > > considering out of date (phoenix-4.11 or even phoenix-4.6).
> > >>> Since I
> > >>> > > still
> > >>> > > > > need time to work on the blocker Jira(PHOENIX-5712,
> > >>> PHOENIX-6241) and
> > >>> > > fix
> > >>> > > > > test flappers, if you want to fix "unusual query" bugs, feel
> > >>> free to
> > >>> > do
> > >>> > > > so.
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > Sincerely,
> > >>> > > > > Xinyi
> > >>> > > > >
> > >>> > > > > On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <
> > ankit@apache.org>
> > >>> > > wrote:
> > >>> > > > >
> > >>> > > > > > Thanks Daniel and appreciate the effort you put in getting
> > the
> > >>> list
> > >>> > > > ready
> > >>> > > > > > for bugs producing wrong results
> > >>> > > > > > but none of them seems to be a blocker to me for 4.16 as
> they
> > >>> are
> > >>> > not
> > >>> > > > the
> > >>> > > > > > regression and doesn't break the general functionality
> > >>> > > > > > except for specific features, RVC/desc as Chenglei also
> > >>> pointed out
> > >>> > > > > (though
> > >>> > > > > > I'll defer the assessment to RM "Xinyi").
> > >>> > > > > > Probably these can be a part of 4.16.1 or we can do 4.17.0
> > soon
> > >>> > maybe
> > >>> > > > > after
> > >>> > > > > > a few weeks/month?
> > >>> > > > > >
> > >>> > > > > > Considering that we have already fixed 137 bugs and done
> 85+
> > >>> > > > > > improvements/features in 4.16,
> > >>> > > > > > it will not be a good idea to deprive the user from such
> > fixes.
> > >>> > > > > > It's been a year since our last 4.15 release, having no
> > release
> > >>> > > brings
> > >>> > > > > more
> > >>> > > > > > questions on the project
> > >>> > > > > > rather than the bugs which affect a certain % of
> > feature/users,
> > >>> > would
> > >>> > > > the
> > >>> > > > > > release notes
> > >>> > > > > > explaining the stability of certain features set the right
> > >>> > > expectation
> > >>> > > > > for
> > >>> > > > > > those users who rely on these features to wait for a future
> > >>> > release?
> > >>> > > > > >
> > >>> > > > > > Regards,
> > >>> > > > > > Ankit Singhal
> > >>> > > > > >
> > >>> > > > > > On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
> > >>> > > > chenglei@apache.org>
> > >>> > > > > > wrote:
> > >>> > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > > In my opinion, we should keep releases light and
> frequent,
> > >>> and
> > >>> > for
> > >>> > > > > some
> > >>> > > > > > > unusual query bugs like RVC and DESC
> > >>> > > > > > > we could delay fix to next release . I think we should
> > >>> release
> > >>> > > 4.16.0
> > >>> > > > > and
> > >>> > > > > > > 5.1.0 as quickly as possible. In China, many users
> > >>> > > > > > > in HBase&Phoenix User Group thought that Phoenix was
> dead
> > >>> > because
> > >>> > > > our
> > >>> > > > > > too
> > >>> > > > > > > long interval release and stopped using
> > >>> > > > > > > Phoenix.
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > > At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
> > >>> > > > chinmayskulkarni@gmail.com
> > >>> > > > > >
> > >>> > > > > > > wrote:
> > >>> > > > > > > >I agree. These are all major bugs and we should aim at
> > >>> solving
> > >>> > > them
> > >>> > > > > > after
> > >>> > > > > > > >checking that they are still issues. I am +1 on 5833
> and I
> > >>> think
> > >>> > > > 5484
> > >>> > > > > > > would
> > >>> > > > > > > >be a great addition to 4.16 as well. We should aim at
> > >>> resolving
> > >>> > > high
> > >>> > > > > > > >priority bugs like this in every release.
> > >>> > > > > > > >
> > >>> > > > > > > >Sometimes we let these bugs slip without a resolution
> > >>> before a
> > >>> > > > > release,
> > >>> > > > > > > >citing that these are "known issues" or "not regressions
> > >>> from
> > >>> > the
> > >>> > > > last
> > >>> > > > > > > >release". In some cases this may be fine since we want
> to
> > >>> keep
> > >>> > > > > releases
> > >>> > > > > > > >light and frequent, but perhaps we can track such issues
> > >>> and aim
> > >>> > > at
> > >>> > > > > > > >reducing the number of bugs by x% in each release? This
> > will
> > >>> > also
> > >>> > > > keep
> > >>> > > > > > old
> > >>> > > > > > > >Jiras alive since we will potentially periodically
> review
> > >>> them.
> > >>> > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > > >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <
> > >>> > > gjacoby@apache.org>
> > >>> > > > > > > wrote:
> > >>> > > > > > > >
> > >>> > > > > > > >> I've got PHOENIX-5435 in review right now, and would
> > like
> > >>> to
> > >>> > get
> > >>> > > > it
> > >>> > > > > in
> > >>> > > > > > > 4.16
> > >>> > > > > > > >> / 5.1.
> > >>> > > > > > > >>
> > >>> > > > > > > >> It's allowing the annotation of Phoenix metadata into
> > >>> HBase
> > >>> > WALs
> > >>> > > > as
> > >>> > > > > a
> > >>> > > > > > > >> pre-req for the Phoenix Change Detection Capture
> > framework
> > >>> > > > > > > (PHOENIX-5442).
> > >>> > > > > > > >> Since it has both client/server logic, and adds a
> field
> > to
> > >>> > > > > > > System.Catalog,
> > >>> > > > > > > >> it can't go in a patch release.
> > >>> > > > > > > >>
> > >>> > > > > > > >> Depending on timing, I'd _like_ to get PHOENIX-6227,
> > >>> which is
> > >>> > > the
> > >>> > > > > last
> > >>> > > > > > > part
> > >>> > > > > > > >> of CDC that will go into core Phoenix, into 4.16, but
> > >>> since
> > >>> > that
> > >>> > > > > _can_
> > >>> > > > > > > go
> > >>> > > > > > > >> in a patch release and I haven't started it yet, if
> the
> > >>> > release
> > >>> > > > gets
> > >>> > > > > > cut
> > >>> > > > > > > >> before it's ready, no big deal. (The rest of CDC will
> go
> > >>> into
> > >>> > > > > > > >> phoenix-connectors for a future release of that
> > project.)
> > >>> > > > > > > >>
> > >>> > > > > > > >> As for the correctness problems that Daniel points
> out,
> > I
> > >>> > think
> > >>> > > we
> > >>> > > > > > > should
> > >>> > > > > > > >> fix the ones that were detected with a recent version
> > >>> (4.14 or
> > >>> > > > > 4.15?),
> > >>> > > > > > > and
> > >>> > > > > > > >> test to see which of the older ones can still be
> > >>> reproduced.
> > >>> > > Once
> > >>> > > > we
> > >>> > > > > > > know
> > >>> > > > > > > >> which bugs are real and which are just historical, we
> > can
> > >>> > better
> > >>> > > > > judge
> > >>> > > > > > > >> scope. And hopefully close a bunch of obsolete bugs.
> > >>> (Thanks,
> > >>> > > > > Daniel,
> > >>> > > > > > > for
> > >>> > > > > > > >> collecting that list!)
> > >>> > > > > > > >>
> > >>> > > > > > > >> Geoffrey
> > >>> > > > > > > >>
> > >>> > > > > > > >>
> > >>> > > > > > > >>
> > >>> > > > > > > >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
> > >>> > > > > > > >> <da...@salesforce.com.invalid> wrote:
> > >>> > > > > > > >>
> > >>> > > > > > > >> > Hi, I wanted to bring up wrong results in Phoenix
> and
> > >>> some
> > >>> > > JIRAs
> > >>> > > > > > > around
> > >>> > > > > > > >> > them that I think we should fix as the wrong result
> > >>> lessens
> > >>> > > the
> > >>> > > > > end
> > >>> > > > > > > >> user's
> > >>> > > > > > > >> > trust in Phoenix. Releasing a new version without
> > >>> > addressing
> > >>> > > > > these
> > >>> > > > > > > in a
> > >>> > > > > > > >> > minor release hurts our visibility in that these
> > >>> critical
> > >>> > > issues
> > >>> > > > > are
> > >>> > > > > > > not
> > >>> > > > > > > >> > addressed.
> > >>> > > > > > > >> >
> > >>> > > > > > > >> > Jira's that I'm involved with for example: I've
> > already
> > >>> > given
> > >>> > > a
> > >>> > > > > > patch
> > >>> > > > > > > >> > several months ago for 5833 and there is a chance it
> > >>> may fix
> > >>> > > > 5484.
> > >>> > > > > > > >> >
> https://issues.apache.org/jira/browse/PHOENIX-5833
> > >>> > > > > > > >> >
> https://issues.apache.org/jira/browse/PHOENIX-5484
> > >>> > > > > > > >> >
> > >>> > > > > > > >> > In addition, inspecting apache JIRA i see several
> > other
> > >>> > wrong
> > >>> > > > > result
> > >>> > > > > > > >> JIRAs
> > >>> > > > > > > >> > from the community. Some of these certainly are
> > >>> probably
> > >>> > old
> > >>> > > > > issues
> > >>> > > > > > > or
> > >>> > > > > > > >> > incorrect understanding but some of these are opened
> > by
> > >>> our
> > >>> > > own
> > >>> > > > > dev
> > >>> > > > > > > >> > community and are likely real problems.
> > >>> > > > > > > >> >
> https://issues.apache.org/jira/browse/PHOENIX-6217
> > >>> > > > > > > >> >
> https://issues.apache.org/jira/browse/PHOENIX-5571
> > >>> > > > > > > >> >
> https://issues.apache.org/jira/browse/PHOENIX-4642
> > >>> > > > > > > >> >
> https://issues.apache.org/jira/browse/PHOENIX-4540
> > >>> > > > > > > >> >
> https://issues.apache.org/jira/browse/PHOENIX-4504
> > >>> > > > > > > >> >
> https://issues.apache.org/jira/browse/PHOENIX-4419
> > >>> > > > > > > >> >
> https://issues.apache.org/jira/browse/PHOENIX-4116
> > >>> > > > > > > >> >
> > >>> > > > > > > >> > What is our stance on this type of issue? Are we
> > going
> > >>> to
> > >>> > say
> > >>> > > > > these
> > >>> > > > > > > were
> > >>> > > > > > > >> > issues prior to 4.15 and not address them? Should
> we
> > >>> have
> > >>> > > > > > > requirements
> > >>> > > > > > > >> for
> > >>> > > > > > > >> > our releases to fix wrong results?
> > >>> > > > > > > >> >
> > >>> > > > > > > >> > Daniel Wong
> > >>> > > > > > > >> >
> > >>> > > > > > > >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <
> > >>> > > yanxinyi@apache.org>
> > >>> > > > > > > wrote:
> > >>> > > > > > > >> >
> > >>> > > > > > > >> > > Hi all,
> > >>> > > > > > > >> > >
> > >>> > > > > > > >> > > It's time to discuss the Phoenix 4.16 release.
> After
> > >>> many
> > >>> > > > > people's
> > >>> > > > > > > >> > > contributions on the bug fixes, new features, and
> > >>> other
> > >>> > > works
> > >>> > > > in
> > >>> > > > > > the
> > >>> > > > > > > >> past
> > >>> > > > > > > >> > > few months, we are kind of close to the point to
> > have
> > >>> a RC
> > >>> > > > > (still
> > >>> > > > > > > need
> > >>> > > > > > > >> to
> > >>> > > > > > > >> > > fix test flappers). Please let me know if you
> think
> > >>> any
> > >>> > JIRA
> > >>> > > > > must
> > >>> > > > > > be
> > >>> > > > > > > >> part
> > >>> > > > > > > >> > > of the Phoenix 4.16 release other than major
> blocker
> > >>> > > > > PHOENIX-5712.
> > >>> > > > > > > >> > >
> > >>> > > > > > > >> > > If no surprise comes up, I will not wait for any
> new
> > >>> major
> > >>> > > > > > features
> > >>> > > > > > > and
> > >>> > > > > > > >> > > focus on the RC as soon as possible.
> > >>> > > > > > > >> > >
> > >>> > > > > > > >> > > Sincerely,
> > >>> > > > > > > >> > > Xinyi
> > >>> > > > > > > >> > >
> > >>> > > > > > > >> >
> > >>> > > > > > > >> >
> > >>> > > > > > > >> > --
> > >>> > > > > > > >> > Daniel Wong
> > >>> > > > > > > >> > Salesforce
> > >>> > > > > > > >> > Mobile: 628.217.1808
> > >>> > > > > > > >> >
> > >>> > > > > > > >>
> > >>> > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > > >--
> > >>> > > > > > > >Chinmay Kulkarni
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>
> > >>
> > >> --
> > >> *István Tóth* | Staff Software Engineer
> > >> stoty@cloudera.com <https://www.cloudera.com>
> > >> [image: Cloudera] <https://www.cloudera.com/>
> > >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > >> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > >> <https://www.cloudera.com/>
> > >> ------------------------------
> > >>
> > >
> >
>
Re: [Discuss] Releasing Phoenix 4.16
Posted by Geoffrey Jacoby <gj...@apache.org>.
Thanks for bringing up the default branch issue, Istvan, I've been meaning
to start a conversation about it on this list.
As part of PHOENIX-5435, I changed the default 4.x HBase release to 1.5 and
the default 5.x HBase to 2.3 because the WAL annotation feature introduced
by 5435 only works with HBase 1.5+ or 2.3+. (It depends on a coproc hook
introduced in HBASE-22623). That means that all tests of that feature must
no-op when run against an earlier HBase, which means that it would never be
tested in our CI pipelines if the default was 1.3 or 2.1.
This has come up quite a few times recently. In particular, the new
indexing framework runs in a degraded state against HBase 2.1 and 2.2
(still better than the old indexing framework though!), because we lack
support in 2.1 for Filters on raw scans and can't use the "max lookback
age" feature with HBase 2.1 or 2.2, which keeps major compactions from
messing with index rebuilds. That's why lots of indexing tests no-op or
have to make different assertions when run against 2.1 or 2.2, so we only
properly test the indexing framework if CI and local developer
tests run against HBase 2.3 or up.
As for the release, don't we usually release separate binaries that are
suitable for all the HBase versions we support? Back before we had a
unified 4.x branch, I believe we used to release from 3 different
4.x-HBase-* branches each time. I've been assuming that part of the release
process would be generating binaries for 4.x-HBase-1.3, 4.x-HBase-1.4,
4.x-HBase-1.5, and 4.x-HBase-1.6 (if that was different from 1.5).
Geoffrey
On Thu, Jan 7, 2021 at 11:19 AM Istvan Toth <st...@apache.org> wrote:
> Two more issues came to my mind:
>
> In PHOENIX-5435 the default profile was changed to HBase 1.5.0
> This causes two conflicts:
> - Our documentation says that the default profile is the lowest supported,
> which is no longer true
> - Unless we change our binary release process, the published maven
> artifacts and binaries will also be built with HBase 1.5, thus they will be
> incompatible with Hbase 1.3 and 1.4
>
> This should be addressed before release.
> One possible solution:
> * Add an "oldest" hbase.profile option which always chooses the oldest
> supported version
> * Update the release scripts to specify this profile
> * Update the docs.
>
> I have also adopted the hbase release scripts to Phoenix, they are present
> in dev/create-release in the master branch, but
> should be able to perform 4.16 release as well. I have used this for the
> phoenix-thirdparty, omid, and tephra releases, but no live phoenix releases
> have been done with them yet, obviously.
>
> regards
> Istvan
>
> On Thu, Jan 7, 2021 at 8:19 AM Istvan Toth <st...@apache.org> wrote:
>
> > Hi!
> >
> > A question of process: Should we backport fixes to the 4.16 branch at our
> > discretion, or is backporting those handled by the release manager
> (Xinyi) ?
> >
> > On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth <st...@cloudera.com> wrote:
> >
> >> Hi!
> >>
> >> Thanks for everyone's effort on fixing the flakey tests.
> >> However, our ASF Jenkins runs are still very unstable, and I am afraid
> >> that the single clean 4.x run was down more to luck than to fixing the
> >> underlying problem.
> >> While I do not consider this a blocker, fixing this would make the pre-
> >> and post commit tests far more useful, and let us start with a clean
> slate
> >> for the next release.
> >> I have created https://issues.apache.org/jira/browse/PHOENIX-6288 to
> >> track the generic cluster setup issue, but my attempts to fix this, or
> at
> >> least figure out the root cause have been unsuccessful so far.
> >>
> >> I ask for your help in fixing this issue. I have added some inconclusive
> >> analysis to the ticket, and the full failsafe output directory is
> >> downloadable as artifacts from the failed multibranch runs (stored for a
> >> few days), which should hold the answer to this riddle.
> >>
> >> regards,
> >> Istvan
> >>
> >> On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan <ya...@apache.org> wrote:
> >>
> >>> Hi,
> >>>
> >>> We finally have a stable 4.x branch build after many flapper test
> fixing
> >>> contributions from the community. I'm going to fork a new branch(4.16)
> >>> from
> >>> the current 4.x branch for the 4.16 release. As a cutoff, I will not
> >>> include any new features other than PHOENIX-6211 and bug fix. Please
> let
> >>> me
> >>> know if you have any questions or concerns.
> >>>
> >>> Thanks,
> >>> Xinyi
> >>>
> >>> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani <vj...@apache.org>
> wrote:
> >>>
> >>> > An update on test stability:
> >>> >
> >>> > As per recent 4.x build results, we are left with very few flappers
> and
> >>> > specifically
> >>> > with HBase profile 1.6, I can see recent 7 build (multibranch + PR
> >>> > precommit results)
> >>> > results without any test failure.
> >>> >
> >>> >
> >>> > On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <ya...@apache.org>
> wrote:
> >>> >
> >>> > > Yes, we are currently working on fixing the 4.x branch test
> flappers
> >>> > while
> >>> > > waiting for the PHOENIX-5435. After that, I will try to have an RC
> >>> ASAP.
> >>> > >
> >>> > > On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <
> >>> ankitsinghal59@gmail.com>
> >>> > > wrote:
> >>> > >
> >>> > > > I see that both the blockers listed here PHOENIX-5712 and
> >>> PHOENIX-6241
> >>> > > > have been resolved(Thanks to Xinyi), and also as per the JIRA
> query
> >>> > there
> >>> > > > is no Jira marked as a blocker for 4.16 except the one related to
> >>> > > > documentation
> >>> > > > for "splittable catalog table".
> >>> > > >
> >>> > > > Xinyi, so are we good to start the release process now?
> >>> > > >
> >>> > > > On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <ya...@apache.org>
> >>> wrote:
> >>> > > >
> >>> > > > > Thanks for replying and providing suggestions. I looked at the
> >>> wrong
> >>> > > > result
> >>> > > > > Jira list that Daniel provided and did some local testing, and
> >>> here
> >>> > is
> >>> > > > the
> >>> > > > > result:
> >>> > > > > [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642: cannot
> >>> > > reproduce
> >>> > > > > it.
> >>> > > > > [More information is required] PHOENIX-4504 cannot reproduce it
> >>> but
> >>> > > > someone
> >>> > > > > claimed that he had a similar issue.
> >>> > > > > [unusual query] PHOENIX-4540 and PHOENIX-6217
> >>> > > > >
> >>> > > > > Based on my finding, I think it's better to have more frequent
> >>> > > > housekeeping
> >>> > > > > and resolve unreproducible bugs especially since many of them
> are
> >>> > > > > considering out of date (phoenix-4.11 or even phoenix-4.6).
> >>> Since I
> >>> > > still
> >>> > > > > need time to work on the blocker Jira(PHOENIX-5712,
> >>> PHOENIX-6241) and
> >>> > > fix
> >>> > > > > test flappers, if you want to fix "unusual query" bugs, feel
> >>> free to
> >>> > do
> >>> > > > so.
> >>> > > > >
> >>> > > > >
> >>> > > > > Sincerely,
> >>> > > > > Xinyi
> >>> > > > >
> >>> > > > > On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <
> ankit@apache.org>
> >>> > > wrote:
> >>> > > > >
> >>> > > > > > Thanks Daniel and appreciate the effort you put in getting
> the
> >>> list
> >>> > > > ready
> >>> > > > > > for bugs producing wrong results
> >>> > > > > > but none of them seems to be a blocker to me for 4.16 as they
> >>> are
> >>> > not
> >>> > > > the
> >>> > > > > > regression and doesn't break the general functionality
> >>> > > > > > except for specific features, RVC/desc as Chenglei also
> >>> pointed out
> >>> > > > > (though
> >>> > > > > > I'll defer the assessment to RM "Xinyi").
> >>> > > > > > Probably these can be a part of 4.16.1 or we can do 4.17.0
> soon
> >>> > maybe
> >>> > > > > after
> >>> > > > > > a few weeks/month?
> >>> > > > > >
> >>> > > > > > Considering that we have already fixed 137 bugs and done 85+
> >>> > > > > > improvements/features in 4.16,
> >>> > > > > > it will not be a good idea to deprive the user from such
> fixes.
> >>> > > > > > It's been a year since our last 4.15 release, having no
> release
> >>> > > brings
> >>> > > > > more
> >>> > > > > > questions on the project
> >>> > > > > > rather than the bugs which affect a certain % of
> feature/users,
> >>> > would
> >>> > > > the
> >>> > > > > > release notes
> >>> > > > > > explaining the stability of certain features set the right
> >>> > > expectation
> >>> > > > > for
> >>> > > > > > those users who rely on these features to wait for a future
> >>> > release?
> >>> > > > > >
> >>> > > > > > Regards,
> >>> > > > > > Ankit Singhal
> >>> > > > > >
> >>> > > > > > On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
> >>> > > > chenglei@apache.org>
> >>> > > > > > wrote:
> >>> > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > > In my opinion, we should keep releases light and frequent,
> >>> and
> >>> > for
> >>> > > > > some
> >>> > > > > > > unusual query bugs like RVC and DESC
> >>> > > > > > > we could delay fix to next release . I think we should
> >>> release
> >>> > > 4.16.0
> >>> > > > > and
> >>> > > > > > > 5.1.0 as quickly as possible. In China, many users
> >>> > > > > > > in HBase&Phoenix User Group thought that Phoenix was dead
> >>> > because
> >>> > > > our
> >>> > > > > > too
> >>> > > > > > > long interval release and stopped using
> >>> > > > > > > Phoenix.
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > > At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
> >>> > > > chinmayskulkarni@gmail.com
> >>> > > > > >
> >>> > > > > > > wrote:
> >>> > > > > > > >I agree. These are all major bugs and we should aim at
> >>> solving
> >>> > > them
> >>> > > > > > after
> >>> > > > > > > >checking that they are still issues. I am +1 on 5833 and I
> >>> think
> >>> > > > 5484
> >>> > > > > > > would
> >>> > > > > > > >be a great addition to 4.16 as well. We should aim at
> >>> resolving
> >>> > > high
> >>> > > > > > > >priority bugs like this in every release.
> >>> > > > > > > >
> >>> > > > > > > >Sometimes we let these bugs slip without a resolution
> >>> before a
> >>> > > > > release,
> >>> > > > > > > >citing that these are "known issues" or "not regressions
> >>> from
> >>> > the
> >>> > > > last
> >>> > > > > > > >release". In some cases this may be fine since we want to
> >>> keep
> >>> > > > > releases
> >>> > > > > > > >light and frequent, but perhaps we can track such issues
> >>> and aim
> >>> > > at
> >>> > > > > > > >reducing the number of bugs by x% in each release? This
> will
> >>> > also
> >>> > > > keep
> >>> > > > > > old
> >>> > > > > > > >Jiras alive since we will potentially periodically review
> >>> them.
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <
> >>> > > gjacoby@apache.org>
> >>> > > > > > > wrote:
> >>> > > > > > > >
> >>> > > > > > > >> I've got PHOENIX-5435 in review right now, and would
> like
> >>> to
> >>> > get
> >>> > > > it
> >>> > > > > in
> >>> > > > > > > 4.16
> >>> > > > > > > >> / 5.1.
> >>> > > > > > > >>
> >>> > > > > > > >> It's allowing the annotation of Phoenix metadata into
> >>> HBase
> >>> > WALs
> >>> > > > as
> >>> > > > > a
> >>> > > > > > > >> pre-req for the Phoenix Change Detection Capture
> framework
> >>> > > > > > > (PHOENIX-5442).
> >>> > > > > > > >> Since it has both client/server logic, and adds a field
> to
> >>> > > > > > > System.Catalog,
> >>> > > > > > > >> it can't go in a patch release.
> >>> > > > > > > >>
> >>> > > > > > > >> Depending on timing, I'd _like_ to get PHOENIX-6227,
> >>> which is
> >>> > > the
> >>> > > > > last
> >>> > > > > > > part
> >>> > > > > > > >> of CDC that will go into core Phoenix, into 4.16, but
> >>> since
> >>> > that
> >>> > > > > _can_
> >>> > > > > > > go
> >>> > > > > > > >> in a patch release and I haven't started it yet, if the
> >>> > release
> >>> > > > gets
> >>> > > > > > cut
> >>> > > > > > > >> before it's ready, no big deal. (The rest of CDC will go
> >>> into
> >>> > > > > > > >> phoenix-connectors for a future release of that
> project.)
> >>> > > > > > > >>
> >>> > > > > > > >> As for the correctness problems that Daniel points out,
> I
> >>> > think
> >>> > > we
> >>> > > > > > > should
> >>> > > > > > > >> fix the ones that were detected with a recent version
> >>> (4.14 or
> >>> > > > > 4.15?),
> >>> > > > > > > and
> >>> > > > > > > >> test to see which of the older ones can still be
> >>> reproduced.
> >>> > > Once
> >>> > > > we
> >>> > > > > > > know
> >>> > > > > > > >> which bugs are real and which are just historical, we
> can
> >>> > better
> >>> > > > > judge
> >>> > > > > > > >> scope. And hopefully close a bunch of obsolete bugs.
> >>> (Thanks,
> >>> > > > > Daniel,
> >>> > > > > > > for
> >>> > > > > > > >> collecting that list!)
> >>> > > > > > > >>
> >>> > > > > > > >> Geoffrey
> >>> > > > > > > >>
> >>> > > > > > > >>
> >>> > > > > > > >>
> >>> > > > > > > >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
> >>> > > > > > > >> <da...@salesforce.com.invalid> wrote:
> >>> > > > > > > >>
> >>> > > > > > > >> > Hi, I wanted to bring up wrong results in Phoenix and
> >>> some
> >>> > > JIRAs
> >>> > > > > > > around
> >>> > > > > > > >> > them that I think we should fix as the wrong result
> >>> lessens
> >>> > > the
> >>> > > > > end
> >>> > > > > > > >> user's
> >>> > > > > > > >> > trust in Phoenix. Releasing a new version without
> >>> > addressing
> >>> > > > > these
> >>> > > > > > > in a
> >>> > > > > > > >> > minor release hurts our visibility in that these
> >>> critical
> >>> > > issues
> >>> > > > > are
> >>> > > > > > > not
> >>> > > > > > > >> > addressed.
> >>> > > > > > > >> >
> >>> > > > > > > >> > Jira's that I'm involved with for example: I've
> already
> >>> > given
> >>> > > a
> >>> > > > > > patch
> >>> > > > > > > >> > several months ago for 5833 and there is a chance it
> >>> may fix
> >>> > > > 5484.
> >>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5833
> >>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5484
> >>> > > > > > > >> >
> >>> > > > > > > >> > In addition, inspecting apache JIRA i see several
> other
> >>> > wrong
> >>> > > > > result
> >>> > > > > > > >> JIRAs
> >>> > > > > > > >> > from the community. Some of these certainly are
> >>> probably
> >>> > old
> >>> > > > > issues
> >>> > > > > > > or
> >>> > > > > > > >> > incorrect understanding but some of these are opened
> by
> >>> our
> >>> > > own
> >>> > > > > dev
> >>> > > > > > > >> > community and are likely real problems.
> >>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-6217
> >>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5571
> >>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4642
> >>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4540
> >>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4504
> >>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4419
> >>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4116
> >>> > > > > > > >> >
> >>> > > > > > > >> > What is our stance on this type of issue? Are we
> going
> >>> to
> >>> > say
> >>> > > > > these
> >>> > > > > > > were
> >>> > > > > > > >> > issues prior to 4.15 and not address them? Should we
> >>> have
> >>> > > > > > > requirements
> >>> > > > > > > >> for
> >>> > > > > > > >> > our releases to fix wrong results?
> >>> > > > > > > >> >
> >>> > > > > > > >> > Daniel Wong
> >>> > > > > > > >> >
> >>> > > > > > > >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <
> >>> > > yanxinyi@apache.org>
> >>> > > > > > > wrote:
> >>> > > > > > > >> >
> >>> > > > > > > >> > > Hi all,
> >>> > > > > > > >> > >
> >>> > > > > > > >> > > It's time to discuss the Phoenix 4.16 release. After
> >>> many
> >>> > > > > people's
> >>> > > > > > > >> > > contributions on the bug fixes, new features, and
> >>> other
> >>> > > works
> >>> > > > in
> >>> > > > > > the
> >>> > > > > > > >> past
> >>> > > > > > > >> > > few months, we are kind of close to the point to
> have
> >>> a RC
> >>> > > > > (still
> >>> > > > > > > need
> >>> > > > > > > >> to
> >>> > > > > > > >> > > fix test flappers). Please let me know if you think
> >>> any
> >>> > JIRA
> >>> > > > > must
> >>> > > > > > be
> >>> > > > > > > >> part
> >>> > > > > > > >> > > of the Phoenix 4.16 release other than major blocker
> >>> > > > > PHOENIX-5712.
> >>> > > > > > > >> > >
> >>> > > > > > > >> > > If no surprise comes up, I will not wait for any new
> >>> major
> >>> > > > > > features
> >>> > > > > > > and
> >>> > > > > > > >> > > focus on the RC as soon as possible.
> >>> > > > > > > >> > >
> >>> > > > > > > >> > > Sincerely,
> >>> > > > > > > >> > > Xinyi
> >>> > > > > > > >> > >
> >>> > > > > > > >> >
> >>> > > > > > > >> >
> >>> > > > > > > >> > --
> >>> > > > > > > >> > Daniel Wong
> >>> > > > > > > >> > Salesforce
> >>> > > > > > > >> > Mobile: 628.217.1808
> >>> > > > > > > >> >
> >>> > > > > > > >>
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > >--
> >>> > > > > > > >Chinmay Kulkarni
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
> >>
> >> --
> >> *István Tóth* | Staff Software Engineer
> >> stoty@cloudera.com <https://www.cloudera.com>
> >> [image: Cloudera] <https://www.cloudera.com/>
> >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> >> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> >> <https://www.cloudera.com/>
> >> ------------------------------
> >>
> >
>
Re: [Discuss] Releasing Phoenix 4.16
Posted by Istvan Toth <st...@apache.org>.
Two more issues came to my mind:
In PHOENIX-5435 the default profile was changed to HBase 1.5.0
This causes two conflicts:
- Our documentation says that the default profile is the lowest supported,
which is no longer true
- Unless we change our binary release process, the published maven
artifacts and binaries will also be built with HBase 1.5, thus they will be
incompatible with Hbase 1.3 and 1.4
This should be addressed before release.
One possible solution:
* Add an "oldest" hbase.profile option which always chooses the oldest
supported version
* Update the release scripts to specify this profile
* Update the docs.
I have also adopted the hbase release scripts to Phoenix, they are present
in dev/create-release in the master branch, but
should be able to perform 4.16 release as well. I have used this for the
phoenix-thirdparty, omid, and tephra releases, but no live phoenix releases
have been done with them yet, obviously.
regards
Istvan
On Thu, Jan 7, 2021 at 8:19 AM Istvan Toth <st...@apache.org> wrote:
> Hi!
>
> A question of process: Should we backport fixes to the 4.16 branch at our
> discretion, or is backporting those handled by the release manager (Xinyi) ?
>
> On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth <st...@cloudera.com> wrote:
>
>> Hi!
>>
>> Thanks for everyone's effort on fixing the flakey tests.
>> However, our ASF Jenkins runs are still very unstable, and I am afraid
>> that the single clean 4.x run was down more to luck than to fixing the
>> underlying problem.
>> While I do not consider this a blocker, fixing this would make the pre-
>> and post commit tests far more useful, and let us start with a clean slate
>> for the next release.
>> I have created https://issues.apache.org/jira/browse/PHOENIX-6288 to
>> track the generic cluster setup issue, but my attempts to fix this, or at
>> least figure out the root cause have been unsuccessful so far.
>>
>> I ask for your help in fixing this issue. I have added some inconclusive
>> analysis to the ticket, and the full failsafe output directory is
>> downloadable as artifacts from the failed multibranch runs (stored for a
>> few days), which should hold the answer to this riddle.
>>
>> regards,
>> Istvan
>>
>> On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan <ya...@apache.org> wrote:
>>
>>> Hi,
>>>
>>> We finally have a stable 4.x branch build after many flapper test fixing
>>> contributions from the community. I'm going to fork a new branch(4.16)
>>> from
>>> the current 4.x branch for the 4.16 release. As a cutoff, I will not
>>> include any new features other than PHOENIX-6211 and bug fix. Please let
>>> me
>>> know if you have any questions or concerns.
>>>
>>> Thanks,
>>> Xinyi
>>>
>>> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani <vj...@apache.org> wrote:
>>>
>>> > An update on test stability:
>>> >
>>> > As per recent 4.x build results, we are left with very few flappers and
>>> > specifically
>>> > with HBase profile 1.6, I can see recent 7 build (multibranch + PR
>>> > precommit results)
>>> > results without any test failure.
>>> >
>>> >
>>> > On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <ya...@apache.org> wrote:
>>> >
>>> > > Yes, we are currently working on fixing the 4.x branch test flappers
>>> > while
>>> > > waiting for the PHOENIX-5435. After that, I will try to have an RC
>>> ASAP.
>>> > >
>>> > > On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <
>>> ankitsinghal59@gmail.com>
>>> > > wrote:
>>> > >
>>> > > > I see that both the blockers listed here PHOENIX-5712 and
>>> PHOENIX-6241
>>> > > > have been resolved(Thanks to Xinyi), and also as per the JIRA query
>>> > there
>>> > > > is no Jira marked as a blocker for 4.16 except the one related to
>>> > > > documentation
>>> > > > for "splittable catalog table".
>>> > > >
>>> > > > Xinyi, so are we good to start the release process now?
>>> > > >
>>> > > > On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <ya...@apache.org>
>>> wrote:
>>> > > >
>>> > > > > Thanks for replying and providing suggestions. I looked at the
>>> wrong
>>> > > > result
>>> > > > > Jira list that Daniel provided and did some local testing, and
>>> here
>>> > is
>>> > > > the
>>> > > > > result:
>>> > > > > [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642: cannot
>>> > > reproduce
>>> > > > > it.
>>> > > > > [More information is required] PHOENIX-4504 cannot reproduce it
>>> but
>>> > > > someone
>>> > > > > claimed that he had a similar issue.
>>> > > > > [unusual query] PHOENIX-4540 and PHOENIX-6217
>>> > > > >
>>> > > > > Based on my finding, I think it's better to have more frequent
>>> > > > housekeeping
>>> > > > > and resolve unreproducible bugs especially since many of them are
>>> > > > > considering out of date (phoenix-4.11 or even phoenix-4.6).
>>> Since I
>>> > > still
>>> > > > > need time to work on the blocker Jira(PHOENIX-5712,
>>> PHOENIX-6241) and
>>> > > fix
>>> > > > > test flappers, if you want to fix "unusual query" bugs, feel
>>> free to
>>> > do
>>> > > > so.
>>> > > > >
>>> > > > >
>>> > > > > Sincerely,
>>> > > > > Xinyi
>>> > > > >
>>> > > > > On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <an...@apache.org>
>>> > > wrote:
>>> > > > >
>>> > > > > > Thanks Daniel and appreciate the effort you put in getting the
>>> list
>>> > > > ready
>>> > > > > > for bugs producing wrong results
>>> > > > > > but none of them seems to be a blocker to me for 4.16 as they
>>> are
>>> > not
>>> > > > the
>>> > > > > > regression and doesn't break the general functionality
>>> > > > > > except for specific features, RVC/desc as Chenglei also
>>> pointed out
>>> > > > > (though
>>> > > > > > I'll defer the assessment to RM "Xinyi").
>>> > > > > > Probably these can be a part of 4.16.1 or we can do 4.17.0 soon
>>> > maybe
>>> > > > > after
>>> > > > > > a few weeks/month?
>>> > > > > >
>>> > > > > > Considering that we have already fixed 137 bugs and done 85+
>>> > > > > > improvements/features in 4.16,
>>> > > > > > it will not be a good idea to deprive the user from such fixes.
>>> > > > > > It's been a year since our last 4.15 release, having no release
>>> > > brings
>>> > > > > more
>>> > > > > > questions on the project
>>> > > > > > rather than the bugs which affect a certain % of feature/users,
>>> > would
>>> > > > the
>>> > > > > > release notes
>>> > > > > > explaining the stability of certain features set the right
>>> > > expectation
>>> > > > > for
>>> > > > > > those users who rely on these features to wait for a future
>>> > release?
>>> > > > > >
>>> > > > > > Regards,
>>> > > > > > Ankit Singhal
>>> > > > > >
>>> > > > > > On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
>>> > > > chenglei@apache.org>
>>> > > > > > wrote:
>>> > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > In my opinion, we should keep releases light and frequent,
>>> and
>>> > for
>>> > > > > some
>>> > > > > > > unusual query bugs like RVC and DESC
>>> > > > > > > we could delay fix to next release . I think we should
>>> release
>>> > > 4.16.0
>>> > > > > and
>>> > > > > > > 5.1.0 as quickly as possible. In China, many users
>>> > > > > > > in HBase&Phoenix User Group thought that Phoenix was dead
>>> > because
>>> > > > our
>>> > > > > > too
>>> > > > > > > long interval release and stopped using
>>> > > > > > > Phoenix.
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
>>> > > > chinmayskulkarni@gmail.com
>>> > > > > >
>>> > > > > > > wrote:
>>> > > > > > > >I agree. These are all major bugs and we should aim at
>>> solving
>>> > > them
>>> > > > > > after
>>> > > > > > > >checking that they are still issues. I am +1 on 5833 and I
>>> think
>>> > > > 5484
>>> > > > > > > would
>>> > > > > > > >be a great addition to 4.16 as well. We should aim at
>>> resolving
>>> > > high
>>> > > > > > > >priority bugs like this in every release.
>>> > > > > > > >
>>> > > > > > > >Sometimes we let these bugs slip without a resolution
>>> before a
>>> > > > > release,
>>> > > > > > > >citing that these are "known issues" or "not regressions
>>> from
>>> > the
>>> > > > last
>>> > > > > > > >release". In some cases this may be fine since we want to
>>> keep
>>> > > > > releases
>>> > > > > > > >light and frequent, but perhaps we can track such issues
>>> and aim
>>> > > at
>>> > > > > > > >reducing the number of bugs by x% in each release? This will
>>> > also
>>> > > > keep
>>> > > > > > old
>>> > > > > > > >Jiras alive since we will potentially periodically review
>>> them.
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <
>>> > > gjacoby@apache.org>
>>> > > > > > > wrote:
>>> > > > > > > >
>>> > > > > > > >> I've got PHOENIX-5435 in review right now, and would like
>>> to
>>> > get
>>> > > > it
>>> > > > > in
>>> > > > > > > 4.16
>>> > > > > > > >> / 5.1.
>>> > > > > > > >>
>>> > > > > > > >> It's allowing the annotation of Phoenix metadata into
>>> HBase
>>> > WALs
>>> > > > as
>>> > > > > a
>>> > > > > > > >> pre-req for the Phoenix Change Detection Capture framework
>>> > > > > > > (PHOENIX-5442).
>>> > > > > > > >> Since it has both client/server logic, and adds a field to
>>> > > > > > > System.Catalog,
>>> > > > > > > >> it can't go in a patch release.
>>> > > > > > > >>
>>> > > > > > > >> Depending on timing, I'd _like_ to get PHOENIX-6227,
>>> which is
>>> > > the
>>> > > > > last
>>> > > > > > > part
>>> > > > > > > >> of CDC that will go into core Phoenix, into 4.16, but
>>> since
>>> > that
>>> > > > > _can_
>>> > > > > > > go
>>> > > > > > > >> in a patch release and I haven't started it yet, if the
>>> > release
>>> > > > gets
>>> > > > > > cut
>>> > > > > > > >> before it's ready, no big deal. (The rest of CDC will go
>>> into
>>> > > > > > > >> phoenix-connectors for a future release of that project.)
>>> > > > > > > >>
>>> > > > > > > >> As for the correctness problems that Daniel points out, I
>>> > think
>>> > > we
>>> > > > > > > should
>>> > > > > > > >> fix the ones that were detected with a recent version
>>> (4.14 or
>>> > > > > 4.15?),
>>> > > > > > > and
>>> > > > > > > >> test to see which of the older ones can still be
>>> reproduced.
>>> > > Once
>>> > > > we
>>> > > > > > > know
>>> > > > > > > >> which bugs are real and which are just historical, we can
>>> > better
>>> > > > > judge
>>> > > > > > > >> scope. And hopefully close a bunch of obsolete bugs.
>>> (Thanks,
>>> > > > > Daniel,
>>> > > > > > > for
>>> > > > > > > >> collecting that list!)
>>> > > > > > > >>
>>> > > > > > > >> Geoffrey
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
>>> > > > > > > >> <da...@salesforce.com.invalid> wrote:
>>> > > > > > > >>
>>> > > > > > > >> > Hi, I wanted to bring up wrong results in Phoenix and
>>> some
>>> > > JIRAs
>>> > > > > > > around
>>> > > > > > > >> > them that I think we should fix as the wrong result
>>> lessens
>>> > > the
>>> > > > > end
>>> > > > > > > >> user's
>>> > > > > > > >> > trust in Phoenix. Releasing a new version without
>>> > addressing
>>> > > > > these
>>> > > > > > > in a
>>> > > > > > > >> > minor release hurts our visibility in that these
>>> critical
>>> > > issues
>>> > > > > are
>>> > > > > > > not
>>> > > > > > > >> > addressed.
>>> > > > > > > >> >
>>> > > > > > > >> > Jira's that I'm involved with for example: I've already
>>> > given
>>> > > a
>>> > > > > > patch
>>> > > > > > > >> > several months ago for 5833 and there is a chance it
>>> may fix
>>> > > > 5484.
>>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5833
>>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5484
>>> > > > > > > >> >
>>> > > > > > > >> > In addition, inspecting apache JIRA i see several other
>>> > wrong
>>> > > > > result
>>> > > > > > > >> JIRAs
>>> > > > > > > >> > from the community. Some of these certainly are
>>> probably
>>> > old
>>> > > > > issues
>>> > > > > > > or
>>> > > > > > > >> > incorrect understanding but some of these are opened by
>>> our
>>> > > own
>>> > > > > dev
>>> > > > > > > >> > community and are likely real problems.
>>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-6217
>>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5571
>>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4642
>>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4540
>>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4504
>>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4419
>>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4116
>>> > > > > > > >> >
>>> > > > > > > >> > What is our stance on this type of issue? Are we going
>>> to
>>> > say
>>> > > > > these
>>> > > > > > > were
>>> > > > > > > >> > issues prior to 4.15 and not address them? Should we
>>> have
>>> > > > > > > requirements
>>> > > > > > > >> for
>>> > > > > > > >> > our releases to fix wrong results?
>>> > > > > > > >> >
>>> > > > > > > >> > Daniel Wong
>>> > > > > > > >> >
>>> > > > > > > >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <
>>> > > yanxinyi@apache.org>
>>> > > > > > > wrote:
>>> > > > > > > >> >
>>> > > > > > > >> > > Hi all,
>>> > > > > > > >> > >
>>> > > > > > > >> > > It's time to discuss the Phoenix 4.16 release. After
>>> many
>>> > > > > people's
>>> > > > > > > >> > > contributions on the bug fixes, new features, and
>>> other
>>> > > works
>>> > > > in
>>> > > > > > the
>>> > > > > > > >> past
>>> > > > > > > >> > > few months, we are kind of close to the point to have
>>> a RC
>>> > > > > (still
>>> > > > > > > need
>>> > > > > > > >> to
>>> > > > > > > >> > > fix test flappers). Please let me know if you think
>>> any
>>> > JIRA
>>> > > > > must
>>> > > > > > be
>>> > > > > > > >> part
>>> > > > > > > >> > > of the Phoenix 4.16 release other than major blocker
>>> > > > > PHOENIX-5712.
>>> > > > > > > >> > >
>>> > > > > > > >> > > If no surprise comes up, I will not wait for any new
>>> major
>>> > > > > > features
>>> > > > > > > and
>>> > > > > > > >> > > focus on the RC as soon as possible.
>>> > > > > > > >> > >
>>> > > > > > > >> > > Sincerely,
>>> > > > > > > >> > > Xinyi
>>> > > > > > > >> > >
>>> > > > > > > >> >
>>> > > > > > > >> >
>>> > > > > > > >> > --
>>> > > > > > > >> > Daniel Wong
>>> > > > > > > >> > Salesforce
>>> > > > > > > >> > Mobile: 628.217.1808
>>> > > > > > > >> >
>>> > > > > > > >>
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >--
>>> > > > > > > >Chinmay Kulkarni
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>> --
>> *István Tóth* | Staff Software Engineer
>> stoty@cloudera.com <https://www.cloudera.com>
>> [image: Cloudera] <https://www.cloudera.com/>
>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>> <https://www.cloudera.com/>
>> ------------------------------
>>
>
Re: [Discuss] Releasing Phoenix 4.16
Posted by Istvan Toth <st...@apache.org>.
Hi!
A question of process: Should we backport fixes to the 4.16 branch at our
discretion, or is backporting those handled by the release manager (Xinyi) ?
On Thu, Jan 7, 2021 at 5:42 AM Istvan Toth <st...@cloudera.com> wrote:
> Hi!
>
> Thanks for everyone's effort on fixing the flakey tests.
> However, our ASF Jenkins runs are still very unstable, and I am afraid
> that the single clean 4.x run was down more to luck than to fixing the
> underlying problem.
> While I do not consider this a blocker, fixing this would make the pre-
> and post commit tests far more useful, and let us start with a clean slate
> for the next release.
> I have created https://issues.apache.org/jira/browse/PHOENIX-6288 to
> track the generic cluster setup issue, but my attempts to fix this, or at
> least figure out the root cause have been unsuccessful so far.
>
> I ask for your help in fixing this issue. I have added some inconclusive
> analysis to the ticket, and the full failsafe output directory is
> downloadable as artifacts from the failed multibranch runs (stored for a
> few days), which should hold the answer to this riddle.
>
> regards,
> Istvan
>
> On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan <ya...@apache.org> wrote:
>
>> Hi,
>>
>> We finally have a stable 4.x branch build after many flapper test fixing
>> contributions from the community. I'm going to fork a new branch(4.16)
>> from
>> the current 4.x branch for the 4.16 release. As a cutoff, I will not
>> include any new features other than PHOENIX-6211 and bug fix. Please let
>> me
>> know if you have any questions or concerns.
>>
>> Thanks,
>> Xinyi
>>
>> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani <vj...@apache.org> wrote:
>>
>> > An update on test stability:
>> >
>> > As per recent 4.x build results, we are left with very few flappers and
>> > specifically
>> > with HBase profile 1.6, I can see recent 7 build (multibranch + PR
>> > precommit results)
>> > results without any test failure.
>> >
>> >
>> > On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <ya...@apache.org> wrote:
>> >
>> > > Yes, we are currently working on fixing the 4.x branch test flappers
>> > while
>> > > waiting for the PHOENIX-5435. After that, I will try to have an RC
>> ASAP.
>> > >
>> > > On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <
>> ankitsinghal59@gmail.com>
>> > > wrote:
>> > >
>> > > > I see that both the blockers listed here PHOENIX-5712 and
>> PHOENIX-6241
>> > > > have been resolved(Thanks to Xinyi), and also as per the JIRA query
>> > there
>> > > > is no Jira marked as a blocker for 4.16 except the one related to
>> > > > documentation
>> > > > for "splittable catalog table".
>> > > >
>> > > > Xinyi, so are we good to start the release process now?
>> > > >
>> > > > On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <ya...@apache.org>
>> wrote:
>> > > >
>> > > > > Thanks for replying and providing suggestions. I looked at the
>> wrong
>> > > > result
>> > > > > Jira list that Daniel provided and did some local testing, and
>> here
>> > is
>> > > > the
>> > > > > result:
>> > > > > [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642: cannot
>> > > reproduce
>> > > > > it.
>> > > > > [More information is required] PHOENIX-4504 cannot reproduce it
>> but
>> > > > someone
>> > > > > claimed that he had a similar issue.
>> > > > > [unusual query] PHOENIX-4540 and PHOENIX-6217
>> > > > >
>> > > > > Based on my finding, I think it's better to have more frequent
>> > > > housekeeping
>> > > > > and resolve unreproducible bugs especially since many of them are
>> > > > > considering out of date (phoenix-4.11 or even phoenix-4.6). Since
>> I
>> > > still
>> > > > > need time to work on the blocker Jira(PHOENIX-5712, PHOENIX-6241)
>> and
>> > > fix
>> > > > > test flappers, if you want to fix "unusual query" bugs, feel free
>> to
>> > do
>> > > > so.
>> > > > >
>> > > > >
>> > > > > Sincerely,
>> > > > > Xinyi
>> > > > >
>> > > > > On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <an...@apache.org>
>> > > wrote:
>> > > > >
>> > > > > > Thanks Daniel and appreciate the effort you put in getting the
>> list
>> > > > ready
>> > > > > > for bugs producing wrong results
>> > > > > > but none of them seems to be a blocker to me for 4.16 as they
>> are
>> > not
>> > > > the
>> > > > > > regression and doesn't break the general functionality
>> > > > > > except for specific features, RVC/desc as Chenglei also pointed
>> out
>> > > > > (though
>> > > > > > I'll defer the assessment to RM "Xinyi").
>> > > > > > Probably these can be a part of 4.16.1 or we can do 4.17.0 soon
>> > maybe
>> > > > > after
>> > > > > > a few weeks/month?
>> > > > > >
>> > > > > > Considering that we have already fixed 137 bugs and done 85+
>> > > > > > improvements/features in 4.16,
>> > > > > > it will not be a good idea to deprive the user from such fixes.
>> > > > > > It's been a year since our last 4.15 release, having no release
>> > > brings
>> > > > > more
>> > > > > > questions on the project
>> > > > > > rather than the bugs which affect a certain % of feature/users,
>> > would
>> > > > the
>> > > > > > release notes
>> > > > > > explaining the stability of certain features set the right
>> > > expectation
>> > > > > for
>> > > > > > those users who rely on these features to wait for a future
>> > release?
>> > > > > >
>> > > > > > Regards,
>> > > > > > Ankit Singhal
>> > > > > >
>> > > > > > On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
>> > > > chenglei@apache.org>
>> > > > > > wrote:
>> > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > In my opinion, we should keep releases light and frequent,
>> and
>> > for
>> > > > > some
>> > > > > > > unusual query bugs like RVC and DESC
>> > > > > > > we could delay fix to next release . I think we should release
>> > > 4.16.0
>> > > > > and
>> > > > > > > 5.1.0 as quickly as possible. In China, many users
>> > > > > > > in HBase&Phoenix User Group thought that Phoenix was dead
>> > because
>> > > > our
>> > > > > > too
>> > > > > > > long interval release and stopped using
>> > > > > > > Phoenix.
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
>> > > > chinmayskulkarni@gmail.com
>> > > > > >
>> > > > > > > wrote:
>> > > > > > > >I agree. These are all major bugs and we should aim at
>> solving
>> > > them
>> > > > > > after
>> > > > > > > >checking that they are still issues. I am +1 on 5833 and I
>> think
>> > > > 5484
>> > > > > > > would
>> > > > > > > >be a great addition to 4.16 as well. We should aim at
>> resolving
>> > > high
>> > > > > > > >priority bugs like this in every release.
>> > > > > > > >
>> > > > > > > >Sometimes we let these bugs slip without a resolution before
>> a
>> > > > > release,
>> > > > > > > >citing that these are "known issues" or "not regressions from
>> > the
>> > > > last
>> > > > > > > >release". In some cases this may be fine since we want to
>> keep
>> > > > > releases
>> > > > > > > >light and frequent, but perhaps we can track such issues and
>> aim
>> > > at
>> > > > > > > >reducing the number of bugs by x% in each release? This will
>> > also
>> > > > keep
>> > > > > > old
>> > > > > > > >Jiras alive since we will potentially periodically review
>> them.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <
>> > > gjacoby@apache.org>
>> > > > > > > wrote:
>> > > > > > > >
>> > > > > > > >> I've got PHOENIX-5435 in review right now, and would like
>> to
>> > get
>> > > > it
>> > > > > in
>> > > > > > > 4.16
>> > > > > > > >> / 5.1.
>> > > > > > > >>
>> > > > > > > >> It's allowing the annotation of Phoenix metadata into HBase
>> > WALs
>> > > > as
>> > > > > a
>> > > > > > > >> pre-req for the Phoenix Change Detection Capture framework
>> > > > > > > (PHOENIX-5442).
>> > > > > > > >> Since it has both client/server logic, and adds a field to
>> > > > > > > System.Catalog,
>> > > > > > > >> it can't go in a patch release.
>> > > > > > > >>
>> > > > > > > >> Depending on timing, I'd _like_ to get PHOENIX-6227, which
>> is
>> > > the
>> > > > > last
>> > > > > > > part
>> > > > > > > >> of CDC that will go into core Phoenix, into 4.16, but since
>> > that
>> > > > > _can_
>> > > > > > > go
>> > > > > > > >> in a patch release and I haven't started it yet, if the
>> > release
>> > > > gets
>> > > > > > cut
>> > > > > > > >> before it's ready, no big deal. (The rest of CDC will go
>> into
>> > > > > > > >> phoenix-connectors for a future release of that project.)
>> > > > > > > >>
>> > > > > > > >> As for the correctness problems that Daniel points out, I
>> > think
>> > > we
>> > > > > > > should
>> > > > > > > >> fix the ones that were detected with a recent version
>> (4.14 or
>> > > > > 4.15?),
>> > > > > > > and
>> > > > > > > >> test to see which of the older ones can still be
>> reproduced.
>> > > Once
>> > > > we
>> > > > > > > know
>> > > > > > > >> which bugs are real and which are just historical, we can
>> > better
>> > > > > judge
>> > > > > > > >> scope. And hopefully close a bunch of obsolete bugs.
>> (Thanks,
>> > > > > Daniel,
>> > > > > > > for
>> > > > > > > >> collecting that list!)
>> > > > > > > >>
>> > > > > > > >> Geoffrey
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
>> > > > > > > >> <da...@salesforce.com.invalid> wrote:
>> > > > > > > >>
>> > > > > > > >> > Hi, I wanted to bring up wrong results in Phoenix and
>> some
>> > > JIRAs
>> > > > > > > around
>> > > > > > > >> > them that I think we should fix as the wrong result
>> lessens
>> > > the
>> > > > > end
>> > > > > > > >> user's
>> > > > > > > >> > trust in Phoenix. Releasing a new version without
>> > addressing
>> > > > > these
>> > > > > > > in a
>> > > > > > > >> > minor release hurts our visibility in that these critical
>> > > issues
>> > > > > are
>> > > > > > > not
>> > > > > > > >> > addressed.
>> > > > > > > >> >
>> > > > > > > >> > Jira's that I'm involved with for example: I've already
>> > given
>> > > a
>> > > > > > patch
>> > > > > > > >> > several months ago for 5833 and there is a chance it may
>> fix
>> > > > 5484.
>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5833
>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5484
>> > > > > > > >> >
>> > > > > > > >> > In addition, inspecting apache JIRA i see several other
>> > wrong
>> > > > > result
>> > > > > > > >> JIRAs
>> > > > > > > >> > from the community. Some of these certainly are probably
>> > old
>> > > > > issues
>> > > > > > > or
>> > > > > > > >> > incorrect understanding but some of these are opened by
>> our
>> > > own
>> > > > > dev
>> > > > > > > >> > community and are likely real problems.
>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-6217
>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5571
>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4642
>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4540
>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4504
>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4419
>> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4116
>> > > > > > > >> >
>> > > > > > > >> > What is our stance on this type of issue? Are we going
>> to
>> > say
>> > > > > these
>> > > > > > > were
>> > > > > > > >> > issues prior to 4.15 and not address them? Should we
>> have
>> > > > > > > requirements
>> > > > > > > >> for
>> > > > > > > >> > our releases to fix wrong results?
>> > > > > > > >> >
>> > > > > > > >> > Daniel Wong
>> > > > > > > >> >
>> > > > > > > >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <
>> > > yanxinyi@apache.org>
>> > > > > > > wrote:
>> > > > > > > >> >
>> > > > > > > >> > > Hi all,
>> > > > > > > >> > >
>> > > > > > > >> > > It's time to discuss the Phoenix 4.16 release. After
>> many
>> > > > > people's
>> > > > > > > >> > > contributions on the bug fixes, new features, and other
>> > > works
>> > > > in
>> > > > > > the
>> > > > > > > >> past
>> > > > > > > >> > > few months, we are kind of close to the point to have
>> a RC
>> > > > > (still
>> > > > > > > need
>> > > > > > > >> to
>> > > > > > > >> > > fix test flappers). Please let me know if you think any
>> > JIRA
>> > > > > must
>> > > > > > be
>> > > > > > > >> part
>> > > > > > > >> > > of the Phoenix 4.16 release other than major blocker
>> > > > > PHOENIX-5712.
>> > > > > > > >> > >
>> > > > > > > >> > > If no surprise comes up, I will not wait for any new
>> major
>> > > > > > features
>> > > > > > > and
>> > > > > > > >> > > focus on the RC as soon as possible.
>> > > > > > > >> > >
>> > > > > > > >> > > Sincerely,
>> > > > > > > >> > > Xinyi
>> > > > > > > >> > >
>> > > > > > > >> >
>> > > > > > > >> >
>> > > > > > > >> > --
>> > > > > > > >> > Daniel Wong
>> > > > > > > >> > Salesforce
>> > > > > > > >> > Mobile: 628.217.1808
>> > > > > > > >> >
>> > > > > > > >>
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >--
>> > > > > > > >Chinmay Kulkarni
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
> --
> *István Tóth* | Staff Software Engineer
> stoty@cloudera.com <https://www.cloudera.com>
> [image: Cloudera] <https://www.cloudera.com/>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> <https://www.cloudera.com/>
> ------------------------------
>
Re: [Discuss] Releasing Phoenix 4.16
Posted by Istvan Toth <st...@cloudera.com.INVALID>.
Hi!
Thanks for everyone's effort on fixing the flakey tests.
However, our ASF Jenkins runs are still very unstable, and I am afraid that
the single clean 4.x run was down more to luck than to fixing the
underlying problem.
While I do not consider this a blocker, fixing this would make the pre- and
post commit tests far more useful, and let us start with a clean slate for
the next release.
I have created https://issues.apache.org/jira/browse/PHOENIX-6288 to track
the generic cluster setup issue, but my attempts to fix this, or at least
figure out the root cause have been unsuccessful so far.
I ask for your help in fixing this issue. I have added some inconclusive
analysis to the ticket, and the full failsafe output directory is
downloadable as artifacts from the failed multibranch runs (stored for a
few days), which should hold the answer to this riddle.
regards,
Istvan
On Thu, Jan 7, 2021 at 4:58 AM Xinyi Yan <ya...@apache.org> wrote:
> Hi,
>
> We finally have a stable 4.x branch build after many flapper test fixing
> contributions from the community. I'm going to fork a new branch(4.16) from
> the current 4.x branch for the 4.16 release. As a cutoff, I will not
> include any new features other than PHOENIX-6211 and bug fix. Please let me
> know if you have any questions or concerns.
>
> Thanks,
> Xinyi
>
> On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani <vj...@apache.org> wrote:
>
> > An update on test stability:
> >
> > As per recent 4.x build results, we are left with very few flappers and
> > specifically
> > with HBase profile 1.6, I can see recent 7 build (multibranch + PR
> > precommit results)
> > results without any test failure.
> >
> >
> > On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <ya...@apache.org> wrote:
> >
> > > Yes, we are currently working on fixing the 4.x branch test flappers
> > while
> > > waiting for the PHOENIX-5435. After that, I will try to have an RC
> ASAP.
> > >
> > > On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <
> ankitsinghal59@gmail.com>
> > > wrote:
> > >
> > > > I see that both the blockers listed here PHOENIX-5712 and
> PHOENIX-6241
> > > > have been resolved(Thanks to Xinyi), and also as per the JIRA query
> > there
> > > > is no Jira marked as a blocker for 4.16 except the one related to
> > > > documentation
> > > > for "splittable catalog table".
> > > >
> > > > Xinyi, so are we good to start the release process now?
> > > >
> > > > On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <ya...@apache.org>
> wrote:
> > > >
> > > > > Thanks for replying and providing suggestions. I looked at the
> wrong
> > > > result
> > > > > Jira list that Daniel provided and did some local testing, and here
> > is
> > > > the
> > > > > result:
> > > > > [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642: cannot
> > > reproduce
> > > > > it.
> > > > > [More information is required] PHOENIX-4504 cannot reproduce it but
> > > > someone
> > > > > claimed that he had a similar issue.
> > > > > [unusual query] PHOENIX-4540 and PHOENIX-6217
> > > > >
> > > > > Based on my finding, I think it's better to have more frequent
> > > > housekeeping
> > > > > and resolve unreproducible bugs especially since many of them are
> > > > > considering out of date (phoenix-4.11 or even phoenix-4.6). Since I
> > > still
> > > > > need time to work on the blocker Jira(PHOENIX-5712, PHOENIX-6241)
> and
> > > fix
> > > > > test flappers, if you want to fix "unusual query" bugs, feel free
> to
> > do
> > > > so.
> > > > >
> > > > >
> > > > > Sincerely,
> > > > > Xinyi
> > > > >
> > > > > On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <an...@apache.org>
> > > wrote:
> > > > >
> > > > > > Thanks Daniel and appreciate the effort you put in getting the
> list
> > > > ready
> > > > > > for bugs producing wrong results
> > > > > > but none of them seems to be a blocker to me for 4.16 as they are
> > not
> > > > the
> > > > > > regression and doesn't break the general functionality
> > > > > > except for specific features, RVC/desc as Chenglei also pointed
> out
> > > > > (though
> > > > > > I'll defer the assessment to RM "Xinyi").
> > > > > > Probably these can be a part of 4.16.1 or we can do 4.17.0 soon
> > maybe
> > > > > after
> > > > > > a few weeks/month?
> > > > > >
> > > > > > Considering that we have already fixed 137 bugs and done 85+
> > > > > > improvements/features in 4.16,
> > > > > > it will not be a good idea to deprive the user from such fixes.
> > > > > > It's been a year since our last 4.15 release, having no release
> > > brings
> > > > > more
> > > > > > questions on the project
> > > > > > rather than the bugs which affect a certain % of feature/users,
> > would
> > > > the
> > > > > > release notes
> > > > > > explaining the stability of certain features set the right
> > > expectation
> > > > > for
> > > > > > those users who rely on these features to wait for a future
> > release?
> > > > > >
> > > > > > Regards,
> > > > > > Ankit Singhal
> > > > > >
> > > > > > On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
> > > > chenglei@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > In my opinion, we should keep releases light and frequent, and
> > for
> > > > > some
> > > > > > > unusual query bugs like RVC and DESC
> > > > > > > we could delay fix to next release . I think we should release
> > > 4.16.0
> > > > > and
> > > > > > > 5.1.0 as quickly as possible. In China, many users
> > > > > > > in HBase&Phoenix User Group thought that Phoenix was dead
> > because
> > > > our
> > > > > > too
> > > > > > > long interval release and stopped using
> > > > > > > Phoenix.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
> > > > chinmayskulkarni@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > > >I agree. These are all major bugs and we should aim at solving
> > > them
> > > > > > after
> > > > > > > >checking that they are still issues. I am +1 on 5833 and I
> think
> > > > 5484
> > > > > > > would
> > > > > > > >be a great addition to 4.16 as well. We should aim at
> resolving
> > > high
> > > > > > > >priority bugs like this in every release.
> > > > > > > >
> > > > > > > >Sometimes we let these bugs slip without a resolution before a
> > > > > release,
> > > > > > > >citing that these are "known issues" or "not regressions from
> > the
> > > > last
> > > > > > > >release". In some cases this may be fine since we want to keep
> > > > > releases
> > > > > > > >light and frequent, but perhaps we can track such issues and
> aim
> > > at
> > > > > > > >reducing the number of bugs by x% in each release? This will
> > also
> > > > keep
> > > > > > old
> > > > > > > >Jiras alive since we will potentially periodically review
> them.
> > > > > > > >
> > > > > > > >
> > > > > > > >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <
> > > gjacoby@apache.org>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > >> I've got PHOENIX-5435 in review right now, and would like to
> > get
> > > > it
> > > > > in
> > > > > > > 4.16
> > > > > > > >> / 5.1.
> > > > > > > >>
> > > > > > > >> It's allowing the annotation of Phoenix metadata into HBase
> > WALs
> > > > as
> > > > > a
> > > > > > > >> pre-req for the Phoenix Change Detection Capture framework
> > > > > > > (PHOENIX-5442).
> > > > > > > >> Since it has both client/server logic, and adds a field to
> > > > > > > System.Catalog,
> > > > > > > >> it can't go in a patch release.
> > > > > > > >>
> > > > > > > >> Depending on timing, I'd _like_ to get PHOENIX-6227, which
> is
> > > the
> > > > > last
> > > > > > > part
> > > > > > > >> of CDC that will go into core Phoenix, into 4.16, but since
> > that
> > > > > _can_
> > > > > > > go
> > > > > > > >> in a patch release and I haven't started it yet, if the
> > release
> > > > gets
> > > > > > cut
> > > > > > > >> before it's ready, no big deal. (The rest of CDC will go
> into
> > > > > > > >> phoenix-connectors for a future release of that project.)
> > > > > > > >>
> > > > > > > >> As for the correctness problems that Daniel points out, I
> > think
> > > we
> > > > > > > should
> > > > > > > >> fix the ones that were detected with a recent version (4.14
> or
> > > > > 4.15?),
> > > > > > > and
> > > > > > > >> test to see which of the older ones can still be reproduced.
> > > Once
> > > > we
> > > > > > > know
> > > > > > > >> which bugs are real and which are just historical, we can
> > better
> > > > > judge
> > > > > > > >> scope. And hopefully close a bunch of obsolete bugs.
> (Thanks,
> > > > > Daniel,
> > > > > > > for
> > > > > > > >> collecting that list!)
> > > > > > > >>
> > > > > > > >> Geoffrey
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
> > > > > > > >> <da...@salesforce.com.invalid> wrote:
> > > > > > > >>
> > > > > > > >> > Hi, I wanted to bring up wrong results in Phoenix and some
> > > JIRAs
> > > > > > > around
> > > > > > > >> > them that I think we should fix as the wrong result
> lessens
> > > the
> > > > > end
> > > > > > > >> user's
> > > > > > > >> > trust in Phoenix. Releasing a new version without
> > addressing
> > > > > these
> > > > > > > in a
> > > > > > > >> > minor release hurts our visibility in that these critical
> > > issues
> > > > > are
> > > > > > > not
> > > > > > > >> > addressed.
> > > > > > > >> >
> > > > > > > >> > Jira's that I'm involved with for example: I've already
> > given
> > > a
> > > > > > patch
> > > > > > > >> > several months ago for 5833 and there is a chance it may
> fix
> > > > 5484.
> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5833
> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5484
> > > > > > > >> >
> > > > > > > >> > In addition, inspecting apache JIRA i see several other
> > wrong
> > > > > result
> > > > > > > >> JIRAs
> > > > > > > >> > from the community. Some of these certainly are probably
> > old
> > > > > issues
> > > > > > > or
> > > > > > > >> > incorrect understanding but some of these are opened by
> our
> > > own
> > > > > dev
> > > > > > > >> > community and are likely real problems.
> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-6217
> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5571
> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4642
> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4540
> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4504
> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4419
> > > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4116
> > > > > > > >> >
> > > > > > > >> > What is our stance on this type of issue? Are we going to
> > say
> > > > > these
> > > > > > > were
> > > > > > > >> > issues prior to 4.15 and not address them? Should we have
> > > > > > > requirements
> > > > > > > >> for
> > > > > > > >> > our releases to fix wrong results?
> > > > > > > >> >
> > > > > > > >> > Daniel Wong
> > > > > > > >> >
> > > > > > > >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <
> > > yanxinyi@apache.org>
> > > > > > > wrote:
> > > > > > > >> >
> > > > > > > >> > > Hi all,
> > > > > > > >> > >
> > > > > > > >> > > It's time to discuss the Phoenix 4.16 release. After
> many
> > > > > people's
> > > > > > > >> > > contributions on the bug fixes, new features, and other
> > > works
> > > > in
> > > > > > the
> > > > > > > >> past
> > > > > > > >> > > few months, we are kind of close to the point to have a
> RC
> > > > > (still
> > > > > > > need
> > > > > > > >> to
> > > > > > > >> > > fix test flappers). Please let me know if you think any
> > JIRA
> > > > > must
> > > > > > be
> > > > > > > >> part
> > > > > > > >> > > of the Phoenix 4.16 release other than major blocker
> > > > > PHOENIX-5712.
> > > > > > > >> > >
> > > > > > > >> > > If no surprise comes up, I will not wait for any new
> major
> > > > > > features
> > > > > > > and
> > > > > > > >> > > focus on the RC as soon as possible.
> > > > > > > >> > >
> > > > > > > >> > > Sincerely,
> > > > > > > >> > > Xinyi
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> > --
> > > > > > > >> > Daniel Wong
> > > > > > > >> > Salesforce
> > > > > > > >> > Mobile: 628.217.1808
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > > >--
> > > > > > > >Chinmay Kulkarni
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--
*István Tóth* | Staff Software Engineer
stoty@cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
<https://www.cloudera.com/>
------------------------------
Re: [Discuss] Releasing Phoenix 4.16
Posted by Xinyi Yan <ya...@apache.org>.
Hi,
We finally have a stable 4.x branch build after many flapper test fixing
contributions from the community. I'm going to fork a new branch(4.16) from
the current 4.x branch for the 4.16 release. As a cutoff, I will not
include any new features other than PHOENIX-6211 and bug fix. Please let me
know if you have any questions or concerns.
Thanks,
Xinyi
On Wed, Dec 23, 2020 at 6:07 AM Viraj Jasani <vj...@apache.org> wrote:
> An update on test stability:
>
> As per recent 4.x build results, we are left with very few flappers and
> specifically
> with HBase profile 1.6, I can see recent 7 build (multibranch + PR
> precommit results)
> results without any test failure.
>
>
> On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <ya...@apache.org> wrote:
>
> > Yes, we are currently working on fixing the 4.x branch test flappers
> while
> > waiting for the PHOENIX-5435. After that, I will try to have an RC ASAP.
> >
> > On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <an...@gmail.com>
> > wrote:
> >
> > > I see that both the blockers listed here PHOENIX-5712 and PHOENIX-6241
> > > have been resolved(Thanks to Xinyi), and also as per the JIRA query
> there
> > > is no Jira marked as a blocker for 4.16 except the one related to
> > > documentation
> > > for "splittable catalog table".
> > >
> > > Xinyi, so are we good to start the release process now?
> > >
> > > On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <ya...@apache.org> wrote:
> > >
> > > > Thanks for replying and providing suggestions. I looked at the wrong
> > > result
> > > > Jira list that Daniel provided and did some local testing, and here
> is
> > > the
> > > > result:
> > > > [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642: cannot
> > reproduce
> > > > it.
> > > > [More information is required] PHOENIX-4504 cannot reproduce it but
> > > someone
> > > > claimed that he had a similar issue.
> > > > [unusual query] PHOENIX-4540 and PHOENIX-6217
> > > >
> > > > Based on my finding, I think it's better to have more frequent
> > > housekeeping
> > > > and resolve unreproducible bugs especially since many of them are
> > > > considering out of date (phoenix-4.11 or even phoenix-4.6). Since I
> > still
> > > > need time to work on the blocker Jira(PHOENIX-5712, PHOENIX-6241) and
> > fix
> > > > test flappers, if you want to fix "unusual query" bugs, feel free to
> do
> > > so.
> > > >
> > > >
> > > > Sincerely,
> > > > Xinyi
> > > >
> > > > On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <an...@apache.org>
> > wrote:
> > > >
> > > > > Thanks Daniel and appreciate the effort you put in getting the list
> > > ready
> > > > > for bugs producing wrong results
> > > > > but none of them seems to be a blocker to me for 4.16 as they are
> not
> > > the
> > > > > regression and doesn't break the general functionality
> > > > > except for specific features, RVC/desc as Chenglei also pointed out
> > > > (though
> > > > > I'll defer the assessment to RM "Xinyi").
> > > > > Probably these can be a part of 4.16.1 or we can do 4.17.0 soon
> maybe
> > > > after
> > > > > a few weeks/month?
> > > > >
> > > > > Considering that we have already fixed 137 bugs and done 85+
> > > > > improvements/features in 4.16,
> > > > > it will not be a good idea to deprive the user from such fixes.
> > > > > It's been a year since our last 4.15 release, having no release
> > brings
> > > > more
> > > > > questions on the project
> > > > > rather than the bugs which affect a certain % of feature/users,
> would
> > > the
> > > > > release notes
> > > > > explaining the stability of certain features set the right
> > expectation
> > > > for
> > > > > those users who rely on these features to wait for a future
> release?
> > > > >
> > > > > Regards,
> > > > > Ankit Singhal
> > > > >
> > > > > On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
> > > chenglei@apache.org>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > In my opinion, we should keep releases light and frequent, and
> for
> > > > some
> > > > > > unusual query bugs like RVC and DESC
> > > > > > we could delay fix to next release . I think we should release
> > 4.16.0
> > > > and
> > > > > > 5.1.0 as quickly as possible. In China, many users
> > > > > > in HBase&Phoenix User Group thought that Phoenix was dead
> because
> > > our
> > > > > too
> > > > > > long interval release and stopped using
> > > > > > Phoenix.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
> > > chinmayskulkarni@gmail.com
> > > > >
> > > > > > wrote:
> > > > > > >I agree. These are all major bugs and we should aim at solving
> > them
> > > > > after
> > > > > > >checking that they are still issues. I am +1 on 5833 and I think
> > > 5484
> > > > > > would
> > > > > > >be a great addition to 4.16 as well. We should aim at resolving
> > high
> > > > > > >priority bugs like this in every release.
> > > > > > >
> > > > > > >Sometimes we let these bugs slip without a resolution before a
> > > > release,
> > > > > > >citing that these are "known issues" or "not regressions from
> the
> > > last
> > > > > > >release". In some cases this may be fine since we want to keep
> > > > releases
> > > > > > >light and frequent, but perhaps we can track such issues and aim
> > at
> > > > > > >reducing the number of bugs by x% in each release? This will
> also
> > > keep
> > > > > old
> > > > > > >Jiras alive since we will potentially periodically review them.
> > > > > > >
> > > > > > >
> > > > > > >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <
> > gjacoby@apache.org>
> > > > > > wrote:
> > > > > > >
> > > > > > >> I've got PHOENIX-5435 in review right now, and would like to
> get
> > > it
> > > > in
> > > > > > 4.16
> > > > > > >> / 5.1.
> > > > > > >>
> > > > > > >> It's allowing the annotation of Phoenix metadata into HBase
> WALs
> > > as
> > > > a
> > > > > > >> pre-req for the Phoenix Change Detection Capture framework
> > > > > > (PHOENIX-5442).
> > > > > > >> Since it has both client/server logic, and adds a field to
> > > > > > System.Catalog,
> > > > > > >> it can't go in a patch release.
> > > > > > >>
> > > > > > >> Depending on timing, I'd _like_ to get PHOENIX-6227, which is
> > the
> > > > last
> > > > > > part
> > > > > > >> of CDC that will go into core Phoenix, into 4.16, but since
> that
> > > > _can_
> > > > > > go
> > > > > > >> in a patch release and I haven't started it yet, if the
> release
> > > gets
> > > > > cut
> > > > > > >> before it's ready, no big deal. (The rest of CDC will go into
> > > > > > >> phoenix-connectors for a future release of that project.)
> > > > > > >>
> > > > > > >> As for the correctness problems that Daniel points out, I
> think
> > we
> > > > > > should
> > > > > > >> fix the ones that were detected with a recent version (4.14 or
> > > > 4.15?),
> > > > > > and
> > > > > > >> test to see which of the older ones can still be reproduced.
> > Once
> > > we
> > > > > > know
> > > > > > >> which bugs are real and which are just historical, we can
> better
> > > > judge
> > > > > > >> scope. And hopefully close a bunch of obsolete bugs. (Thanks,
> > > > Daniel,
> > > > > > for
> > > > > > >> collecting that list!)
> > > > > > >>
> > > > > > >> Geoffrey
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
> > > > > > >> <da...@salesforce.com.invalid> wrote:
> > > > > > >>
> > > > > > >> > Hi, I wanted to bring up wrong results in Phoenix and some
> > JIRAs
> > > > > > around
> > > > > > >> > them that I think we should fix as the wrong result lessens
> > the
> > > > end
> > > > > > >> user's
> > > > > > >> > trust in Phoenix. Releasing a new version without
> addressing
> > > > these
> > > > > > in a
> > > > > > >> > minor release hurts our visibility in that these critical
> > issues
> > > > are
> > > > > > not
> > > > > > >> > addressed.
> > > > > > >> >
> > > > > > >> > Jira's that I'm involved with for example: I've already
> given
> > a
> > > > > patch
> > > > > > >> > several months ago for 5833 and there is a chance it may fix
> > > 5484.
> > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5833
> > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5484
> > > > > > >> >
> > > > > > >> > In addition, inspecting apache JIRA i see several other
> wrong
> > > > result
> > > > > > >> JIRAs
> > > > > > >> > from the community. Some of these certainly are probably
> old
> > > > issues
> > > > > > or
> > > > > > >> > incorrect understanding but some of these are opened by our
> > own
> > > > dev
> > > > > > >> > community and are likely real problems.
> > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-6217
> > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5571
> > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4642
> > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4540
> > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4504
> > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4419
> > > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4116
> > > > > > >> >
> > > > > > >> > What is our stance on this type of issue? Are we going to
> say
> > > > these
> > > > > > were
> > > > > > >> > issues prior to 4.15 and not address them? Should we have
> > > > > > requirements
> > > > > > >> for
> > > > > > >> > our releases to fix wrong results?
> > > > > > >> >
> > > > > > >> > Daniel Wong
> > > > > > >> >
> > > > > > >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <
> > yanxinyi@apache.org>
> > > > > > wrote:
> > > > > > >> >
> > > > > > >> > > Hi all,
> > > > > > >> > >
> > > > > > >> > > It's time to discuss the Phoenix 4.16 release. After many
> > > > people's
> > > > > > >> > > contributions on the bug fixes, new features, and other
> > works
> > > in
> > > > > the
> > > > > > >> past
> > > > > > >> > > few months, we are kind of close to the point to have a RC
> > > > (still
> > > > > > need
> > > > > > >> to
> > > > > > >> > > fix test flappers). Please let me know if you think any
> JIRA
> > > > must
> > > > > be
> > > > > > >> part
> > > > > > >> > > of the Phoenix 4.16 release other than major blocker
> > > > PHOENIX-5712.
> > > > > > >> > >
> > > > > > >> > > If no surprise comes up, I will not wait for any new major
> > > > > features
> > > > > > and
> > > > > > >> > > focus on the RC as soon as possible.
> > > > > > >> > >
> > > > > > >> > > Sincerely,
> > > > > > >> > > Xinyi
> > > > > > >> > >
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > --
> > > > > > >> > Daniel Wong
> > > > > > >> > Salesforce
> > > > > > >> > Mobile: 628.217.1808
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > >--
> > > > > > >Chinmay Kulkarni
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [Discuss] Releasing Phoenix 4.16
Posted by Viraj Jasani <vj...@apache.org>.
An update on test stability:
As per recent 4.x build results, we are left with very few flappers and
specifically
with HBase profile 1.6, I can see recent 7 build (multibranch + PR
precommit results)
results without any test failure.
On Tue, Dec 15, 2020 at 5:52 AM Xinyi Yan <ya...@apache.org> wrote:
> Yes, we are currently working on fixing the 4.x branch test flappers while
> waiting for the PHOENIX-5435. After that, I will try to have an RC ASAP.
>
> On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <an...@gmail.com>
> wrote:
>
> > I see that both the blockers listed here PHOENIX-5712 and PHOENIX-6241
> > have been resolved(Thanks to Xinyi), and also as per the JIRA query there
> > is no Jira marked as a blocker for 4.16 except the one related to
> > documentation
> > for "splittable catalog table".
> >
> > Xinyi, so are we good to start the release process now?
> >
> > On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <ya...@apache.org> wrote:
> >
> > > Thanks for replying and providing suggestions. I looked at the wrong
> > result
> > > Jira list that Daniel provided and did some local testing, and here is
> > the
> > > result:
> > > [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642: cannot
> reproduce
> > > it.
> > > [More information is required] PHOENIX-4504 cannot reproduce it but
> > someone
> > > claimed that he had a similar issue.
> > > [unusual query] PHOENIX-4540 and PHOENIX-6217
> > >
> > > Based on my finding, I think it's better to have more frequent
> > housekeeping
> > > and resolve unreproducible bugs especially since many of them are
> > > considering out of date (phoenix-4.11 or even phoenix-4.6). Since I
> still
> > > need time to work on the blocker Jira(PHOENIX-5712, PHOENIX-6241) and
> fix
> > > test flappers, if you want to fix "unusual query" bugs, feel free to do
> > so.
> > >
> > >
> > > Sincerely,
> > > Xinyi
> > >
> > > On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <an...@apache.org>
> wrote:
> > >
> > > > Thanks Daniel and appreciate the effort you put in getting the list
> > ready
> > > > for bugs producing wrong results
> > > > but none of them seems to be a blocker to me for 4.16 as they are not
> > the
> > > > regression and doesn't break the general functionality
> > > > except for specific features, RVC/desc as Chenglei also pointed out
> > > (though
> > > > I'll defer the assessment to RM "Xinyi").
> > > > Probably these can be a part of 4.16.1 or we can do 4.17.0 soon maybe
> > > after
> > > > a few weeks/month?
> > > >
> > > > Considering that we have already fixed 137 bugs and done 85+
> > > > improvements/features in 4.16,
> > > > it will not be a good idea to deprive the user from such fixes.
> > > > It's been a year since our last 4.15 release, having no release
> brings
> > > more
> > > > questions on the project
> > > > rather than the bugs which affect a certain % of feature/users, would
> > the
> > > > release notes
> > > > explaining the stability of certain features set the right
> expectation
> > > for
> > > > those users who rely on these features to wait for a future release?
> > > >
> > > > Regards,
> > > > Ankit Singhal
> > > >
> > > > On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
> > chenglei@apache.org>
> > > > wrote:
> > > >
> > > > >
> > > > >
> > > > >
> > > > > In my opinion, we should keep releases light and frequent, and for
> > > some
> > > > > unusual query bugs like RVC and DESC
> > > > > we could delay fix to next release . I think we should release
> 4.16.0
> > > and
> > > > > 5.1.0 as quickly as possible. In China, many users
> > > > > in HBase&Phoenix User Group thought that Phoenix was dead because
> > our
> > > > too
> > > > > long interval release and stopped using
> > > > > Phoenix.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
> > chinmayskulkarni@gmail.com
> > > >
> > > > > wrote:
> > > > > >I agree. These are all major bugs and we should aim at solving
> them
> > > > after
> > > > > >checking that they are still issues. I am +1 on 5833 and I think
> > 5484
> > > > > would
> > > > > >be a great addition to 4.16 as well. We should aim at resolving
> high
> > > > > >priority bugs like this in every release.
> > > > > >
> > > > > >Sometimes we let these bugs slip without a resolution before a
> > > release,
> > > > > >citing that these are "known issues" or "not regressions from the
> > last
> > > > > >release". In some cases this may be fine since we want to keep
> > > releases
> > > > > >light and frequent, but perhaps we can track such issues and aim
> at
> > > > > >reducing the number of bugs by x% in each release? This will also
> > keep
> > > > old
> > > > > >Jiras alive since we will potentially periodically review them.
> > > > > >
> > > > > >
> > > > > >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <
> gjacoby@apache.org>
> > > > > wrote:
> > > > > >
> > > > > >> I've got PHOENIX-5435 in review right now, and would like to get
> > it
> > > in
> > > > > 4.16
> > > > > >> / 5.1.
> > > > > >>
> > > > > >> It's allowing the annotation of Phoenix metadata into HBase WALs
> > as
> > > a
> > > > > >> pre-req for the Phoenix Change Detection Capture framework
> > > > > (PHOENIX-5442).
> > > > > >> Since it has both client/server logic, and adds a field to
> > > > > System.Catalog,
> > > > > >> it can't go in a patch release.
> > > > > >>
> > > > > >> Depending on timing, I'd _like_ to get PHOENIX-6227, which is
> the
> > > last
> > > > > part
> > > > > >> of CDC that will go into core Phoenix, into 4.16, but since that
> > > _can_
> > > > > go
> > > > > >> in a patch release and I haven't started it yet, if the release
> > gets
> > > > cut
> > > > > >> before it's ready, no big deal. (The rest of CDC will go into
> > > > > >> phoenix-connectors for a future release of that project.)
> > > > > >>
> > > > > >> As for the correctness problems that Daniel points out, I think
> we
> > > > > should
> > > > > >> fix the ones that were detected with a recent version (4.14 or
> > > 4.15?),
> > > > > and
> > > > > >> test to see which of the older ones can still be reproduced.
> Once
> > we
> > > > > know
> > > > > >> which bugs are real and which are just historical, we can better
> > > judge
> > > > > >> scope. And hopefully close a bunch of obsolete bugs. (Thanks,
> > > Daniel,
> > > > > for
> > > > > >> collecting that list!)
> > > > > >>
> > > > > >> Geoffrey
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
> > > > > >> <da...@salesforce.com.invalid> wrote:
> > > > > >>
> > > > > >> > Hi, I wanted to bring up wrong results in Phoenix and some
> JIRAs
> > > > > around
> > > > > >> > them that I think we should fix as the wrong result lessens
> the
> > > end
> > > > > >> user's
> > > > > >> > trust in Phoenix. Releasing a new version without addressing
> > > these
> > > > > in a
> > > > > >> > minor release hurts our visibility in that these critical
> issues
> > > are
> > > > > not
> > > > > >> > addressed.
> > > > > >> >
> > > > > >> > Jira's that I'm involved with for example: I've already given
> a
> > > > patch
> > > > > >> > several months ago for 5833 and there is a chance it may fix
> > 5484.
> > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5833
> > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5484
> > > > > >> >
> > > > > >> > In addition, inspecting apache JIRA i see several other wrong
> > > result
> > > > > >> JIRAs
> > > > > >> > from the community. Some of these certainly are probably old
> > > issues
> > > > > or
> > > > > >> > incorrect understanding but some of these are opened by our
> own
> > > dev
> > > > > >> > community and are likely real problems.
> > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-6217
> > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5571
> > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4642
> > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4540
> > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4504
> > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4419
> > > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4116
> > > > > >> >
> > > > > >> > What is our stance on this type of issue? Are we going to say
> > > these
> > > > > were
> > > > > >> > issues prior to 4.15 and not address them? Should we have
> > > > > requirements
> > > > > >> for
> > > > > >> > our releases to fix wrong results?
> > > > > >> >
> > > > > >> > Daniel Wong
> > > > > >> >
> > > > > >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <
> yanxinyi@apache.org>
> > > > > wrote:
> > > > > >> >
> > > > > >> > > Hi all,
> > > > > >> > >
> > > > > >> > > It's time to discuss the Phoenix 4.16 release. After many
> > > people's
> > > > > >> > > contributions on the bug fixes, new features, and other
> works
> > in
> > > > the
> > > > > >> past
> > > > > >> > > few months, we are kind of close to the point to have a RC
> > > (still
> > > > > need
> > > > > >> to
> > > > > >> > > fix test flappers). Please let me know if you think any JIRA
> > > must
> > > > be
> > > > > >> part
> > > > > >> > > of the Phoenix 4.16 release other than major blocker
> > > PHOENIX-5712.
> > > > > >> > >
> > > > > >> > > If no surprise comes up, I will not wait for any new major
> > > > features
> > > > > and
> > > > > >> > > focus on the RC as soon as possible.
> > > > > >> > >
> > > > > >> > > Sincerely,
> > > > > >> > > Xinyi
> > > > > >> > >
> > > > > >> >
> > > > > >> >
> > > > > >> > --
> > > > > >> > Daniel Wong
> > > > > >> > Salesforce
> > > > > >> > Mobile: 628.217.1808
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > > >--
> > > > > >Chinmay Kulkarni
> > > > >
> > > >
> > >
> >
>
Re: [Discuss] Releasing Phoenix 4.16
Posted by Xinyi Yan <ya...@apache.org>.
Yes, we are currently working on fixing the 4.x branch test flappers while
waiting for the PHOENIX-5435. After that, I will try to have an RC ASAP.
On Sun, Dec 13, 2020 at 3:29 PM Ankit Singhal <an...@gmail.com>
wrote:
> I see that both the blockers listed here PHOENIX-5712 and PHOENIX-6241
> have been resolved(Thanks to Xinyi), and also as per the JIRA query there
> is no Jira marked as a blocker for 4.16 except the one related to
> documentation
> for "splittable catalog table".
>
> Xinyi, so are we good to start the release process now?
>
> On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <ya...@apache.org> wrote:
>
> > Thanks for replying and providing suggestions. I looked at the wrong
> result
> > Jira list that Daniel provided and did some local testing, and here is
> the
> > result:
> > [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642: cannot reproduce
> > it.
> > [More information is required] PHOENIX-4504 cannot reproduce it but
> someone
> > claimed that he had a similar issue.
> > [unusual query] PHOENIX-4540 and PHOENIX-6217
> >
> > Based on my finding, I think it's better to have more frequent
> housekeeping
> > and resolve unreproducible bugs especially since many of them are
> > considering out of date (phoenix-4.11 or even phoenix-4.6). Since I still
> > need time to work on the blocker Jira(PHOENIX-5712, PHOENIX-6241) and fix
> > test flappers, if you want to fix "unusual query" bugs, feel free to do
> so.
> >
> >
> > Sincerely,
> > Xinyi
> >
> > On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <an...@apache.org> wrote:
> >
> > > Thanks Daniel and appreciate the effort you put in getting the list
> ready
> > > for bugs producing wrong results
> > > but none of them seems to be a blocker to me for 4.16 as they are not
> the
> > > regression and doesn't break the general functionality
> > > except for specific features, RVC/desc as Chenglei also pointed out
> > (though
> > > I'll defer the assessment to RM "Xinyi").
> > > Probably these can be a part of 4.16.1 or we can do 4.17.0 soon maybe
> > after
> > > a few weeks/month?
> > >
> > > Considering that we have already fixed 137 bugs and done 85+
> > > improvements/features in 4.16,
> > > it will not be a good idea to deprive the user from such fixes.
> > > It's been a year since our last 4.15 release, having no release brings
> > more
> > > questions on the project
> > > rather than the bugs which affect a certain % of feature/users, would
> the
> > > release notes
> > > explaining the stability of certain features set the right expectation
> > for
> > > those users who rely on these features to wait for a future release?
> > >
> > > Regards,
> > > Ankit Singhal
> > >
> > > On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <
> chenglei@apache.org>
> > > wrote:
> > >
> > > >
> > > >
> > > >
> > > > In my opinion, we should keep releases light and frequent, and for
> > some
> > > > unusual query bugs like RVC and DESC
> > > > we could delay fix to next release . I think we should release 4.16.0
> > and
> > > > 5.1.0 as quickly as possible. In China, many users
> > > > in HBase&Phoenix User Group thought that Phoenix was dead because
> our
> > > too
> > > > long interval release and stopped using
> > > > Phoenix.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > At 2020-12-02 08:45:46, "Chinmay Kulkarni" <
> chinmayskulkarni@gmail.com
> > >
> > > > wrote:
> > > > >I agree. These are all major bugs and we should aim at solving them
> > > after
> > > > >checking that they are still issues. I am +1 on 5833 and I think
> 5484
> > > > would
> > > > >be a great addition to 4.16 as well. We should aim at resolving high
> > > > >priority bugs like this in every release.
> > > > >
> > > > >Sometimes we let these bugs slip without a resolution before a
> > release,
> > > > >citing that these are "known issues" or "not regressions from the
> last
> > > > >release". In some cases this may be fine since we want to keep
> > releases
> > > > >light and frequent, but perhaps we can track such issues and aim at
> > > > >reducing the number of bugs by x% in each release? This will also
> keep
> > > old
> > > > >Jiras alive since we will potentially periodically review them.
> > > > >
> > > > >
> > > > >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <gj...@apache.org>
> > > > wrote:
> > > > >
> > > > >> I've got PHOENIX-5435 in review right now, and would like to get
> it
> > in
> > > > 4.16
> > > > >> / 5.1.
> > > > >>
> > > > >> It's allowing the annotation of Phoenix metadata into HBase WALs
> as
> > a
> > > > >> pre-req for the Phoenix Change Detection Capture framework
> > > > (PHOENIX-5442).
> > > > >> Since it has both client/server logic, and adds a field to
> > > > System.Catalog,
> > > > >> it can't go in a patch release.
> > > > >>
> > > > >> Depending on timing, I'd _like_ to get PHOENIX-6227, which is the
> > last
> > > > part
> > > > >> of CDC that will go into core Phoenix, into 4.16, but since that
> > _can_
> > > > go
> > > > >> in a patch release and I haven't started it yet, if the release
> gets
> > > cut
> > > > >> before it's ready, no big deal. (The rest of CDC will go into
> > > > >> phoenix-connectors for a future release of that project.)
> > > > >>
> > > > >> As for the correctness problems that Daniel points out, I think we
> > > > should
> > > > >> fix the ones that were detected with a recent version (4.14 or
> > 4.15?),
> > > > and
> > > > >> test to see which of the older ones can still be reproduced. Once
> we
> > > > know
> > > > >> which bugs are real and which are just historical, we can better
> > judge
> > > > >> scope. And hopefully close a bunch of obsolete bugs. (Thanks,
> > Daniel,
> > > > for
> > > > >> collecting that list!)
> > > > >>
> > > > >> Geoffrey
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
> > > > >> <da...@salesforce.com.invalid> wrote:
> > > > >>
> > > > >> > Hi, I wanted to bring up wrong results in Phoenix and some JIRAs
> > > > around
> > > > >> > them that I think we should fix as the wrong result lessens the
> > end
> > > > >> user's
> > > > >> > trust in Phoenix. Releasing a new version without addressing
> > these
> > > > in a
> > > > >> > minor release hurts our visibility in that these critical issues
> > are
> > > > not
> > > > >> > addressed.
> > > > >> >
> > > > >> > Jira's that I'm involved with for example: I've already given a
> > > patch
> > > > >> > several months ago for 5833 and there is a chance it may fix
> 5484.
> > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5833
> > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5484
> > > > >> >
> > > > >> > In addition, inspecting apache JIRA i see several other wrong
> > result
> > > > >> JIRAs
> > > > >> > from the community. Some of these certainly are probably old
> > issues
> > > > or
> > > > >> > incorrect understanding but some of these are opened by our own
> > dev
> > > > >> > community and are likely real problems.
> > > > >> > https://issues.apache.org/jira/browse/PHOENIX-6217
> > > > >> > https://issues.apache.org/jira/browse/PHOENIX-5571
> > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4642
> > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4540
> > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4504
> > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4419
> > > > >> > https://issues.apache.org/jira/browse/PHOENIX-4116
> > > > >> >
> > > > >> > What is our stance on this type of issue? Are we going to say
> > these
> > > > were
> > > > >> > issues prior to 4.15 and not address them? Should we have
> > > > requirements
> > > > >> for
> > > > >> > our releases to fix wrong results?
> > > > >> >
> > > > >> > Daniel Wong
> > > > >> >
> > > > >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <ya...@apache.org>
> > > > wrote:
> > > > >> >
> > > > >> > > Hi all,
> > > > >> > >
> > > > >> > > It's time to discuss the Phoenix 4.16 release. After many
> > people's
> > > > >> > > contributions on the bug fixes, new features, and other works
> in
> > > the
> > > > >> past
> > > > >> > > few months, we are kind of close to the point to have a RC
> > (still
> > > > need
> > > > >> to
> > > > >> > > fix test flappers). Please let me know if you think any JIRA
> > must
> > > be
> > > > >> part
> > > > >> > > of the Phoenix 4.16 release other than major blocker
> > PHOENIX-5712.
> > > > >> > >
> > > > >> > > If no surprise comes up, I will not wait for any new major
> > > features
> > > > and
> > > > >> > > focus on the RC as soon as possible.
> > > > >> > >
> > > > >> > > Sincerely,
> > > > >> > > Xinyi
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >> > --
> > > > >> > Daniel Wong
> > > > >> > Salesforce
> > > > >> > Mobile: 628.217.1808
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > > >--
> > > > >Chinmay Kulkarni
> > > >
> > >
> >
>
Re: [Discuss] Releasing Phoenix 4.16
Posted by Ankit Singhal <an...@gmail.com>.
I see that both the blockers listed here PHOENIX-5712 and PHOENIX-6241
have been resolved(Thanks to Xinyi), and also as per the JIRA query there
is no Jira marked as a blocker for 4.16 except the one related to
documentation
for "splittable catalog table".
Xinyi, so are we good to start the release process now?
On Wed, Dec 2, 2020 at 9:32 PM Xinyi Yan <ya...@apache.org> wrote:
> Thanks for replying and providing suggestions. I looked at the wrong result
> Jira list that Daniel provided and did some local testing, and here is the
> result:
> [resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642: cannot reproduce
> it.
> [More information is required] PHOENIX-4504 cannot reproduce it but someone
> claimed that he had a similar issue.
> [unusual query] PHOENIX-4540 and PHOENIX-6217
>
> Based on my finding, I think it's better to have more frequent housekeeping
> and resolve unreproducible bugs especially since many of them are
> considering out of date (phoenix-4.11 or even phoenix-4.6). Since I still
> need time to work on the blocker Jira(PHOENIX-5712, PHOENIX-6241) and fix
> test flappers, if you want to fix "unusual query" bugs, feel free to do so.
>
>
> Sincerely,
> Xinyi
>
> On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <an...@apache.org> wrote:
>
> > Thanks Daniel and appreciate the effort you put in getting the list ready
> > for bugs producing wrong results
> > but none of them seems to be a blocker to me for 4.16 as they are not the
> > regression and doesn't break the general functionality
> > except for specific features, RVC/desc as Chenglei also pointed out
> (though
> > I'll defer the assessment to RM "Xinyi").
> > Probably these can be a part of 4.16.1 or we can do 4.17.0 soon maybe
> after
> > a few weeks/month?
> >
> > Considering that we have already fixed 137 bugs and done 85+
> > improvements/features in 4.16,
> > it will not be a good idea to deprive the user from such fixes.
> > It's been a year since our last 4.15 release, having no release brings
> more
> > questions on the project
> > rather than the bugs which affect a certain % of feature/users, would the
> > release notes
> > explaining the stability of certain features set the right expectation
> for
> > those users who rely on these features to wait for a future release?
> >
> > Regards,
> > Ankit Singhal
> >
> > On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <ch...@apache.org>
> > wrote:
> >
> > >
> > >
> > >
> > > In my opinion, we should keep releases light and frequent, and for
> some
> > > unusual query bugs like RVC and DESC
> > > we could delay fix to next release . I think we should release 4.16.0
> and
> > > 5.1.0 as quickly as possible. In China, many users
> > > in HBase&Phoenix User Group thought that Phoenix was dead because our
> > too
> > > long interval release and stopped using
> > > Phoenix.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > At 2020-12-02 08:45:46, "Chinmay Kulkarni" <chinmayskulkarni@gmail.com
> >
> > > wrote:
> > > >I agree. These are all major bugs and we should aim at solving them
> > after
> > > >checking that they are still issues. I am +1 on 5833 and I think 5484
> > > would
> > > >be a great addition to 4.16 as well. We should aim at resolving high
> > > >priority bugs like this in every release.
> > > >
> > > >Sometimes we let these bugs slip without a resolution before a
> release,
> > > >citing that these are "known issues" or "not regressions from the last
> > > >release". In some cases this may be fine since we want to keep
> releases
> > > >light and frequent, but perhaps we can track such issues and aim at
> > > >reducing the number of bugs by x% in each release? This will also keep
> > old
> > > >Jiras alive since we will potentially periodically review them.
> > > >
> > > >
> > > >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <gj...@apache.org>
> > > wrote:
> > > >
> > > >> I've got PHOENIX-5435 in review right now, and would like to get it
> in
> > > 4.16
> > > >> / 5.1.
> > > >>
> > > >> It's allowing the annotation of Phoenix metadata into HBase WALs as
> a
> > > >> pre-req for the Phoenix Change Detection Capture framework
> > > (PHOENIX-5442).
> > > >> Since it has both client/server logic, and adds a field to
> > > System.Catalog,
> > > >> it can't go in a patch release.
> > > >>
> > > >> Depending on timing, I'd _like_ to get PHOENIX-6227, which is the
> last
> > > part
> > > >> of CDC that will go into core Phoenix, into 4.16, but since that
> _can_
> > > go
> > > >> in a patch release and I haven't started it yet, if the release gets
> > cut
> > > >> before it's ready, no big deal. (The rest of CDC will go into
> > > >> phoenix-connectors for a future release of that project.)
> > > >>
> > > >> As for the correctness problems that Daniel points out, I think we
> > > should
> > > >> fix the ones that were detected with a recent version (4.14 or
> 4.15?),
> > > and
> > > >> test to see which of the older ones can still be reproduced. Once we
> > > know
> > > >> which bugs are real and which are just historical, we can better
> judge
> > > >> scope. And hopefully close a bunch of obsolete bugs. (Thanks,
> Daniel,
> > > for
> > > >> collecting that list!)
> > > >>
> > > >> Geoffrey
> > > >>
> > > >>
> > > >>
> > > >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
> > > >> <da...@salesforce.com.invalid> wrote:
> > > >>
> > > >> > Hi, I wanted to bring up wrong results in Phoenix and some JIRAs
> > > around
> > > >> > them that I think we should fix as the wrong result lessens the
> end
> > > >> user's
> > > >> > trust in Phoenix. Releasing a new version without addressing
> these
> > > in a
> > > >> > minor release hurts our visibility in that these critical issues
> are
> > > not
> > > >> > addressed.
> > > >> >
> > > >> > Jira's that I'm involved with for example: I've already given a
> > patch
> > > >> > several months ago for 5833 and there is a chance it may fix 5484.
> > > >> > https://issues.apache.org/jira/browse/PHOENIX-5833
> > > >> > https://issues.apache.org/jira/browse/PHOENIX-5484
> > > >> >
> > > >> > In addition, inspecting apache JIRA i see several other wrong
> result
> > > >> JIRAs
> > > >> > from the community. Some of these certainly are probably old
> issues
> > > or
> > > >> > incorrect understanding but some of these are opened by our own
> dev
> > > >> > community and are likely real problems.
> > > >> > https://issues.apache.org/jira/browse/PHOENIX-6217
> > > >> > https://issues.apache.org/jira/browse/PHOENIX-5571
> > > >> > https://issues.apache.org/jira/browse/PHOENIX-4642
> > > >> > https://issues.apache.org/jira/browse/PHOENIX-4540
> > > >> > https://issues.apache.org/jira/browse/PHOENIX-4504
> > > >> > https://issues.apache.org/jira/browse/PHOENIX-4419
> > > >> > https://issues.apache.org/jira/browse/PHOENIX-4116
> > > >> >
> > > >> > What is our stance on this type of issue? Are we going to say
> these
> > > were
> > > >> > issues prior to 4.15 and not address them? Should we have
> > > requirements
> > > >> for
> > > >> > our releases to fix wrong results?
> > > >> >
> > > >> > Daniel Wong
> > > >> >
> > > >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <ya...@apache.org>
> > > wrote:
> > > >> >
> > > >> > > Hi all,
> > > >> > >
> > > >> > > It's time to discuss the Phoenix 4.16 release. After many
> people's
> > > >> > > contributions on the bug fixes, new features, and other works in
> > the
> > > >> past
> > > >> > > few months, we are kind of close to the point to have a RC
> (still
> > > need
> > > >> to
> > > >> > > fix test flappers). Please let me know if you think any JIRA
> must
> > be
> > > >> part
> > > >> > > of the Phoenix 4.16 release other than major blocker
> PHOENIX-5712.
> > > >> > >
> > > >> > > If no surprise comes up, I will not wait for any new major
> > features
> > > and
> > > >> > > focus on the RC as soon as possible.
> > > >> > >
> > > >> > > Sincerely,
> > > >> > > Xinyi
> > > >> > >
> > > >> >
> > > >> >
> > > >> > --
> > > >> > Daniel Wong
> > > >> > Salesforce
> > > >> > Mobile: 628.217.1808
> > > >> >
> > > >>
> > > >
> > > >
> > > >--
> > > >Chinmay Kulkarni
> > >
> >
>
Re: [Discuss] Releasing Phoenix 4.16
Posted by Xinyi Yan <ya...@apache.org>.
Thanks for replying and providing suggestions. I looked at the wrong result
Jira list that Daniel provided and did some local testing, and here is the
result:
[resolved] PHOENIX-4116, PHOENIX-4419, and PHOENIX-4642: cannot reproduce
it.
[More information is required] PHOENIX-4504 cannot reproduce it but someone
claimed that he had a similar issue.
[unusual query] PHOENIX-4540 and PHOENIX-6217
Based on my finding, I think it's better to have more frequent housekeeping
and resolve unreproducible bugs especially since many of them are
considering out of date (phoenix-4.11 or even phoenix-4.6). Since I still
need time to work on the blocker Jira(PHOENIX-5712, PHOENIX-6241) and fix
test flappers, if you want to fix "unusual query" bugs, feel free to do so.
Sincerely,
Xinyi
On Wed, Dec 2, 2020 at 12:41 AM Ankit Singhal <an...@apache.org> wrote:
> Thanks Daniel and appreciate the effort you put in getting the list ready
> for bugs producing wrong results
> but none of them seems to be a blocker to me for 4.16 as they are not the
> regression and doesn't break the general functionality
> except for specific features, RVC/desc as Chenglei also pointed out (though
> I'll defer the assessment to RM "Xinyi").
> Probably these can be a part of 4.16.1 or we can do 4.17.0 soon maybe after
> a few weeks/month?
>
> Considering that we have already fixed 137 bugs and done 85+
> improvements/features in 4.16,
> it will not be a good idea to deprive the user from such fixes.
> It's been a year since our last 4.15 release, having no release brings more
> questions on the project
> rather than the bugs which affect a certain % of feature/users, would the
> release notes
> explaining the stability of certain features set the right expectation for
> those users who rely on these features to wait for a future release?
>
> Regards,
> Ankit Singhal
>
> On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <ch...@apache.org>
> wrote:
>
> >
> >
> >
> > In my opinion, we should keep releases light and frequent, and for some
> > unusual query bugs like RVC and DESC
> > we could delay fix to next release . I think we should release 4.16.0 and
> > 5.1.0 as quickly as possible. In China, many users
> > in HBase&Phoenix User Group thought that Phoenix was dead because our
> too
> > long interval release and stopped using
> > Phoenix.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > At 2020-12-02 08:45:46, "Chinmay Kulkarni" <ch...@gmail.com>
> > wrote:
> > >I agree. These are all major bugs and we should aim at solving them
> after
> > >checking that they are still issues. I am +1 on 5833 and I think 5484
> > would
> > >be a great addition to 4.16 as well. We should aim at resolving high
> > >priority bugs like this in every release.
> > >
> > >Sometimes we let these bugs slip without a resolution before a release,
> > >citing that these are "known issues" or "not regressions from the last
> > >release". In some cases this may be fine since we want to keep releases
> > >light and frequent, but perhaps we can track such issues and aim at
> > >reducing the number of bugs by x% in each release? This will also keep
> old
> > >Jiras alive since we will potentially periodically review them.
> > >
> > >
> > >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <gj...@apache.org>
> > wrote:
> > >
> > >> I've got PHOENIX-5435 in review right now, and would like to get it in
> > 4.16
> > >> / 5.1.
> > >>
> > >> It's allowing the annotation of Phoenix metadata into HBase WALs as a
> > >> pre-req for the Phoenix Change Detection Capture framework
> > (PHOENIX-5442).
> > >> Since it has both client/server logic, and adds a field to
> > System.Catalog,
> > >> it can't go in a patch release.
> > >>
> > >> Depending on timing, I'd _like_ to get PHOENIX-6227, which is the last
> > part
> > >> of CDC that will go into core Phoenix, into 4.16, but since that _can_
> > go
> > >> in a patch release and I haven't started it yet, if the release gets
> cut
> > >> before it's ready, no big deal. (The rest of CDC will go into
> > >> phoenix-connectors for a future release of that project.)
> > >>
> > >> As for the correctness problems that Daniel points out, I think we
> > should
> > >> fix the ones that were detected with a recent version (4.14 or 4.15?),
> > and
> > >> test to see which of the older ones can still be reproduced. Once we
> > know
> > >> which bugs are real and which are just historical, we can better judge
> > >> scope. And hopefully close a bunch of obsolete bugs. (Thanks, Daniel,
> > for
> > >> collecting that list!)
> > >>
> > >> Geoffrey
> > >>
> > >>
> > >>
> > >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
> > >> <da...@salesforce.com.invalid> wrote:
> > >>
> > >> > Hi, I wanted to bring up wrong results in Phoenix and some JIRAs
> > around
> > >> > them that I think we should fix as the wrong result lessens the end
> > >> user's
> > >> > trust in Phoenix. Releasing a new version without addressing these
> > in a
> > >> > minor release hurts our visibility in that these critical issues are
> > not
> > >> > addressed.
> > >> >
> > >> > Jira's that I'm involved with for example: I've already given a
> patch
> > >> > several months ago for 5833 and there is a chance it may fix 5484.
> > >> > https://issues.apache.org/jira/browse/PHOENIX-5833
> > >> > https://issues.apache.org/jira/browse/PHOENIX-5484
> > >> >
> > >> > In addition, inspecting apache JIRA i see several other wrong result
> > >> JIRAs
> > >> > from the community. Some of these certainly are probably old issues
> > or
> > >> > incorrect understanding but some of these are opened by our own dev
> > >> > community and are likely real problems.
> > >> > https://issues.apache.org/jira/browse/PHOENIX-6217
> > >> > https://issues.apache.org/jira/browse/PHOENIX-5571
> > >> > https://issues.apache.org/jira/browse/PHOENIX-4642
> > >> > https://issues.apache.org/jira/browse/PHOENIX-4540
> > >> > https://issues.apache.org/jira/browse/PHOENIX-4504
> > >> > https://issues.apache.org/jira/browse/PHOENIX-4419
> > >> > https://issues.apache.org/jira/browse/PHOENIX-4116
> > >> >
> > >> > What is our stance on this type of issue? Are we going to say these
> > were
> > >> > issues prior to 4.15 and not address them? Should we have
> > requirements
> > >> for
> > >> > our releases to fix wrong results?
> > >> >
> > >> > Daniel Wong
> > >> >
> > >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <ya...@apache.org>
> > wrote:
> > >> >
> > >> > > Hi all,
> > >> > >
> > >> > > It's time to discuss the Phoenix 4.16 release. After many people's
> > >> > > contributions on the bug fixes, new features, and other works in
> the
> > >> past
> > >> > > few months, we are kind of close to the point to have a RC (still
> > need
> > >> to
> > >> > > fix test flappers). Please let me know if you think any JIRA must
> be
> > >> part
> > >> > > of the Phoenix 4.16 release other than major blocker PHOENIX-5712.
> > >> > >
> > >> > > If no surprise comes up, I will not wait for any new major
> features
> > and
> > >> > > focus on the RC as soon as possible.
> > >> > >
> > >> > > Sincerely,
> > >> > > Xinyi
> > >> > >
> > >> >
> > >> >
> > >> > --
> > >> > Daniel Wong
> > >> > Salesforce
> > >> > Mobile: 628.217.1808
> > >> >
> > >>
> > >
> > >
> > >--
> > >Chinmay Kulkarni
> >
>
Re: [Discuss] Releasing Phoenix 4.16
Posted by Ankit Singhal <an...@apache.org>.
Thanks Daniel and appreciate the effort you put in getting the list ready
for bugs producing wrong results
but none of them seems to be a blocker to me for 4.16 as they are not the
regression and doesn't break the general functionality
except for specific features, RVC/desc as Chenglei also pointed out (though
I'll defer the assessment to RM "Xinyi").
Probably these can be a part of 4.16.1 or we can do 4.17.0 soon maybe after
a few weeks/month?
Considering that we have already fixed 137 bugs and done 85+
improvements/features in 4.16,
it will not be a good idea to deprive the user from such fixes.
It's been a year since our last 4.15 release, having no release brings more
questions on the project
rather than the bugs which affect a certain % of feature/users, would the
release notes
explaining the stability of certain features set the right expectation for
those users who rely on these features to wait for a future release?
Regards,
Ankit Singhal
On Tue, Dec 1, 2020 at 8:21 PM chenglei@apache.org <ch...@apache.org>
wrote:
>
>
>
> In my opinion, we should keep releases light and frequent, and for some
> unusual query bugs like RVC and DESC
> we could delay fix to next release . I think we should release 4.16.0 and
> 5.1.0 as quickly as possible. In China, many users
> in HBase&Phoenix User Group thought that Phoenix was dead because our too
> long interval release and stopped using
> Phoenix.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> At 2020-12-02 08:45:46, "Chinmay Kulkarni" <ch...@gmail.com>
> wrote:
> >I agree. These are all major bugs and we should aim at solving them after
> >checking that they are still issues. I am +1 on 5833 and I think 5484
> would
> >be a great addition to 4.16 as well. We should aim at resolving high
> >priority bugs like this in every release.
> >
> >Sometimes we let these bugs slip without a resolution before a release,
> >citing that these are "known issues" or "not regressions from the last
> >release". In some cases this may be fine since we want to keep releases
> >light and frequent, but perhaps we can track such issues and aim at
> >reducing the number of bugs by x% in each release? This will also keep old
> >Jiras alive since we will potentially periodically review them.
> >
> >
> >On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <gj...@apache.org>
> wrote:
> >
> >> I've got PHOENIX-5435 in review right now, and would like to get it in
> 4.16
> >> / 5.1.
> >>
> >> It's allowing the annotation of Phoenix metadata into HBase WALs as a
> >> pre-req for the Phoenix Change Detection Capture framework
> (PHOENIX-5442).
> >> Since it has both client/server logic, and adds a field to
> System.Catalog,
> >> it can't go in a patch release.
> >>
> >> Depending on timing, I'd _like_ to get PHOENIX-6227, which is the last
> part
> >> of CDC that will go into core Phoenix, into 4.16, but since that _can_
> go
> >> in a patch release and I haven't started it yet, if the release gets cut
> >> before it's ready, no big deal. (The rest of CDC will go into
> >> phoenix-connectors for a future release of that project.)
> >>
> >> As for the correctness problems that Daniel points out, I think we
> should
> >> fix the ones that were detected with a recent version (4.14 or 4.15?),
> and
> >> test to see which of the older ones can still be reproduced. Once we
> know
> >> which bugs are real and which are just historical, we can better judge
> >> scope. And hopefully close a bunch of obsolete bugs. (Thanks, Daniel,
> for
> >> collecting that list!)
> >>
> >> Geoffrey
> >>
> >>
> >>
> >> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
> >> <da...@salesforce.com.invalid> wrote:
> >>
> >> > Hi, I wanted to bring up wrong results in Phoenix and some JIRAs
> around
> >> > them that I think we should fix as the wrong result lessens the end
> >> user's
> >> > trust in Phoenix. Releasing a new version without addressing these
> in a
> >> > minor release hurts our visibility in that these critical issues are
> not
> >> > addressed.
> >> >
> >> > Jira's that I'm involved with for example: I've already given a patch
> >> > several months ago for 5833 and there is a chance it may fix 5484.
> >> > https://issues.apache.org/jira/browse/PHOENIX-5833
> >> > https://issues.apache.org/jira/browse/PHOENIX-5484
> >> >
> >> > In addition, inspecting apache JIRA i see several other wrong result
> >> JIRAs
> >> > from the community. Some of these certainly are probably old issues
> or
> >> > incorrect understanding but some of these are opened by our own dev
> >> > community and are likely real problems.
> >> > https://issues.apache.org/jira/browse/PHOENIX-6217
> >> > https://issues.apache.org/jira/browse/PHOENIX-5571
> >> > https://issues.apache.org/jira/browse/PHOENIX-4642
> >> > https://issues.apache.org/jira/browse/PHOENIX-4540
> >> > https://issues.apache.org/jira/browse/PHOENIX-4504
> >> > https://issues.apache.org/jira/browse/PHOENIX-4419
> >> > https://issues.apache.org/jira/browse/PHOENIX-4116
> >> >
> >> > What is our stance on this type of issue? Are we going to say these
> were
> >> > issues prior to 4.15 and not address them? Should we have
> requirements
> >> for
> >> > our releases to fix wrong results?
> >> >
> >> > Daniel Wong
> >> >
> >> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <ya...@apache.org>
> wrote:
> >> >
> >> > > Hi all,
> >> > >
> >> > > It's time to discuss the Phoenix 4.16 release. After many people's
> >> > > contributions on the bug fixes, new features, and other works in the
> >> past
> >> > > few months, we are kind of close to the point to have a RC (still
> need
> >> to
> >> > > fix test flappers). Please let me know if you think any JIRA must be
> >> part
> >> > > of the Phoenix 4.16 release other than major blocker PHOENIX-5712.
> >> > >
> >> > > If no surprise comes up, I will not wait for any new major features
> and
> >> > > focus on the RC as soon as possible.
> >> > >
> >> > > Sincerely,
> >> > > Xinyi
> >> > >
> >> >
> >> >
> >> > --
> >> > Daniel Wong
> >> > Salesforce
> >> > Mobile: 628.217.1808
> >> >
> >>
> >
> >
> >--
> >Chinmay Kulkarni
>
Re:[Discuss] Releasing Phoenix 4.16
Posted by "chenglei@apache.org" <ch...@apache.org>.
In my opinion, we should keep releases light and frequent, and for some unusual query bugs like RVC and DESC
we could delay fix to next release . I think we should release 4.16.0 and 5.1.0 as quickly as possible. In China, many users
in HBase&Phoenix User Group thought that Phoenix was dead because our too long interval release and stopped using
Phoenix.
At 2020-12-02 08:45:46, "Chinmay Kulkarni" <ch...@gmail.com> wrote:
>I agree. These are all major bugs and we should aim at solving them after
>checking that they are still issues. I am +1 on 5833 and I think 5484 would
>be a great addition to 4.16 as well. We should aim at resolving high
>priority bugs like this in every release.
>
>Sometimes we let these bugs slip without a resolution before a release,
>citing that these are "known issues" or "not regressions from the last
>release". In some cases this may be fine since we want to keep releases
>light and frequent, but perhaps we can track such issues and aim at
>reducing the number of bugs by x% in each release? This will also keep old
>Jiras alive since we will potentially periodically review them.
>
>
>On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <gj...@apache.org> wrote:
>
>> I've got PHOENIX-5435 in review right now, and would like to get it in 4.16
>> / 5.1.
>>
>> It's allowing the annotation of Phoenix metadata into HBase WALs as a
>> pre-req for the Phoenix Change Detection Capture framework (PHOENIX-5442).
>> Since it has both client/server logic, and adds a field to System.Catalog,
>> it can't go in a patch release.
>>
>> Depending on timing, I'd _like_ to get PHOENIX-6227, which is the last part
>> of CDC that will go into core Phoenix, into 4.16, but since that _can_ go
>> in a patch release and I haven't started it yet, if the release gets cut
>> before it's ready, no big deal. (The rest of CDC will go into
>> phoenix-connectors for a future release of that project.)
>>
>> As for the correctness problems that Daniel points out, I think we should
>> fix the ones that were detected with a recent version (4.14 or 4.15?), and
>> test to see which of the older ones can still be reproduced. Once we know
>> which bugs are real and which are just historical, we can better judge
>> scope. And hopefully close a bunch of obsolete bugs. (Thanks, Daniel, for
>> collecting that list!)
>>
>> Geoffrey
>>
>>
>>
>> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
>> <da...@salesforce.com.invalid> wrote:
>>
>> > Hi, I wanted to bring up wrong results in Phoenix and some JIRAs around
>> > them that I think we should fix as the wrong result lessens the end
>> user's
>> > trust in Phoenix. Releasing a new version without addressing these in a
>> > minor release hurts our visibility in that these critical issues are not
>> > addressed.
>> >
>> > Jira's that I'm involved with for example: I've already given a patch
>> > several months ago for 5833 and there is a chance it may fix 5484.
>> > https://issues.apache.org/jira/browse/PHOENIX-5833
>> > https://issues.apache.org/jira/browse/PHOENIX-5484
>> >
>> > In addition, inspecting apache JIRA i see several other wrong result
>> JIRAs
>> > from the community. Some of these certainly are probably old issues or
>> > incorrect understanding but some of these are opened by our own dev
>> > community and are likely real problems.
>> > https://issues.apache.org/jira/browse/PHOENIX-6217
>> > https://issues.apache.org/jira/browse/PHOENIX-5571
>> > https://issues.apache.org/jira/browse/PHOENIX-4642
>> > https://issues.apache.org/jira/browse/PHOENIX-4540
>> > https://issues.apache.org/jira/browse/PHOENIX-4504
>> > https://issues.apache.org/jira/browse/PHOENIX-4419
>> > https://issues.apache.org/jira/browse/PHOENIX-4116
>> >
>> > What is our stance on this type of issue? Are we going to say these were
>> > issues prior to 4.15 and not address them? Should we have requirements
>> for
>> > our releases to fix wrong results?
>> >
>> > Daniel Wong
>> >
>> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <ya...@apache.org> wrote:
>> >
>> > > Hi all,
>> > >
>> > > It's time to discuss the Phoenix 4.16 release. After many people's
>> > > contributions on the bug fixes, new features, and other works in the
>> past
>> > > few months, we are kind of close to the point to have a RC (still need
>> to
>> > > fix test flappers). Please let me know if you think any JIRA must be
>> part
>> > > of the Phoenix 4.16 release other than major blocker PHOENIX-5712.
>> > >
>> > > If no surprise comes up, I will not wait for any new major features and
>> > > focus on the RC as soon as possible.
>> > >
>> > > Sincerely,
>> > > Xinyi
>> > >
>> >
>> >
>> > --
>> > Daniel Wong
>> > Salesforce
>> > Mobile: 628.217.1808
>> >
>>
>
>
>--
>Chinmay Kulkarni
Re: [Discuss] Releasing Phoenix 4.16
Posted by Chinmay Kulkarni <ch...@gmail.com>.
I agree. These are all major bugs and we should aim at solving them after
checking that they are still issues. I am +1 on 5833 and I think 5484 would
be a great addition to 4.16 as well. We should aim at resolving high
priority bugs like this in every release.
Sometimes we let these bugs slip without a resolution before a release,
citing that these are "known issues" or "not regressions from the last
release". In some cases this may be fine since we want to keep releases
light and frequent, but perhaps we can track such issues and aim at
reducing the number of bugs by x% in each release? This will also keep old
Jiras alive since we will potentially periodically review them.
On Tue, Dec 1, 2020 at 4:01 PM Geoffrey Jacoby <gj...@apache.org> wrote:
> I've got PHOENIX-5435 in review right now, and would like to get it in 4.16
> / 5.1.
>
> It's allowing the annotation of Phoenix metadata into HBase WALs as a
> pre-req for the Phoenix Change Detection Capture framework (PHOENIX-5442).
> Since it has both client/server logic, and adds a field to System.Catalog,
> it can't go in a patch release.
>
> Depending on timing, I'd _like_ to get PHOENIX-6227, which is the last part
> of CDC that will go into core Phoenix, into 4.16, but since that _can_ go
> in a patch release and I haven't started it yet, if the release gets cut
> before it's ready, no big deal. (The rest of CDC will go into
> phoenix-connectors for a future release of that project.)
>
> As for the correctness problems that Daniel points out, I think we should
> fix the ones that were detected with a recent version (4.14 or 4.15?), and
> test to see which of the older ones can still be reproduced. Once we know
> which bugs are real and which are just historical, we can better judge
> scope. And hopefully close a bunch of obsolete bugs. (Thanks, Daniel, for
> collecting that list!)
>
> Geoffrey
>
>
>
> On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
> <da...@salesforce.com.invalid> wrote:
>
> > Hi, I wanted to bring up wrong results in Phoenix and some JIRAs around
> > them that I think we should fix as the wrong result lessens the end
> user's
> > trust in Phoenix. Releasing a new version without addressing these in a
> > minor release hurts our visibility in that these critical issues are not
> > addressed.
> >
> > Jira's that I'm involved with for example: I've already given a patch
> > several months ago for 5833 and there is a chance it may fix 5484.
> > https://issues.apache.org/jira/browse/PHOENIX-5833
> > https://issues.apache.org/jira/browse/PHOENIX-5484
> >
> > In addition, inspecting apache JIRA i see several other wrong result
> JIRAs
> > from the community. Some of these certainly are probably old issues or
> > incorrect understanding but some of these are opened by our own dev
> > community and are likely real problems.
> > https://issues.apache.org/jira/browse/PHOENIX-6217
> > https://issues.apache.org/jira/browse/PHOENIX-5571
> > https://issues.apache.org/jira/browse/PHOENIX-4642
> > https://issues.apache.org/jira/browse/PHOENIX-4540
> > https://issues.apache.org/jira/browse/PHOENIX-4504
> > https://issues.apache.org/jira/browse/PHOENIX-4419
> > https://issues.apache.org/jira/browse/PHOENIX-4116
> >
> > What is our stance on this type of issue? Are we going to say these were
> > issues prior to 4.15 and not address them? Should we have requirements
> for
> > our releases to fix wrong results?
> >
> > Daniel Wong
> >
> > On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <ya...@apache.org> wrote:
> >
> > > Hi all,
> > >
> > > It's time to discuss the Phoenix 4.16 release. After many people's
> > > contributions on the bug fixes, new features, and other works in the
> past
> > > few months, we are kind of close to the point to have a RC (still need
> to
> > > fix test flappers). Please let me know if you think any JIRA must be
> part
> > > of the Phoenix 4.16 release other than major blocker PHOENIX-5712.
> > >
> > > If no surprise comes up, I will not wait for any new major features and
> > > focus on the RC as soon as possible.
> > >
> > > Sincerely,
> > > Xinyi
> > >
> >
> >
> > --
> > Daniel Wong
> > Salesforce
> > Mobile: 628.217.1808
> >
>
--
Chinmay Kulkarni
Re: [Discuss] Releasing Phoenix 4.16
Posted by Geoffrey Jacoby <gj...@apache.org>.
I've got PHOENIX-5435 in review right now, and would like to get it in 4.16
/ 5.1.
It's allowing the annotation of Phoenix metadata into HBase WALs as a
pre-req for the Phoenix Change Detection Capture framework (PHOENIX-5442).
Since it has both client/server logic, and adds a field to System.Catalog,
it can't go in a patch release.
Depending on timing, I'd _like_ to get PHOENIX-6227, which is the last part
of CDC that will go into core Phoenix, into 4.16, but since that _can_ go
in a patch release and I haven't started it yet, if the release gets cut
before it's ready, no big deal. (The rest of CDC will go into
phoenix-connectors for a future release of that project.)
As for the correctness problems that Daniel points out, I think we should
fix the ones that were detected with a recent version (4.14 or 4.15?), and
test to see which of the older ones can still be reproduced. Once we know
which bugs are real and which are just historical, we can better judge
scope. And hopefully close a bunch of obsolete bugs. (Thanks, Daniel, for
collecting that list!)
Geoffrey
On Tue, Dec 1, 2020 at 1:33 PM Daniel Wong
<da...@salesforce.com.invalid> wrote:
> Hi, I wanted to bring up wrong results in Phoenix and some JIRAs around
> them that I think we should fix as the wrong result lessens the end user's
> trust in Phoenix. Releasing a new version without addressing these in a
> minor release hurts our visibility in that these critical issues are not
> addressed.
>
> Jira's that I'm involved with for example: I've already given a patch
> several months ago for 5833 and there is a chance it may fix 5484.
> https://issues.apache.org/jira/browse/PHOENIX-5833
> https://issues.apache.org/jira/browse/PHOENIX-5484
>
> In addition, inspecting apache JIRA i see several other wrong result JIRAs
> from the community. Some of these certainly are probably old issues or
> incorrect understanding but some of these are opened by our own dev
> community and are likely real problems.
> https://issues.apache.org/jira/browse/PHOENIX-6217
> https://issues.apache.org/jira/browse/PHOENIX-5571
> https://issues.apache.org/jira/browse/PHOENIX-4642
> https://issues.apache.org/jira/browse/PHOENIX-4540
> https://issues.apache.org/jira/browse/PHOENIX-4504
> https://issues.apache.org/jira/browse/PHOENIX-4419
> https://issues.apache.org/jira/browse/PHOENIX-4116
>
> What is our stance on this type of issue? Are we going to say these were
> issues prior to 4.15 and not address them? Should we have requirements for
> our releases to fix wrong results?
>
> Daniel Wong
>
> On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <ya...@apache.org> wrote:
>
> > Hi all,
> >
> > It's time to discuss the Phoenix 4.16 release. After many people's
> > contributions on the bug fixes, new features, and other works in the past
> > few months, we are kind of close to the point to have a RC (still need to
> > fix test flappers). Please let me know if you think any JIRA must be part
> > of the Phoenix 4.16 release other than major blocker PHOENIX-5712.
> >
> > If no surprise comes up, I will not wait for any new major features and
> > focus on the RC as soon as possible.
> >
> > Sincerely,
> > Xinyi
> >
>
>
> --
> Daniel Wong
> Salesforce
> Mobile: 628.217.1808
>
Re: [Discuss] Releasing Phoenix 4.16
Posted by Daniel Wong <da...@salesforce.com.INVALID>.
Hi, I wanted to bring up wrong results in Phoenix and some JIRAs around
them that I think we should fix as the wrong result lessens the end user's
trust in Phoenix. Releasing a new version without addressing these in a
minor release hurts our visibility in that these critical issues are not
addressed.
Jira's that I'm involved with for example: I've already given a patch
several months ago for 5833 and there is a chance it may fix 5484.
https://issues.apache.org/jira/browse/PHOENIX-5833
https://issues.apache.org/jira/browse/PHOENIX-5484
In addition, inspecting apache JIRA i see several other wrong result JIRAs
from the community. Some of these certainly are probably old issues or
incorrect understanding but some of these are opened by our own dev
community and are likely real problems.
https://issues.apache.org/jira/browse/PHOENIX-6217
https://issues.apache.org/jira/browse/PHOENIX-5571
https://issues.apache.org/jira/browse/PHOENIX-4642
https://issues.apache.org/jira/browse/PHOENIX-4540
https://issues.apache.org/jira/browse/PHOENIX-4504
https://issues.apache.org/jira/browse/PHOENIX-4419
https://issues.apache.org/jira/browse/PHOENIX-4116
What is our stance on this type of issue? Are we going to say these were
issues prior to 4.15 and not address them? Should we have requirements for
our releases to fix wrong results?
Daniel Wong
On Mon, Nov 30, 2020 at 7:30 PM Xinyi Yan <ya...@apache.org> wrote:
> Hi all,
>
> It's time to discuss the Phoenix 4.16 release. After many people's
> contributions on the bug fixes, new features, and other works in the past
> few months, we are kind of close to the point to have a RC (still need to
> fix test flappers). Please let me know if you think any JIRA must be part
> of the Phoenix 4.16 release other than major blocker PHOENIX-5712.
>
> If no surprise comes up, I will not wait for any new major features and
> focus on the RC as soon as possible.
>
> Sincerely,
> Xinyi
>
--
Daniel Wong
Salesforce
Mobile: 628.217.1808