You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "张铎(Duo Zhang)" <pa...@gmail.com> on 2026/03/30 11:28:43 UTC
Re: [DISCUSS] For 3.0.0-beta-2 and the final 3.0.0 release
Some updates here, there is no blocker issue for 3.0.0-beta-2 release
now, but the junit5 upgrading is in its last stage, where we want to
migrate all the tests in hbase-server, so I plan to finish the
migration before cutting the final release.
I will also start to run several rounds of 10B ITBLL to check the
stability of branch-3.
Thanks.
张铎(Duo Zhang) <pa...@gmail.com> 于2026年1月22日周四 19:15写道:
>
> Thanks Istvan for the detailed analysis!
>
> For HBASE-29834, I've already put up a PR to fix the problem. See
>
> https://github.com/apache/hbase/pull/7660
>
> And on HBASE-29843, I do not fully understand the problem and need
> more context about the usage on Phoenix side. I think we should be
> able to fix it.
>
> Thanks.
>
> Istvan Toth <st...@cloudera.com> 于2026年1月22日周四 14:33写道:
> >
> > This seems to have been caused by forward synchronous RPC changes between
> > Hbase 2 and 3 (where HBase 2 simulates non-inclusive start keys by
> > adding a \00 byte and setting them as inclusive, while HBase 3 handles them
> > natively.
> > This is something that we can handle on the Phoenix side.
> >
> > On the other hand, I've found a major and a minor breaking change in HBase
> > 3:
> >
> > The big problem is HBASE-29843
> > <https://issues.apache.org/jira/browse/HBASE-29843>
> > HBaseRPCController.setPriority(final TableName tn) is never called on
> > branch-3+
> > Phoenix relies on setting it its own HBaseRPCController to set custom
> > priorities, but HBase 3 has mostly abandoned using HBaseRPCController,
> > leaving no
> > working mechanism for Phoenix to use.
> > HBase needs to restore using it to maintain compatibility. (or provide some
> > alternative method)
> >
> > The smaller problem is HBASE-29834
> > <https://issues.apache.org/jira/browse/HBASE-29834> MILLIS_BETWEEN_NEXTS
> > metric is not updated on branch-3+
> > This does not break Phoenix, it just happens to be detected by a Phoenix
> > test.
> >
> > Istvan
> >
> >
> > On Mon, Nov 3, 2025 at 10:22 AM Istvan Toth <st...@cloudera.com> wrote:
> >
> > > Sorry, I missed your reply eralier, Duo.
> > >
> > > I still haven't figured out the issue (though at least now I am now able
> > > to run those tests from a debugger)
> > > No ticket yet, as I'm still not sure which project to use.
> > >
> > > To repro this, we need to
> > > * install a current HBase 3 snapshot locally from branch-3 head,
> > > * check out my WIP Phoenix HBase 3 branch:
> > > https://github.com/stoty/phoenix/tree/PHOENIX-7164-2025
> > > * then run a faling test:
> > >
> > > *mvn clean verify -am -pl phoenix-core
> > > -Dit.test=BaseAggregateWithRegionMoves2IT#testGroupByArray
> > > -Dhbase.profile=3.0 -Dtest=foo*
> > > There are other similar tests failures, missing expected ResultSet rows
> > > after aggressively moving around the table regions during the query.
> > >
> > > Istvan
> > >
> > > On Thu, Oct 16, 2025 at 3:49 PM 张铎(Duo Zhang) <pa...@gmail.com>
> > > wrote:
> > >
> > >> Is there an issue in HBase or Phoenix for tracking this problem?
> > >>
> > >> Thanks.
> > >>
> > >> Istvan Toth <st...@cloudera.com.invalid> 于2025年10月16日周四 11:58写道:
> > >> >
> > >> > Quick heads up:
> > >> > I'm investigating a regression in the Phoenix test suite where scans are
> > >> > not reset when the processed region is moved, leading to missing data
> > >> from
> > >> > the scan.
> > >> >
> > >> > I am not positive yet that the issue is on the HBase side, will report
> > >> here
> > >> > (and open a ticket if needed) when I know more.
> > >> >
> > >> > Istvan
> > >> >
> > >> > On Sat, Sep 13, 2025 at 2:18 PM 张铎(Duo Zhang) <pa...@gmail.com>
> > >> wrote:
> > >> >
> > >> > > Only 4 issues left on 3.0.0-beta-2 now, will wait for one more week to
> > >> > > see if we can get these issues done, otherwise will move them out of
> > >> > > 3.0.0.
> > >> > >
> > >> > > And the HBCK2 issue has been fixed.
> > >> > >
> > >> > > Plan to cut a new hbase-operator-tools release before cutting
> > >> 3.0.0-beta-2.
> > >> > >
> > >> > > Thanks.
> > >> > >
> > >> > > 张铎(Duo Zhang) <pa...@gmail.com> 于2025年8月31日周日 22:57写道:
> > >> > > >
> > >> > > > I've finished one round of reviewing pending issues for
> > >> 3.0.0-beta-2.
> > >> > > >
> > >> > > > There are still some necessary ones we need to get in before the
> > >> final
> > >> > > > 3.0.0 release, like make HBCK2 work against hbase3.
> > >> > > >
> > >> > > > Will keep working on them.
> > >> > > >
> > >> > > > Thanks.
> > >> > > >
> > >> > > > Nihal Jain <ni...@gmail.com> 于2025年6月19日周四 16:06写道:
> > >> > > > >
> > >> > > > > Yes, from hbase-thirdparty side, release is done and also
> > >> consumed in
> > >> > > hbase
> > >> > > > > branch-2.5 and branch-2.6 and we are good here.
> > >> > > > >
> > >> > > > > On Wed, 18 Jun 2025 at 8:48 PM, 张铎(Duo Zhang) <
> > >> palomino219@gmail.com>
> > >> > > wrote:
> > >> > > > >
> > >> > > > > > Oh, I think we should cut 2.6.3 and 2.5.12 too? Since we have
> > >> already
> > >> > > > > > made the new thirdparty release without jetty 12.
> > >> > > > > >
> > >> > > > > > Thanks.
> > >> > > > > >
> > >> > > > > > Nihal Jain <ni...@gmail.com> 于2025年6月18日周三 01:51写道:
> > >> > > > > > >
> > >> > > > > > > Ah yes, let me restart work for hbase thirdparty release for
> > >> jetty
> > >> > > 12
> > >> > > > > > soon.
> > >> > > > > > >
> > >> > > > > > > On Tue, 17 Jun 2025 at 10:37 PM, Istvan Toth
> > >> > > <stoty@cloudera.com.invalid
> > >> > > > > > >
> > >> > > > > > > wrote:
> > >> > > > > > >
> > >> > > > > > > > I think we should make sure we're using Jetty 12 in the next
> > >> > > beta. (
> > >> > > > > > so we
> > >> > > > > > > > need a thirdparty release for Jetty 12)
> > >> > > > > > > > We could also review other dependencies, and bump the ones
> > >> (if
> > >> > > any)
> > >> > > > > > that
> > >> > > > > > > > were kept on old versions because of the Java 8 requirement.
> > >> > > > > > > >
> > >> > > > > > > > Istvan
> > >> > > > > > > >
> > >> > > > > > > > On Tue, Jun 17, 2025 at 11:08 AM 张铎(Duo Zhang) <
> > >> > > palomino219@gmail.com>
> > >> > > > > > > > wrote:
> > >> > > > > > > >
> > >> > > > > > > > > The good news is that I've finally got a successful 10B
> > >> ITBLL
> > >> > > run.
> > >> > > > > > > > >
> > >> > > > > > > > > You can check HBASE-27919 for more details.
> > >> > > > > > > > >
> > >> > > > > > > > > Next let me cleanup the jira issues and nightly runs.
> > >> > > > > > > > >
> > >> > > > > > > > > This time I think we can see 3.0.0 out soon.
> > >> > > > > > > > >
> > >> > > > > > > > > Thanks.
> > >> > > > > > > > >
> > >> > > > > > > > > Nihal Jain <ni...@apache.org> 于2025年3月12日周三 03:37写道:
> > >> > > > > > > > > >
> > >> > > > > > > > > > Hi Duo,
> > >> > > > > > > > > >
> > >> > > > > > > > > > Have you tried hbase-3.x on a kerberized env? I
> > >> encountered
> > >> > > > > > HBASE-29144
> > >> > > > > > > > > with kerberos. Posting here to just bring to attention.
> > >> > > > > > > > > >
> > >> > > > > > > > > > Regards
> > >> > > > > > > > > > Nihal
> > >> > > > > > > > > >
> > >> > > > > > > > > > On 2025/03/08 01:49:12 "张铎(Duo Zhang)" wrote:
> > >> > > > > > > > > > > A good news, cluster replication worked fine while
> > >> rolling
> > >> > > > > > upgrading
> > >> > > > > > > > > in my test.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > Let me focus on the region replication stuff. After
> > >> fixing
> > >> > > this
> > >> > > > > > > > issue,
> > >> > > > > > > > > > > the only blocker is a stable 10B ITBLL run.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > Hope we could get 3.0.0 out soon.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > Thanks.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > 张铎(Duo Zhang) <pa...@gmail.com> 于2025年3月6日周四
> > >> > > 22:55写道:
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > I've finished one round of rolling upgrading test.
> > >> It
> > >> > > worked
> > >> > > > > > fine.
> > >> > > > > > > > > > > > Please see HBASE-29161.
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > But soon I found a problem when memstore
> > >> replication is
> > >> > > > > > enabled,
> > >> > > > > > > > and
> > >> > > > > > > > > > > > filed HBASE-29169.
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > I will test upgrading with cluster replication too,
> > >> to
> > >> > > see if
> > >> > > > > > there
> > >> > > > > > > > > > > > are any other issues.
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > Thanks.
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > Istvan Toth <st...@apache.org> 于2025年1月20日周一
> > >> 14:24写道:
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > For HBASE-28431, only the final step of adding the
> > >> > > > > > Hadoop-less
> > >> > > > > > > > > assembly is
> > >> > > > > > > > > > > > > missing.
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > Please review
> > >> > > https://github.com/apache/hbase/pull/5766 if
> > >> > > > > > you
> > >> > > > > > > > > have the
> > >> > > > > > > > > > > > > time.
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > Istvan
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > On Tue, Dec 31, 2024 at 2:43 PM 张铎(Duo Zhang) <
> > >> > > > > > > > > palomino219@gmail.com> wrote:
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > I've finished the first round of issue
> > >> alignment, it
> > >> > > took
> > >> > > > > > > > > several months...
> > >> > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > And I've created a release audit tool for 3.0.0
> > >> > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > https://github.com/apache9/hbase-release-audit
> > >> > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > where we can have several ignore files, so we
> > >> do not
> > >> > > need
> > >> > > > > > to
> > >> > > > > > > > > manually
> > >> > > > > > > > > > > > > > check the result every time.
> > >> > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > So now, there are still several things to be
> > >> done
> > >> > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > 1. Make sure we have all the features and fixes
> > >> we
> > >> > > want to
> > >> > > > > > land
> > >> > > > > > > > > on
> > >> > > > > > > > > > > > > > branch-3. For example, I think Istvan's assembly
> > >> > > cleanup
> > >> > > > > > should
> > >> > > > > > > > > be
> > >> > > > > > > > > > > > > > included. Please report other issues you want to
> > >> > > land in
> > >> > > > > > final
> > >> > > > > > > > > 3.0.0.
> > >> > > > > > > > > > > > > > 2. Several stable 10B ITBLL runs. I will try to
> > >> see
> > >> > > if I
> > >> > > > > > can
> > >> > > > > > > > get
> > >> > > > > > > > > > > > > > enough resources but it will be great if others
> > >> > > could help
> > >> > > > > > > > here.
> > >> > > > > > > > > > > > > > 3. Rolling upgrading test. I plan to do rolling
> > >> > > upgrade
> > >> > > > > > test
> > >> > > > > > > > > with the
> > >> > > > > > > > > > > > > > latest 2.6.x and 2.5.x releases. Since we have
> > >> big
> > >> > > changes
> > >> > > > > > on
> > >> > > > > > > > > > > > > > replication and region replica replication, I
> > >> plan
> > >> > > to have
> > >> > > > > > > > tests
> > >> > > > > > > > > with
> > >> > > > > > > > > > > > > > these two features enabled too if possible.
> > >> > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > Any other suggestions?
> > >> > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > Thanks.
> > >> > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > 张铎(Duo Zhang) <pa...@gmail.com>
> > >> 于2024年9月19日周四
> > >> > > > > > 22:35写道:
> > >> > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > And do we want to apply the same trick to
> > >> > > MasterObserver
> > >> > > > > > and
> > >> > > > > > > > > > > > > > > RegionServerObserver?
> > >> > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > For RegionObserver, we'd better do this
> > >> because
> > >> > > there are
> > >> > > > > > > > > already
> > >> > > > > > > > > > > > > > > methods in it that use the ObserverContext<?
> > >> > > extends
> > >> > > > > > > > > > > > > > > RegionCoprocessorEnvironment> declaration,
> > >> which
> > >> > > were
> > >> > > > > > added
> > >> > > > > > > > in
> > >> > > > > > > > > > > > > > > HBASE-10378.
> > >> > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > For other coprocessor observers, we do not
> > >> have
> > >> > > such
> > >> > > > > > methods
> > >> > > > > > > > > yet.
> > >> > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > Nick Dimiduk <nd...@apache.org>
> > >> 于2024年9月19日周四
> > >> > > > > > 22:03写道:
> > >> > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > We do it here or we wait for 4.0. I'm
> > >> inclined
> > >> > > toward
> > >> > > > > > doing
> > >> > > > > > > > > it here
> > >> > > > > > > > > > > > > > > > because (i think) we expect coprocessor
> > >> > > > > > implementations to
> > >> > > > > > > > > necessarily
> > >> > > > > > > > > > > > > > > > recompile anyway. Though I admit I've not
> > >> thought
> > >> > > > > > through
> > >> > > > > > > > > what
> > >> > > > > > > > > > > > > > features are
> > >> > > > > > > > > > > > > > > > enabled by this new generic signature.
> > >> > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > On Thu, Sep 19, 2024 at 3:57 PM 张铎(Duo
> > >> Zhang) <
> > >> > > > > > > > > palomino219@gmail.com>
> > >> > > > > > > > > > > > > > wrote:
> > >> > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > Filed HBASE-28862.
> > >> > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > It will break old coprocessor
> > >> implementations
> > >> > > even if
> > >> > > > > > > > they
> > >> > > > > > > > > do not use
> > >> > > > > > > > > > > > > > > > > any deprecated method. But since it does
> > >> not
> > >> > > change
> > >> > > > > > any
> > >> > > > > > > > > abilities, I
> > >> > > > > > > > > > > > > > > > > think it is OK to include this in a major
> > >> > > release as
> > >> > > > > > we
> > >> > > > > > > > > have already
> > >> > > > > > > > > > > > > > > > > done bunch of breaking changes, like
> > >> changing
> > >> > > the
> > >> > > > > > > > protobuf
> > >> > > > > > > > > from
> > >> > > > > > > > > > > > > > > > > unshaded one to shaded and relocated
> > >> one...
> > >> > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > Thanks.
> > >> > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > 张铎(Duo Zhang) <pa...@gmail.com>
> > >> > > 于2024年9月19日周四
> > >> > > > > > > > > 17:36写道:
> > >> > > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > > Thanks for pointing this TODO out.
> > >> > > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > > Let me file an issue abort this. For me
> > >> I
> > >> > > think we
> > >> > > > > > > > > should do this
> > >> > > > > > > > > > > > > > in
> > >> > > > > > > > > > > > > > > > > 3.0.0.
> > >> > > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > > Nick Dimiduk <nd...@apache.org>
> > >> > > 于2024年9月19日周四
> > >> > > > > > > > > 16:46写道:
> > >> > > > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > > > Should we take the opportunity to
> > >> upgrade
> > >> > > the
> > >> > > > > > generic
> > >> > > > > > > > > > > > > > definitions in
> > >> > > > > > > > > > > > > > > > > our
> > >> > > > > > > > > > > > > > > > > > > coprocessor signatures as well?
> > >> > > > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > >
> > >> > >
> > >> https://github.com/apache/hbase/blob/edbb145a3f0c6847f444fefdadea5b82d02b0bdb/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java#L101
> > >> > > > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > > > On Tue, Sep 17, 2024 at 4:41 PM Duo
> > >> Zhang <
> > >> > > > > > > > > zhangduo@apache.org>
> > >> > > > > > > > > > > > > > wrote:
> > >> > > > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > > > > I've done several rounds of API
> > >> cleanup
> > >> > > in
> > >> > > > > > > > > HBASE-24888 and now
> > >> > > > > > > > > > > > > > for me
> > >> > > > > > > > > > > > > > > > > > > > I think it is enough for making the
> > >> new
> > >> > > major
> > >> > > > > > > > > release.
> > >> > > > > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > > > > I've filed HBASE-28844 for aligning
> > >> the
> > >> > > commit
> > >> > > > > > > > > history and jira
> > >> > > > > > > > > > > > > > > > > > > > issues, it will be a huge project
> > >> for a
> > >> > > major
> > >> > > > > > > > > release and may
> > >> > > > > > > > > > > > > > take
> > >> > > > > > > > > > > > > > > > > > > > several weeks.
> > >> > > > > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > > > > After that, I think we are good to
> > >> go.
> > >> > > And I
> > >> > > > > > will
> > >> > > > > > > > > also try to
> > >> > > > > > > > > > > > > > find
> > >> > > > > > > > > > > > > > > > > > > > some resources to run several
> > >> rounds of
> > >> > > ITBLL
> > >> > > > > > and
> > >> > > > > > > > > also some
> > >> > > > > > > > > > > > > > upgrading
> > >> > > > > > > > > > > > > > > > > > > > tests. Finally, after everything is
> > >> OK,
> > >> > > I will
> > >> > > > > > > > > release
> > >> > > > > > > > > > > > > > 3.0.0-beta-2,
> > >> > > > > > > > > > > > > > > > > > > > and then cut branch-3.0 from the
> > >> > > 3.0.0-beta-2
> > >> > > > > > tag
> > >> > > > > > > > > and make the
> > >> > > > > > > > > > > > > > 3.0.0
> > >> > > > > > > > > > > > > > > > > > > > release soon.
> > >> > > > > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > > > > Hope I can get all these things done
> > >> > > within the
> > >> > > > > > > > year
> > >> > > > > > > > > 2024.
> > >> > > > > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > > > > > Thanks.
> > >> > > > > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > > > > >
> > >> > > > > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > --
> > >> > > > > > > > *István Tóth* | Sr. Staff Software Engineer
> > >> > > > > > > > *Email*: stoty@cloudera.com
> > >> > > > > > > > cloudera.com <https://www.cloudera.com>
> > >> > > > > > > > [image: Cloudera] <https://www.cloudera.com/>
> > >> > > > > > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera>
> > >> > > [image:
> > >> > > > > > > > Cloudera on Facebook] <https://www.facebook.com/cloudera>
> > >> > > [image:
> > >> > > > > > Cloudera
> > >> > > > > > > > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > >> > > > > > > > ------------------------------
> > >> > > > > > > > ------------------------------
> > >> > > > > > > >
> > >> > > > > >
> > >> > >
> > >> >
> > >> >
> > >> > --
> > >> > *István Tóth* | Sr. Staff Software Engineer
> > >> > *Email*: stoty@cloudera.com
> > >> > cloudera.com <https://www.cloudera.com>
> > >> > [image: Cloudera] <https://www.cloudera.com/>
> > >> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > >> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > >> Cloudera
> > >> > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > >> > ------------------------------
> > >> > ------------------------------
> > >>
> > >
> > >
> > > --
> > > *István Tóth* | Sr. Staff Software Engineer
> > > *Email*: stoty@cloudera.com
> > > cloudera.com <https://www.cloudera.com>
> > > [image: Cloudera] <https://www.cloudera.com/>
> > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > > Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > ------------------------------
> > > ------------------------------
> > >
> >
> >
> > --
> > *István Tóth* | Sr. Staff Software Engineer
> > *Email*: stoty@cloudera.com
> > cloudera.com <https://www.cloudera.com>
> > [image: Cloudera] <https://www.cloudera.com/>
> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > ------------------------------
> > ------------------------------
Re: [DISCUSS] For 3.0.0-beta-2 and the final 3.0.0 release
Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
Going to cut 3.0.0-beta-2 right now.
Moved some issues out of 3.0.0-beta-2.
Thanks.
张铎(Duo Zhang) <pa...@gmail.com> 于2026年6月6日周六 17:12写道:
>
> The JUnit5 migration is nearly done.
>
> I will start new rounds of 10B ITBLL to verify the stability of branch-3.
>
> After merging the last several PRs for migration and the ITBLL result
> is good, I will start cutting the 3.0.0-beta-2 release.
>
> Thanks.
>
> 张铎(Duo Zhang) <pa...@gmail.com> 于2026年3月30日周一 19:28写道:
> >
> > Some updates here, there is no blocker issue for 3.0.0-beta-2 release
> > now, but the junit5 upgrading is in its last stage, where we want to
> > migrate all the tests in hbase-server, so I plan to finish the
> > migration before cutting the final release.
> >
> > I will also start to run several rounds of 10B ITBLL to check the
> > stability of branch-3.
> >
> > Thanks.
> >
> > 张铎(Duo Zhang) <pa...@gmail.com> 于2026年1月22日周四 19:15写道:
> > >
> > > Thanks Istvan for the detailed analysis!
> > >
> > > For HBASE-29834, I've already put up a PR to fix the problem. See
> > >
> > > https://github.com/apache/hbase/pull/7660
> > >
> > > And on HBASE-29843, I do not fully understand the problem and need
> > > more context about the usage on Phoenix side. I think we should be
> > > able to fix it.
> > >
> > > Thanks.
> > >
> > > Istvan Toth <st...@cloudera.com> 于2026年1月22日周四 14:33写道:
> > > >
> > > > This seems to have been caused by forward synchronous RPC changes between
> > > > Hbase 2 and 3 (where HBase 2 simulates non-inclusive start keys by
> > > > adding a \00 byte and setting them as inclusive, while HBase 3 handles them
> > > > natively.
> > > > This is something that we can handle on the Phoenix side.
> > > >
> > > > On the other hand, I've found a major and a minor breaking change in HBase
> > > > 3:
> > > >
> > > > The big problem is HBASE-29843
> > > > <https://issues.apache.org/jira/browse/HBASE-29843>
> > > > HBaseRPCController.setPriority(final TableName tn) is never called on
> > > > branch-3+
> > > > Phoenix relies on setting it its own HBaseRPCController to set custom
> > > > priorities, but HBase 3 has mostly abandoned using HBaseRPCController,
> > > > leaving no
> > > > working mechanism for Phoenix to use.
> > > > HBase needs to restore using it to maintain compatibility. (or provide some
> > > > alternative method)
> > > >
> > > > The smaller problem is HBASE-29834
> > > > <https://issues.apache.org/jira/browse/HBASE-29834> MILLIS_BETWEEN_NEXTS
> > > > metric is not updated on branch-3+
> > > > This does not break Phoenix, it just happens to be detected by a Phoenix
> > > > test.
> > > >
> > > > Istvan
> > > >
> > > >
> > > > On Mon, Nov 3, 2025 at 10:22 AM Istvan Toth <st...@cloudera.com> wrote:
> > > >
> > > > > Sorry, I missed your reply eralier, Duo.
> > > > >
> > > > > I still haven't figured out the issue (though at least now I am now able
> > > > > to run those tests from a debugger)
> > > > > No ticket yet, as I'm still not sure which project to use.
> > > > >
> > > > > To repro this, we need to
> > > > > * install a current HBase 3 snapshot locally from branch-3 head,
> > > > > * check out my WIP Phoenix HBase 3 branch:
> > > > > https://github.com/stoty/phoenix/tree/PHOENIX-7164-2025
> > > > > * then run a faling test:
> > > > >
> > > > > *mvn clean verify -am -pl phoenix-core
> > > > > -Dit.test=BaseAggregateWithRegionMoves2IT#testGroupByArray
> > > > > -Dhbase.profile=3.0 -Dtest=foo*
> > > > > There are other similar tests failures, missing expected ResultSet rows
> > > > > after aggressively moving around the table regions during the query.
> > > > >
> > > > > Istvan
> > > > >
> > > > > On Thu, Oct 16, 2025 at 3:49 PM 张铎(Duo Zhang) <pa...@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> Is there an issue in HBase or Phoenix for tracking this problem?
> > > > >>
> > > > >> Thanks.
> > > > >>
> > > > >> Istvan Toth <st...@cloudera.com.invalid> 于2025年10月16日周四 11:58写道:
> > > > >> >
> > > > >> > Quick heads up:
> > > > >> > I'm investigating a regression in the Phoenix test suite where scans are
> > > > >> > not reset when the processed region is moved, leading to missing data
> > > > >> from
> > > > >> > the scan.
> > > > >> >
> > > > >> > I am not positive yet that the issue is on the HBase side, will report
> > > > >> here
> > > > >> > (and open a ticket if needed) when I know more.
> > > > >> >
> > > > >> > Istvan
> > > > >> >
> > > > >> > On Sat, Sep 13, 2025 at 2:18 PM 张铎(Duo Zhang) <pa...@gmail.com>
> > > > >> wrote:
> > > > >> >
> > > > >> > > Only 4 issues left on 3.0.0-beta-2 now, will wait for one more week to
> > > > >> > > see if we can get these issues done, otherwise will move them out of
> > > > >> > > 3.0.0.
> > > > >> > >
> > > > >> > > And the HBCK2 issue has been fixed.
> > > > >> > >
> > > > >> > > Plan to cut a new hbase-operator-tools release before cutting
> > > > >> 3.0.0-beta-2.
> > > > >> > >
> > > > >> > > Thanks.
> > > > >> > >
> > > > >> > > 张铎(Duo Zhang) <pa...@gmail.com> 于2025年8月31日周日 22:57写道:
> > > > >> > > >
> > > > >> > > > I've finished one round of reviewing pending issues for
> > > > >> 3.0.0-beta-2.
> > > > >> > > >
> > > > >> > > > There are still some necessary ones we need to get in before the
> > > > >> final
> > > > >> > > > 3.0.0 release, like make HBCK2 work against hbase3.
> > > > >> > > >
> > > > >> > > > Will keep working on them.
> > > > >> > > >
> > > > >> > > > Thanks.
> > > > >> > > >
> > > > >> > > > Nihal Jain <ni...@gmail.com> 于2025年6月19日周四 16:06写道:
> > > > >> > > > >
> > > > >> > > > > Yes, from hbase-thirdparty side, release is done and also
> > > > >> consumed in
> > > > >> > > hbase
> > > > >> > > > > branch-2.5 and branch-2.6 and we are good here.
> > > > >> > > > >
> > > > >> > > > > On Wed, 18 Jun 2025 at 8:48 PM, 张铎(Duo Zhang) <
> > > > >> palomino219@gmail.com>
> > > > >> > > wrote:
> > > > >> > > > >
> > > > >> > > > > > Oh, I think we should cut 2.6.3 and 2.5.12 too? Since we have
> > > > >> already
> > > > >> > > > > > made the new thirdparty release without jetty 12.
> > > > >> > > > > >
> > > > >> > > > > > Thanks.
> > > > >> > > > > >
> > > > >> > > > > > Nihal Jain <ni...@gmail.com> 于2025年6月18日周三 01:51写道:
> > > > >> > > > > > >
> > > > >> > > > > > > Ah yes, let me restart work for hbase thirdparty release for
> > > > >> jetty
> > > > >> > > 12
> > > > >> > > > > > soon.
> > > > >> > > > > > >
> > > > >> > > > > > > On Tue, 17 Jun 2025 at 10:37 PM, Istvan Toth
> > > > >> > > <stoty@cloudera.com.invalid
> > > > >> > > > > > >
> > > > >> > > > > > > wrote:
> > > > >> > > > > > >
> > > > >> > > > > > > > I think we should make sure we're using Jetty 12 in the next
> > > > >> > > beta. (
> > > > >> > > > > > so we
> > > > >> > > > > > > > need a thirdparty release for Jetty 12)
> > > > >> > > > > > > > We could also review other dependencies, and bump the ones
> > > > >> (if
> > > > >> > > any)
> > > > >> > > > > > that
> > > > >> > > > > > > > were kept on old versions because of the Java 8 requirement.
> > > > >> > > > > > > >
> > > > >> > > > > > > > Istvan
> > > > >> > > > > > > >
> > > > >> > > > > > > > On Tue, Jun 17, 2025 at 11:08 AM 张铎(Duo Zhang) <
> > > > >> > > palomino219@gmail.com>
> > > > >> > > > > > > > wrote:
> > > > >> > > > > > > >
> > > > >> > > > > > > > > The good news is that I've finally got a successful 10B
> > > > >> ITBLL
> > > > >> > > run.
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > You can check HBASE-27919 for more details.
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > Next let me cleanup the jira issues and nightly runs.
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > This time I think we can see 3.0.0 out soon.
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > Thanks.
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > Nihal Jain <ni...@apache.org> 于2025年3月12日周三 03:37写道:
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > Hi Duo,
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > Have you tried hbase-3.x on a kerberized env? I
> > > > >> encountered
> > > > >> > > > > > HBASE-29144
> > > > >> > > > > > > > > with kerberos. Posting here to just bring to attention.
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > Regards
> > > > >> > > > > > > > > > Nihal
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > On 2025/03/08 01:49:12 "张铎(Duo Zhang)" wrote:
> > > > >> > > > > > > > > > > A good news, cluster replication worked fine while
> > > > >> rolling
> > > > >> > > > > > upgrading
> > > > >> > > > > > > > > in my test.
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > Let me focus on the region replication stuff. After
> > > > >> fixing
> > > > >> > > this
> > > > >> > > > > > > > issue,
> > > > >> > > > > > > > > > > the only blocker is a stable 10B ITBLL run.
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > Hope we could get 3.0.0 out soon.
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > Thanks.
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > 张铎(Duo Zhang) <pa...@gmail.com> 于2025年3月6日周四
> > > > >> > > 22:55写道:
> > > > >> > > > > > > > > > > >
> > > > >> > > > > > > > > > > > I've finished one round of rolling upgrading test.
> > > > >> It
> > > > >> > > worked
> > > > >> > > > > > fine.
> > > > >> > > > > > > > > > > > Please see HBASE-29161.
> > > > >> > > > > > > > > > > >
> > > > >> > > > > > > > > > > > But soon I found a problem when memstore
> > > > >> replication is
> > > > >> > > > > > enabled,
> > > > >> > > > > > > > and
> > > > >> > > > > > > > > > > > filed HBASE-29169.
> > > > >> > > > > > > > > > > >
> > > > >> > > > > > > > > > > > I will test upgrading with cluster replication too,
> > > > >> to
> > > > >> > > see if
> > > > >> > > > > > there
> > > > >> > > > > > > > > > > > are any other issues.
> > > > >> > > > > > > > > > > >
> > > > >> > > > > > > > > > > > Thanks.
> > > > >> > > > > > > > > > > >
> > > > >> > > > > > > > > > > > Istvan Toth <st...@apache.org> 于2025年1月20日周一
> > > > >> 14:24写道:
> > > > >> > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > For HBASE-28431, only the final step of adding the
> > > > >> > > > > > Hadoop-less
> > > > >> > > > > > > > > assembly is
> > > > >> > > > > > > > > > > > > missing.
> > > > >> > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > Please review
> > > > >> > > https://github.com/apache/hbase/pull/5766 if
> > > > >> > > > > > you
> > > > >> > > > > > > > > have the
> > > > >> > > > > > > > > > > > > time.
> > > > >> > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > Istvan
> > > > >> > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > On Tue, Dec 31, 2024 at 2:43 PM 张铎(Duo Zhang) <
> > > > >> > > > > > > > > palomino219@gmail.com> wrote:
> > > > >> > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > I've finished the first round of issue
> > > > >> alignment, it
> > > > >> > > took
> > > > >> > > > > > > > > several months...
> > > > >> > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > And I've created a release audit tool for 3.0.0
> > > > >> > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > https://github.com/apache9/hbase-release-audit
> > > > >> > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > where we can have several ignore files, so we
> > > > >> do not
> > > > >> > > need
> > > > >> > > > > > to
> > > > >> > > > > > > > > manually
> > > > >> > > > > > > > > > > > > > check the result every time.
> > > > >> > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > So now, there are still several things to be
> > > > >> done
> > > > >> > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > 1. Make sure we have all the features and fixes
> > > > >> we
> > > > >> > > want to
> > > > >> > > > > > land
> > > > >> > > > > > > > > on
> > > > >> > > > > > > > > > > > > > branch-3. For example, I think Istvan's assembly
> > > > >> > > cleanup
> > > > >> > > > > > should
> > > > >> > > > > > > > > be
> > > > >> > > > > > > > > > > > > > included. Please report other issues you want to
> > > > >> > > land in
> > > > >> > > > > > final
> > > > >> > > > > > > > > 3.0.0.
> > > > >> > > > > > > > > > > > > > 2. Several stable 10B ITBLL runs. I will try to
> > > > >> see
> > > > >> > > if I
> > > > >> > > > > > can
> > > > >> > > > > > > > get
> > > > >> > > > > > > > > > > > > > enough resources but it will be great if others
> > > > >> > > could help
> > > > >> > > > > > > > here.
> > > > >> > > > > > > > > > > > > > 3. Rolling upgrading test. I plan to do rolling
> > > > >> > > upgrade
> > > > >> > > > > > test
> > > > >> > > > > > > > > with the
> > > > >> > > > > > > > > > > > > > latest 2.6.x and 2.5.x releases. Since we have
> > > > >> big
> > > > >> > > changes
> > > > >> > > > > > on
> > > > >> > > > > > > > > > > > > > replication and region replica replication, I
> > > > >> plan
> > > > >> > > to have
> > > > >> > > > > > > > tests
> > > > >> > > > > > > > > with
> > > > >> > > > > > > > > > > > > > these two features enabled too if possible.
> > > > >> > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > Any other suggestions?
> > > > >> > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > Thanks.
> > > > >> > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > 张铎(Duo Zhang) <pa...@gmail.com>
> > > > >> 于2024年9月19日周四
> > > > >> > > > > > 22:35写道:
> > > > >> > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > And do we want to apply the same trick to
> > > > >> > > MasterObserver
> > > > >> > > > > > and
> > > > >> > > > > > > > > > > > > > > RegionServerObserver?
> > > > >> > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > For RegionObserver, we'd better do this
> > > > >> because
> > > > >> > > there are
> > > > >> > > > > > > > > already
> > > > >> > > > > > > > > > > > > > > methods in it that use the ObserverContext<?
> > > > >> > > extends
> > > > >> > > > > > > > > > > > > > > RegionCoprocessorEnvironment> declaration,
> > > > >> which
> > > > >> > > were
> > > > >> > > > > > added
> > > > >> > > > > > > > in
> > > > >> > > > > > > > > > > > > > > HBASE-10378.
> > > > >> > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > For other coprocessor observers, we do not
> > > > >> have
> > > > >> > > such
> > > > >> > > > > > methods
> > > > >> > > > > > > > > yet.
> > > > >> > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > Nick Dimiduk <nd...@apache.org>
> > > > >> 于2024年9月19日周四
> > > > >> > > > > > 22:03写道:
> > > > >> > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > We do it here or we wait for 4.0. I'm
> > > > >> inclined
> > > > >> > > toward
> > > > >> > > > > > doing
> > > > >> > > > > > > > > it here
> > > > >> > > > > > > > > > > > > > > > because (i think) we expect coprocessor
> > > > >> > > > > > implementations to
> > > > >> > > > > > > > > necessarily
> > > > >> > > > > > > > > > > > > > > > recompile anyway. Though I admit I've not
> > > > >> thought
> > > > >> > > > > > through
> > > > >> > > > > > > > > what
> > > > >> > > > > > > > > > > > > > features are
> > > > >> > > > > > > > > > > > > > > > enabled by this new generic signature.
> > > > >> > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > On Thu, Sep 19, 2024 at 3:57 PM 张铎(Duo
> > > > >> Zhang) <
> > > > >> > > > > > > > > palomino219@gmail.com>
> > > > >> > > > > > > > > > > > > > wrote:
> > > > >> > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > Filed HBASE-28862.
> > > > >> > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > It will break old coprocessor
> > > > >> implementations
> > > > >> > > even if
> > > > >> > > > > > > > they
> > > > >> > > > > > > > > do not use
> > > > >> > > > > > > > > > > > > > > > > any deprecated method. But since it does
> > > > >> not
> > > > >> > > change
> > > > >> > > > > > any
> > > > >> > > > > > > > > abilities, I
> > > > >> > > > > > > > > > > > > > > > > think it is OK to include this in a major
> > > > >> > > release as
> > > > >> > > > > > we
> > > > >> > > > > > > > > have already
> > > > >> > > > > > > > > > > > > > > > > done bunch of breaking changes, like
> > > > >> changing
> > > > >> > > the
> > > > >> > > > > > > > protobuf
> > > > >> > > > > > > > > from
> > > > >> > > > > > > > > > > > > > > > > unshaded one to shaded and relocated
> > > > >> one...
> > > > >> > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > Thanks.
> > > > >> > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > 张铎(Duo Zhang) <pa...@gmail.com>
> > > > >> > > 于2024年9月19日周四
> > > > >> > > > > > > > > 17:36写道:
> > > > >> > > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > > Thanks for pointing this TODO out.
> > > > >> > > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > > Let me file an issue abort this. For me
> > > > >> I
> > > > >> > > think we
> > > > >> > > > > > > > > should do this
> > > > >> > > > > > > > > > > > > > in
> > > > >> > > > > > > > > > > > > > > > > 3.0.0.
> > > > >> > > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > > Nick Dimiduk <nd...@apache.org>
> > > > >> > > 于2024年9月19日周四
> > > > >> > > > > > > > > 16:46写道:
> > > > >> > > > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > > > Should we take the opportunity to
> > > > >> upgrade
> > > > >> > > the
> > > > >> > > > > > generic
> > > > >> > > > > > > > > > > > > > definitions in
> > > > >> > > > > > > > > > > > > > > > > our
> > > > >> > > > > > > > > > > > > > > > > > > coprocessor signatures as well?
> > > > >> > > > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > >
> > > > >> > >
> > > > >> https://github.com/apache/hbase/blob/edbb145a3f0c6847f444fefdadea5b82d02b0bdb/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java#L101
> > > > >> > > > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > > > On Tue, Sep 17, 2024 at 4:41 PM Duo
> > > > >> Zhang <
> > > > >> > > > > > > > > zhangduo@apache.org>
> > > > >> > > > > > > > > > > > > > wrote:
> > > > >> > > > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > > > > I've done several rounds of API
> > > > >> cleanup
> > > > >> > > in
> > > > >> > > > > > > > > HBASE-24888 and now
> > > > >> > > > > > > > > > > > > > for me
> > > > >> > > > > > > > > > > > > > > > > > > > I think it is enough for making the
> > > > >> new
> > > > >> > > major
> > > > >> > > > > > > > > release.
> > > > >> > > > > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > > > > I've filed HBASE-28844 for aligning
> > > > >> the
> > > > >> > > commit
> > > > >> > > > > > > > > history and jira
> > > > >> > > > > > > > > > > > > > > > > > > > issues, it will be a huge project
> > > > >> for a
> > > > >> > > major
> > > > >> > > > > > > > > release and may
> > > > >> > > > > > > > > > > > > > take
> > > > >> > > > > > > > > > > > > > > > > > > > several weeks.
> > > > >> > > > > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > > > > After that, I think we are good to
> > > > >> go.
> > > > >> > > And I
> > > > >> > > > > > will
> > > > >> > > > > > > > > also try to
> > > > >> > > > > > > > > > > > > > find
> > > > >> > > > > > > > > > > > > > > > > > > > some resources to run several
> > > > >> rounds of
> > > > >> > > ITBLL
> > > > >> > > > > > and
> > > > >> > > > > > > > > also some
> > > > >> > > > > > > > > > > > > > upgrading
> > > > >> > > > > > > > > > > > > > > > > > > > tests. Finally, after everything is
> > > > >> OK,
> > > > >> > > I will
> > > > >> > > > > > > > > release
> > > > >> > > > > > > > > > > > > > 3.0.0-beta-2,
> > > > >> > > > > > > > > > > > > > > > > > > > and then cut branch-3.0 from the
> > > > >> > > 3.0.0-beta-2
> > > > >> > > > > > tag
> > > > >> > > > > > > > > and make the
> > > > >> > > > > > > > > > > > > > 3.0.0
> > > > >> > > > > > > > > > > > > > > > > > > > release soon.
> > > > >> > > > > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > > > > Hope I can get all these things done
> > > > >> > > within the
> > > > >> > > > > > > > year
> > > > >> > > > > > > > > 2024.
> > > > >> > > > > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > > > > > Thanks.
> > > > >> > > > > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > > > > >
> > > > >> > > > > > > > > > > > > >
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > > --
> > > > >> > > > > > > > *István Tóth* | Sr. Staff Software Engineer
> > > > >> > > > > > > > *Email*: stoty@cloudera.com
> > > > >> > > > > > > > cloudera.com <https://www.cloudera.com>
> > > > >> > > > > > > > [image: Cloudera] <https://www.cloudera.com/>
> > > > >> > > > > > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera>
> > > > >> > > [image:
> > > > >> > > > > > > > Cloudera on Facebook] <https://www.facebook.com/cloudera>
> > > > >> > > [image:
> > > > >> > > > > > Cloudera
> > > > >> > > > > > > > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > > >> > > > > > > > ------------------------------
> > > > >> > > > > > > > ------------------------------
> > > > >> > > > > > > >
> > > > >> > > > > >
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >> > --
> > > > >> > *István Tóth* | Sr. Staff Software Engineer
> > > > >> > *Email*: stoty@cloudera.com
> > > > >> > cloudera.com <https://www.cloudera.com>
> > > > >> > [image: Cloudera] <https://www.cloudera.com/>
> > > > >> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > > > >> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > > > >> Cloudera
> > > > >> > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > > >> > ------------------------------
> > > > >> > ------------------------------
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > *István Tóth* | Sr. Staff Software Engineer
> > > > > *Email*: stoty@cloudera.com
> > > > > cloudera.com <https://www.cloudera.com>
> > > > > [image: Cloudera] <https://www.cloudera.com/>
> > > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > > > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > > > > Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > > > ------------------------------
> > > > > ------------------------------
> > > > >
> > > >
> > > >
> > > > --
> > > > *István Tóth* | Sr. Staff Software Engineer
> > > > *Email*: stoty@cloudera.com
> > > > cloudera.com <https://www.cloudera.com>
> > > > [image: Cloudera] <https://www.cloudera.com/>
> > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> > > > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > > ------------------------------
> > > > ------------------------------
Re: [DISCUSS] For 3.0.0-beta-2 and the final 3.0.0 release
Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
The JUnit5 migration is nearly done.
I will start new rounds of 10B ITBLL to verify the stability of branch-3.
After merging the last several PRs for migration and the ITBLL result
is good, I will start cutting the 3.0.0-beta-2 release.
Thanks.
张铎(Duo Zhang) <pa...@gmail.com> 于2026年3月30日周一 19:28写道:
>
> Some updates here, there is no blocker issue for 3.0.0-beta-2 release
> now, but the junit5 upgrading is in its last stage, where we want to
> migrate all the tests in hbase-server, so I plan to finish the
> migration before cutting the final release.
>
> I will also start to run several rounds of 10B ITBLL to check the
> stability of branch-3.
>
> Thanks.
>
> 张铎(Duo Zhang) <pa...@gmail.com> 于2026年1月22日周四 19:15写道:
> >
> > Thanks Istvan for the detailed analysis!
> >
> > For HBASE-29834, I've already put up a PR to fix the problem. See
> >
> > https://github.com/apache/hbase/pull/7660
> >
> > And on HBASE-29843, I do not fully understand the problem and need
> > more context about the usage on Phoenix side. I think we should be
> > able to fix it.
> >
> > Thanks.
> >
> > Istvan Toth <st...@cloudera.com> 于2026年1月22日周四 14:33写道:
> > >
> > > This seems to have been caused by forward synchronous RPC changes between
> > > Hbase 2 and 3 (where HBase 2 simulates non-inclusive start keys by
> > > adding a \00 byte and setting them as inclusive, while HBase 3 handles them
> > > natively.
> > > This is something that we can handle on the Phoenix side.
> > >
> > > On the other hand, I've found a major and a minor breaking change in HBase
> > > 3:
> > >
> > > The big problem is HBASE-29843
> > > <https://issues.apache.org/jira/browse/HBASE-29843>
> > > HBaseRPCController.setPriority(final TableName tn) is never called on
> > > branch-3+
> > > Phoenix relies on setting it its own HBaseRPCController to set custom
> > > priorities, but HBase 3 has mostly abandoned using HBaseRPCController,
> > > leaving no
> > > working mechanism for Phoenix to use.
> > > HBase needs to restore using it to maintain compatibility. (or provide some
> > > alternative method)
> > >
> > > The smaller problem is HBASE-29834
> > > <https://issues.apache.org/jira/browse/HBASE-29834> MILLIS_BETWEEN_NEXTS
> > > metric is not updated on branch-3+
> > > This does not break Phoenix, it just happens to be detected by a Phoenix
> > > test.
> > >
> > > Istvan
> > >
> > >
> > > On Mon, Nov 3, 2025 at 10:22 AM Istvan Toth <st...@cloudera.com> wrote:
> > >
> > > > Sorry, I missed your reply eralier, Duo.
> > > >
> > > > I still haven't figured out the issue (though at least now I am now able
> > > > to run those tests from a debugger)
> > > > No ticket yet, as I'm still not sure which project to use.
> > > >
> > > > To repro this, we need to
> > > > * install a current HBase 3 snapshot locally from branch-3 head,
> > > > * check out my WIP Phoenix HBase 3 branch:
> > > > https://github.com/stoty/phoenix/tree/PHOENIX-7164-2025
> > > > * then run a faling test:
> > > >
> > > > *mvn clean verify -am -pl phoenix-core
> > > > -Dit.test=BaseAggregateWithRegionMoves2IT#testGroupByArray
> > > > -Dhbase.profile=3.0 -Dtest=foo*
> > > > There are other similar tests failures, missing expected ResultSet rows
> > > > after aggressively moving around the table regions during the query.
> > > >
> > > > Istvan
> > > >
> > > > On Thu, Oct 16, 2025 at 3:49 PM 张铎(Duo Zhang) <pa...@gmail.com>
> > > > wrote:
> > > >
> > > >> Is there an issue in HBase or Phoenix for tracking this problem?
> > > >>
> > > >> Thanks.
> > > >>
> > > >> Istvan Toth <st...@cloudera.com.invalid> 于2025年10月16日周四 11:58写道:
> > > >> >
> > > >> > Quick heads up:
> > > >> > I'm investigating a regression in the Phoenix test suite where scans are
> > > >> > not reset when the processed region is moved, leading to missing data
> > > >> from
> > > >> > the scan.
> > > >> >
> > > >> > I am not positive yet that the issue is on the HBase side, will report
> > > >> here
> > > >> > (and open a ticket if needed) when I know more.
> > > >> >
> > > >> > Istvan
> > > >> >
> > > >> > On Sat, Sep 13, 2025 at 2:18 PM 张铎(Duo Zhang) <pa...@gmail.com>
> > > >> wrote:
> > > >> >
> > > >> > > Only 4 issues left on 3.0.0-beta-2 now, will wait for one more week to
> > > >> > > see if we can get these issues done, otherwise will move them out of
> > > >> > > 3.0.0.
> > > >> > >
> > > >> > > And the HBCK2 issue has been fixed.
> > > >> > >
> > > >> > > Plan to cut a new hbase-operator-tools release before cutting
> > > >> 3.0.0-beta-2.
> > > >> > >
> > > >> > > Thanks.
> > > >> > >
> > > >> > > 张铎(Duo Zhang) <pa...@gmail.com> 于2025年8月31日周日 22:57写道:
> > > >> > > >
> > > >> > > > I've finished one round of reviewing pending issues for
> > > >> 3.0.0-beta-2.
> > > >> > > >
> > > >> > > > There are still some necessary ones we need to get in before the
> > > >> final
> > > >> > > > 3.0.0 release, like make HBCK2 work against hbase3.
> > > >> > > >
> > > >> > > > Will keep working on them.
> > > >> > > >
> > > >> > > > Thanks.
> > > >> > > >
> > > >> > > > Nihal Jain <ni...@gmail.com> 于2025年6月19日周四 16:06写道:
> > > >> > > > >
> > > >> > > > > Yes, from hbase-thirdparty side, release is done and also
> > > >> consumed in
> > > >> > > hbase
> > > >> > > > > branch-2.5 and branch-2.6 and we are good here.
> > > >> > > > >
> > > >> > > > > On Wed, 18 Jun 2025 at 8:48 PM, 张铎(Duo Zhang) <
> > > >> palomino219@gmail.com>
> > > >> > > wrote:
> > > >> > > > >
> > > >> > > > > > Oh, I think we should cut 2.6.3 and 2.5.12 too? Since we have
> > > >> already
> > > >> > > > > > made the new thirdparty release without jetty 12.
> > > >> > > > > >
> > > >> > > > > > Thanks.
> > > >> > > > > >
> > > >> > > > > > Nihal Jain <ni...@gmail.com> 于2025年6月18日周三 01:51写道:
> > > >> > > > > > >
> > > >> > > > > > > Ah yes, let me restart work for hbase thirdparty release for
> > > >> jetty
> > > >> > > 12
> > > >> > > > > > soon.
> > > >> > > > > > >
> > > >> > > > > > > On Tue, 17 Jun 2025 at 10:37 PM, Istvan Toth
> > > >> > > <stoty@cloudera.com.invalid
> > > >> > > > > > >
> > > >> > > > > > > wrote:
> > > >> > > > > > >
> > > >> > > > > > > > I think we should make sure we're using Jetty 12 in the next
> > > >> > > beta. (
> > > >> > > > > > so we
> > > >> > > > > > > > need a thirdparty release for Jetty 12)
> > > >> > > > > > > > We could also review other dependencies, and bump the ones
> > > >> (if
> > > >> > > any)
> > > >> > > > > > that
> > > >> > > > > > > > were kept on old versions because of the Java 8 requirement.
> > > >> > > > > > > >
> > > >> > > > > > > > Istvan
> > > >> > > > > > > >
> > > >> > > > > > > > On Tue, Jun 17, 2025 at 11:08 AM 张铎(Duo Zhang) <
> > > >> > > palomino219@gmail.com>
> > > >> > > > > > > > wrote:
> > > >> > > > > > > >
> > > >> > > > > > > > > The good news is that I've finally got a successful 10B
> > > >> ITBLL
> > > >> > > run.
> > > >> > > > > > > > >
> > > >> > > > > > > > > You can check HBASE-27919 for more details.
> > > >> > > > > > > > >
> > > >> > > > > > > > > Next let me cleanup the jira issues and nightly runs.
> > > >> > > > > > > > >
> > > >> > > > > > > > > This time I think we can see 3.0.0 out soon.
> > > >> > > > > > > > >
> > > >> > > > > > > > > Thanks.
> > > >> > > > > > > > >
> > > >> > > > > > > > > Nihal Jain <ni...@apache.org> 于2025年3月12日周三 03:37写道:
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > Hi Duo,
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > Have you tried hbase-3.x on a kerberized env? I
> > > >> encountered
> > > >> > > > > > HBASE-29144
> > > >> > > > > > > > > with kerberos. Posting here to just bring to attention.
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > Regards
> > > >> > > > > > > > > > Nihal
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > On 2025/03/08 01:49:12 "张铎(Duo Zhang)" wrote:
> > > >> > > > > > > > > > > A good news, cluster replication worked fine while
> > > >> rolling
> > > >> > > > > > upgrading
> > > >> > > > > > > > > in my test.
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > Let me focus on the region replication stuff. After
> > > >> fixing
> > > >> > > this
> > > >> > > > > > > > issue,
> > > >> > > > > > > > > > > the only blocker is a stable 10B ITBLL run.
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > Hope we could get 3.0.0 out soon.
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > Thanks.
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > 张铎(Duo Zhang) <pa...@gmail.com> 于2025年3月6日周四
> > > >> > > 22:55写道:
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > > I've finished one round of rolling upgrading test.
> > > >> It
> > > >> > > worked
> > > >> > > > > > fine.
> > > >> > > > > > > > > > > > Please see HBASE-29161.
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > > But soon I found a problem when memstore
> > > >> replication is
> > > >> > > > > > enabled,
> > > >> > > > > > > > and
> > > >> > > > > > > > > > > > filed HBASE-29169.
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > > I will test upgrading with cluster replication too,
> > > >> to
> > > >> > > see if
> > > >> > > > > > there
> > > >> > > > > > > > > > > > are any other issues.
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > > Thanks.
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > > Istvan Toth <st...@apache.org> 于2025年1月20日周一
> > > >> 14:24写道:
> > > >> > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > For HBASE-28431, only the final step of adding the
> > > >> > > > > > Hadoop-less
> > > >> > > > > > > > > assembly is
> > > >> > > > > > > > > > > > > missing.
> > > >> > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > Please review
> > > >> > > https://github.com/apache/hbase/pull/5766 if
> > > >> > > > > > you
> > > >> > > > > > > > > have the
> > > >> > > > > > > > > > > > > time.
> > > >> > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > Istvan
> > > >> > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > On Tue, Dec 31, 2024 at 2:43 PM 张铎(Duo Zhang) <
> > > >> > > > > > > > > palomino219@gmail.com> wrote:
> > > >> > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > I've finished the first round of issue
> > > >> alignment, it
> > > >> > > took
> > > >> > > > > > > > > several months...
> > > >> > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > And I've created a release audit tool for 3.0.0
> > > >> > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > https://github.com/apache9/hbase-release-audit
> > > >> > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > where we can have several ignore files, so we
> > > >> do not
> > > >> > > need
> > > >> > > > > > to
> > > >> > > > > > > > > manually
> > > >> > > > > > > > > > > > > > check the result every time.
> > > >> > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > So now, there are still several things to be
> > > >> done
> > > >> > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > 1. Make sure we have all the features and fixes
> > > >> we
> > > >> > > want to
> > > >> > > > > > land
> > > >> > > > > > > > > on
> > > >> > > > > > > > > > > > > > branch-3. For example, I think Istvan's assembly
> > > >> > > cleanup
> > > >> > > > > > should
> > > >> > > > > > > > > be
> > > >> > > > > > > > > > > > > > included. Please report other issues you want to
> > > >> > > land in
> > > >> > > > > > final
> > > >> > > > > > > > > 3.0.0.
> > > >> > > > > > > > > > > > > > 2. Several stable 10B ITBLL runs. I will try to
> > > >> see
> > > >> > > if I
> > > >> > > > > > can
> > > >> > > > > > > > get
> > > >> > > > > > > > > > > > > > enough resources but it will be great if others
> > > >> > > could help
> > > >> > > > > > > > here.
> > > >> > > > > > > > > > > > > > 3. Rolling upgrading test. I plan to do rolling
> > > >> > > upgrade
> > > >> > > > > > test
> > > >> > > > > > > > > with the
> > > >> > > > > > > > > > > > > > latest 2.6.x and 2.5.x releases. Since we have
> > > >> big
> > > >> > > changes
> > > >> > > > > > on
> > > >> > > > > > > > > > > > > > replication and region replica replication, I
> > > >> plan
> > > >> > > to have
> > > >> > > > > > > > tests
> > > >> > > > > > > > > with
> > > >> > > > > > > > > > > > > > these two features enabled too if possible.
> > > >> > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > Any other suggestions?
> > > >> > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > Thanks.
> > > >> > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > 张铎(Duo Zhang) <pa...@gmail.com>
> > > >> 于2024年9月19日周四
> > > >> > > > > > 22:35写道:
> > > >> > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > And do we want to apply the same trick to
> > > >> > > MasterObserver
> > > >> > > > > > and
> > > >> > > > > > > > > > > > > > > RegionServerObserver?
> > > >> > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > For RegionObserver, we'd better do this
> > > >> because
> > > >> > > there are
> > > >> > > > > > > > > already
> > > >> > > > > > > > > > > > > > > methods in it that use the ObserverContext<?
> > > >> > > extends
> > > >> > > > > > > > > > > > > > > RegionCoprocessorEnvironment> declaration,
> > > >> which
> > > >> > > were
> > > >> > > > > > added
> > > >> > > > > > > > in
> > > >> > > > > > > > > > > > > > > HBASE-10378.
> > > >> > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > For other coprocessor observers, we do not
> > > >> have
> > > >> > > such
> > > >> > > > > > methods
> > > >> > > > > > > > > yet.
> > > >> > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > Nick Dimiduk <nd...@apache.org>
> > > >> 于2024年9月19日周四
> > > >> > > > > > 22:03写道:
> > > >> > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > We do it here or we wait for 4.0. I'm
> > > >> inclined
> > > >> > > toward
> > > >> > > > > > doing
> > > >> > > > > > > > > it here
> > > >> > > > > > > > > > > > > > > > because (i think) we expect coprocessor
> > > >> > > > > > implementations to
> > > >> > > > > > > > > necessarily
> > > >> > > > > > > > > > > > > > > > recompile anyway. Though I admit I've not
> > > >> thought
> > > >> > > > > > through
> > > >> > > > > > > > > what
> > > >> > > > > > > > > > > > > > features are
> > > >> > > > > > > > > > > > > > > > enabled by this new generic signature.
> > > >> > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > On Thu, Sep 19, 2024 at 3:57 PM 张铎(Duo
> > > >> Zhang) <
> > > >> > > > > > > > > palomino219@gmail.com>
> > > >> > > > > > > > > > > > > > wrote:
> > > >> > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > Filed HBASE-28862.
> > > >> > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > It will break old coprocessor
> > > >> implementations
> > > >> > > even if
> > > >> > > > > > > > they
> > > >> > > > > > > > > do not use
> > > >> > > > > > > > > > > > > > > > > any deprecated method. But since it does
> > > >> not
> > > >> > > change
> > > >> > > > > > any
> > > >> > > > > > > > > abilities, I
> > > >> > > > > > > > > > > > > > > > > think it is OK to include this in a major
> > > >> > > release as
> > > >> > > > > > we
> > > >> > > > > > > > > have already
> > > >> > > > > > > > > > > > > > > > > done bunch of breaking changes, like
> > > >> changing
> > > >> > > the
> > > >> > > > > > > > protobuf
> > > >> > > > > > > > > from
> > > >> > > > > > > > > > > > > > > > > unshaded one to shaded and relocated
> > > >> one...
> > > >> > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > Thanks.
> > > >> > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > 张铎(Duo Zhang) <pa...@gmail.com>
> > > >> > > 于2024年9月19日周四
> > > >> > > > > > > > > 17:36写道:
> > > >> > > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > > Thanks for pointing this TODO out.
> > > >> > > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > > Let me file an issue abort this. For me
> > > >> I
> > > >> > > think we
> > > >> > > > > > > > > should do this
> > > >> > > > > > > > > > > > > > in
> > > >> > > > > > > > > > > > > > > > > 3.0.0.
> > > >> > > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > > Nick Dimiduk <nd...@apache.org>
> > > >> > > 于2024年9月19日周四
> > > >> > > > > > > > > 16:46写道:
> > > >> > > > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > > > Should we take the opportunity to
> > > >> upgrade
> > > >> > > the
> > > >> > > > > > generic
> > > >> > > > > > > > > > > > > > definitions in
> > > >> > > > > > > > > > > > > > > > > our
> > > >> > > > > > > > > > > > > > > > > > > coprocessor signatures as well?
> > > >> > > > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > >
> > > >> > >
> > > >> https://github.com/apache/hbase/blob/edbb145a3f0c6847f444fefdadea5b82d02b0bdb/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java#L101
> > > >> > > > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > > > On Tue, Sep 17, 2024 at 4:41 PM Duo
> > > >> Zhang <
> > > >> > > > > > > > > zhangduo@apache.org>
> > > >> > > > > > > > > > > > > > wrote:
> > > >> > > > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > > > > I've done several rounds of API
> > > >> cleanup
> > > >> > > in
> > > >> > > > > > > > > HBASE-24888 and now
> > > >> > > > > > > > > > > > > > for me
> > > >> > > > > > > > > > > > > > > > > > > > I think it is enough for making the
> > > >> new
> > > >> > > major
> > > >> > > > > > > > > release.
> > > >> > > > > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > > > > I've filed HBASE-28844 for aligning
> > > >> the
> > > >> > > commit
> > > >> > > > > > > > > history and jira
> > > >> > > > > > > > > > > > > > > > > > > > issues, it will be a huge project
> > > >> for a
> > > >> > > major
> > > >> > > > > > > > > release and may
> > > >> > > > > > > > > > > > > > take
> > > >> > > > > > > > > > > > > > > > > > > > several weeks.
> > > >> > > > > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > > > > After that, I think we are good to
> > > >> go.
> > > >> > > And I
> > > >> > > > > > will
> > > >> > > > > > > > > also try to
> > > >> > > > > > > > > > > > > > find
> > > >> > > > > > > > > > > > > > > > > > > > some resources to run several
> > > >> rounds of
> > > >> > > ITBLL
> > > >> > > > > > and
> > > >> > > > > > > > > also some
> > > >> > > > > > > > > > > > > > upgrading
> > > >> > > > > > > > > > > > > > > > > > > > tests. Finally, after everything is
> > > >> OK,
> > > >> > > I will
> > > >> > > > > > > > > release
> > > >> > > > > > > > > > > > > > 3.0.0-beta-2,
> > > >> > > > > > > > > > > > > > > > > > > > and then cut branch-3.0 from the
> > > >> > > 3.0.0-beta-2
> > > >> > > > > > tag
> > > >> > > > > > > > > and make the
> > > >> > > > > > > > > > > > > > 3.0.0
> > > >> > > > > > > > > > > > > > > > > > > > release soon.
> > > >> > > > > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > > > > Hope I can get all these things done
> > > >> > > within the
> > > >> > > > > > > > year
> > > >> > > > > > > > > 2024.
> > > >> > > > > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > > > > > Thanks.
> > > >> > > > > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > > > > >
> > > >> > > > > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > > --
> > > >> > > > > > > > *István Tóth* | Sr. Staff Software Engineer
> > > >> > > > > > > > *Email*: stoty@cloudera.com
> > > >> > > > > > > > cloudera.com <https://www.cloudera.com>
> > > >> > > > > > > > [image: Cloudera] <https://www.cloudera.com/>
> > > >> > > > > > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera>
> > > >> > > [image:
> > > >> > > > > > > > Cloudera on Facebook] <https://www.facebook.com/cloudera>
> > > >> > > [image:
> > > >> > > > > > Cloudera
> > > >> > > > > > > > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > >> > > > > > > > ------------------------------
> > > >> > > > > > > > ------------------------------
> > > >> > > > > > > >
> > > >> > > > > >
> > > >> > >
> > > >> >
> > > >> >
> > > >> > --
> > > >> > *István Tóth* | Sr. Staff Software Engineer
> > > >> > *Email*: stoty@cloudera.com
> > > >> > cloudera.com <https://www.cloudera.com>
> > > >> > [image: Cloudera] <https://www.cloudera.com/>
> > > >> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > > >> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > > >> Cloudera
> > > >> > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > >> > ------------------------------
> > > >> > ------------------------------
> > > >>
> > > >
> > > >
> > > > --
> > > > *István Tóth* | Sr. Staff Software Engineer
> > > > *Email*: stoty@cloudera.com
> > > > cloudera.com <https://www.cloudera.com>
> > > > [image: Cloudera] <https://www.cloudera.com/>
> > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > > > Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > > ------------------------------
> > > > ------------------------------
> > > >
> > >
> > >
> > > --
> > > *István Tóth* | Sr. Staff Software Engineer
> > > *Email*: stoty@cloudera.com
> > > cloudera.com <https://www.cloudera.com>
> > > [image: Cloudera] <https://www.cloudera.com/>
> > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> > > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > ------------------------------
> > > ------------------------------