You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Ayush Saxena <ay...@gmail.com> on 2025/06/06 09:17:13 UTC

Re: Re: [DISCUSS] Hive 4.1.x release

Hi folks,

With the JDK-17 update now behind us, I think it's a good time to
start discussing the 4.1.0 release.

From what I see, HIVE-28984 appears to be the only significant blocker
left — and it doesn’t look like a major hurdle code wise. Should be
doable without much trouble.

Let’s begin thinking about timelines and whether there are any
additional features or nice-to-haves folks would like to see included
— especially those that can be wrapped up in the next week or so.
Just wanted to get this conversation going.

Best,
Ayush

On Mon, 23 Dec 2024 at 12:17, Butao Zhang <zh...@apache.org> wrote:
>
> Hi all,
>
> I believe the main factors blocking the release of version Hive 4.1.0 are the upgrade to JDK 17 and the upgrade to Tez 0.10.5[2]. Both of these will require some time. Ideally, it could be ready within a month, but considering the upcoming holiday season, it might take 1-2 months to prepare the prerequisites for the Hive 4.1.0 release.
>
> Therefore, I have decided to postpone the creation of the Hive 4.1.0 branch.
>
> As for version Hive 4.0.2, I currently don't see a strong demand for it, nor am I sure which patches would need to be included in the 4.0.2 version. So, personally, I don't think there's a significant need to release a minor 4.0.2 version. Of course, if the community strongly desires it, I would be more than happy to support the release of a minor 4.0.2 version.
>
> [1]https://lists.apache.org/thread/r42zl037xyt8kyqr27qol4yky818mmfg
>
>
> On 2024/12/18 09:38:10 Ayush Saxena wrote:
> > Hi Folks,
> > I strongly believe that the 4.1 release should include JDK-17 support, as
> > it has been a key feature we’ve been planning for a while. JDK-17 is a
> > compelling reason for users to upgrade, whereas other features, though
> > significant, might not be enough to drive widespread adoption.
> >
> > If JDK-17 integration requires additional time and we need an immediate
> > release, we could consider a 4.0.2 bug-fix release next month. This could
> > include all major bugs along with some safe & significant improvements.
> > Following that, we could target a 4.1 release in 1–2 months with JDK-17
> > support as its highlight.
> >
> > That said, I still feel strongly that the 4.1 release should include
> > JDK-17.
> >
> > Best,
> > Ayush
> >
> > On Wed, 18 Dec 2024 at 13:40, Butao Zhang <bu...@163.com> wrote:
> >
> > > Thanks all,
> > >
> > >           JDK17 is a really big feature, and we should carefully evaluate
> > > whether this feature should be included in this 4.1.0 release. Hope to have
> > > other folks to share their advice.
> > >           BTW, i want to cut the new branch 4.1.0 next week. If you want
> > > to include some tickets into 4.1.0, please add hive-4.1.0-must label in
> > > the JIRA. Some open tickets for 4.1.0 list in [1].
> > >
> > > [1]
> > > https://issues.apache.org/jira/browse/HIVE-28669?jql=status%20in%20(Open%2C%20%22In%20Progress%22)%20AND%20labels%20%3D%20hive-4.1.0-must
> > >
> > >
> > > Thanks,
> > > Butao Zhang
> > > ---- Replied Message ----
> > > From Stamatis Zampetakis<za...@gmail.com> <undefined>
> > > Date 12/16/2024 17:33
> > > To <de...@hive.apache.org> <de...@hive.apache.org>
> > > Subject Re: Re: [DISCUSS] Hive 4.1.x release
> > > The master branch has enough content to justify a release and make
> > > many end-users happy. The naming depends (4.1.0, 4.0.2, 5.0.0) on the
> > > actual content and without doing an exhaustive check I feel that if we
> > > release now we should do a minor bump i.e., 4.1.0. If we start
> > > compiling with JDK17, it is probably an important breaking change thus
> > > I feel that it deserves a major version bump (i.e., 5.0.0). Just to be
> > > clear, compiling and running with JDK17 are two very different topics
> > > as other people also highlighted in the respective thread.
> > >
> > > In other words, we can cut and make a release whenever we want
> > > (assuming that master is not broken). The version identifier will and
> > > should depend on the content of the release branch.
> > >
> > > Best,
> > > Stamatis
> > >
> > > On Sun, Dec 15, 2024 at 10:13 AM lisoda <li...@yeah.net> wrote:
> > >
> > >
> > > I agree with Butao that since the community has previously promised that
> > > jdk17 will be supported in 4.1.0, it would be best if we keep that promise.
> > > But maybe we can go ahead and release a 4.0.2 version?And then wait for
> > > HIVE-28665 to be merged before releasing a 4.1.0 version?
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > At 2024-12-13 10:33:47, "Butao Zhang" <zh...@apache.org> wrote:
> > >
> > > Thanks Stamatis for your insightful suggestion! I basically agree with
> > > you, and what you said is also what i have been thinking recently.
> > >
> > > However, i think we've had a lot of discussion about the release of jdk17
> > > features in version Hive 4.1.x. At some point, many community users will
> > > expect Hive 4.1.x  to inlcude jdk17 features. For example, we have some
> > > disscussion in Apache iceberg about the Hive support, especially the which
> > > JDK supporting in Hive
> > > https://lists.apache.org/thread/jfcqfw9vhq4j7h0kwnlf338jgyzcq8s4.
> > > BTW, Apache iceberg has been used Jdk11&JDK17 since 1.7.x, so we can not
> > > upgrade Iceberg dependency in Hive if we can not complete the JDK17 upgrade
> > > (HIVE-28665).
> > >
> > > Since JDK17 is a major feature, if we can not include it in Hive4.1.0,
> > > then we also can not include it in Hive 4.1.1 ->4.1.x. User must wait for
> > > another big release, e.g. 4.2.0, but I am concerned that this big release
> > > 4.2.0 will come too late and affect the use of community users(Apache
> > > iceberg or other OSS which need the Hive with high jdk version).
> > >
> > > I see that JDK-17(HIVE-26473) is under active review, and it seems that it
> > > does not have big problem. So i want to wait a short time to complete this
> > > feature, maybe two weeks or one month? Meantime, we can check if there are
> > > any other PR(label hive-4.1.0-must) ) that need to be added to this version.
> > > Of course, if we find some major issues when reviewing JDK-17(HIVE-26473),
> > > we don't have to wait for it, and Hive4.1 branch will be cut soon.
> > >
> > > Thanks,
> > > Butao Zhang
> > >
> > > On 2024/12/12 13:14:16 Stamatis Zampetakis wrote:
> > >
> > > Thanks for volunteering Butao!
> > >
> > > As usual I have a slightly different perspective regarding the scope
> > > for the release. If there are no regressions or other serious blockers
> > > in the current master we could cut the release ASAP. We don't need to
> > > delay the release for getting new features in. Once JDK-17 (or any
> > > other kind of major feature) is ready we can merge it to master and
> > > cut a new release (that being 4.2, 4.3, 5.0 or whatever makes sense at
> > > that point). Nobody will complain if we get a shiny new release out
> > > really soon :)
> > >
> > > Best,
> > > Stamatis
> > >
> > > On Thu, Dec 12, 2024 at 1:00 PM Akshat m <ak...@gmail.com> wrote:
> > >
> > >
> > > Hi Team,
> > >
> > > Thanks for the productive discussion!
> > > I’d like to provide a quick update on JDK-17. The initial review comments
> > > have been addressed, and to stay aligned with ongoing upstream changes,
> > > we’ve rebased the PR onto the latest master. After the recent rebase, we
> > > did encounter multiple test failures, but that issue has now been resolved
> > > and the CI tests are running again.
> > >
> > > While we await a green build, we’d really appreciate it if the community
> > > could begin the secondary review process. Your insights are invaluable, and
> > > we start to address any new feedback as soon as possible.
> > >
> > > Regards
> > > Akshat
> > >
> > > On Thu, Dec 12, 2024 at 11:56 AM Butao Zhang <zh...@apache.org>
> > > wrote:
> > >
> > >
> > > Thanks!
> > > IMO, JDK-17(HIVE-26473) should definitely be included in this release. I
> > > am also watching it. Maybe we can cut the branch-4.1 at some point after
> > > the JDK-17 is finished.
> > >
> > >
> > > Thanks,
> > > Butao Zhang
> > >
> > > On 2024/12/11 14:22:55 Denys Kuzmenko wrote:
> > >
> > > Thanks Butao!
> > > We'll support you all the way.
> > >
> > > ATM we should probably finalize the scope for the release (create new
> > > label hive-4.1.0-must) and focus of getting those items prioritised.
> > >
> > > There is also ongoing discussion on JDK-8 and JDK-17. Would be great if we
> > > manage to add support for JDK-17 in that release.
> > >
> > >
> > >
> >

Re: [DISCUSS] Hive 4.1.x release

Posted by Shohei Okumiya <ok...@apache.org>.
I also think we can discuss when it turns out that we really need
4.1.1, for example, due to a critical CVE. Essentially, cherry-picking
is not straightforward thanks to our recent excellent momentum and
changes, such as JDK 21 or the restructuring of IMetaStoreClient.
However, if it is genuinely essential, I can assist the community in
making the decision and cutting a patch version.

Regards,
Okumin

On Thu, Jul 10, 2025 at 1:13 AM Ayush Saxena <ay...@gmail.com> wrote:
>
> +1 to initiating the discussions once this is wrapped up. That said, I
> echo the concerns raised by Stamatis — we simply don't have the
> resources or bandwidth to maintain multiple release lines. We
> shouldn't plan with 4.1.1 in mind from the outset. Instead, let’s keep
> the door open for a potential 4.1.1 only if something critical comes
> up in this release.
>
> Otherwise, the focus should shift toward 4.2.0, which can include more
> substantial changes. Let’s treat 4.1.0 as a complete package, not
> something that's expecting an immediate follow-up.
>
> Thanks, Butao, for sharing the release status. Looking forward to the RC :)
>
> –Ayush
>
> -Ayush
>
> On Wed, 9 Jul 2025 at 16:17, Denys Kuzmenko <dk...@apache.org> wrote:
> >
> > Not sure. ideally, 4.1.1 should include the critical fixes. However, we might consider a major release that includes JDK 21 support, Iceberg auto-compaction, and other improvements from the master branch.
> >
> > But let’s discuss it once we get there.

Re: [DISCUSS] Hive 4.1.x release

Posted by Ayush Saxena <ay...@gmail.com>.
+1 to initiating the discussions once this is wrapped up. That said, I
echo the concerns raised by Stamatis — we simply don't have the
resources or bandwidth to maintain multiple release lines. We
shouldn't plan with 4.1.1 in mind from the outset. Instead, let’s keep
the door open for a potential 4.1.1 only if something critical comes
up in this release.

Otherwise, the focus should shift toward 4.2.0, which can include more
substantial changes. Let’s treat 4.1.0 as a complete package, not
something that's expecting an immediate follow-up.

Thanks, Butao, for sharing the release status. Looking forward to the RC :)

–Ayush

-Ayush

On Wed, 9 Jul 2025 at 16:17, Denys Kuzmenko <dk...@apache.org> wrote:
>
> Not sure. ideally, 4.1.1 should include the critical fixes. However, we might consider a major release that includes JDK 21 support, Iceberg auto-compaction, and other improvements from the master branch.
>
> But let’s discuss it once we get there.

Re: [DISCUSS] Hive 4.1.x release

Posted by Denys Kuzmenko <dk...@apache.org>.
Not sure. ideally, 4.1.1 should include the critical fixes. However, we might consider a major release that includes JDK 21 support, Iceberg auto-compaction, and other improvements from the master branch. 

But let’s discuss it once we get there.

Re: [DISCUSS] Hive 4.1.x release

Posted by Stamatis Zampetakis <za...@gmail.com>.
Hey team,

Great progress, glad to see that we are close to the 4.1.0 release.

I've noticed that people were talking about the stability of branch-4.1 and
the potential of a 4.1.1 release so I was wondering what is the plan with
respect to subsequent releases. Are we planning to keep the branch-4.1 open
to improvements and cherry-picks after the release of Hive 4.1.0? I am not
sure if we have sufficient resources (both machine and human) to properly
maintain multiple branches.

Best,
Stamatis

On Tue, Jul 8, 2025 at 4:32 PM Butao Zhang <zh...@apache.org> wrote:

> Hi Ayush,
>
>          Recently, with the help of Denys and Zhihua, we have optimized
> the packaging and release process for the standalone metastore over the
> past week, which is now largely completed. Several critical PRs have also
> been cherry-picked. Personally, I believe there are no major issues
> remaining on the branch-4.1 branch at this point. There are still a few
> issues [1] under consideration for merging into branch-4.1, but a decision
> should be made soon.
>         Additionally, two minor issues have emerged on branch-4.1, which
> have somewhat slowed down the PR merging process—one being a flaky test [2]
> and the other related to JavaDoc generation with JDK-17 [3]. Perhaps we can
> consider optimizing these in the 4.1.1 release.
>       In the next couple of days, I will perform further pre-release
> validation locally. Ideally, I think we could initiate the RC vote by the
> end of this week, or at the latest, early next week.
>
>       If you have any suggestions, please feel free to share them. Many
> thanks!
>
> [1]
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HIVE%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20hive-4.1.0-must%20ORDER%20BY%20updated%20DESC%2C%20priority%20DESC
> [2] https://issues.apache.org/jira/browse/HIVE-29061
> [3] https://issues.apache.org/jira/browse/HIVE-29063
>
> Thanks,
> Butao Zhang
> ---- Replied Message ----
> From Ayush Saxena<ay...@gmail.com> <ay...@gmail.com>
> Date 7/8/2025 18:32
> To <de...@hive.apache.org> <de...@hive.apache.org>
> Subject Re: [DISCUSS] Hive 4.1.x release
>
> Hi Butao,
> Just checking in — do you have any pointers on what’s still pending,
> or when we might expect to have an RC ready for vote? Are there any
> major blockers we should be aware of?
>
> Thanks,
> Ayush
>
> On Thu, 26 Jun 2025 at 00:02, Denys Kuzmenko <dk...@apache.org> wrote:
>
>
>  I've updated the Release guide with instructions for the HMS:
>  https://github.com/apache/hive-site/pull/51
>
>

Re: [DISCUSS] Hive 4.1.x release

Posted by Butao Zhang <zh...@apache.org>.
Hi Ayush,


         Recently, with the help of Denys and Zhihua, we have optimized the packaging and release process for the standalone metastore over the past week, which is now largely completed. Several critical PRs have also been cherry-picked. Personally, I believe there are no major issues remaining on the branch-4.1 branch at this point. There are still a few issues [1] under consideration for merging into branch-4.1, but a decision should be made soon.
        Additionally, two minor issues have emerged on branch-4.1, which have somewhat slowed down the PR merging process—one being a flaky test [2] and the other related to JavaDoc generation with JDK-17 [3]. Perhaps we can consider optimizing these in the 4.1.1 release.
      In the next couple of days, I will perform further pre-release validation locally. Ideally, I think we could initiate the RC vote by the end of this week, or at the latest, early next week.
 
      If you have any suggestions, please feel free to share them. Many thanks!


[1] https://issues.apache.org/jira/issues/?jql=project%20%3D%20HIVE%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20hive-4.1.0-must%20ORDER%20BY%20updated%20DESC%2C%20priority%20DESC
[2] https://issues.apache.org/jira/browse/HIVE-29061
[3] https://issues.apache.org/jira/browse/HIVE-29063


Thanks,
Butao Zhang
---- Replied Message ----
| From | Ayush Saxena<ay...@gmail.com> |
| Date | 7/8/2025 18:32 |
| To | <de...@hive.apache.org> |
| Subject | Re: [DISCUSS] Hive 4.1.x release |
Hi Butao,
Just checking in — do you have any pointers on what’s still pending,
or when we might expect to have an RC ready for vote? Are there any
major blockers we should be aware of?

Thanks,
Ayush

On Thu, 26 Jun 2025 at 00:02, Denys Kuzmenko <dk...@apache.org> wrote:

 I've updated the Release guide with instructions for the HMS:
 https://github.com/apache/hive-site/pull/51

Re: [DISCUSS] Hive 4.1.x release

Posted by Ayush Saxena <ay...@gmail.com>.
Hi Butao,
Just checking in — do you have any pointers on what’s still pending,
or when we might expect to have an RC ready for vote? Are there any
major blockers we should be aware of?

Thanks,
Ayush

On Thu, 26 Jun 2025 at 00:02, Denys Kuzmenko <dk...@apache.org> wrote:
>
> I've updated the Release guide with instructions for the HMS:
> https://github.com/apache/hive-site/pull/51

Re: [DISCUSS] Hive 4.1.x release

Posted by Denys Kuzmenko <dk...@apache.org>.
I've updated the Release guide with instructions for the HMS:
https://github.com/apache/hive-site/pull/51

Re: [DISCUSS] Hive 4.1.x release

Posted by Zhihua Deng <de...@apache.org>.
In my opinion, we should include the standalone Metastore, given:
 - The Metastore binary/src tarball can be built and provided;
 - We have been requested for the Metastore tarball and Docker image in the user email or jira.
Regarding the package name, the HIVE-20451 has the context, we can just keep them as it is.

Thanks,
Zhihua 

On 2025/06/25 02:07:01 Butao Zhang wrote:
> I would like to confirm whether it is necessary to release the standalone metastore version this time?
> 
> 
> I locally built the code and found that the standalone hms package is located at the path [1], and its source code tar is at the path [2]. Am I right? 
> 
> 
> If there is a need to release an HMS version, is there any difference between it and the regular Hive release?
> 
> 
> If we are to release the HMS version, do we need to change the package name to apache-hive-standalone-metastore-4.1.0-SNAPSHOT-bin.tar.gz? I just think it's a bit strange to include the word "server" in it. Because the previously released HMS versions did not include the word "server" at all, like[3].
> 
> 
> [1] hive/standalone-metastore/metastore-server/target/apache-hive-standalone-metastore-server-4.1.0-SNAPSHOT-bin.tar.gz
> [2] hive/standalone-metastore/target/apache-hive-standalone-metastore-4.1.0-SNAPSHOT-src.tar.gz
> [3] https://downloads.apache.org/hive/hive-standalone-metastore-3.0.0/hive-standalone-metastore-3.0.0-bin.tar.gz
> 
> 
> 
> 
> Thanks,
> Butao Zhang
> ---- Replied Message ----
> | From | Denys Kuzmenko<dk...@apache.org> |
> | Date | 6/25/2025 02:26 |
> | To | <de...@hive.apache.org> |
> | Subject | Re: [DISCUSS] Hive 4.1.x release |
> Would be nice to resolve https://sonarcloud.io/project/issues?impactSeverities=BLOCKER&issueStatuses=OPEN%2CCONFIRMED&id=apache_hive before the Hive-5.x release
> 

Re: [DISCUSS] Hive 4.1.x release

Posted by Butao Zhang <bu...@163.com>.
I would like to confirm whether it is necessary to release the standalone metastore version this time?


I locally built the code and found that the standalone hms package is located at the path [1], and its source code tar is at the path [2]. Am I right? 


If there is a need to release an HMS version, is there any difference between it and the regular Hive release?


If we are to release the HMS version, do we need to change the package name to apache-hive-standalone-metastore-4.1.0-SNAPSHOT-bin.tar.gz? I just think it's a bit strange to include the word "server" in it. Because the previously released HMS versions did not include the word "server" at all, like[3].


[1] hive/standalone-metastore/metastore-server/target/apache-hive-standalone-metastore-server-4.1.0-SNAPSHOT-bin.tar.gz
[2] hive/standalone-metastore/target/apache-hive-standalone-metastore-4.1.0-SNAPSHOT-src.tar.gz
[3] https://downloads.apache.org/hive/hive-standalone-metastore-3.0.0/hive-standalone-metastore-3.0.0-bin.tar.gz




Thanks,
Butao Zhang
---- Replied Message ----
| From | Denys Kuzmenko<dk...@apache.org> |
| Date | 6/25/2025 02:26 |
| To | <de...@hive.apache.org> |
| Subject | Re: [DISCUSS] Hive 4.1.x release |
Would be nice to resolve https://sonarcloud.io/project/issues?impactSeverities=BLOCKER&issueStatuses=OPEN%2CCONFIRMED&id=apache_hive before the Hive-5.x release

Re: [DISCUSS] Hive 4.1.x release

Posted by Denys Kuzmenko <dk...@apache.org>.
Would be nice to resolve https://sonarcloud.io/project/issues?impactSeverities=BLOCKER&issueStatuses=OPEN%2CCONFIRMED&id=apache_hive before the Hive-5.x release

Re: [DISCUSS] Hive 4.1.x release

Posted by Ayush Saxena <ay...@gmail.com>.
Master moves to 4.2.0-SNAPSHOT, no changes as such. If we decide to name
the next release 5.0.0, the Release Manager for that version can handle the
versioning changes accordingly.

-Ayush

On Tue, 24 Jun 2025 at 13:36, Butao Zhang <bu...@163.com> wrote:

> I would like to reconfirm a question. If the 4.1.0 branch is released,
> should the version number of the master branch be changed to 5.0.0-SNAPSHOT
> or 4.2.0-SNAPSHOT?
> It seems that everyone believes that the name of the next major version
> should be 5.0.0-SNAPSHOT.
>
> Thanks,
> Butao Zhang
> ---- Replied Message ----
> From Attila Turoczy<at...@cloudera.com.INVALID>
> <at...@cloudera.com.INVALID>
> Date 6/23/2025 19:47
> To <de...@hive.apache.org> <de...@hive.apache.org>
> Subject Re: [DISCUSS] Hive 4.1.x release
> In my view, Hive 5 (high five! 🙂) should represent a major milestone.
> Ideally, it would include more impactful changes such as:
>
>    - Making Iceberg the default table format (pending community
>    consensus),
>    - Removing legacy configurations and integrations
>    - Integrating new compaction strategies, among other forward-looking
>    improvements.
>
> Given the current scope and feature set, the existing package feels more
> aligned with a 4.1 release rather than a true version 5.
>
> -Attila
>
>
> On Mon, Jun 23, 2025 at 1:36 PM lisoda <li...@yeah.net> wrote:
>
>> What if we integrate JDK 21 and then release version 5.x?
>>
>>
>> ---- Replied Message ----
>> From Ayush Saxena<ay...@gmail.com> <ay...@gmail.com>
>> Date 06/23/2025 19:31
>> To dev@hive.apache.org
>> Cc
>> Subject Re: [DISCUSS] Hive 4.1.x release
>> I don’t have a strict preference around the version number — the
>> important thing is that we get the release out. That said, this feels
>> quite last-minute. Most of us have been planning with 4.1 in mind.
>>
>> While 5.x does open the door to removing deprecated methods/configs
>> and gives us more freedom to make breaking changes, I think we should
>> proceed with 4.1 as originally planned. Even if the work for 5.x seems
>> small now, properly thinking through what all we might want to include
>> in a major release could end up pushing timelines further.
>>
>> Unless we decide to do a last-minute major version bump without
>> looking out what more can be scoped — which I don’t think would be
>> very community-friendly — it's better to keep things simple and
>> focused. From what I can tell, most of the folks don't have a
>> particularly strong preference, so let’s not overcomplicate things 2
>> days before the code freeze.
>>
>> Let’s aim to release 4.1 next week as planned, and we can save the
>> more ambitious ideas for 5.x 🙂
>>
>> (Outside scope info): As far as I know, Hadoop is also planning to
>> drop JDK 8 only in the 3.4 release [1].
>>
>> -Ayush
>>
>> [1] https://lists.apache.org/thread/5543x33nv6196bc5xyxdqjoh2dht0kbg
>>
>>
>> On Mon, 23 Jun 2025 at 16:48, Shohei Okumiya <ok...@apache.org> wrote:
>> >
>> > Thank you all!
>> >
>> > I also think Iceberg REST Catalog is not a blocker. It is nice to have,
>> and HIVE-29019 verified that no critical bugs, such as metadata
>> corruptions, exist in the current effective endpoint. Improving the release
>> cadence sounds more important.
>> >
>> > Regarding 4.1 vs 5.0, I also don't have a firm preference. If we go
>> with 5.0, isn't the extra work significant? We may have to change the fix
>> versions of all tickets, update some documents mentioning 4.1.0(such as the
>> following one), that's all? In that case, it sounds technically possible.
>> > - https://hive.apache.org/docs/latest/capture-lineage-info/
>> >
>> > I'm excited about the next release. Hopefully, everyone shares the
>> thought!
>> >
>> > Best,
>> > Okumin
>> >
>> >
>> > On Mon, Jun 23, 2025 at 7:13 PM Butao Zhang <bu...@163.com>
>> wrote:
>> >>
>> >> Thanks for the advice!
>> >> I just sent out the notification email regarding the code freeze[1].
>> >>
>> >> Additionally, regarding Stamatis' suggestion: "I feel that the current
>> content deserves a major version bump (instead of 4.1.0 we should probably
>> consider 5.0.0)." I don't have strong feelings either for or against this.
>> Given the significant changes in this release, whether it would be more
>> meaningful to define it as version 5.0 may require further discussion.
>> Perhaps we should initiate a community email vote on whether to name it
>> 4.1.0 or 5.0.0?
>> >>
>> >> [1]https://lists.apache.org/thread/c166fz9hr3yft75qo3lcmhrbm1howpol
>> >>
>> >> Thanks,
>> >> Butao Zhang
>> >> ---- Replied Message ----
>> >> FromAyush Saxena<ay...@gmail.com>
>> >> Date6/23/2025 17:23
>> >> To<de...@hive.apache.org>
>> >> SubjectRe: [DISCUSS] Hive 4.1.x release
>> >>
>> >> Could we consider sending out a notification email?
>> >>
>> >>
>> >> Let’s have Butao Zhang, with their Release Manager hat on, do the
>> >> honors here. :-)
>> >>
>> >> Butao Zhang, could you please send out a [NOTICE] email announcing
>> >> that the code freeze for 4.1.0 is set for the 25th?
>> >> Post that, only critical items will be considered for the release and
>> >> must be cherry-picked to the release branch (alongside master, where
>> >> applicable).
>> >> Feel free to phrase it accordingly.
>> >>
>> >> –Ayush
>> >>
>> >> On Mon, 23 Jun 2025 at 14:33, Denys Kuzmenko <dk...@apache.org>
>> wrote:
>> >>
>> >>
>> >> HIVE-29038 is more like an open discussion.
>> >>
>> >> 25th sounds good! Could we consider sending out a notification email?
>> >>
>> >> Thanks!
>>
>

Re: [DISCUSS] Hive 4.1.x release

Posted by Butao Zhang <bu...@163.com>.
I would like to reconfirm a question. If the 4.1.0 branch is released, should the version number of the master branch be changed to 5.0.0-SNAPSHOT or 4.2.0-SNAPSHOT?
It seems that everyone believes that the name of the next major version should be 5.0.0-SNAPSHOT.


Thanks,
Butao Zhang
---- Replied Message ----
| From | Attila Turoczy<at...@cloudera.com.INVALID> |
| Date | 6/23/2025 19:47 |
| To | <de...@hive.apache.org> |
| Subject | Re: [DISCUSS] Hive 4.1.x release |
In my view, Hive 5 (high five! 🙂) should represent a major milestone. Ideally, it would include more impactful changes such as:

Making Iceberg the default table format (pending community consensus),
Removing legacy configurations and integrations
Integrating new compaction strategies, among other forward-looking improvements.
Given the current scope and feature set, the existing package feels more aligned with a 4.1 release rather than a true version 5.


-Attila




On Mon, Jun 23, 2025 at 1:36 PM lisoda <li...@yeah.net> wrote:

What if we integrate JDK 21 and then release version 5.x?



---- Replied Message ----
| From | Ayush Saxena<ay...@gmail.com> |
| Date | 06/23/2025 19:31 |
| To | dev@hive.apache.org |
| Cc | |
| Subject | Re: [DISCUSS] Hive 4.1.x release |
I don’t have a strict preference around the version number — the
important thing is that we get the release out. That said, this feels
quite last-minute. Most of us have been planning with 4.1 in mind.

While 5.x does open the door to removing deprecated methods/configs
and gives us more freedom to make breaking changes, I think we should
proceed with 4.1 as originally planned. Even if the work for 5.x seems
small now, properly thinking through what all we might want to include
in a major release could end up pushing timelines further.

Unless we decide to do a last-minute major version bump without
looking out what more can be scoped — which I don’t think would be
very community-friendly — it's better to keep things simple and
focused. From what I can tell, most of the folks don't have a
particularly strong preference, so let’s not overcomplicate things 2
days before the code freeze.

Let’s aim to release 4.1 next week as planned, and we can save the
more ambitious ideas for 5.x 🙂

(Outside scope info): As far as I know, Hadoop is also planning to
drop JDK 8 only in the 3.4 release [1].

-Ayush

[1] https://lists.apache.org/thread/5543x33nv6196bc5xyxdqjoh2dht0kbg


On Mon, 23 Jun 2025 at 16:48, Shohei Okumiya <ok...@apache.org> wrote:
>
> Thank you all!
>
> I also think Iceberg REST Catalog is not a blocker. It is nice to have, and HIVE-29019 verified that no critical bugs, such as metadata corruptions, exist in the current effective endpoint. Improving the release cadence sounds more important.
>
> Regarding 4.1 vs 5.0, I also don't have a firm preference. If we go with 5.0, isn't the extra work significant? We may have to change the fix versions of all tickets, update some documents mentioning 4.1.0(such as the following one), that's all? In that case, it sounds technically possible.
> - https://hive.apache.org/docs/latest/capture-lineage-info/
>
> I'm excited about the next release. Hopefully, everyone shares the thought!
>
> Best,
> Okumin
>
>
> On Mon, Jun 23, 2025 at 7:13 PM Butao Zhang <bu...@163.com> wrote:
>>
>> Thanks for the advice!
>> I just sent out the notification email regarding the code freeze[1].
>>
>> Additionally, regarding Stamatis' suggestion: "I feel that the current content deserves a major version bump (instead of 4.1.0 we should probably consider 5.0.0)." I don't have strong feelings either for or against this. Given the significant changes in this release, whether it would be more meaningful to define it as version 5.0 may require further discussion. Perhaps we should initiate a community email vote on whether to name it 4.1.0 or 5.0.0?
>>
>> [1]https://lists.apache.org/thread/c166fz9hr3yft75qo3lcmhrbm1howpol
>>
>> Thanks,
>> Butao Zhang
>> ---- Replied Message ----
>> FromAyush Saxena<ay...@gmail.com>
>> Date6/23/2025 17:23
>> To<de...@hive.apache.org>
>> SubjectRe: [DISCUSS] Hive 4.1.x release
>>
>> Could we consider sending out a notification email?
>>
>>
>> Let’s have Butao Zhang, with their Release Manager hat on, do the
>> honors here. :-)
>>
>> Butao Zhang, could you please send out a [NOTICE] email announcing
>> that the code freeze for 4.1.0 is set for the 25th?
>> Post that, only critical items will be considered for the release and
>> must be cherry-picked to the release branch (alongside master, where
>> applicable).
>> Feel free to phrase it accordingly.
>>
>> –Ayush
>>
>> On Mon, 23 Jun 2025 at 14:33, Denys Kuzmenko <dk...@apache.org> wrote:
>>
>>
>> HIVE-29038 is more like an open discussion.
>>
>> 25th sounds good! Could we consider sending out a notification email?
>>
>> Thanks!

Re: [DISCUSS] Hive 4.1.x release

Posted by Attila Turoczy <at...@cloudera.com.INVALID>.
In my view, Hive 5 (high five! 🙂) should represent a major milestone.
Ideally, it would include more impactful changes such as:

   - Making Iceberg the default table format (pending community consensus),
   - Removing legacy configurations and integrations
   - Integrating new compaction strategies, among other forward-looking
   improvements.

Given the current scope and feature set, the existing package feels more
aligned with a 4.1 release rather than a true version 5.

-Attila


On Mon, Jun 23, 2025 at 1:36 PM lisoda <li...@yeah.net> wrote:

> What if we integrate JDK 21 and then release version 5.x?
>
>
> ---- Replied Message ----
> From Ayush Saxena<ay...@gmail.com> <ay...@gmail.com>
> Date 06/23/2025 19:31
> To dev@hive.apache.org
> Cc
> Subject Re: [DISCUSS] Hive 4.1.x release
> I don’t have a strict preference around the version number — the
> important thing is that we get the release out. That said, this feels
> quite last-minute. Most of us have been planning with 4.1 in mind.
>
> While 5.x does open the door to removing deprecated methods/configs
> and gives us more freedom to make breaking changes, I think we should
> proceed with 4.1 as originally planned. Even if the work for 5.x seems
> small now, properly thinking through what all we might want to include
> in a major release could end up pushing timelines further.
>
> Unless we decide to do a last-minute major version bump without
> looking out what more can be scoped — which I don’t think would be
> very community-friendly — it's better to keep things simple and
> focused. From what I can tell, most of the folks don't have a
> particularly strong preference, so let’s not overcomplicate things 2
> days before the code freeze.
>
> Let’s aim to release 4.1 next week as planned, and we can save the
> more ambitious ideas for 5.x 🙂
>
> (Outside scope info): As far as I know, Hadoop is also planning to
> drop JDK 8 only in the 3.4 release [1].
>
> -Ayush
>
> [1] https://lists.apache.org/thread/5543x33nv6196bc5xyxdqjoh2dht0kbg
>
>
> On Mon, 23 Jun 2025 at 16:48, Shohei Okumiya <ok...@apache.org> wrote:
> >
> > Thank you all!
> >
> > I also think Iceberg REST Catalog is not a blocker. It is nice to have,
> and HIVE-29019 verified that no critical bugs, such as metadata
> corruptions, exist in the current effective endpoint. Improving the release
> cadence sounds more important.
> >
> > Regarding 4.1 vs 5.0, I also don't have a firm preference. If we go with
> 5.0, isn't the extra work significant? We may have to change the fix
> versions of all tickets, update some documents mentioning 4.1.0(such as the
> following one), that's all? In that case, it sounds technically possible.
> > - https://hive.apache.org/docs/latest/capture-lineage-info/
> >
> > I'm excited about the next release. Hopefully, everyone shares the
> thought!
> >
> > Best,
> > Okumin
> >
> >
> > On Mon, Jun 23, 2025 at 7:13 PM Butao Zhang <bu...@163.com>
> wrote:
> >>
> >> Thanks for the advice!
> >> I just sent out the notification email regarding the code freeze[1].
> >>
> >> Additionally, regarding Stamatis' suggestion: "I feel that the current
> content deserves a major version bump (instead of 4.1.0 we should probably
> consider 5.0.0)." I don't have strong feelings either for or against this.
> Given the significant changes in this release, whether it would be more
> meaningful to define it as version 5.0 may require further discussion.
> Perhaps we should initiate a community email vote on whether to name it
> 4.1.0 or 5.0.0?
> >>
> >> [1]https://lists.apache.org/thread/c166fz9hr3yft75qo3lcmhrbm1howpol
> >>
> >> Thanks,
> >> Butao Zhang
> >> ---- Replied Message ----
> >> FromAyush Saxena<ay...@gmail.com>
> >> Date6/23/2025 17:23
> >> To<de...@hive.apache.org>
> >> SubjectRe: [DISCUSS] Hive 4.1.x release
> >>
> >> Could we consider sending out a notification email?
> >>
> >>
> >> Let’s have Butao Zhang, with their Release Manager hat on, do the
> >> honors here. :-)
> >>
> >> Butao Zhang, could you please send out a [NOTICE] email announcing
> >> that the code freeze for 4.1.0 is set for the 25th?
> >> Post that, only critical items will be considered for the release and
> >> must be cherry-picked to the release branch (alongside master, where
> >> applicable).
> >> Feel free to phrase it accordingly.
> >>
> >> –Ayush
> >>
> >> On Mon, 23 Jun 2025 at 14:33, Denys Kuzmenko <dk...@apache.org>
> wrote:
> >>
> >>
> >> HIVE-29038 is more like an open discussion.
> >>
> >> 25th sounds good! Could we consider sending out a notification email?
> >>
> >> Thanks!
>

Re: [DISCUSS] Hive 4.1.x release

Posted by lisoda <li...@yeah.net>.
What if we integrate JDK 21 and then release version 5.x?



---- Replied Message ----
| From | Ayush Saxena<ay...@gmail.com> |
| Date | 06/23/2025 19:31 |
| To | dev@hive.apache.org |
| Cc | |
| Subject | Re: [DISCUSS] Hive 4.1.x release |
I don’t have a strict preference around the version number — the
important thing is that we get the release out. That said, this feels
quite last-minute. Most of us have been planning with 4.1 in mind.

While 5.x does open the door to removing deprecated methods/configs
and gives us more freedom to make breaking changes, I think we should
proceed with 4.1 as originally planned. Even if the work for 5.x seems
small now, properly thinking through what all we might want to include
in a major release could end up pushing timelines further.

Unless we decide to do a last-minute major version bump without
looking out what more can be scoped — which I don’t think would be
very community-friendly — it's better to keep things simple and
focused. From what I can tell, most of the folks don't have a
particularly strong preference, so let’s not overcomplicate things 2
days before the code freeze.

Let’s aim to release 4.1 next week as planned, and we can save the
more ambitious ideas for 5.x 🙂

(Outside scope info): As far as I know, Hadoop is also planning to
drop JDK 8 only in the 3.4 release [1].

-Ayush

[1] https://lists.apache.org/thread/5543x33nv6196bc5xyxdqjoh2dht0kbg


On Mon, 23 Jun 2025 at 16:48, Shohei Okumiya <ok...@apache.org> wrote:
>
> Thank you all!
>
> I also think Iceberg REST Catalog is not a blocker. It is nice to have, and HIVE-29019 verified that no critical bugs, such as metadata corruptions, exist in the current effective endpoint. Improving the release cadence sounds more important.
>
> Regarding 4.1 vs 5.0, I also don't have a firm preference. If we go with 5.0, isn't the extra work significant? We may have to change the fix versions of all tickets, update some documents mentioning 4.1.0(such as the following one), that's all? In that case, it sounds technically possible.
> - https://hive.apache.org/docs/latest/capture-lineage-info/
>
> I'm excited about the next release. Hopefully, everyone shares the thought!
>
> Best,
> Okumin
>
>
> On Mon, Jun 23, 2025 at 7:13 PM Butao Zhang <bu...@163.com> wrote:
>>
>> Thanks for the advice!
>> I just sent out the notification email regarding the code freeze[1].
>>
>> Additionally, regarding Stamatis' suggestion: "I feel that the current content deserves a major version bump (instead of 4.1.0 we should probably consider 5.0.0)." I don't have strong feelings either for or against this. Given the significant changes in this release, whether it would be more meaningful to define it as version 5.0 may require further discussion. Perhaps we should initiate a community email vote on whether to name it 4.1.0 or 5.0.0?
>>
>> [1]https://lists.apache.org/thread/c166fz9hr3yft75qo3lcmhrbm1howpol
>>
>> Thanks,
>> Butao Zhang
>> ---- Replied Message ----
>> FromAyush Saxena<ay...@gmail.com>
>> Date6/23/2025 17:23
>> To<de...@hive.apache.org>
>> SubjectRe: [DISCUSS] Hive 4.1.x release
>>
>> Could we consider sending out a notification email?
>>
>>
>> Let’s have Butao Zhang, with their Release Manager hat on, do the
>> honors here. :-)
>>
>> Butao Zhang, could you please send out a [NOTICE] email announcing
>> that the code freeze for 4.1.0 is set for the 25th?
>> Post that, only critical items will be considered for the release and
>> must be cherry-picked to the release branch (alongside master, where
>> applicable).
>> Feel free to phrase it accordingly.
>>
>> –Ayush
>>
>> On Mon, 23 Jun 2025 at 14:33, Denys Kuzmenko <dk...@apache.org> wrote:
>>
>>
>> HIVE-29038 is more like an open discussion.
>>
>> 25th sounds good! Could we consider sending out a notification email?
>>
>> Thanks!

Re: [DISCUSS] Hive 4.1.x release

Posted by Ayush Saxena <ay...@gmail.com>.
I don’t have a strict preference around the version number — the
important thing is that we get the release out. That said, this feels
quite last-minute. Most of us have been planning with 4.1 in mind.

While 5.x does open the door to removing deprecated methods/configs
and gives us more freedom to make breaking changes, I think we should
proceed with 4.1 as originally planned. Even if the work for 5.x seems
small now, properly thinking through what all we might want to include
in a major release could end up pushing timelines further.

Unless we decide to do a last-minute major version bump without
looking out what more can be scoped — which I don’t think would be
very community-friendly — it's better to keep things simple and
focused. From what I can tell, most of the folks don't have a
particularly strong preference, so let’s not overcomplicate things 2
days before the code freeze.

Let’s aim to release 4.1 next week as planned, and we can save the
more ambitious ideas for 5.x 🙂

(Outside scope info): As far as I know, Hadoop is also planning to
drop JDK 8 only in the 3.4 release [1].

-Ayush

[1] https://lists.apache.org/thread/5543x33nv6196bc5xyxdqjoh2dht0kbg


On Mon, 23 Jun 2025 at 16:48, Shohei Okumiya <ok...@apache.org> wrote:
>
> Thank you all!
>
> I also think Iceberg REST Catalog is not a blocker. It is nice to have, and HIVE-29019 verified that no critical bugs, such as metadata corruptions, exist in the current effective endpoint. Improving the release cadence sounds more important.
>
> Regarding 4.1 vs 5.0, I also don't have a firm preference. If we go with 5.0, isn't the extra work significant? We may have to change the fix versions of all tickets, update some documents mentioning 4.1.0(such as the following one), that's all? In that case, it sounds technically possible.
> - https://hive.apache.org/docs/latest/capture-lineage-info/
>
> I'm excited about the next release. Hopefully, everyone shares the thought!
>
> Best,
> Okumin
>
>
> On Mon, Jun 23, 2025 at 7:13 PM Butao Zhang <bu...@163.com> wrote:
>>
>> Thanks for the advice!
>> I just sent out the notification email regarding the code freeze[1].
>>
>> Additionally, regarding Stamatis' suggestion: "I feel that the current content deserves a major version bump (instead of 4.1.0 we should probably consider 5.0.0)." I don't have strong feelings either for or against this. Given the significant changes in this release, whether it would be more meaningful to define it as version 5.0 may require further discussion. Perhaps we should initiate a community email vote on whether to name it 4.1.0 or 5.0.0?
>>
>> [1]https://lists.apache.org/thread/c166fz9hr3yft75qo3lcmhrbm1howpol
>>
>> Thanks,
>> Butao Zhang
>> ---- Replied Message ----
>> FromAyush Saxena<ay...@gmail.com>
>> Date6/23/2025 17:23
>> To<de...@hive.apache.org>
>> SubjectRe: [DISCUSS] Hive 4.1.x release
>>
>> Could we consider sending out a notification email?
>>
>>
>> Let’s have Butao Zhang, with their Release Manager hat on, do the
>> honors here. :-)
>>
>> Butao Zhang, could you please send out a [NOTICE] email announcing
>> that the code freeze for 4.1.0 is set for the 25th?
>> Post that, only critical items will be considered for the release and
>> must be cherry-picked to the release branch (alongside master, where
>> applicable).
>> Feel free to phrase it accordingly.
>>
>> –Ayush
>>
>> On Mon, 23 Jun 2025 at 14:33, Denys Kuzmenko <dk...@apache.org> wrote:
>>
>>
>> HIVE-29038 is more like an open discussion.
>>
>> 25th sounds good! Could we consider sending out a notification email?
>>
>> Thanks!

Re: [DISCUSS] Hive 4.1.x release

Posted by Shohei Okumiya <ok...@apache.org>.
Thank you all!

I also think Iceberg REST Catalog is not a blocker. It is nice to have,
and HIVE-29019 verified that no critical bugs, such as metadata
corruptions, exist in the current effective endpoint. Improving the release
cadence sounds more important.

Regarding 4.1 vs 5.0, I also don't have a firm preference. If we go with
5.0, isn't the extra work significant? We may have to change the fix
versions of all tickets, update some documents mentioning 4.1.0(such as the
following one), that's all? In that case, it sounds technically possible.
- https://hive.apache.org/docs/latest/capture-lineage-info/

I'm excited about the next release. Hopefully, everyone shares the thought!

Best,
Okumin


On Mon, Jun 23, 2025 at 7:13 PM Butao Zhang <bu...@163.com> wrote:

> Thanks for the advice!
> I just sent out the notification email regarding the code freeze[1].
>
> Additionally, regarding Stamatis' suggestion: "I feel that the current
> content deserves a major version bump (instead of 4.1.0 we should probably
> consider 5.0.0)." I don't have strong feelings either for or against this.
> Given the significant changes in this release, whether it would be more
> meaningful to define it as version 5.0 may require further discussion. Perhaps
> we should initiate a community email vote on whether to name it 4.1.0 or
> 5.0.0?
>
> [1]https://lists.apache.org/thread/c166fz9hr3yft75qo3lcmhrbm1howpol
>
> Thanks,
> Butao Zhang
> ---- Replied Message ----
> From Ayush Saxena<ay...@gmail.com> <ay...@gmail.com>
> Date 6/23/2025 17:23
> To <de...@hive.apache.org> <de...@hive.apache.org>
> Subject Re: [DISCUSS] Hive 4.1.x release
>
> Could we consider sending out a notification email?
>
>
> Let’s have Butao Zhang, with their Release Manager hat on, do the
> honors here. :-)
>
> Butao Zhang, could you please send out a [NOTICE] email announcing
> that the code freeze for 4.1.0 is set for the 25th?
> Post that, only critical items will be considered for the release and
> must be cherry-picked to the release branch (alongside master, where
> applicable).
> Feel free to phrase it accordingly.
>
> –Ayush
>
> On Mon, 23 Jun 2025 at 14:33, Denys Kuzmenko <dk...@apache.org> wrote:
>
>
> HIVE-29038 is more like an open discussion.
>
> 25th sounds good! Could we consider sending out a notification email?
>
> Thanks!
>
>

Re: [DISCUSS] Hive 4.1.x release

Posted by Butao Zhang <bu...@163.com>.
Thanks for the advice!
I just sent out the notification email regarding the code freeze[1].


Additionally, regarding Stamatis' suggestion: "I feel that the current content deserves a major version bump (instead of 4.1.0 we should probably consider 5.0.0)." I don't have strong feelings either for or against this. Given the significant changes in this release, whether it would be more meaningful to define it as version 5.0 may require further discussion. Perhaps we should initiate a community email vote on whether to name it 4.1.0 or 5.0.0?


[1]https://lists.apache.org/thread/c166fz9hr3yft75qo3lcmhrbm1howpol


Thanks,
Butao Zhang
---- Replied Message ----
| From | Ayush Saxena<ay...@gmail.com> |
| Date | 6/23/2025 17:23 |
| To | <de...@hive.apache.org> |
| Subject | Re: [DISCUSS] Hive 4.1.x release |
Could we consider sending out a notification email?

Let’s have Butao Zhang, with their Release Manager hat on, do the
honors here. :-)

Butao Zhang, could you please send out a [NOTICE] email announcing
that the code freeze for 4.1.0 is set for the 25th?
Post that, only critical items will be considered for the release and
must be cherry-picked to the release branch (alongside master, where
applicable).
Feel free to phrase it accordingly.

–Ayush

On Mon, 23 Jun 2025 at 14:33, Denys Kuzmenko <dk...@apache.org> wrote:

HIVE-29038 is more like an open discussion.

25th sounds good! Could we consider sending out a notification email?

Thanks!

Re: [DISCUSS] Hive 4.1.x release

Posted by Ayush Saxena <ay...@gmail.com>.
> Could we consider sending out a notification email?

Let’s have Butao Zhang, with their Release Manager hat on, do the
honors here. :-)

Butao Zhang, could you please send out a [NOTICE] email announcing
that the code freeze for 4.1.0 is set for the 25th?
Post that, only critical items will be considered for the release and
must be cherry-picked to the release branch (alongside master, where
applicable).
Feel free to phrase it accordingly.

–Ayush

On Mon, 23 Jun 2025 at 14:33, Denys Kuzmenko <dk...@apache.org> wrote:
>
> HIVE-29038 is more like an open discussion.
>
> 25th sounds good! Could we consider sending out a notification email?
>
> Thanks!

Re: [DISCUSS] Hive 4.1.x release

Posted by Denys Kuzmenko <dk...@apache.org>.
HIVE-29038 is more like an open discussion.

25th sounds good! Could we consider sending out a notification email?

Thanks! 

Re: [DISCUSS] Hive 4.1.x release

Posted by Ayush Saxena <ay...@gmail.com>.
Having the 25th as the branch creation date sounds good to me.

Regarding HIVE-29038 — if there's a quick fix available, I’m fine
proceeding with it. Once the branch is created, we can wait for the DN
regressions to be addressed there. In the meantime, any other fixes
that land can also be cherry-picked to the branch as needed.

Just to clarify: my suggestion is simply to go ahead with branch
creation and start the release process in parallel. We can continue
discussing with folks whether specific tickets should be part of this
release or deferred to the next — the intent is not to override
anyone’s opinion. We should only push the tag once everyone is
aligned.

–Ayush

On Mon, 23 Jun 2025 at 13:26, Stamatis Zampetakis <za...@gmail.com> wrote:
>
> It's great to see that we are so close in cutting the next Hive release!
>
> My take as usual is to not hold off the release unless there are
> serious issues, regressions, and legally blocking stuff. Anything that
> falls into the category nice to have can go into the next release
> assuming that it will follow-up in the next few months.
>
> Regarding the versioning scheme, I feel that the current content
> deserves a major version bump (instead of 4.1.0 we should probably
> consider 5.0.0). There have been many changes (~500 commits) since the
> 4.0.1 and among them we definitely have breaking changes, behavior
> changes, new features that are gonna be noticeable by end-users. The
> JDK17 upgrade and the drop of support for 8 is one example, the
> upgrade of Calcite from 1.25.0 to 1.33.0 highlighting affecting the
> query optimizer is another one, and I am sure we can find many more.
>
> Best,
> Stamatis
>
> On Mon, Jun 23, 2025 at 8:38 AM Denys Kuzmenko <dk...@apache.org> wrote:
> >
> > RestCatalog is definitely not a blocker, however, would be good to reach an agreement on HIVE-29038 before releasing the new configs.
> > For now, we can move forward with Iceberg-specific configs and handle unification in a follow-up release.
> > WDYT?
> >
> > HIVE-28744 <https://issues.apache.org/jira/browse/HIVE-28744>: Fix the row count estimation for INT and BIGINT types
> > is not a regression, but should improve row count estimates and performance
> >
> > Only HIVE-29030 and HIVE-29034 are regression-related items

Re: [DISCUSS] Hive 4.1.x release

Posted by Stamatis Zampetakis <za...@gmail.com>.
It's great to see that we are so close in cutting the next Hive release!

My take as usual is to not hold off the release unless there are
serious issues, regressions, and legally blocking stuff. Anything that
falls into the category nice to have can go into the next release
assuming that it will follow-up in the next few months.

Regarding the versioning scheme, I feel that the current content
deserves a major version bump (instead of 4.1.0 we should probably
consider 5.0.0). There have been many changes (~500 commits) since the
4.0.1 and among them we definitely have breaking changes, behavior
changes, new features that are gonna be noticeable by end-users. The
JDK17 upgrade and the drop of support for 8 is one example, the
upgrade of Calcite from 1.25.0 to 1.33.0 highlighting affecting the
query optimizer is another one, and I am sure we can find many more.

Best,
Stamatis

On Mon, Jun 23, 2025 at 8:38 AM Denys Kuzmenko <dk...@apache.org> wrote:
>
> RestCatalog is definitely not a blocker, however, would be good to reach an agreement on HIVE-29038 before releasing the new configs.
> For now, we can move forward with Iceberg-specific configs and handle unification in a follow-up release.
> WDYT?
>
> HIVE-28744 <https://issues.apache.org/jira/browse/HIVE-28744>: Fix the row count estimation for INT and BIGINT types
> is not a regression, but should improve row count estimates and performance
>
> Only HIVE-29030 and HIVE-29034 are regression-related items

Re: [DISCUSS] Hive 4.1.x release

Posted by Denys Kuzmenko <dk...@apache.org>.
RestCatalog is definitely not a blocker, however, would be good to reach an agreement on HIVE-29038 before releasing the new configs.
For now, we can move forward with Iceberg-specific configs and handle unification in a follow-up release. 
WDYT?

HIVE-28744 <https://issues.apache.org/jira/browse/HIVE-28744>: Fix the row count estimation for INT and BIGINT types
is not a regression, but should improve row count estimates and performance

Only HIVE-29030 and HIVE-29034 are regression-related items

Re: [DISCUSS] Hive 4.1.x release

Posted by Butao Zhang <bu...@163.com>.
Given that there are currently only a few relatively minor fixes, I think it would be appropriate to set a specific branch-cutting date.


How about June 25, 2025, at UTC+8 (East Eighth Zone Time)? After this date, if there are still any code that needs to be fixed on version 4.1.0, we can solve it through cherry-pick.


Thanks,
Butao Zhang
---- Replied Message ----
| From | Ayush Saxena<ay...@gmail.com> |
| Date | 6/23/2025 14:09 |
| To | <de...@hive.apache.org> |
| Subject | Re: [DISCUSS] Hive 4.1.x release |
Do we consider the two REST catalog issues as blockers? To me, they
seem more like improvements that we could potentially defer,
especially if the other item gets addressed.

Denys — is HIVE-28744 a regression, or was it present in earlier releases too?

The other two do appear to be blockers. Personally, I still feel we
should move ahead with branching for Hive 4.1.0 and treat this as a
code freeze point. We can cherry-pick these, or any other relevant
fixes, into the branch afterward — otherwise, code will continue to
evolve and drift.

Just my 2 cents.
– Ayush

On Mon, 23 Jun 2025 at 10:50, Denys Kuzmenko <dk...@apache.org> wrote:

 Currently, at least three items remain incomplete:
 https://issues.apache.org/jira/issues/?jql=project%20%3D%20HIVE%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20hive-4.1.0-must%20ORDER%20BY%20updated%20DESC%2C%20priority%20DESC

Re: [DISCUSS] Hive 4.1.x release

Posted by Ayush Saxena <ay...@gmail.com>.
Do we consider the two REST catalog issues as blockers? To me, they
seem more like improvements that we could potentially defer,
especially if the other item gets addressed.

Denys — is HIVE-28744 a regression, or was it present in earlier releases too?

The other two do appear to be blockers. Personally, I still feel we
should move ahead with branching for Hive 4.1.0 and treat this as a
code freeze point. We can cherry-pick these, or any other relevant
fixes, into the branch afterward — otherwise, code will continue to
evolve and drift.

Just my 2 cents.
– Ayush

On Mon, 23 Jun 2025 at 10:50, Denys Kuzmenko <dk...@apache.org> wrote:
>
> Currently, at least three items remain incomplete:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HIVE%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20hive-4.1.0-must%20ORDER%20BY%20updated%20DESC%2C%20priority%20DESC

Re: [DISCUSS] Hive 4.1.x release

Posted by Denys Kuzmenko <dk...@apache.org>.
Currently, at least three items remain incomplete: 
https://issues.apache.org/jira/issues/?jql=project%20%3D%20HIVE%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20hive-4.1.0-must%20ORDER%20BY%20updated%20DESC%2C%20priority%20DESC

Re: [DISCUSS] Hive 4.1.x release

Posted by Ayush Saxena <ay...@gmail.com>.
Hi Butao Zhang,

I believe all blockers have now been addressed. The remaining items, as far as I can tell, are nice-to-haves rather than critical.

You can go ahead with the branch creation and initiate the release process. If any pending items get resolved before the tag is pushed, they can still be included in the branch.

Given that the entire process will take 2–3 days, there’s some buffer for folks to get things in. Anything that doesn’t make it can be targeted for the next release.

Best regards,
Ayush

> On 20 Jun 2025, at 1:32 PM, Butao Zhang <zh...@apache.org> wrote:
> 
> Thanks Denys! I am also watching these tickets.
> 
>> On 2025/06/19 09:32:28 Denys Kuzmenko wrote:
>> Hi everyone,
>> 
>> Short Status UPDATE:
>> 
>> RESOLVED:
>> 1) HIVE-28987 <https://issues.apache.org/jira/browse/HIVE-28987>: Iceberg: A faulty query predicate can compromise transaction isolation.
>> 2) HIVE-27275 <https://issues.apache.org/jira/browse/HIVE-27275>: Provide docker image for HMS
>> 3) HIVE-28665 <https://issues.apache.org/jira/browse/HIVE-28665>: Iceberg: Upgrade iceberg version to 1.9.1
>> 4) HIVE-28930 <https://issues.apache.org/jira/browse/HIVE-28930>: Implement
>> a metastore service that expires iceberg table snapshots periodically
>> 5) HIVE-28986 <https://issues.apache.org/jira/browse/HIVE-28986>: Upgrade
>> to tez 0.10.5
>> 
>> PENDING:
>> 1) RestCatalog
>>  - HIVE-29016 <https://issues.apache.org/jira/browse/HIVE-29016>: Disable caching on the Iceberg REST Catalog
>>  - HIVE-28998 <https://issues.apache.org/jira/browse/HIVE-28998>: Support Iceberg REST Catalog without authentication
>>  - HIVE-29018 <https://issues.apache.org/jira/browse/HIVE-29018>: Build Iceberg REST Catalog with Java 17
>> 2) HIVE-28984 <https://issues.apache.org/jira/browse/HIVE-28984>: Replace nashorn-core with graalvm which is compatible with ASF license
>> 3) HIVE-29003 <https://issues.apache.org/jira/browse/HIVE-29003>: DatePartitionedLogger from Tez is always needed for hive logging hook
>> 4) HIVE-28744 <https://issues.apache.org/jira/browse/HIVE-28744>: Wrong Int, Bigint type range estimation in estimateColStats
>> 
>> REMOVED:
>> 1) HIVE-28658 <https://issues.apache.org/jira/browse/HIVE-28658>: Iceberg REST Catalog client
>> Reason: unclear ETA, please let me know if we should still try to sqeeze it in
>> 2) HIVE-26505
>> Case When Some result data is lost when there are common column conditions and partitioned column conditions
>> Reason: no updates on either the Jira issue or the PR
>> 
>> Let me know if I’ve overlooked anything. Once the blockers are addressed, we can proceed with branching
>> 
>> Thanks,
>> Denys
>> 

Re: Re: [DISCUSS] Hive 4.1.x release

Posted by Butao Zhang <zh...@apache.org>.
Thanks Denys! I am also watching these tickets.

On 2025/06/19 09:32:28 Denys Kuzmenko wrote:
> Hi everyone,
> 
> Short Status UPDATE:
> 
> RESOLVED:
> 1) HIVE-28987 <https://issues.apache.org/jira/browse/HIVE-28987>: Iceberg: A faulty query predicate can compromise transaction isolation.
> 2) HIVE-27275 <https://issues.apache.org/jira/browse/HIVE-27275>: Provide docker image for HMS
> 3) HIVE-28665 <https://issues.apache.org/jira/browse/HIVE-28665>: Iceberg: Upgrade iceberg version to 1.9.1
> 4) HIVE-28930 <https://issues.apache.org/jira/browse/HIVE-28930>: Implement
> a metastore service that expires iceberg table snapshots periodically
> 5) HIVE-28986 <https://issues.apache.org/jira/browse/HIVE-28986>: Upgrade
> to tez 0.10.5
> 
> PENDING:
> 1) RestCatalog
>   - HIVE-29016 <https://issues.apache.org/jira/browse/HIVE-29016>: Disable caching on the Iceberg REST Catalog
>   - HIVE-28998 <https://issues.apache.org/jira/browse/HIVE-28998>: Support Iceberg REST Catalog without authentication
>   - HIVE-29018 <https://issues.apache.org/jira/browse/HIVE-29018>: Build Iceberg REST Catalog with Java 17
> 2) HIVE-28984 <https://issues.apache.org/jira/browse/HIVE-28984>: Replace nashorn-core with graalvm which is compatible with ASF license
> 3) HIVE-29003 <https://issues.apache.org/jira/browse/HIVE-29003>: DatePartitionedLogger from Tez is always needed for hive logging hook
> 4) HIVE-28744 <https://issues.apache.org/jira/browse/HIVE-28744>: Wrong Int, Bigint type range estimation in estimateColStats
>  
> REMOVED:
> 1) HIVE-28658 <https://issues.apache.org/jira/browse/HIVE-28658>: Iceberg REST Catalog client
> Reason: unclear ETA, please let me know if we should still try to sqeeze it in
> 2) HIVE-26505
> Case When Some result data is lost when there are common column conditions and partitioned column conditions
> Reason: no updates on either the Jira issue or the PR
> 
> Let me know if I’ve overlooked anything. Once the blockers are addressed, we can proceed with branching
> 
> Thanks,
> Denys
> 

Re: Re: [DISCUSS] Hive 4.1.x release

Posted by Denys Kuzmenko <dk...@apache.org>.
Hi everyone,

Short Status UPDATE:

RESOLVED:
1) HIVE-28987 <https://issues.apache.org/jira/browse/HIVE-28987>: Iceberg: A faulty query predicate can compromise transaction isolation.
2) HIVE-27275 <https://issues.apache.org/jira/browse/HIVE-27275>: Provide docker image for HMS
3) HIVE-28665 <https://issues.apache.org/jira/browse/HIVE-28665>: Iceberg: Upgrade iceberg version to 1.9.1
4) HIVE-28930 <https://issues.apache.org/jira/browse/HIVE-28930>: Implement
a metastore service that expires iceberg table snapshots periodically
5) HIVE-28986 <https://issues.apache.org/jira/browse/HIVE-28986>: Upgrade
to tez 0.10.5

PENDING:
1) RestCatalog
  - HIVE-29016 <https://issues.apache.org/jira/browse/HIVE-29016>: Disable caching on the Iceberg REST Catalog
  - HIVE-28998 <https://issues.apache.org/jira/browse/HIVE-28998>: Support Iceberg REST Catalog without authentication
  - HIVE-29018 <https://issues.apache.org/jira/browse/HIVE-29018>: Build Iceberg REST Catalog with Java 17
2) HIVE-28984 <https://issues.apache.org/jira/browse/HIVE-28984>: Replace nashorn-core with graalvm which is compatible with ASF license
3) HIVE-29003 <https://issues.apache.org/jira/browse/HIVE-29003>: DatePartitionedLogger from Tez is always needed for hive logging hook
4) HIVE-28744 <https://issues.apache.org/jira/browse/HIVE-28744>: Wrong Int, Bigint type range estimation in estimateColStats
 
REMOVED:
1) HIVE-28658 <https://issues.apache.org/jira/browse/HIVE-28658>: Iceberg REST Catalog client
Reason: unclear ETA, please let me know if we should still try to sqeeze it in
2) HIVE-26505
Case When Some result data is lost when there are common column conditions and partitioned column conditions
Reason: no updates on either the Jira issue or the PR

Let me know if I’ve overlooked anything. Once the blockers are addressed, we can proceed with branching

Thanks,
Denys

Re: Re: [DISCUSS] Hive 4.1.x release

Posted by László Bodor <bo...@gmail.com>.
I just added hive-4.1.0-must label on a recently created HIVE-29003
<https://issues.apache.org/jira/browse/HIVE-29003>: DatePartitionedLogger
from Tez is always needed for hive logging hook, please consider this also

On Mon, 9 Jun 2025 at 14:33, Okumin <ma...@okumin.com> wrote:

> Hi,
>
> I plan to test "HIVE-28059: Iceberg REST Catalog" via another query
> engine quickly once Iceberg 1.9.1 is merged. When I tested it last
> time, a set of basic commands didn't work because our HiveCatalog
> lacked some features. Now, it should be ok.
> I'm also aware that the current REST Catalog only implements
> authentication methods native to Hadoop or Hive, which are hard for
> another query engine to use. I created HIVE-28998 and plan to push the
> change in my local branch. I expect we can support the minimal
> functionality of the REST Catalog with 4.1.0, but I may reconsider if
> I cannot easily make the feature work.
>
> I'm sure 4.1.0 will be as fantastic a release as 4.0.0!
>
> Regards,
> Okumin
>
> On Mon, Jun 9, 2025 at 7:33 PM Simhadri G <si...@apache.org> wrote:
> >
> > Hi,
> > If we are adding standalone-metastore docker image, i think it will be
> good  to add a github action to build and push this image to hive dockerhub
> repo as well.
> >
> > Thanks!
> > Simhadri G
> >
> > On Mon, Jun 9, 2025 at 2:44 PM Denys Kuzmenko <dk...@apache.org>
> wrote:
> >>
> >> Hi,
> >>
> >> We reviewed and cleaned up the `hive-must` items with Ayush. A few
> tasks remain, which should be resolved soon, along with some pending
> responses.
> >>
> >> Regarding the `standalone-metastore`, I suggest releasing a tarball and
> a lightweight Docker image, as there has been strong interest. I plan to
> update the release page with additional instructions.
> >>
> >> Regards,
> >> Denys
>

Re: Re: [DISCUSS] Hive 4.1.x release

Posted by Okumin <ma...@okumin.com>.
Hi,

I plan to test "HIVE-28059: Iceberg REST Catalog" via another query
engine quickly once Iceberg 1.9.1 is merged. When I tested it last
time, a set of basic commands didn't work because our HiveCatalog
lacked some features. Now, it should be ok.
I'm also aware that the current REST Catalog only implements
authentication methods native to Hadoop or Hive, which are hard for
another query engine to use. I created HIVE-28998 and plan to push the
change in my local branch. I expect we can support the minimal
functionality of the REST Catalog with 4.1.0, but I may reconsider if
I cannot easily make the feature work.

I'm sure 4.1.0 will be as fantastic a release as 4.0.0!

Regards,
Okumin

On Mon, Jun 9, 2025 at 7:33 PM Simhadri G <si...@apache.org> wrote:
>
> Hi,
> If we are adding standalone-metastore docker image, i think it will be good  to add a github action to build and push this image to hive dockerhub repo as well.
>
> Thanks!
> Simhadri G
>
> On Mon, Jun 9, 2025 at 2:44 PM Denys Kuzmenko <dk...@apache.org> wrote:
>>
>> Hi,
>>
>> We reviewed and cleaned up the `hive-must` items with Ayush. A few tasks remain, which should be resolved soon, along with some pending responses.
>>
>> Regarding the `standalone-metastore`, I suggest releasing a tarball and a lightweight Docker image, as there has been strong interest. I plan to update the release page with additional instructions.
>>
>> Regards,
>> Denys

Re: Re: [DISCUSS] Hive 4.1.x release

Posted by Simhadri G <si...@apache.org>.
Hi,
If we are adding standalone-metastore docker image, i think it will be
good  to add a github action
<https://github.com/apache/hive/blob/master/.github/workflows/docker-GA-images.yml>
to build and push this image to hive dockerhub repo as well.

Thanks!
Simhadri G

On Mon, Jun 9, 2025 at 2:44 PM Denys Kuzmenko <dk...@apache.org> wrote:

> Hi,
>
> We reviewed and cleaned up the `hive-must` items with Ayush. A few tasks
> remain, which should be resolved soon, along with some pending responses.
>
> Regarding the `standalone-metastore`, I suggest releasing a tarball and a
> lightweight Docker image, as there has been strong interest. I plan to
> update the release page with additional instructions.
>
> Regards,
> Denys
>

Re: Re: [DISCUSS] Hive 4.1.x release

Posted by Denys Kuzmenko <dk...@apache.org>.
Hi,

We reviewed and cleaned up the `hive-must` items with Ayush. A few tasks remain, which should be resolved soon, along with some pending responses. 

Regarding the `standalone-metastore`, I suggest releasing a tarball and a lightweight Docker image, as there has been strong interest. I plan to update the release page with additional instructions.

Regards,
Denys

Re: Re: [DISCUSS] Hive 4.1.x release

Posted by Butao Zhang <zh...@apache.org>.
I'm very glad to see that the jdk17 issues related to the version 4.1.0 have been resolved. 

As the release manager of version 4.1.0, I have studied the documentation for hive version releases[1].

I think the next step is to cut the 4.1.0 branch from the master branch. From the PRs mentioned by the above folks that we want to include in version 4.1.0, personally, I think the branch cutting time should be this weekend or at the latest next week. The specific branch cutting time will be updated in this thread.

BTW, The hive-standalone-metastore has not been released since version 3.0.0. I'm not sure if we need to release the hive-standalone-metastore version. Or, could we release the hive-standalone-metastore version independently instead of releasing it together with Hive4.1.0?

[1]https://cwiki.apache.org/confluence/display/Hive/HowToRelease#HowToRelease-HiveRelease

Butao Zhang,
Thanks.

On 2025/06/06 11:34:36 Denys Kuzmenko wrote:
> The following filter returns 17 'hive-4.1.0-must' items, some with the abandon PR:
> 
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HIVE%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20hive-4.1.0-must%20ORDER%20BY%20updated%20DESC%2C%20priority%20DESC
> 
> 

Re: Re: [DISCUSS] Hive 4.1.x release

Posted by Denys Kuzmenko <dk...@apache.org>.
The following filter returns 17 'hive-4.1.0-must' items, some with the abandon PR:

https://issues.apache.org/jira/issues/?jql=project%20%3D%20HIVE%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20hive-4.1.0-must%20ORDER%20BY%20updated%20DESC%2C%20priority%20DESC


Re: Re: [DISCUSS] Hive 4.1.x release

Posted by László Bodor <bo...@gmail.com>.
Hi Team!

If we consider non-blockers too (idea came from Denys' suggestion above),
then I would love to include these below (both are 99% finished/approved,
I'm just fighting with the precommit):


   1. HIVE-28930 <https://issues.apache.org/jira/browse/HIVE-28930>: Implement
   a metastore service that expires iceberg table snapshots periodically
   1. HIVE-28986 <https://issues.apache.org/jira/browse/HIVE-28986>: Upgrade
      to tez 0.10.5

      Regards,
      Laszlo Bodor



On Fri, 6 Jun 2025 at 13:00, Denys Kuzmenko <dk...@apache.org> wrote:

> Hi Ayush,
>
> Thanks for restarting the conversation!
>
> I have a few additional items to consider, with only the first one being
> mandatory.
>
> 1) HIVE-28987: Iceberg: A faulty query predicate can compromise
> transaction isolation.
> 2) HIVE-27275: Provide docker image for HMS (needs approving review)
> 3) HIVE-28665: Iceberg: Upgrade iceberg version to 1.9.1
>
> Regards,
> Denys
>

Re: Re: [DISCUSS] Hive 4.1.x release

Posted by Denys Kuzmenko <dk...@apache.org>.
Hi Ayush,

Thanks for restarting the conversation!

I have a few additional items to consider, with only the first one being mandatory.

1) HIVE-28987: Iceberg: A faulty query predicate can compromise transaction isolation. 
2) HIVE-27275: Provide docker image for HMS (needs approving review)
3) HIVE-28665: Iceberg: Upgrade iceberg version to 1.9.1

Regards,
Denys