You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Denys Kuzmenko <dk...@cloudera.com.INVALID> on 2024/03/01 13:33:00 UTC
Re: Release of Hive 4 and TPC-DS benchmark
Hi All,
Some updates on Hive-4.0 status:
All known 4.0 release blockers were addressed and merged to the master.
Thanks to everyone involved!!!
We've raised a new PR to rebase branch-4.0. After that, we'll accept only
bug fixes or some major improvements.
Feel free to check and test branch-4.0 in a couple of days. We'll continue
running number of functional & performance tests before the RC.
Regards,
Denys
On Tue, Jan 30, 2024 at 6:40 AM Okumin <ma...@okumin.com> wrote:
> Hi,
>
> We deployed a test version of Hive 4 in our production. It is mostly
> working well. I appreciate the community's effort.
>
> Let me bump this thread so that community members can share the latest
> situation, and we can contribute to the remaining problems.
>
> Currently, we list the following tickets labeled with hive-4.0.0-must[1].
> - HIVE-27858: OOM happens when selecting many columns and JOIN
> - HIVE-26654: Test with the TPC-DS benchmark
> - HIVE-24167: No.14 (Postponed until 4.1.0)
> - HIVE-26986: No.71 (No labels)
>
> We have already applied a workaround[2] for HIVE-24167. Also, Denys
> says HIVE-26986 is not a blocker of Hive 4.0.0. So, I expect
> HIVE-27858 to be the only blocker.
>
> I have some proposals about the next action.
> - Can we take over HIVE-27858? We are the reporter of the ticket. It
> would be reasonable If Naveen is not working on it yet
> - We may review HIVE-27984[3]. As I mentioned in another thread[4], it
> would have a fairly big impact on the Hive ecosystem
>
> And I would like someone to take a look at the following PR for
> HIVE-24167. It is not a blocker thanks to
> `hive.optimize.cte.materialize.threshold=-1`. However, we plan to use
> Hive 4 with CTE materialization.
> https://github.com/apache/hive/pull/5037
>
> Thanks!
>
> - [1]
> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20hive-4.0.0-must
> - [2] https://github.com/apache/hive/pull/4858
> - [3] https://github.com/apache/hive/pull/4984
> - [4] https://www.mail-archive.com/dev@hive.apache.org/msg144280.html
>
> Regards,
> Okumin
>
> On Thu, Nov 23, 2023 at 11:42 PM Butao Zhang <bu...@163.com> wrote:
> >
> > Cool! Thank you Denys for driving this!
> >
> >
> > Thanks,
> > Butao Zhang
> > ---- Replied Message ----
> > | From | Attila Turoczy<at...@cloudera.com.INVALID> |
> > | Date | 11/23/2023 22:29 |
> > | To | <de...@hive.apache.org> |
> > | Subject | Re: Release of Hive 4 and TPC-DS benchmark |
> > Excellent new Denys! Hive 4 is here! Can't wait :)
> >
> > -Attila
> >
> >
> > On Thu, Nov 23, 2023 at 3:20 PM Denys Kuzmenko <dk...@apache.org>
> wrote:
> >
> > Update:
> > 1. Query 2, 71: Resolved in HIVE-27006 [1];
> >
> > 2. Query 97: Under review in HIVE-27269 [2];
> > Thanks, Seonggon for providing a reproduce qfile.
> >
> > 3. Query 14: Reported in HIVE-24167 [3];
> > set hive.optimize.cte.materialize.threshold to -1 by default in Hive 4
> and
> > fix it in the next versions
> >
> > 4. HIVE-26986 [4] is a performance improvement that is nice to have, but
> > not a blocker for the release.
> >
> > Be advised, next week we plan to cut 4.0.0 release branch from master and
> > start testing.
> >
> > Thanks, Denys
> >
> > [1] https://issues.apache.org/jira/browse/HIVE-27006
> > [2] https://issues.apache.org/jira/browse/HIVE-27269
> > [3] https://issues.apache.org/jira/browse/HIVE-24167
> > [4] https://issues.apache.org/jira/browse/HIVE-26986
> >
> >
>
Re: Release of Hive 4 and TPC-DS benchmark
Posted by Denys Kuzmenko <dk...@apache.org>.
Hi Pravin,
TPC-DS tracking Jira: HIVE-27929.
The functional test run is completed, waiting for the TPC-DS execution. There were some glitches with the infra last week, expecting results in the next 2-3 days. After that, we should be ready with RC.
Regards,
Denys
Re: Release of Hive 4 and TPC-DS benchmark
Posted by Pravin sinha <pk...@gmail.com>.
Thanks for the update on this, Denys. Do we have any tracking jira for the
plan on performance test on the branch-4 build and is that the only
activity pending for releasing Hive-4.0.0?
Regards,
Pravin
On Fri, Mar 1, 2024 at 7:03 PM Denys Kuzmenko
<dk...@cloudera.com.invalid> wrote:
> Hi All,
>
> Some updates on Hive-4.0 status:
>
> All known 4.0 release blockers were addressed and merged to the master.
> Thanks to everyone involved!!!
> We've raised a new PR to rebase branch-4.0. After that, we'll accept only
> bug fixes or some major improvements.
>
> Feel free to check and test branch-4.0 in a couple of days. We'll continue
> running number of functional & performance tests before the RC.
>
> Regards,
> Denys
>
> On Tue, Jan 30, 2024 at 6:40 AM Okumin <ma...@okumin.com> wrote:
>
> > Hi,
> >
> > We deployed a test version of Hive 4 in our production. It is mostly
> > working well. I appreciate the community's effort.
> >
> > Let me bump this thread so that community members can share the latest
> > situation, and we can contribute to the remaining problems.
> >
> > Currently, we list the following tickets labeled with hive-4.0.0-must[1].
> > - HIVE-27858: OOM happens when selecting many columns and JOIN
> > - HIVE-26654: Test with the TPC-DS benchmark
> > - HIVE-24167: No.14 (Postponed until 4.1.0)
> > - HIVE-26986: No.71 (No labels)
> >
> > We have already applied a workaround[2] for HIVE-24167. Also, Denys
> > says HIVE-26986 is not a blocker of Hive 4.0.0. So, I expect
> > HIVE-27858 to be the only blocker.
> >
> > I have some proposals about the next action.
> > - Can we take over HIVE-27858? We are the reporter of the ticket. It
> > would be reasonable If Naveen is not working on it yet
> > - We may review HIVE-27984[3]. As I mentioned in another thread[4], it
> > would have a fairly big impact on the Hive ecosystem
> >
> > And I would like someone to take a look at the following PR for
> > HIVE-24167. It is not a blocker thanks to
> > `hive.optimize.cte.materialize.threshold=-1`. However, we plan to use
> > Hive 4 with CTE materialization.
> > https://github.com/apache/hive/pull/5037
> >
> > Thanks!
> >
> > - [1]
> >
> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20hive-4.0.0-must
> > - [2] https://github.com/apache/hive/pull/4858
> > - [3] https://github.com/apache/hive/pull/4984
> > - [4] https://www.mail-archive.com/dev@hive.apache.org/msg144280.html
> >
> > Regards,
> > Okumin
> >
> > On Thu, Nov 23, 2023 at 11:42 PM Butao Zhang <bu...@163.com>
> wrote:
> > >
> > > Cool! Thank you Denys for driving this!
> > >
> > >
> > > Thanks,
> > > Butao Zhang
> > > ---- Replied Message ----
> > > | From | Attila Turoczy<at...@cloudera.com.INVALID> |
> > > | Date | 11/23/2023 22:29 |
> > > | To | <de...@hive.apache.org> |
> > > | Subject | Re: Release of Hive 4 and TPC-DS benchmark |
> > > Excellent new Denys! Hive 4 is here! Can't wait :)
> > >
> > > -Attila
> > >
> > >
> > > On Thu, Nov 23, 2023 at 3:20 PM Denys Kuzmenko <dk...@apache.org>
> > wrote:
> > >
> > > Update:
> > > 1. Query 2, 71: Resolved in HIVE-27006 [1];
> > >
> > > 2. Query 97: Under review in HIVE-27269 [2];
> > > Thanks, Seonggon for providing a reproduce qfile.
> > >
> > > 3. Query 14: Reported in HIVE-24167 [3];
> > > set hive.optimize.cte.materialize.threshold to -1 by default in Hive 4
> > and
> > > fix it in the next versions
> > >
> > > 4. HIVE-26986 [4] is a performance improvement that is nice to have,
> but
> > > not a blocker for the release.
> > >
> > > Be advised, next week we plan to cut 4.0.0 release branch from master
> and
> > > start testing.
> > >
> > > Thanks, Denys
> > >
> > > [1] https://issues.apache.org/jira/browse/HIVE-27006
> > > [2] https://issues.apache.org/jira/browse/HIVE-27269
> > > [3] https://issues.apache.org/jira/browse/HIVE-24167
> > > [4] https://issues.apache.org/jira/browse/HIVE-26986
> > >
> > >
> >
>