You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iceberg.apache.org by Manu Zhang <ow...@gmail.com> on 2024/08/27 00:34:01 UTC
[DISCUSS] Drop Hive 2 support
Hi all,
I'd like to start a discussion on dropping Hive 2 support, which reached
EOL three months ago[1]. It's also a prerequisite for migration to Hadoop
3, as shown by Steve's PR[2]. For your reference, I have a draft PR[3] to
show the needed changes.
Since we've not deprecated Hive 2 support yet, I think the path is to
deprecate Hive 2 in 1.7 and drop Hive 2 in 1.8. What do you think?
[1] https://hive.apache.org/general/downloads/
[2] https://github.com/apache/iceberg/pull/10932
[3] https://github.com/apache/iceberg/pull/10996
Regards,
Manu
Re: [DISCUSS] Drop Hive 2 support
Posted by Manu Zhang <ow...@gmail.com>.
Hi Anton,
The goal is to drop Hive 2 dependencies across all modules.
Manu
On Wed, Aug 28, 2024 at 4:47 AM Anton Okolnychyi <ao...@gmail.com>
wrote:
> Is the goal to get rid of the Hive 2 connector that currently lives in
> Iceberg or any dependency on Hive 2 across all modules?
>
> - Anton
>
> вт, 27 серп. 2024 р. о 08:29 Ryan Blue <bl...@databricks.com.invalid> пише:
>
>> +1 for this. Anyone using Hive 2 can still use older released versions. I
>> don't think that we do much maintenance on the Hive codebase anyway (I'd
>> also support deprecating/removing Hive 3).
>>
>> On Tue, Aug 27, 2024 at 12:17 AM Piotr Findeisen <
>> piotr.findeisen@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Trino query engine won't be affected by the drop.
>>> These are the artifacts Trino depends on
>>>
>>> iceberg-api
>>> iceberg-bundled-guava
>>> iceberg-core
>>> iceberg-nessie
>>> iceberg-orc
>>> iceberg-parquet
>>> iceberg-snowflake
>>>
>>> Best
>>> Piotr
>>>
>>>
>>>
>>> On Tue, 27 Aug 2024 at 02:51, Anton Okolnychyi <ao...@gmail.com>
>>> wrote:
>>>
>>>> Are we aware of any engines that can be affected by this change?
>>>>
>>>> пн, 26 серп. 2024 р. о 17:34 Manu Zhang <ow...@gmail.com> пише:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I'd like to start a discussion on dropping Hive 2 support, which
>>>>> reached EOL three months ago[1]. It's also a prerequisite for migration to
>>>>> Hadoop 3, as shown by Steve's PR[2]. For your reference, I have a draft
>>>>> PR[3] to show the needed changes.
>>>>>
>>>>> Since we've not deprecated Hive 2 support yet, I think the path is to
>>>>> deprecate Hive 2 in 1.7 and drop Hive 2 in 1.8. What do you think?
>>>>>
>>>>> [1] https://hive.apache.org/general/downloads/
>>>>> [2] https://github.com/apache/iceberg/pull/10932
>>>>> [3] https://github.com/apache/iceberg/pull/10996
>>>>>
>>>>> Regards,
>>>>> Manu
>>>>>
>>>>
>>
>> --
>> Ryan Blue
>> Databricks
>>
>
Re: [DISCUSS] Drop Hive 2 support
Posted by Anton Okolnychyi <ao...@gmail.com>.
Is the goal to get rid of the Hive 2 connector that currently lives in
Iceberg or any dependency on Hive 2 across all modules?
- Anton
вт, 27 серп. 2024 р. о 08:29 Ryan Blue <bl...@databricks.com.invalid> пише:
> +1 for this. Anyone using Hive 2 can still use older released versions. I
> don't think that we do much maintenance on the Hive codebase anyway (I'd
> also support deprecating/removing Hive 3).
>
> On Tue, Aug 27, 2024 at 12:17 AM Piotr Findeisen <
> piotr.findeisen@gmail.com> wrote:
>
>> Hi,
>>
>> Trino query engine won't be affected by the drop.
>> These are the artifacts Trino depends on
>>
>> iceberg-api
>> iceberg-bundled-guava
>> iceberg-core
>> iceberg-nessie
>> iceberg-orc
>> iceberg-parquet
>> iceberg-snowflake
>>
>> Best
>> Piotr
>>
>>
>>
>> On Tue, 27 Aug 2024 at 02:51, Anton Okolnychyi <ao...@gmail.com>
>> wrote:
>>
>>> Are we aware of any engines that can be affected by this change?
>>>
>>> пн, 26 серп. 2024 р. о 17:34 Manu Zhang <ow...@gmail.com> пише:
>>>
>>>> Hi all,
>>>>
>>>> I'd like to start a discussion on dropping Hive 2 support, which
>>>> reached EOL three months ago[1]. It's also a prerequisite for migration to
>>>> Hadoop 3, as shown by Steve's PR[2]. For your reference, I have a draft
>>>> PR[3] to show the needed changes.
>>>>
>>>> Since we've not deprecated Hive 2 support yet, I think the path is to
>>>> deprecate Hive 2 in 1.7 and drop Hive 2 in 1.8. What do you think?
>>>>
>>>> [1] https://hive.apache.org/general/downloads/
>>>> [2] https://github.com/apache/iceberg/pull/10932
>>>> [3] https://github.com/apache/iceberg/pull/10996
>>>>
>>>> Regards,
>>>> Manu
>>>>
>>>
>
> --
> Ryan Blue
> Databricks
>
Re: [DISCUSS] Drop Hive 2 support
Posted by Ryan Blue <bl...@databricks.com.INVALID>.
+1 for this. Anyone using Hive 2 can still use older released versions. I
don't think that we do much maintenance on the Hive codebase anyway (I'd
also support deprecating/removing Hive 3).
On Tue, Aug 27, 2024 at 12:17 AM Piotr Findeisen <pi...@gmail.com>
wrote:
> Hi,
>
> Trino query engine won't be affected by the drop.
> These are the artifacts Trino depends on
>
> iceberg-api
> iceberg-bundled-guava
> iceberg-core
> iceberg-nessie
> iceberg-orc
> iceberg-parquet
> iceberg-snowflake
>
> Best
> Piotr
>
>
>
> On Tue, 27 Aug 2024 at 02:51, Anton Okolnychyi <ao...@gmail.com>
> wrote:
>
>> Are we aware of any engines that can be affected by this change?
>>
>> пн, 26 серп. 2024 р. о 17:34 Manu Zhang <ow...@gmail.com> пише:
>>
>>> Hi all,
>>>
>>> I'd like to start a discussion on dropping Hive 2 support, which reached
>>> EOL three months ago[1]. It's also a prerequisite for migration to Hadoop
>>> 3, as shown by Steve's PR[2]. For your reference, I have a draft PR[3] to
>>> show the needed changes.
>>>
>>> Since we've not deprecated Hive 2 support yet, I think the path is to
>>> deprecate Hive 2 in 1.7 and drop Hive 2 in 1.8. What do you think?
>>>
>>> [1] https://hive.apache.org/general/downloads/
>>> [2] https://github.com/apache/iceberg/pull/10932
>>> [3] https://github.com/apache/iceberg/pull/10996
>>>
>>> Regards,
>>> Manu
>>>
>>
--
Ryan Blue
Databricks
Re: [DISCUSS] Drop Hive 2 support
Posted by Piotr Findeisen <pi...@gmail.com>.
Hi,
Trino query engine won't be affected by the drop.
These are the artifacts Trino depends on
iceberg-api
iceberg-bundled-guava
iceberg-core
iceberg-nessie
iceberg-orc
iceberg-parquet
iceberg-snowflake
Best
Piotr
On Tue, 27 Aug 2024 at 02:51, Anton Okolnychyi <ao...@gmail.com>
wrote:
> Are we aware of any engines that can be affected by this change?
>
> пн, 26 серп. 2024 р. о 17:34 Manu Zhang <ow...@gmail.com> пише:
>
>> Hi all,
>>
>> I'd like to start a discussion on dropping Hive 2 support, which reached
>> EOL three months ago[1]. It's also a prerequisite for migration to Hadoop
>> 3, as shown by Steve's PR[2]. For your reference, I have a draft PR[3] to
>> show the needed changes.
>>
>> Since we've not deprecated Hive 2 support yet, I think the path is to
>> deprecate Hive 2 in 1.7 and drop Hive 2 in 1.8. What do you think?
>>
>> [1] https://hive.apache.org/general/downloads/
>> [2] https://github.com/apache/iceberg/pull/10932
>> [3] https://github.com/apache/iceberg/pull/10996
>>
>> Regards,
>> Manu
>>
>
Re: [DISCUSS] Drop Hive 2 support
Posted by Anton Okolnychyi <ao...@gmail.com>.
Are we aware of any engines that can be affected by this change?
пн, 26 серп. 2024 р. о 17:34 Manu Zhang <ow...@gmail.com> пише:
> Hi all,
>
> I'd like to start a discussion on dropping Hive 2 support, which reached
> EOL three months ago[1]. It's also a prerequisite for migration to Hadoop
> 3, as shown by Steve's PR[2]. For your reference, I have a draft PR[3] to
> show the needed changes.
>
> Since we've not deprecated Hive 2 support yet, I think the path is to
> deprecate Hive 2 in 1.7 and drop Hive 2 in 1.8. What do you think?
>
> [1] https://hive.apache.org/general/downloads/
> [2] https://github.com/apache/iceberg/pull/10932
> [3] https://github.com/apache/iceberg/pull/10996
>
> Regards,
> Manu
>
Re: [DISCUSS] Drop Hive 2 support
Posted by Daniel Weeks <dw...@apache.org>.
+1
On Mon, Sep 9, 2024 at 4:24 PM Steve Zhang <ho...@apple.com.invalid>
wrote:
> +1
>
> Thanks,
> Steve Zhang
>
>
>
> On Sep 9, 2024, at 11:45 AM, Russell Spitzer <ru...@gmail.com>
> wrote:
>
> +1
>
> On Mon, Sep 9, 2024 at 7:59 AM Eduard Tudenhöfner <
> etudenhoefner@apache.org> wrote:
>
>> +1 on deprecating Hive 2 in Iceberg 1.7 and removing it in 1.8
>>
>> On Mon, Sep 2, 2024 at 8:53 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> It sounds good.
>>>
>>> Thanks !
>>> Regards
>>> JB
>>>
>>> On Mon, Sep 2, 2024 at 5:06 AM Manu Zhang <ow...@gmail.com>
>>> wrote:
>>> >
>>> > Thanks JB. I think the last Hive 2 support will be Iceberg 1.7.x
>>> although we don't recommend users to use a deprecated version.
>>> >
>>> > Regards,
>>> > Manu
>>> >
>>> > On Fri, Aug 30, 2024 at 8:12 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>> >>
>>> >> Hi Manu
>>> >>
>>> >> Your plan makes sense to me (deprecated Hive 2 in 1.7 and remove in
>>> 1.8).
>>> >> Engines still depending to Hive 2 will have to use Iceberg <= 1.6,
>>> which is OK.
>>> >>
>>> >> Regards
>>> >> JB
>>> >>
>>> >> On Tue, Aug 27, 2024 at 2:34 AM Manu Zhang <ow...@gmail.com>
>>> wrote:
>>> >> >
>>> >> > Hi all,
>>> >> >
>>> >> > I'd like to start a discussion on dropping Hive 2 support, which
>>> reached EOL three months ago[1]. It's also a prerequisite for migration to
>>> Hadoop 3, as shown by Steve's PR[2]. For your reference, I have a draft
>>> PR[3] to show the needed changes.
>>> >> >
>>> >> > Since we've not deprecated Hive 2 support yet, I think the path is
>>> to deprecate Hive 2 in 1.7 and drop Hive 2 in 1.8. What do you think?
>>> >> >
>>> >> > [1] https://hive.apache.org/general/downloads/
>>> >> > [2] https://github.com/apache/iceberg/pull/10932
>>> >> > [3] https://github.com/apache/iceberg/pull/10996
>>> >> >
>>> >> > Regards,
>>> >> > Manu
>>>
>>
>
Re: [DISCUSS] Drop Hive 2 support
Posted by Steve Zhang <ho...@apple.com.INVALID>.
+1
Thanks,
Steve Zhang
> On Sep 9, 2024, at 11:45 AM, Russell Spitzer <ru...@gmail.com> wrote:
>
> +1
>
> On Mon, Sep 9, 2024 at 7:59 AM Eduard Tudenhöfner <etudenhoefner@apache.org <ma...@apache.org>> wrote:
>> +1 on deprecating Hive 2 in Iceberg 1.7 and removing it in 1.8
>>
>> On Mon, Sep 2, 2024 at 8:53 AM Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>> wrote:
>>> It sounds good.
>>>
>>> Thanks !
>>> Regards
>>> JB
>>>
>>> On Mon, Sep 2, 2024 at 5:06 AM Manu Zhang <owenzhang1990@gmail.com <ma...@gmail.com>> wrote:
>>> >
>>> > Thanks JB. I think the last Hive 2 support will be Iceberg 1.7.x although we don't recommend users to use a deprecated version.
>>> >
>>> > Regards,
>>> > Manu
>>> >
>>> > On Fri, Aug 30, 2024 at 8:12 PM Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>> wrote:
>>> >>
>>> >> Hi Manu
>>> >>
>>> >> Your plan makes sense to me (deprecated Hive 2 in 1.7 and remove in 1.8).
>>> >> Engines still depending to Hive 2 will have to use Iceberg <= 1.6, which is OK.
>>> >>
>>> >> Regards
>>> >> JB
>>> >>
>>> >> On Tue, Aug 27, 2024 at 2:34 AM Manu Zhang <owenzhang1990@gmail.com <ma...@gmail.com>> wrote:
>>> >> >
>>> >> > Hi all,
>>> >> >
>>> >> > I'd like to start a discussion on dropping Hive 2 support, which reached EOL three months ago[1]. It's also a prerequisite for migration to Hadoop 3, as shown by Steve's PR[2]. For your reference, I have a draft PR[3] to show the needed changes.
>>> >> >
>>> >> > Since we've not deprecated Hive 2 support yet, I think the path is to deprecate Hive 2 in 1.7 and drop Hive 2 in 1.8. What do you think?
>>> >> >
>>> >> > [1] https://hive.apache.org/general/downloads/
>>> >> > [2] https://github.com/apache/iceberg/pull/10932
>>> >> > [3] https://github.com/apache/iceberg/pull/10996
>>> >> >
>>> >> > Regards,
>>> >> > Manu
Re: [DISCUSS] Drop Hive 2 support
Posted by Russell Spitzer <ru...@gmail.com>.
+1
On Mon, Sep 9, 2024 at 7:59 AM Eduard Tudenhöfner <et...@apache.org>
wrote:
> +1 on deprecating Hive 2 in Iceberg 1.7 and removing it in 1.8
>
> On Mon, Sep 2, 2024 at 8:53 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> It sounds good.
>>
>> Thanks !
>> Regards
>> JB
>>
>> On Mon, Sep 2, 2024 at 5:06 AM Manu Zhang <ow...@gmail.com>
>> wrote:
>> >
>> > Thanks JB. I think the last Hive 2 support will be Iceberg 1.7.x
>> although we don't recommend users to use a deprecated version.
>> >
>> > Regards,
>> > Manu
>> >
>> > On Fri, Aug 30, 2024 at 8:12 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>> >>
>> >> Hi Manu
>> >>
>> >> Your plan makes sense to me (deprecated Hive 2 in 1.7 and remove in
>> 1.8).
>> >> Engines still depending to Hive 2 will have to use Iceberg <= 1.6,
>> which is OK.
>> >>
>> >> Regards
>> >> JB
>> >>
>> >> On Tue, Aug 27, 2024 at 2:34 AM Manu Zhang <ow...@gmail.com>
>> wrote:
>> >> >
>> >> > Hi all,
>> >> >
>> >> > I'd like to start a discussion on dropping Hive 2 support, which
>> reached EOL three months ago[1]. It's also a prerequisite for migration to
>> Hadoop 3, as shown by Steve's PR[2]. For your reference, I have a draft
>> PR[3] to show the needed changes.
>> >> >
>> >> > Since we've not deprecated Hive 2 support yet, I think the path is
>> to deprecate Hive 2 in 1.7 and drop Hive 2 in 1.8. What do you think?
>> >> >
>> >> > [1] https://hive.apache.org/general/downloads/
>> >> > [2] https://github.com/apache/iceberg/pull/10932
>> >> > [3] https://github.com/apache/iceberg/pull/10996
>> >> >
>> >> > Regards,
>> >> > Manu
>>
>
Re: [DISCUSS] Drop Hive 2 support
Posted by Eduard Tudenhöfner <et...@apache.org>.
+1 on deprecating Hive 2 in Iceberg 1.7 and removing it in 1.8
On Mon, Sep 2, 2024 at 8:53 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> It sounds good.
>
> Thanks !
> Regards
> JB
>
> On Mon, Sep 2, 2024 at 5:06 AM Manu Zhang <ow...@gmail.com> wrote:
> >
> > Thanks JB. I think the last Hive 2 support will be Iceberg 1.7.x
> although we don't recommend users to use a deprecated version.
> >
> > Regards,
> > Manu
> >
> > On Fri, Aug 30, 2024 at 8:12 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
> >>
> >> Hi Manu
> >>
> >> Your plan makes sense to me (deprecated Hive 2 in 1.7 and remove in
> 1.8).
> >> Engines still depending to Hive 2 will have to use Iceberg <= 1.6,
> which is OK.
> >>
> >> Regards
> >> JB
> >>
> >> On Tue, Aug 27, 2024 at 2:34 AM Manu Zhang <ow...@gmail.com>
> wrote:
> >> >
> >> > Hi all,
> >> >
> >> > I'd like to start a discussion on dropping Hive 2 support, which
> reached EOL three months ago[1]. It's also a prerequisite for migration to
> Hadoop 3, as shown by Steve's PR[2]. For your reference, I have a draft
> PR[3] to show the needed changes.
> >> >
> >> > Since we've not deprecated Hive 2 support yet, I think the path is to
> deprecate Hive 2 in 1.7 and drop Hive 2 in 1.8. What do you think?
> >> >
> >> > [1] https://hive.apache.org/general/downloads/
> >> > [2] https://github.com/apache/iceberg/pull/10932
> >> > [3] https://github.com/apache/iceberg/pull/10996
> >> >
> >> > Regards,
> >> > Manu
>
Re: [DISCUSS] Drop Hive 2 support
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
It sounds good.
Thanks !
Regards
JB
On Mon, Sep 2, 2024 at 5:06 AM Manu Zhang <ow...@gmail.com> wrote:
>
> Thanks JB. I think the last Hive 2 support will be Iceberg 1.7.x although we don't recommend users to use a deprecated version.
>
> Regards,
> Manu
>
> On Fri, Aug 30, 2024 at 8:12 PM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>>
>> Hi Manu
>>
>> Your plan makes sense to me (deprecated Hive 2 in 1.7 and remove in 1.8).
>> Engines still depending to Hive 2 will have to use Iceberg <= 1.6, which is OK.
>>
>> Regards
>> JB
>>
>> On Tue, Aug 27, 2024 at 2:34 AM Manu Zhang <ow...@gmail.com> wrote:
>> >
>> > Hi all,
>> >
>> > I'd like to start a discussion on dropping Hive 2 support, which reached EOL three months ago[1]. It's also a prerequisite for migration to Hadoop 3, as shown by Steve's PR[2]. For your reference, I have a draft PR[3] to show the needed changes.
>> >
>> > Since we've not deprecated Hive 2 support yet, I think the path is to deprecate Hive 2 in 1.7 and drop Hive 2 in 1.8. What do you think?
>> >
>> > [1] https://hive.apache.org/general/downloads/
>> > [2] https://github.com/apache/iceberg/pull/10932
>> > [3] https://github.com/apache/iceberg/pull/10996
>> >
>> > Regards,
>> > Manu
Re: [DISCUSS] Drop Hive 2 support
Posted by Manu Zhang <ow...@gmail.com>.
Thanks JB. I think the last Hive 2 support will be Iceberg 1.7.x although
we don't recommend users to use a deprecated version.
Regards,
Manu
On Fri, Aug 30, 2024 at 8:12 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi Manu
>
> Your plan makes sense to me (deprecated Hive 2 in 1.7 and remove in 1.8).
> Engines still depending to Hive 2 will have to use Iceberg <= 1.6, which
> is OK.
>
> Regards
> JB
>
> On Tue, Aug 27, 2024 at 2:34 AM Manu Zhang <ow...@gmail.com>
> wrote:
> >
> > Hi all,
> >
> > I'd like to start a discussion on dropping Hive 2 support, which reached
> EOL three months ago[1]. It's also a prerequisite for migration to Hadoop
> 3, as shown by Steve's PR[2]. For your reference, I have a draft PR[3] to
> show the needed changes.
> >
> > Since we've not deprecated Hive 2 support yet, I think the path is to
> deprecate Hive 2 in 1.7 and drop Hive 2 in 1.8. What do you think?
> >
> > [1] https://hive.apache.org/general/downloads/
> > [2] https://github.com/apache/iceberg/pull/10932
> > [3] https://github.com/apache/iceberg/pull/10996
> >
> > Regards,
> > Manu
>
Re: [DISCUSS] Drop Hive 2 support
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Manu
Your plan makes sense to me (deprecated Hive 2 in 1.7 and remove in 1.8).
Engines still depending to Hive 2 will have to use Iceberg <= 1.6, which is OK.
Regards
JB
On Tue, Aug 27, 2024 at 2:34 AM Manu Zhang <ow...@gmail.com> wrote:
>
> Hi all,
>
> I'd like to start a discussion on dropping Hive 2 support, which reached EOL three months ago[1]. It's also a prerequisite for migration to Hadoop 3, as shown by Steve's PR[2]. For your reference, I have a draft PR[3] to show the needed changes.
>
> Since we've not deprecated Hive 2 support yet, I think the path is to deprecate Hive 2 in 1.7 and drop Hive 2 in 1.8. What do you think?
>
> [1] https://hive.apache.org/general/downloads/
> [2] https://github.com/apache/iceberg/pull/10932
> [3] https://github.com/apache/iceberg/pull/10996
>
> Regards,
> Manu