You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iceberg.apache.org by Ajantha Bhat <aj...@gmail.com> on 2023/12/13 18:10:45 UTC
[Discuss] Spark 3.2 support?
Hi All,
In our recent discussion, we deprecated Spark 3.2 support in Iceberg 1.4.0
release
as outlined in this thread:
https://lists.apache.org/thread/zw1blng2d1bbrlcftxwmmhb2l7jxbxqx
As we're gearing up for the 1.5.0 release, I suggest that we go ahead and
completely remove the Spark 3.2 support with Iceberg 1.5.0.
Maintaining the deprecated folder has become burdensome, especially when
making changes in the core (used with Spark),
as it necessitates fixing all Spark versions for compilability. With this
in mind, removing Spark 3.2 support in the upcoming 1.5.0 release seems
timely.
It's important to note that users currently relying on Spark 3.2 can still
utilize older Iceberg versions, as discussed previously.
I will open a PR to remove the Spark 3.2 folder tomorrow.
Let me know your thoughts on this.
Thanks,
Ajantha
Re: [Discuss] Spark 3.2 support?
Posted by John Zhuge <jz...@apache.org>.
+1
John Zhuge
On Thu, Dec 14, 2023 at 4:03 PM Jack Ye <ye...@gmail.com> wrote:
> +1
>
> -Jack
>
> On Wed, Dec 13, 2023 at 11:25 PM Eduard Tudenhoefner <ed...@tabular.io>
> wrote:
>
>> +1 on removing Spark 3.2
>>
>> On Wed, Dec 13, 2023 at 8:01 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> +1
>>>
>>> Regards
>>> JB
>>>
>>> On Wed, Dec 13, 2023 at 7:10 PM Ajantha Bhat <aj...@gmail.com>
>>> wrote:
>>> >
>>> > Hi All,
>>> >
>>> > In our recent discussion, we deprecated Spark 3.2 support in Iceberg
>>> 1.4.0 release
>>> > as outlined in this thread:
>>> https://lists.apache.org/thread/zw1blng2d1bbrlcftxwmmhb2l7jxbxqx
>>> >
>>> > As we're gearing up for the 1.5.0 release, I suggest that we go ahead
>>> and completely remove the Spark 3.2 support with Iceberg 1.5.0.
>>> > Maintaining the deprecated folder has become burdensome, especially
>>> when making changes in the core (used with Spark),
>>> > as it necessitates fixing all Spark versions for compilability. With
>>> this in mind, removing Spark 3.2 support in the upcoming 1.5.0 release
>>> seems timely.
>>> >
>>> > It's important to note that users currently relying on Spark 3.2 can
>>> still utilize older Iceberg versions, as discussed previously.
>>> >
>>> > I will open a PR to remove the Spark 3.2 folder tomorrow.
>>> >
>>> > Let me know your thoughts on this.
>>> >
>>> > Thanks,
>>> > Ajantha
>>>
>>
Re: [Discuss] Spark 3.2 support?
Posted by Jack Ye <ye...@gmail.com>.
+1
-Jack
On Wed, Dec 13, 2023 at 11:25 PM Eduard Tudenhoefner <ed...@tabular.io>
wrote:
> +1 on removing Spark 3.2
>
> On Wed, Dec 13, 2023 at 8:01 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> +1
>>
>> Regards
>> JB
>>
>> On Wed, Dec 13, 2023 at 7:10 PM Ajantha Bhat <aj...@gmail.com>
>> wrote:
>> >
>> > Hi All,
>> >
>> > In our recent discussion, we deprecated Spark 3.2 support in Iceberg
>> 1.4.0 release
>> > as outlined in this thread:
>> https://lists.apache.org/thread/zw1blng2d1bbrlcftxwmmhb2l7jxbxqx
>> >
>> > As we're gearing up for the 1.5.0 release, I suggest that we go ahead
>> and completely remove the Spark 3.2 support with Iceberg 1.5.0.
>> > Maintaining the deprecated folder has become burdensome, especially
>> when making changes in the core (used with Spark),
>> > as it necessitates fixing all Spark versions for compilability. With
>> this in mind, removing Spark 3.2 support in the upcoming 1.5.0 release
>> seems timely.
>> >
>> > It's important to note that users currently relying on Spark 3.2 can
>> still utilize older Iceberg versions, as discussed previously.
>> >
>> > I will open a PR to remove the Spark 3.2 folder tomorrow.
>> >
>> > Let me know your thoughts on this.
>> >
>> > Thanks,
>> > Ajantha
>>
>
Re: [Discuss] Spark 3.2 support?
Posted by Eduard Tudenhoefner <ed...@tabular.io>.
+1 on removing Spark 3.2
On Wed, Dec 13, 2023 at 8:01 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> +1
>
> Regards
> JB
>
> On Wed, Dec 13, 2023 at 7:10 PM Ajantha Bhat <aj...@gmail.com>
> wrote:
> >
> > Hi All,
> >
> > In our recent discussion, we deprecated Spark 3.2 support in Iceberg
> 1.4.0 release
> > as outlined in this thread:
> https://lists.apache.org/thread/zw1blng2d1bbrlcftxwmmhb2l7jxbxqx
> >
> > As we're gearing up for the 1.5.0 release, I suggest that we go ahead
> and completely remove the Spark 3.2 support with Iceberg 1.5.0.
> > Maintaining the deprecated folder has become burdensome, especially when
> making changes in the core (used with Spark),
> > as it necessitates fixing all Spark versions for compilability. With
> this in mind, removing Spark 3.2 support in the upcoming 1.5.0 release
> seems timely.
> >
> > It's important to note that users currently relying on Spark 3.2 can
> still utilize older Iceberg versions, as discussed previously.
> >
> > I will open a PR to remove the Spark 3.2 folder tomorrow.
> >
> > Let me know your thoughts on this.
> >
> > Thanks,
> > Ajantha
>
Re: [Discuss] Spark 3.2 support?
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
+1
Regards
JB
On Wed, Dec 13, 2023 at 7:10 PM Ajantha Bhat <aj...@gmail.com> wrote:
>
> Hi All,
>
> In our recent discussion, we deprecated Spark 3.2 support in Iceberg 1.4.0 release
> as outlined in this thread: https://lists.apache.org/thread/zw1blng2d1bbrlcftxwmmhb2l7jxbxqx
>
> As we're gearing up for the 1.5.0 release, I suggest that we go ahead and completely remove the Spark 3.2 support with Iceberg 1.5.0.
> Maintaining the deprecated folder has become burdensome, especially when making changes in the core (used with Spark),
> as it necessitates fixing all Spark versions for compilability. With this in mind, removing Spark 3.2 support in the upcoming 1.5.0 release seems timely.
>
> It's important to note that users currently relying on Spark 3.2 can still utilize older Iceberg versions, as discussed previously.
>
> I will open a PR to remove the Spark 3.2 folder tomorrow.
>
> Let me know your thoughts on this.
>
> Thanks,
> Ajantha