You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iceberg.apache.org by Aihua Xu <ai...@gmail.com> on 2026/02/02 23:19:10 UTC
Re: Spark 4.1 support
Hi Anton,
I want to keep track of this work item for the Iceberg 1.11.0 release. Can
you share the status for the work?
Thanks,
Aihua
On Mon, Jan 26, 2026 at 2:55 PM Romain Manni-Bucau <rm...@gmail.com>
wrote:
> Hi Anton,
>
> Didn't want to say the work done was 1h, I literally meant the fix to make
> it work is 1h of work.
> I'm not sure I get your opportunity point since deprecating/reworking
> fundamentally something might need a new minor of iceberg for end users
> IMHO.
>
> That said happy to test a PR on my use cases (mainly maintenance and SQL
> as of today) once you think it is worth it too.
>
> Romain Manni-Bucau
> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
> <https://dotnetbirdie.github.io/> | Blog <https://rmannibucau.github.io/> |
> Old Blog <http://rmannibucau.wordpress.com> | Github
> <https://github.com/rmannibucau> | LinkedIn
> <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
> Javaccino founder (Java/.NET service - contact via linkedin)
>
>
> Le lun. 26 janv. 2026 à 23:27, Anton Okolnychyi <ao...@gmail.com> a
> écrit :
>
>> What may seem like 1 hr of work is actually quite complicated. I am sure
>> Manu dedicated a lot of time in setting the new module and validating all
>> failed tests, not to mention fixes that were made after testing Spark RCs.
>>
>> Anyway, a new Spark release is always a rare opportunity to
>> deprecate/improve something fundamentally. In Spark 4.1, one of these
>> things is the new version framework in DSv2 that requires us to rethink a
>> large chunk of the connector. We shouldn't rush this. We will see
>> pre-requisite PRs this week. Help with reviews is always appreciated.
>>
>> сб, 17 січ. 2026 р. о 01:02 Romain Manni-Bucau <rm...@gmail.com>
>> пише:
>>
>>> Hi JB,
>>>
>>> As said it is 100% an user standpoint and since enbling it is literally
>>> 1h I think it is worth if next week is missed but still happy to avoir.
>>>
>>>
>>> Romain Manni-Bucau
>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>> <https://dotnetbirdie.github.io/> | Blog
>>> <https://rmannibucau.github.io/> | Old Blog
>>> <http://rmannibucau.wordpress.com> | Github
>>> <https://github.com/rmannibucau> | LinkedIn
>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>
>>> Le sam. 17 janv. 2026, 06:21, Jean-Baptiste Onofré <jb...@nanthrax.net> a
>>> écrit :
>>>
>>>> Hi Romain
>>>>
>>>> I'm not in favor of 1.10.2 with Spark 4.1 support.
>>>>
>>>> As I think we are pretty close to having the cleanup/enhancement done
>>>> on main, by chance, it will be included in 1.11.0, so worth to wait for it.
>>>>
>>>> Regards
>>>> JB
>>>>
>>>> On Fri, Jan 16, 2026 at 10:06 PM Romain Manni-Bucau <
>>>> rmannibucau@gmail.com> wrote:
>>>>
>>>>> @Steven sure, my point was just that delaying Spark 4.1 support to get
>>>>> new feature can not be the best so branch latest 1.10 tag and just fixing
>>>>> the 2-3 compilation issues to get a 1.10.2 and Spark 4.1 working can be an
>>>>> intermediate step.
>>>>>
>>>>> let's see next week what Anton comes back with :)
>>>>>
>>>>> Romain Manni-Bucau
>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>
>>>>>
>>>>> Le ven. 16 janv. 2026 à 20:28, Steven Wu <st...@gmail.com> a
>>>>> écrit :
>>>>>
>>>>>> Romain, patch releases (like 1.10.2) are only for bug fixes. New
>>>>>> features (like Spark 4.1) would have to happen in minor version releases
>>>>>> (like 1.11.0)
>>>>>>
>>>>>> On Mon, Jan 12, 2026 at 9:22 AM Romain Manni-Bucau <
>>>>>> rmannibucau@gmail.com> wrote:
>>>>>>
>>>>>>> Ok, good news Anton, if it is longer than expected (hoping not)
>>>>>>> maybe a 1.10.2 with the new module be a compromise, impatient to get your
>>>>>>> feedback next week.
>>>>>>>
>>>>>>> Thanks for the hard work.
>>>>>>>
>>>>>>> Romain Manni-Bucau
>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>
>>>>>>>
>>>>>>> Le lun. 12 janv. 2026 à 17:54, Anton Okolnychyi <
>>>>>>> aokolnychyi@gmail.com> a écrit :
>>>>>>>
>>>>>>>> We already have a module for Spark 4.1 in main. That said, Spark
>>>>>>>> 4.1 has introduced substantial improvements in table lifecycle that should
>>>>>>>> let us remove some legacy hacks that may cause data correctness issues.
>>>>>>>> Given that this work may result in some properties being no longer
>>>>>>>> supported, we can do that only in the first Iceberg release for Spark 4.1.
>>>>>>>> I would strongly oppose releasing support for Spark 4.1 prior to embracing
>>>>>>>> the new version framework in DSv2.
>>>>>>>>
>>>>>>>> Good news, it still shouldn't take too long. I am working on a
>>>>>>>> prototype as we speak. It should be available next week.
>>>>>>>>
>>>>>>>> - Anton
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> пн, 12 січ. 2026 р. о 07:33 Romain Manni-Bucau <
>>>>>>>> rmannibucau@gmail.com> пише:
>>>>>>>>
>>>>>>>>> Hi everyone,
>>>>>>>>>
>>>>>>>>> I wonder if you do have some plan to release Iceberg 1.11 (I
>>>>>>>>> assume next one?) with Spark 4.1 support.
>>>>>>>>> The datasource v2 API breakage was very localized but is just
>>>>>>>>> blocking to upgrade and some spark 4.1 features are game changers (like the
>>>>>>>>> SQL enhancement) so I'm really looking forward being able to move on it
>>>>>>>>> with iceberg support.
>>>>>>>>> If you need some help please let me know.
>>>>>>>>>
>>>>>>>>> Romain Manni-Bucau
>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>
>>>>>>>>
Re: Spark 4.1 support
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi Steve,
I'll likely not go with the custom bundling cause it is not just building
your own, it is also repackaging it, generatig custom SBOM and a tons of
broken solution in enterprises so i'll prefer to await for a valid release
to upgrade prod branches.
That said I just validated all IT are passing with the main branch freshly
build so impatient a new release pops out :).
Thanks guys for the hard work.
Romain Manni-Bucau
@rmannibucau <https://x.com/rmannibucau> | .NET Blog
<https://dotnetbirdie.github.io/> | Blog <https://rmannibucau.github.io/> | Old
Blog <http://rmannibucau.wordpress.com> | Github
<https://github.com/rmannibucau> | LinkedIn
<https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
Javaccino founder (Java/.NET service - contact via linkedin)
Le ven. 13 mars 2026 à 23:02, Steve Loughran <st...@cloudera.com> a écrit :
>
> Romain,
>
> Downloading and building your own release is now the best way to get this
> update -and it's the way to test that it works before the release ships. If
> there's a problem: it'll get fixed almost immediately
>
> -Steve
>
> On Fri, 13 Mar 2026 at 12:12, Romain Manni-Bucau <rm...@gmail.com>
> wrote:
>
>> Aweomse Anton, thanks a lot for the feedback.
>>
>> Romain Manni-Bucau
>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>> <https://dotnetbirdie.github.io/> | Blog <https://rmannibucau.github.io/> |
>> Old Blog <http://rmannibucau.wordpress.com> | Github
>> <https://github.com/rmannibucau> | LinkedIn
>> <https://www.linkedin.com/in/rmannibucau> | Book
>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>> Javaccino founder (Java/.NET service - contact via linkedin)
>>
>>
>> Le ven. 13 mars 2026 à 13:08, Anton Okolnychyi <ao...@gmail.com> a
>> écrit :
>>
>>> All Spark 4.1 related work that I discussed earlier has been merged and
>>> is awaiting a release candidate
>>>
>>> On Fri, Mar 13, 2026 at 11:48 AM Romain Manni-Bucau <
>>> rmannibucau@gmail.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Wonder if we made some progress or if we should fall back on the spark
>>>> 3 like compatibility without leveraging new feature just to enable the
>>>> community to upgrade Apache Spark?
>>>>
>>>> Thanks in advance,
>>>>
>>>> Romain Manni-Bucau
>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>> <https://dotnetbirdie.github.io/> | Blog
>>>> <https://rmannibucau.github.io/> | Old Blog
>>>> <http://rmannibucau.wordpress.com> | Github
>>>> <https://github.com/rmannibucau> | LinkedIn
>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>
>>>>
>>>> Le mer. 4 févr. 2026 à 19:59, Anton Okolnychyi <ao...@gmail.com>
>>>> a écrit :
>>>>
>>>>> Will share the WIP PR for the new version framework by the end of
>>>>> today. I will have to then split it into manageable pieces and iterate on
>>>>> still failing tests.
>>>>>
>>>>> пн, 2 лют. 2026 р. о 15:20 Aihua Xu <ai...@gmail.com> пише:
>>>>>
>>>>>> Hi Anton,
>>>>>>
>>>>>> I want to keep track of this work item for the Iceberg 1.11.0
>>>>>> release. Can you share the status for the work?
>>>>>>
>>>>>> Thanks,
>>>>>> Aihua
>>>>>>
>>>>>> On Mon, Jan 26, 2026 at 2:55 PM Romain Manni-Bucau <
>>>>>> rmannibucau@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Anton,
>>>>>>>
>>>>>>> Didn't want to say the work done was 1h, I literally meant the fix
>>>>>>> to make it work is 1h of work.
>>>>>>> I'm not sure I get your opportunity point since
>>>>>>> deprecating/reworking fundamentally something might need a new minor of
>>>>>>> iceberg for end users IMHO.
>>>>>>>
>>>>>>> That said happy to test a PR on my use cases (mainly maintenance and
>>>>>>> SQL as of today) once you think it is worth it too.
>>>>>>>
>>>>>>> Romain Manni-Bucau
>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>
>>>>>>>
>>>>>>> Le lun. 26 janv. 2026 à 23:27, Anton Okolnychyi <
>>>>>>> aokolnychyi@gmail.com> a écrit :
>>>>>>>
>>>>>>>> What may seem like 1 hr of work is actually quite complicated. I am
>>>>>>>> sure Manu dedicated a lot of time in setting the new module and validating
>>>>>>>> all failed tests, not to mention fixes that were made after testing Spark
>>>>>>>> RCs.
>>>>>>>>
>>>>>>>> Anyway, a new Spark release is always a rare opportunity to
>>>>>>>> deprecate/improve something fundamentally. In Spark 4.1, one of these
>>>>>>>> things is the new version framework in DSv2 that requires us to rethink a
>>>>>>>> large chunk of the connector. We shouldn't rush this. We will see
>>>>>>>> pre-requisite PRs this week. Help with reviews is always appreciated.
>>>>>>>>
>>>>>>>> сб, 17 січ. 2026 р. о 01:02 Romain Manni-Bucau <
>>>>>>>> rmannibucau@gmail.com> пише:
>>>>>>>>
>>>>>>>>> Hi JB,
>>>>>>>>>
>>>>>>>>> As said it is 100% an user standpoint and since enbling it is
>>>>>>>>> literally 1h I think it is worth if next week is missed but still happy to
>>>>>>>>> avoir.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Romain Manni-Bucau
>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>
>>>>>>>>> Le sam. 17 janv. 2026, 06:21, Jean-Baptiste Onofré <
>>>>>>>>> jb@nanthrax.net> a écrit :
>>>>>>>>>
>>>>>>>>>> Hi Romain
>>>>>>>>>>
>>>>>>>>>> I'm not in favor of 1.10.2 with Spark 4.1 support.
>>>>>>>>>>
>>>>>>>>>> As I think we are pretty close to having the cleanup/enhancement
>>>>>>>>>> done on main, by chance, it will be included in 1.11.0, so worth to wait
>>>>>>>>>> for it.
>>>>>>>>>>
>>>>>>>>>> Regards
>>>>>>>>>> JB
>>>>>>>>>>
>>>>>>>>>> On Fri, Jan 16, 2026 at 10:06 PM Romain Manni-Bucau <
>>>>>>>>>> rmannibucau@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> @Steven sure, my point was just that delaying Spark 4.1 support
>>>>>>>>>>> to get new feature can not be the best so branch latest 1.10 tag and just
>>>>>>>>>>> fixing the 2-3 compilation issues to get a 1.10.2 and Spark 4.1 working can
>>>>>>>>>>> be an intermediate step.
>>>>>>>>>>>
>>>>>>>>>>> let's see next week what Anton comes back with :)
>>>>>>>>>>>
>>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Le ven. 16 janv. 2026 à 20:28, Steven Wu <st...@gmail.com>
>>>>>>>>>>> a écrit :
>>>>>>>>>>>
>>>>>>>>>>>> Romain, patch releases (like 1.10.2) are only for bug fixes.
>>>>>>>>>>>> New features (like Spark 4.1) would have to happen in minor version
>>>>>>>>>>>> releases (like 1.11.0)
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jan 12, 2026 at 9:22 AM Romain Manni-Bucau <
>>>>>>>>>>>> rmannibucau@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Ok, good news Anton, if it is longer than expected (hoping
>>>>>>>>>>>>> not) maybe a 1.10.2 with the new module be a compromise, impatient to get
>>>>>>>>>>>>> your feedback next week.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for the hard work.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Le lun. 12 janv. 2026 à 17:54, Anton Okolnychyi <
>>>>>>>>>>>>> aokolnychyi@gmail.com> a écrit :
>>>>>>>>>>>>>
>>>>>>>>>>>>>> We already have a module for Spark 4.1 in main. That said,
>>>>>>>>>>>>>> Spark 4.1 has introduced substantial improvements in table lifecycle that
>>>>>>>>>>>>>> should let us remove some legacy hacks that may cause data correctness
>>>>>>>>>>>>>> issues. Given that this work may result in some properties being no longer
>>>>>>>>>>>>>> supported, we can do that only in the first Iceberg release for Spark 4.1.
>>>>>>>>>>>>>> I would strongly oppose releasing support for Spark 4.1 prior to embracing
>>>>>>>>>>>>>> the new version framework in DSv2.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Good news, it still shouldn't take too long. I am working on
>>>>>>>>>>>>>> a prototype as we speak. It should be available next week.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Anton
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> пн, 12 січ. 2026 р. о 07:33 Romain Manni-Bucau <
>>>>>>>>>>>>>> rmannibucau@gmail.com> пише:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I wonder if you do have some plan to release Iceberg 1.11 (I
>>>>>>>>>>>>>>> assume next one?) with Spark 4.1 support.
>>>>>>>>>>>>>>> The datasource v2 API breakage was very localized but is
>>>>>>>>>>>>>>> just blocking to upgrade and some spark 4.1 features are game changers
>>>>>>>>>>>>>>> (like the SQL enhancement) so I'm really looking forward being able to move
>>>>>>>>>>>>>>> on it with iceberg support.
>>>>>>>>>>>>>>> If you need some help please let me know.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
Re: Spark 4.1 support
Posted by Steve Loughran <st...@cloudera.com>.
Romain,
Downloading and building your own release is now the best way to get this
update -and it's the way to test that it works before the release ships. If
there's a problem: it'll get fixed almost immediately
-Steve
On Fri, 13 Mar 2026 at 12:12, Romain Manni-Bucau <rm...@gmail.com>
wrote:
> Aweomse Anton, thanks a lot for the feedback.
>
> Romain Manni-Bucau
> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
> <https://dotnetbirdie.github.io/> | Blog <https://rmannibucau.github.io/> |
> Old Blog <http://rmannibucau.wordpress.com> | Github
> <https://github.com/rmannibucau> | LinkedIn
> <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
> Javaccino founder (Java/.NET service - contact via linkedin)
>
>
> Le ven. 13 mars 2026 à 13:08, Anton Okolnychyi <ao...@gmail.com> a
> écrit :
>
>> All Spark 4.1 related work that I discussed earlier has been merged and
>> is awaiting a release candidate
>>
>> On Fri, Mar 13, 2026 at 11:48 AM Romain Manni-Bucau <
>> rmannibucau@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> Wonder if we made some progress or if we should fall back on the spark 3
>>> like compatibility without leveraging new feature just to enable the
>>> community to upgrade Apache Spark?
>>>
>>> Thanks in advance,
>>>
>>> Romain Manni-Bucau
>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>> <https://dotnetbirdie.github.io/> | Blog
>>> <https://rmannibucau.github.io/> | Old Blog
>>> <http://rmannibucau.wordpress.com> | Github
>>> <https://github.com/rmannibucau> | LinkedIn
>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>
>>>
>>> Le mer. 4 févr. 2026 à 19:59, Anton Okolnychyi <ao...@gmail.com>
>>> a écrit :
>>>
>>>> Will share the WIP PR for the new version framework by the end of
>>>> today. I will have to then split it into manageable pieces and iterate on
>>>> still failing tests.
>>>>
>>>> пн, 2 лют. 2026 р. о 15:20 Aihua Xu <ai...@gmail.com> пише:
>>>>
>>>>> Hi Anton,
>>>>>
>>>>> I want to keep track of this work item for the Iceberg 1.11.0 release.
>>>>> Can you share the status for the work?
>>>>>
>>>>> Thanks,
>>>>> Aihua
>>>>>
>>>>> On Mon, Jan 26, 2026 at 2:55 PM Romain Manni-Bucau <
>>>>> rmannibucau@gmail.com> wrote:
>>>>>
>>>>>> Hi Anton,
>>>>>>
>>>>>> Didn't want to say the work done was 1h, I literally meant the fix to
>>>>>> make it work is 1h of work.
>>>>>> I'm not sure I get your opportunity point since deprecating/reworking
>>>>>> fundamentally something might need a new minor of iceberg for end users
>>>>>> IMHO.
>>>>>>
>>>>>> That said happy to test a PR on my use cases (mainly maintenance and
>>>>>> SQL as of today) once you think it is worth it too.
>>>>>>
>>>>>> Romain Manni-Bucau
>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>
>>>>>>
>>>>>> Le lun. 26 janv. 2026 à 23:27, Anton Okolnychyi <
>>>>>> aokolnychyi@gmail.com> a écrit :
>>>>>>
>>>>>>> What may seem like 1 hr of work is actually quite complicated. I am
>>>>>>> sure Manu dedicated a lot of time in setting the new module and validating
>>>>>>> all failed tests, not to mention fixes that were made after testing Spark
>>>>>>> RCs.
>>>>>>>
>>>>>>> Anyway, a new Spark release is always a rare opportunity to
>>>>>>> deprecate/improve something fundamentally. In Spark 4.1, one of these
>>>>>>> things is the new version framework in DSv2 that requires us to rethink a
>>>>>>> large chunk of the connector. We shouldn't rush this. We will see
>>>>>>> pre-requisite PRs this week. Help with reviews is always appreciated.
>>>>>>>
>>>>>>> сб, 17 січ. 2026 р. о 01:02 Romain Manni-Bucau <
>>>>>>> rmannibucau@gmail.com> пише:
>>>>>>>
>>>>>>>> Hi JB,
>>>>>>>>
>>>>>>>> As said it is 100% an user standpoint and since enbling it is
>>>>>>>> literally 1h I think it is worth if next week is missed but still happy to
>>>>>>>> avoir.
>>>>>>>>
>>>>>>>>
>>>>>>>> Romain Manni-Bucau
>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>
>>>>>>>> Le sam. 17 janv. 2026, 06:21, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>>>>> a écrit :
>>>>>>>>
>>>>>>>>> Hi Romain
>>>>>>>>>
>>>>>>>>> I'm not in favor of 1.10.2 with Spark 4.1 support.
>>>>>>>>>
>>>>>>>>> As I think we are pretty close to having the cleanup/enhancement
>>>>>>>>> done on main, by chance, it will be included in 1.11.0, so worth to wait
>>>>>>>>> for it.
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>> JB
>>>>>>>>>
>>>>>>>>> On Fri, Jan 16, 2026 at 10:06 PM Romain Manni-Bucau <
>>>>>>>>> rmannibucau@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> @Steven sure, my point was just that delaying Spark 4.1 support
>>>>>>>>>> to get new feature can not be the best so branch latest 1.10 tag and just
>>>>>>>>>> fixing the 2-3 compilation issues to get a 1.10.2 and Spark 4.1 working can
>>>>>>>>>> be an intermediate step.
>>>>>>>>>>
>>>>>>>>>> let's see next week what Anton comes back with :)
>>>>>>>>>>
>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Le ven. 16 janv. 2026 à 20:28, Steven Wu <st...@gmail.com>
>>>>>>>>>> a écrit :
>>>>>>>>>>
>>>>>>>>>>> Romain, patch releases (like 1.10.2) are only for bug fixes. New
>>>>>>>>>>> features (like Spark 4.1) would have to happen in minor version releases
>>>>>>>>>>> (like 1.11.0)
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jan 12, 2026 at 9:22 AM Romain Manni-Bucau <
>>>>>>>>>>> rmannibucau@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Ok, good news Anton, if it is longer than expected (hoping
>>>>>>>>>>>> not) maybe a 1.10.2 with the new module be a compromise, impatient to get
>>>>>>>>>>>> your feedback next week.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the hard work.
>>>>>>>>>>>>
>>>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Le lun. 12 janv. 2026 à 17:54, Anton Okolnychyi <
>>>>>>>>>>>> aokolnychyi@gmail.com> a écrit :
>>>>>>>>>>>>
>>>>>>>>>>>>> We already have a module for Spark 4.1 in main. That said,
>>>>>>>>>>>>> Spark 4.1 has introduced substantial improvements in table lifecycle that
>>>>>>>>>>>>> should let us remove some legacy hacks that may cause data correctness
>>>>>>>>>>>>> issues. Given that this work may result in some properties being no longer
>>>>>>>>>>>>> supported, we can do that only in the first Iceberg release for Spark 4.1.
>>>>>>>>>>>>> I would strongly oppose releasing support for Spark 4.1 prior to embracing
>>>>>>>>>>>>> the new version framework in DSv2.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Good news, it still shouldn't take too long. I am working on a
>>>>>>>>>>>>> prototype as we speak. It should be available next week.
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Anton
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> пн, 12 січ. 2026 р. о 07:33 Romain Manni-Bucau <
>>>>>>>>>>>>> rmannibucau@gmail.com> пише:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I wonder if you do have some plan to release Iceberg 1.11 (I
>>>>>>>>>>>>>> assume next one?) with Spark 4.1 support.
>>>>>>>>>>>>>> The datasource v2 API breakage was very localized but is just
>>>>>>>>>>>>>> blocking to upgrade and some spark 4.1 features are game changers (like the
>>>>>>>>>>>>>> SQL enhancement) so I'm really looking forward being able to move on it
>>>>>>>>>>>>>> with iceberg support.
>>>>>>>>>>>>>> If you need some help please let me know.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
Re: Spark 4.1 support
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Aweomse Anton, thanks a lot for the feedback.
Romain Manni-Bucau
@rmannibucau <https://x.com/rmannibucau> | .NET Blog
<https://dotnetbirdie.github.io/> | Blog <https://rmannibucau.github.io/> | Old
Blog <http://rmannibucau.wordpress.com> | Github
<https://github.com/rmannibucau> | LinkedIn
<https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
Javaccino founder (Java/.NET service - contact via linkedin)
Le ven. 13 mars 2026 à 13:08, Anton Okolnychyi <ao...@gmail.com> a
écrit :
> All Spark 4.1 related work that I discussed earlier has been merged and is
> awaiting a release candidate
>
> On Fri, Mar 13, 2026 at 11:48 AM Romain Manni-Bucau <rm...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> Wonder if we made some progress or if we should fall back on the spark 3
>> like compatibility without leveraging new feature just to enable the
>> community to upgrade Apache Spark?
>>
>> Thanks in advance,
>>
>> Romain Manni-Bucau
>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>> <https://dotnetbirdie.github.io/> | Blog <https://rmannibucau.github.io/> |
>> Old Blog <http://rmannibucau.wordpress.com> | Github
>> <https://github.com/rmannibucau> | LinkedIn
>> <https://www.linkedin.com/in/rmannibucau> | Book
>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>> Javaccino founder (Java/.NET service - contact via linkedin)
>>
>>
>> Le mer. 4 févr. 2026 à 19:59, Anton Okolnychyi <ao...@gmail.com> a
>> écrit :
>>
>>> Will share the WIP PR for the new version framework by the end of today.
>>> I will have to then split it into manageable pieces and iterate on still
>>> failing tests.
>>>
>>> пн, 2 лют. 2026 р. о 15:20 Aihua Xu <ai...@gmail.com> пише:
>>>
>>>> Hi Anton,
>>>>
>>>> I want to keep track of this work item for the Iceberg 1.11.0 release.
>>>> Can you share the status for the work?
>>>>
>>>> Thanks,
>>>> Aihua
>>>>
>>>> On Mon, Jan 26, 2026 at 2:55 PM Romain Manni-Bucau <
>>>> rmannibucau@gmail.com> wrote:
>>>>
>>>>> Hi Anton,
>>>>>
>>>>> Didn't want to say the work done was 1h, I literally meant the fix to
>>>>> make it work is 1h of work.
>>>>> I'm not sure I get your opportunity point since deprecating/reworking
>>>>> fundamentally something might need a new minor of iceberg for end users
>>>>> IMHO.
>>>>>
>>>>> That said happy to test a PR on my use cases (mainly maintenance and
>>>>> SQL as of today) once you think it is worth it too.
>>>>>
>>>>> Romain Manni-Bucau
>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>
>>>>>
>>>>> Le lun. 26 janv. 2026 à 23:27, Anton Okolnychyi <ao...@gmail.com>
>>>>> a écrit :
>>>>>
>>>>>> What may seem like 1 hr of work is actually quite complicated. I am
>>>>>> sure Manu dedicated a lot of time in setting the new module and validating
>>>>>> all failed tests, not to mention fixes that were made after testing Spark
>>>>>> RCs.
>>>>>>
>>>>>> Anyway, a new Spark release is always a rare opportunity to
>>>>>> deprecate/improve something fundamentally. In Spark 4.1, one of these
>>>>>> things is the new version framework in DSv2 that requires us to rethink a
>>>>>> large chunk of the connector. We shouldn't rush this. We will see
>>>>>> pre-requisite PRs this week. Help with reviews is always appreciated.
>>>>>>
>>>>>> сб, 17 січ. 2026 р. о 01:02 Romain Manni-Bucau <rm...@gmail.com>
>>>>>> пише:
>>>>>>
>>>>>>> Hi JB,
>>>>>>>
>>>>>>> As said it is 100% an user standpoint and since enbling it is
>>>>>>> literally 1h I think it is worth if next week is missed but still happy to
>>>>>>> avoir.
>>>>>>>
>>>>>>>
>>>>>>> Romain Manni-Bucau
>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>
>>>>>>> Le sam. 17 janv. 2026, 06:21, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>>>> a écrit :
>>>>>>>
>>>>>>>> Hi Romain
>>>>>>>>
>>>>>>>> I'm not in favor of 1.10.2 with Spark 4.1 support.
>>>>>>>>
>>>>>>>> As I think we are pretty close to having the cleanup/enhancement
>>>>>>>> done on main, by chance, it will be included in 1.11.0, so worth to wait
>>>>>>>> for it.
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> JB
>>>>>>>>
>>>>>>>> On Fri, Jan 16, 2026 at 10:06 PM Romain Manni-Bucau <
>>>>>>>> rmannibucau@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> @Steven sure, my point was just that delaying Spark 4.1 support to
>>>>>>>>> get new feature can not be the best so branch latest 1.10 tag and just
>>>>>>>>> fixing the 2-3 compilation issues to get a 1.10.2 and Spark 4.1 working can
>>>>>>>>> be an intermediate step.
>>>>>>>>>
>>>>>>>>> let's see next week what Anton comes back with :)
>>>>>>>>>
>>>>>>>>> Romain Manni-Bucau
>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Le ven. 16 janv. 2026 à 20:28, Steven Wu <st...@gmail.com> a
>>>>>>>>> écrit :
>>>>>>>>>
>>>>>>>>>> Romain, patch releases (like 1.10.2) are only for bug fixes. New
>>>>>>>>>> features (like Spark 4.1) would have to happen in minor version releases
>>>>>>>>>> (like 1.11.0)
>>>>>>>>>>
>>>>>>>>>> On Mon, Jan 12, 2026 at 9:22 AM Romain Manni-Bucau <
>>>>>>>>>> rmannibucau@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Ok, good news Anton, if it is longer than expected (hoping not)
>>>>>>>>>>> maybe a 1.10.2 with the new module be a compromise, impatient to get your
>>>>>>>>>>> feedback next week.
>>>>>>>>>>>
>>>>>>>>>>> Thanks for the hard work.
>>>>>>>>>>>
>>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Le lun. 12 janv. 2026 à 17:54, Anton Okolnychyi <
>>>>>>>>>>> aokolnychyi@gmail.com> a écrit :
>>>>>>>>>>>
>>>>>>>>>>>> We already have a module for Spark 4.1 in main. That said,
>>>>>>>>>>>> Spark 4.1 has introduced substantial improvements in table lifecycle that
>>>>>>>>>>>> should let us remove some legacy hacks that may cause data correctness
>>>>>>>>>>>> issues. Given that this work may result in some properties being no longer
>>>>>>>>>>>> supported, we can do that only in the first Iceberg release for Spark 4.1.
>>>>>>>>>>>> I would strongly oppose releasing support for Spark 4.1 prior to embracing
>>>>>>>>>>>> the new version framework in DSv2.
>>>>>>>>>>>>
>>>>>>>>>>>> Good news, it still shouldn't take too long. I am working on a
>>>>>>>>>>>> prototype as we speak. It should be available next week.
>>>>>>>>>>>>
>>>>>>>>>>>> - Anton
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> пн, 12 січ. 2026 р. о 07:33 Romain Manni-Bucau <
>>>>>>>>>>>> rmannibucau@gmail.com> пише:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I wonder if you do have some plan to release Iceberg 1.11 (I
>>>>>>>>>>>>> assume next one?) with Spark 4.1 support.
>>>>>>>>>>>>> The datasource v2 API breakage was very localized but is just
>>>>>>>>>>>>> blocking to upgrade and some spark 4.1 features are game changers (like the
>>>>>>>>>>>>> SQL enhancement) so I'm really looking forward being able to move on it
>>>>>>>>>>>>> with iceberg support.
>>>>>>>>>>>>> If you need some help please let me know.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>>>>>
>>>>>>>>>>>>
Re: Spark 4.1 support
Posted by Anton Okolnychyi <ao...@gmail.com>.
All Spark 4.1 related work that I discussed earlier has been merged and is
awaiting a release candidate
On Fri, Mar 13, 2026 at 11:48 AM Romain Manni-Bucau <rm...@gmail.com>
wrote:
> Hi all,
>
> Wonder if we made some progress or if we should fall back on the spark 3
> like compatibility without leveraging new feature just to enable the
> community to upgrade Apache Spark?
>
> Thanks in advance,
>
> Romain Manni-Bucau
> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
> <https://dotnetbirdie.github.io/> | Blog <https://rmannibucau.github.io/> |
> Old Blog <http://rmannibucau.wordpress.com> | Github
> <https://github.com/rmannibucau> | LinkedIn
> <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
> Javaccino founder (Java/.NET service - contact via linkedin)
>
>
> Le mer. 4 févr. 2026 à 19:59, Anton Okolnychyi <ao...@gmail.com> a
> écrit :
>
>> Will share the WIP PR for the new version framework by the end of today.
>> I will have to then split it into manageable pieces and iterate on still
>> failing tests.
>>
>> пн, 2 лют. 2026 р. о 15:20 Aihua Xu <ai...@gmail.com> пише:
>>
>>> Hi Anton,
>>>
>>> I want to keep track of this work item for the Iceberg 1.11.0 release.
>>> Can you share the status for the work?
>>>
>>> Thanks,
>>> Aihua
>>>
>>> On Mon, Jan 26, 2026 at 2:55 PM Romain Manni-Bucau <
>>> rmannibucau@gmail.com> wrote:
>>>
>>>> Hi Anton,
>>>>
>>>> Didn't want to say the work done was 1h, I literally meant the fix to
>>>> make it work is 1h of work.
>>>> I'm not sure I get your opportunity point since deprecating/reworking
>>>> fundamentally something might need a new minor of iceberg for end users
>>>> IMHO.
>>>>
>>>> That said happy to test a PR on my use cases (mainly maintenance and
>>>> SQL as of today) once you think it is worth it too.
>>>>
>>>> Romain Manni-Bucau
>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>> <https://dotnetbirdie.github.io/> | Blog
>>>> <https://rmannibucau.github.io/> | Old Blog
>>>> <http://rmannibucau.wordpress.com> | Github
>>>> <https://github.com/rmannibucau> | LinkedIn
>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>
>>>>
>>>> Le lun. 26 janv. 2026 à 23:27, Anton Okolnychyi <ao...@gmail.com>
>>>> a écrit :
>>>>
>>>>> What may seem like 1 hr of work is actually quite complicated. I am
>>>>> sure Manu dedicated a lot of time in setting the new module and validating
>>>>> all failed tests, not to mention fixes that were made after testing Spark
>>>>> RCs.
>>>>>
>>>>> Anyway, a new Spark release is always a rare opportunity to
>>>>> deprecate/improve something fundamentally. In Spark 4.1, one of these
>>>>> things is the new version framework in DSv2 that requires us to rethink a
>>>>> large chunk of the connector. We shouldn't rush this. We will see
>>>>> pre-requisite PRs this week. Help with reviews is always appreciated.
>>>>>
>>>>> сб, 17 січ. 2026 р. о 01:02 Romain Manni-Bucau <rm...@gmail.com>
>>>>> пише:
>>>>>
>>>>>> Hi JB,
>>>>>>
>>>>>> As said it is 100% an user standpoint and since enbling it is
>>>>>> literally 1h I think it is worth if next week is missed but still happy to
>>>>>> avoir.
>>>>>>
>>>>>>
>>>>>> Romain Manni-Bucau
>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>
>>>>>> Le sam. 17 janv. 2026, 06:21, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>>> a écrit :
>>>>>>
>>>>>>> Hi Romain
>>>>>>>
>>>>>>> I'm not in favor of 1.10.2 with Spark 4.1 support.
>>>>>>>
>>>>>>> As I think we are pretty close to having the cleanup/enhancement
>>>>>>> done on main, by chance, it will be included in 1.11.0, so worth to wait
>>>>>>> for it.
>>>>>>>
>>>>>>> Regards
>>>>>>> JB
>>>>>>>
>>>>>>> On Fri, Jan 16, 2026 at 10:06 PM Romain Manni-Bucau <
>>>>>>> rmannibucau@gmail.com> wrote:
>>>>>>>
>>>>>>>> @Steven sure, my point was just that delaying Spark 4.1 support to
>>>>>>>> get new feature can not be the best so branch latest 1.10 tag and just
>>>>>>>> fixing the 2-3 compilation issues to get a 1.10.2 and Spark 4.1 working can
>>>>>>>> be an intermediate step.
>>>>>>>>
>>>>>>>> let's see next week what Anton comes back with :)
>>>>>>>>
>>>>>>>> Romain Manni-Bucau
>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>
>>>>>>>>
>>>>>>>> Le ven. 16 janv. 2026 à 20:28, Steven Wu <st...@gmail.com> a
>>>>>>>> écrit :
>>>>>>>>
>>>>>>>>> Romain, patch releases (like 1.10.2) are only for bug fixes. New
>>>>>>>>> features (like Spark 4.1) would have to happen in minor version releases
>>>>>>>>> (like 1.11.0)
>>>>>>>>>
>>>>>>>>> On Mon, Jan 12, 2026 at 9:22 AM Romain Manni-Bucau <
>>>>>>>>> rmannibucau@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Ok, good news Anton, if it is longer than expected (hoping not)
>>>>>>>>>> maybe a 1.10.2 with the new module be a compromise, impatient to get your
>>>>>>>>>> feedback next week.
>>>>>>>>>>
>>>>>>>>>> Thanks for the hard work.
>>>>>>>>>>
>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Le lun. 12 janv. 2026 à 17:54, Anton Okolnychyi <
>>>>>>>>>> aokolnychyi@gmail.com> a écrit :
>>>>>>>>>>
>>>>>>>>>>> We already have a module for Spark 4.1 in main. That said, Spark
>>>>>>>>>>> 4.1 has introduced substantial improvements in table lifecycle that should
>>>>>>>>>>> let us remove some legacy hacks that may cause data correctness issues.
>>>>>>>>>>> Given that this work may result in some properties being no longer
>>>>>>>>>>> supported, we can do that only in the first Iceberg release for Spark 4.1.
>>>>>>>>>>> I would strongly oppose releasing support for Spark 4.1 prior to embracing
>>>>>>>>>>> the new version framework in DSv2.
>>>>>>>>>>>
>>>>>>>>>>> Good news, it still shouldn't take too long. I am working on a
>>>>>>>>>>> prototype as we speak. It should be available next week.
>>>>>>>>>>>
>>>>>>>>>>> - Anton
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> пн, 12 січ. 2026 р. о 07:33 Romain Manni-Bucau <
>>>>>>>>>>> rmannibucau@gmail.com> пише:
>>>>>>>>>>>
>>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>>
>>>>>>>>>>>> I wonder if you do have some plan to release Iceberg 1.11 (I
>>>>>>>>>>>> assume next one?) with Spark 4.1 support.
>>>>>>>>>>>> The datasource v2 API breakage was very localized but is just
>>>>>>>>>>>> blocking to upgrade and some spark 4.1 features are game changers (like the
>>>>>>>>>>>> SQL enhancement) so I'm really looking forward being able to move on it
>>>>>>>>>>>> with iceberg support.
>>>>>>>>>>>> If you need some help please let me know.
>>>>>>>>>>>>
>>>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>>>>
>>>>>>>>>>>
Re: Spark 4.1 support
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi all,
Wonder if we made some progress or if we should fall back on the spark 3
like compatibility without leveraging new feature just to enable the
community to upgrade Apache Spark?
Thanks in advance,
Romain Manni-Bucau
@rmannibucau <https://x.com/rmannibucau> | .NET Blog
<https://dotnetbirdie.github.io/> | Blog <https://rmannibucau.github.io/> | Old
Blog <http://rmannibucau.wordpress.com> | Github
<https://github.com/rmannibucau> | LinkedIn
<https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
Javaccino founder (Java/.NET service - contact via linkedin)
Le mer. 4 févr. 2026 à 19:59, Anton Okolnychyi <ao...@gmail.com> a
écrit :
> Will share the WIP PR for the new version framework by the end of today. I
> will have to then split it into manageable pieces and iterate on still
> failing tests.
>
> пн, 2 лют. 2026 р. о 15:20 Aihua Xu <ai...@gmail.com> пише:
>
>> Hi Anton,
>>
>> I want to keep track of this work item for the Iceberg 1.11.0 release.
>> Can you share the status for the work?
>>
>> Thanks,
>> Aihua
>>
>> On Mon, Jan 26, 2026 at 2:55 PM Romain Manni-Bucau <rm...@gmail.com>
>> wrote:
>>
>>> Hi Anton,
>>>
>>> Didn't want to say the work done was 1h, I literally meant the fix to
>>> make it work is 1h of work.
>>> I'm not sure I get your opportunity point since deprecating/reworking
>>> fundamentally something might need a new minor of iceberg for end users
>>> IMHO.
>>>
>>> That said happy to test a PR on my use cases (mainly maintenance and SQL
>>> as of today) once you think it is worth it too.
>>>
>>> Romain Manni-Bucau
>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>> <https://dotnetbirdie.github.io/> | Blog
>>> <https://rmannibucau.github.io/> | Old Blog
>>> <http://rmannibucau.wordpress.com> | Github
>>> <https://github.com/rmannibucau> | LinkedIn
>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>
>>>
>>> Le lun. 26 janv. 2026 à 23:27, Anton Okolnychyi <ao...@gmail.com>
>>> a écrit :
>>>
>>>> What may seem like 1 hr of work is actually quite complicated. I am
>>>> sure Manu dedicated a lot of time in setting the new module and validating
>>>> all failed tests, not to mention fixes that were made after testing Spark
>>>> RCs.
>>>>
>>>> Anyway, a new Spark release is always a rare opportunity to
>>>> deprecate/improve something fundamentally. In Spark 4.1, one of these
>>>> things is the new version framework in DSv2 that requires us to rethink a
>>>> large chunk of the connector. We shouldn't rush this. We will see
>>>> pre-requisite PRs this week. Help with reviews is always appreciated.
>>>>
>>>> сб, 17 січ. 2026 р. о 01:02 Romain Manni-Bucau <rm...@gmail.com>
>>>> пише:
>>>>
>>>>> Hi JB,
>>>>>
>>>>> As said it is 100% an user standpoint and since enbling it is
>>>>> literally 1h I think it is worth if next week is missed but still happy to
>>>>> avoir.
>>>>>
>>>>>
>>>>> Romain Manni-Bucau
>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>
>>>>> Le sam. 17 janv. 2026, 06:21, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>> a écrit :
>>>>>
>>>>>> Hi Romain
>>>>>>
>>>>>> I'm not in favor of 1.10.2 with Spark 4.1 support.
>>>>>>
>>>>>> As I think we are pretty close to having the cleanup/enhancement done
>>>>>> on main, by chance, it will be included in 1.11.0, so worth to wait for it.
>>>>>>
>>>>>> Regards
>>>>>> JB
>>>>>>
>>>>>> On Fri, Jan 16, 2026 at 10:06 PM Romain Manni-Bucau <
>>>>>> rmannibucau@gmail.com> wrote:
>>>>>>
>>>>>>> @Steven sure, my point was just that delaying Spark 4.1 support to
>>>>>>> get new feature can not be the best so branch latest 1.10 tag and just
>>>>>>> fixing the 2-3 compilation issues to get a 1.10.2 and Spark 4.1 working can
>>>>>>> be an intermediate step.
>>>>>>>
>>>>>>> let's see next week what Anton comes back with :)
>>>>>>>
>>>>>>> Romain Manni-Bucau
>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>
>>>>>>>
>>>>>>> Le ven. 16 janv. 2026 à 20:28, Steven Wu <st...@gmail.com> a
>>>>>>> écrit :
>>>>>>>
>>>>>>>> Romain, patch releases (like 1.10.2) are only for bug fixes. New
>>>>>>>> features (like Spark 4.1) would have to happen in minor version releases
>>>>>>>> (like 1.11.0)
>>>>>>>>
>>>>>>>> On Mon, Jan 12, 2026 at 9:22 AM Romain Manni-Bucau <
>>>>>>>> rmannibucau@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Ok, good news Anton, if it is longer than expected (hoping not)
>>>>>>>>> maybe a 1.10.2 with the new module be a compromise, impatient to get your
>>>>>>>>> feedback next week.
>>>>>>>>>
>>>>>>>>> Thanks for the hard work.
>>>>>>>>>
>>>>>>>>> Romain Manni-Bucau
>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Le lun. 12 janv. 2026 à 17:54, Anton Okolnychyi <
>>>>>>>>> aokolnychyi@gmail.com> a écrit :
>>>>>>>>>
>>>>>>>>>> We already have a module for Spark 4.1 in main. That said, Spark
>>>>>>>>>> 4.1 has introduced substantial improvements in table lifecycle that should
>>>>>>>>>> let us remove some legacy hacks that may cause data correctness issues.
>>>>>>>>>> Given that this work may result in some properties being no longer
>>>>>>>>>> supported, we can do that only in the first Iceberg release for Spark 4.1.
>>>>>>>>>> I would strongly oppose releasing support for Spark 4.1 prior to embracing
>>>>>>>>>> the new version framework in DSv2.
>>>>>>>>>>
>>>>>>>>>> Good news, it still shouldn't take too long. I am working on a
>>>>>>>>>> prototype as we speak. It should be available next week.
>>>>>>>>>>
>>>>>>>>>> - Anton
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> пн, 12 січ. 2026 р. о 07:33 Romain Manni-Bucau <
>>>>>>>>>> rmannibucau@gmail.com> пише:
>>>>>>>>>>
>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>
>>>>>>>>>>> I wonder if you do have some plan to release Iceberg 1.11 (I
>>>>>>>>>>> assume next one?) with Spark 4.1 support.
>>>>>>>>>>> The datasource v2 API breakage was very localized but is just
>>>>>>>>>>> blocking to upgrade and some spark 4.1 features are game changers (like the
>>>>>>>>>>> SQL enhancement) so I'm really looking forward being able to move on it
>>>>>>>>>>> with iceberg support.
>>>>>>>>>>> If you need some help please let me know.
>>>>>>>>>>>
>>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>>>
>>>>>>>>>>
Re: Spark 4.1 support
Posted by Anton Okolnychyi <ao...@gmail.com>.
Will share the WIP PR for the new version framework by the end of today. I
will have to then split it into manageable pieces and iterate on still
failing tests.
пн, 2 лют. 2026 р. о 15:20 Aihua Xu <ai...@gmail.com> пише:
> Hi Anton,
>
> I want to keep track of this work item for the Iceberg 1.11.0 release. Can
> you share the status for the work?
>
> Thanks,
> Aihua
>
> On Mon, Jan 26, 2026 at 2:55 PM Romain Manni-Bucau <rm...@gmail.com>
> wrote:
>
>> Hi Anton,
>>
>> Didn't want to say the work done was 1h, I literally meant the fix to
>> make it work is 1h of work.
>> I'm not sure I get your opportunity point since deprecating/reworking
>> fundamentally something might need a new minor of iceberg for end users
>> IMHO.
>>
>> That said happy to test a PR on my use cases (mainly maintenance and SQL
>> as of today) once you think it is worth it too.
>>
>> Romain Manni-Bucau
>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>> <https://dotnetbirdie.github.io/> | Blog <https://rmannibucau.github.io/> |
>> Old Blog <http://rmannibucau.wordpress.com> | Github
>> <https://github.com/rmannibucau> | LinkedIn
>> <https://www.linkedin.com/in/rmannibucau> | Book
>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>> Javaccino founder (Java/.NET service - contact via linkedin)
>>
>>
>> Le lun. 26 janv. 2026 à 23:27, Anton Okolnychyi <ao...@gmail.com>
>> a écrit :
>>
>>> What may seem like 1 hr of work is actually quite complicated. I am sure
>>> Manu dedicated a lot of time in setting the new module and validating all
>>> failed tests, not to mention fixes that were made after testing Spark RCs.
>>>
>>> Anyway, a new Spark release is always a rare opportunity to
>>> deprecate/improve something fundamentally. In Spark 4.1, one of these
>>> things is the new version framework in DSv2 that requires us to rethink a
>>> large chunk of the connector. We shouldn't rush this. We will see
>>> pre-requisite PRs this week. Help with reviews is always appreciated.
>>>
>>> сб, 17 січ. 2026 р. о 01:02 Romain Manni-Bucau <rm...@gmail.com>
>>> пише:
>>>
>>>> Hi JB,
>>>>
>>>> As said it is 100% an user standpoint and since enbling it is literally
>>>> 1h I think it is worth if next week is missed but still happy to avoir.
>>>>
>>>>
>>>> Romain Manni-Bucau
>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>> <https://dotnetbirdie.github.io/> | Blog
>>>> <https://rmannibucau.github.io/> | Old Blog
>>>> <http://rmannibucau.wordpress.com> | Github
>>>> <https://github.com/rmannibucau> | LinkedIn
>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>
>>>> Le sam. 17 janv. 2026, 06:21, Jean-Baptiste Onofré <jb...@nanthrax.net> a
>>>> écrit :
>>>>
>>>>> Hi Romain
>>>>>
>>>>> I'm not in favor of 1.10.2 with Spark 4.1 support.
>>>>>
>>>>> As I think we are pretty close to having the cleanup/enhancement done
>>>>> on main, by chance, it will be included in 1.11.0, so worth to wait for it.
>>>>>
>>>>> Regards
>>>>> JB
>>>>>
>>>>> On Fri, Jan 16, 2026 at 10:06 PM Romain Manni-Bucau <
>>>>> rmannibucau@gmail.com> wrote:
>>>>>
>>>>>> @Steven sure, my point was just that delaying Spark 4.1 support to
>>>>>> get new feature can not be the best so branch latest 1.10 tag and just
>>>>>> fixing the 2-3 compilation issues to get a 1.10.2 and Spark 4.1 working can
>>>>>> be an intermediate step.
>>>>>>
>>>>>> let's see next week what Anton comes back with :)
>>>>>>
>>>>>> Romain Manni-Bucau
>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>
>>>>>>
>>>>>> Le ven. 16 janv. 2026 à 20:28, Steven Wu <st...@gmail.com> a
>>>>>> écrit :
>>>>>>
>>>>>>> Romain, patch releases (like 1.10.2) are only for bug fixes. New
>>>>>>> features (like Spark 4.1) would have to happen in minor version releases
>>>>>>> (like 1.11.0)
>>>>>>>
>>>>>>> On Mon, Jan 12, 2026 at 9:22 AM Romain Manni-Bucau <
>>>>>>> rmannibucau@gmail.com> wrote:
>>>>>>>
>>>>>>>> Ok, good news Anton, if it is longer than expected (hoping not)
>>>>>>>> maybe a 1.10.2 with the new module be a compromise, impatient to get your
>>>>>>>> feedback next week.
>>>>>>>>
>>>>>>>> Thanks for the hard work.
>>>>>>>>
>>>>>>>> Romain Manni-Bucau
>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>
>>>>>>>>
>>>>>>>> Le lun. 12 janv. 2026 à 17:54, Anton Okolnychyi <
>>>>>>>> aokolnychyi@gmail.com> a écrit :
>>>>>>>>
>>>>>>>>> We already have a module for Spark 4.1 in main. That said, Spark
>>>>>>>>> 4.1 has introduced substantial improvements in table lifecycle that should
>>>>>>>>> let us remove some legacy hacks that may cause data correctness issues.
>>>>>>>>> Given that this work may result in some properties being no longer
>>>>>>>>> supported, we can do that only in the first Iceberg release for Spark 4.1.
>>>>>>>>> I would strongly oppose releasing support for Spark 4.1 prior to embracing
>>>>>>>>> the new version framework in DSv2.
>>>>>>>>>
>>>>>>>>> Good news, it still shouldn't take too long. I am working on a
>>>>>>>>> prototype as we speak. It should be available next week.
>>>>>>>>>
>>>>>>>>> - Anton
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> пн, 12 січ. 2026 р. о 07:33 Romain Manni-Bucau <
>>>>>>>>> rmannibucau@gmail.com> пише:
>>>>>>>>>
>>>>>>>>>> Hi everyone,
>>>>>>>>>>
>>>>>>>>>> I wonder if you do have some plan to release Iceberg 1.11 (I
>>>>>>>>>> assume next one?) with Spark 4.1 support.
>>>>>>>>>> The datasource v2 API breakage was very localized but is just
>>>>>>>>>> blocking to upgrade and some spark 4.1 features are game changers (like the
>>>>>>>>>> SQL enhancement) so I'm really looking forward being able to move on it
>>>>>>>>>> with iceberg support.
>>>>>>>>>> If you need some help please let me know.
>>>>>>>>>>
>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>> @rmannibucau <https://x.com/rmannibucau> | .NET Blog
>>>>>>>>>> <https://dotnetbirdie.github.io/> | Blog
>>>>>>>>>> <https://rmannibucau.github.io/> | Old Blog
>>>>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>>>>> <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064>
>>>>>>>>>> Javaccino founder (Java/.NET service - contact via linkedin)
>>>>>>>>>>
>>>>>>>>>