Spark Merge-on-Read Feature

[Aman Rawat <ra...@gmail.com>]

Hello devs,

We are trying to implement Spark support for the merge-on-read feature in
Iceberg. Can you please share the elaborate plan here, ongoing work and
tentative timelines for the same (both from spark and iceberg repos side).
We are following the priority board that has been set up for this feature
in roadmap <https://github.com/apache/iceberg/projects/11> (currently is
empty) and the draft PRs shared by Anton for Spark Row level delete
support. Please let us know if there are any pieces which can be taken up
or any kind of support we can provide to move things faster here.

Best Regards,
Aman.

Re: Spark Merge-on-Read Feature

[Jack Ye <ye...@gmail.com>]

Sorry the link does not actually directly jump to the thread... it's the
thread of [DISCUSS] Spark version support strategy.
-Jack

On Fri, Sep 24, 2021 at 10:32 AM Jack Ye <ye...@gmail.com> wrote:

> Hi Aman,
>
> Thank you for the interest! I would refer you to this mailing list
> discussion thread:
> https://mail-archives.apache.org/mod_mbox/iceberg-dev/202109.mbox/browser
> for more details about Spark MoR support and why there is no PR open for
> that yet. The MoR compaction is tracked in a separated project
> https://github.com/apache/iceberg/projects/10.
>
> Best,
> Jack Ye
>
> On Fri, Sep 24, 2021 at 10:23 AM Aman Rawat <ra...@gmail.com>
> wrote:
>
>> Hello devs,
>>
>> We are trying to implement Spark support for the merge-on-read feature in
>> Iceberg. Can you please share the elaborate plan here, ongoing work and
>> tentative timelines for the same (both from spark and iceberg repos side).
>> We are following the priority board that has been set up for this feature
>> in roadmap <https://github.com/apache/iceberg/projects/11> (currently is
>> empty) and the draft PRs shared by Anton for Spark Row level delete
>> support. Please let us know if there are any pieces which can be taken up
>> or any kind of support we can provide to move things faster here.
>>
>> Best Regards,
>> Aman.
>>
>

Re: Spark Merge-on-Read Feature

[Jack Ye <ye...@gmail.com>]

Hi Aman,

Thank you for the interest! I would refer you to this mailing list
discussion thread:
https://mail-archives.apache.org/mod_mbox/iceberg-dev/202109.mbox/browser
for more details about Spark MoR support and why there is no PR open for
that yet. The MoR compaction is tracked in a separated project
https://github.com/apache/iceberg/projects/10.

Best,
Jack Ye

On Fri, Sep 24, 2021 at 10:23 AM Aman Rawat <ra...@gmail.com> wrote:

> Hello devs,
>
> We are trying to implement Spark support for the merge-on-read feature in
> Iceberg. Can you please share the elaborate plan here, ongoing work and
> tentative timelines for the same (both from spark and iceberg repos side).
> We are following the priority board that has been set up for this feature
> in roadmap <https://github.com/apache/iceberg/projects/11> (currently is
> empty) and the draft PRs shared by Anton for Spark Row level delete
> support. Please let us know if there are any pieces which can be taken up
> or any kind of support we can provide to move things faster here.
>
> Best Regards,
> Aman.
>