You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iceberg.apache.org by Jean-Baptiste Onofré <jb...@nanthrax.net> on 2024/09/05 08:39:00 UTC
Re: [DISCUSS] Improving Position Deletes in V3
Hi Anton
Sorry for the late reply on this proposal.
I like it ! It looks good to me (I have a few minor comments).
It would be great to include this spec update in V3.
Please let me know if I can help on this !
Regards
JB
On Wed, Aug 21, 2024 at 11:28 PM Anton Okolnychyi <ao...@gmail.com> wrote:
>
> Hey folks,
>
> As discussed during the sync, I've been working on a proposal to improve the handling of position deletes in V3. It builds on lessons learned from deploying the current approach at scale and addresses all unresolved questions from past community discussions and proposals around this topic.
>
> In particular, the proposal attempts to address the following shortcomings we observe today:
>
> Choosing between fewer delete files on disk or targeted deletes.
> Dependence on external maintenance for consistent write and read performance.
> Writing and reading overhead as in-memory and on-disk representations differ.
>
> Please, take a look at the doc [1] and let me know what you think. Any feedback is highly appreciated!
>
> - Anton
>
> [1] - https://docs.google.com/document/d/18Bqhr-vnzFfQk1S4AgRISkA_5_m5m32Nnc2Cw0zn2XM
>
>
Re: [DISCUSS] Improving Position Deletes in V3
Posted by Russell Spitzer <ru...@gmail.com>.
+1 I am on board, all of my doubts are resolved
On Thu, Sep 5, 2024 at 3:42 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> Hi Anton
>
> Sorry for the late reply on this proposal.
> I like it ! It looks good to me (I have a few minor comments).
>
> It would be great to include this spec update in V3.
> Please let me know if I can help on this !
>
> Regards
> JB
>
> On Wed, Aug 21, 2024 at 11:28 PM Anton Okolnychyi <ao...@gmail.com>
> wrote:
> >
> > Hey folks,
> >
> > As discussed during the sync, I've been working on a proposal to improve
> the handling of position deletes in V3. It builds on lessons learned from
> deploying the current approach at scale and addresses all unresolved
> questions from past community discussions and proposals around this topic.
> >
> > In particular, the proposal attempts to address the following
> shortcomings we observe today:
> >
> > Choosing between fewer delete files on disk or targeted deletes.
> > Dependence on external maintenance for consistent write and read
> performance.
> > Writing and reading overhead as in-memory and on-disk representations
> differ.
> >
> > Please, take a look at the doc [1] and let me know what you think. Any
> feedback is highly appreciated!
> >
> > - Anton
> >
> > [1] -
> https://docs.google.com/document/d/18Bqhr-vnzFfQk1S4AgRISkA_5_m5m32Nnc2Cw0zn2XM
> >
> >
>