You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hudi.apache.org by Shiyan Xu <xu...@gmail.com> on 2020/06/27 04:01:57 UTC
[DISCUSS] Make delete marker configurable?
Hi all,
A small suggestion: as delta streamer relies on `_hoodie_is_deleted` to do
hard delete, can we make it configurable? as in users can specify any
boolean field for delete marker and `_hoodie_is_deleted` remains as default.
Regards,
Raymond
Re: [DISCUSS] Make delete marker configurable?
Posted by Bhavani Sudha <bh...@gmail.com>.
+1 as well. Thanks Raymond for explaining.
On Sun, Jun 28, 2020 at 11:35 AM Shiyan Xu <xu...@gmail.com>
wrote:
> Hi Sudha, the delete marker being configurable can give more flexibility to
> users when process delete events; they can check any bool field they may
> have on their own schema.
>
> On Sat, Jun 27, 2020 at 5:32 PM Bhavani Sudha <bh...@gmail.com>
> wrote:
>
> > Hi Raymond,
> >
> > I am trying to understand the use case . Can you please provide more
> > context on what problem this addresses ?
> >
> >
> > Thanks,
> > Sudha
> >
> > On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu <xu...@gmail.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > A small suggestion: as delta streamer relies on `_hoodie_is_deleted` to
> > do
> > > hard delete, can we make it configurable? as in users can specify any
> > > boolean field for delete marker and `_hoodie_is_deleted` remains as
> > > default.
> > >
> > > Regards,
> > > Raymond
> > >
> >
>
Re: [DISCUSS] Make delete marker configurable?
Posted by Shiyan Xu <xu...@gmail.com>.
Hi Sudha, the delete marker being configurable can give more flexibility to
users when process delete events; they can check any bool field they may
have on their own schema.
On Sat, Jun 27, 2020 at 5:32 PM Bhavani Sudha <bh...@gmail.com>
wrote:
> Hi Raymond,
>
> I am trying to understand the use case . Can you please provide more
> context on what problem this addresses ?
>
>
> Thanks,
> Sudha
>
> On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu <xu...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > A small suggestion: as delta streamer relies on `_hoodie_is_deleted` to
> do
> > hard delete, can we make it configurable? as in users can specify any
> > boolean field for delete marker and `_hoodie_is_deleted` remains as
> > default.
> >
> > Regards,
> > Raymond
> >
>
Re: [DISCUSS] Make delete marker configurable?
Posted by Balaji Varadarajan <v....@ymail.com.INVALID>.
+1
Sent from Yahoo Mail for iPhone
On Monday, June 29, 2020, 5:34 PM, Vinoth Chandar <vi...@apache.org> wrote:
+1 as well. (sorry , for jumping in late)
On Sun, Jun 28, 2020 at 11:36 AM Shiyan Xu <xu...@gmail.com>
wrote:
> Thanks for the +1. Filed https://issues.apache.org/jira/browse/HUDI-1058
>
> On Sat, Jun 27, 2020 at 11:34 PM Pratyaksh Sharma <pr...@gmail.com>
> wrote:
>
> > The suggestion looks good to me as well.
> >
> > On Sun, Jun 28, 2020 at 8:17 AM Sivabalan <n....@gmail.com> wrote:
> >
> > > +1, I just left it as a todo for future patch when I worked on it.
> > >
> > > On Sat, Jun 27, 2020 at 8:32 PM Bhavani Sudha <bhavanisudhas@gmail.com
> >
> > > wrote:
> > >
> > > > Hi Raymond,
> > > >
> > > > I am trying to understand the use case . Can you please provide more
> > > > context on what problem this addresses ?
> > > >
> > > >
> > > > Thanks,
> > > > Sudha
> > > >
> > > > On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu <
> xu.shiyan.raymond@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > A small suggestion: as delta streamer relies on
> `_hoodie_is_deleted`
> > to
> > > > do
> > > > > hard delete, can we make it configurable? as in users can specify
> any
> > > > > boolean field for delete marker and `_hoodie_is_deleted` remains as
> > > > > default.
> > > > >
> > > > > Regards,
> > > > > Raymond
> > > > >
> > > >
> > > --
> > > Regards,
> > > -Sivabalan
> > >
> >
>
Re: [DISCUSS] Make delete marker configurable?
Posted by Vinoth Chandar <vi...@apache.org>.
+1 as well. (sorry , for jumping in late)
On Sun, Jun 28, 2020 at 11:36 AM Shiyan Xu <xu...@gmail.com>
wrote:
> Thanks for the +1. Filed https://issues.apache.org/jira/browse/HUDI-1058
>
> On Sat, Jun 27, 2020 at 11:34 PM Pratyaksh Sharma <pr...@gmail.com>
> wrote:
>
> > The suggestion looks good to me as well.
> >
> > On Sun, Jun 28, 2020 at 8:17 AM Sivabalan <n....@gmail.com> wrote:
> >
> > > +1, I just left it as a todo for future patch when I worked on it.
> > >
> > > On Sat, Jun 27, 2020 at 8:32 PM Bhavani Sudha <bhavanisudhas@gmail.com
> >
> > > wrote:
> > >
> > > > Hi Raymond,
> > > >
> > > > I am trying to understand the use case . Can you please provide more
> > > > context on what problem this addresses ?
> > > >
> > > >
> > > > Thanks,
> > > > Sudha
> > > >
> > > > On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu <
> xu.shiyan.raymond@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > A small suggestion: as delta streamer relies on
> `_hoodie_is_deleted`
> > to
> > > > do
> > > > > hard delete, can we make it configurable? as in users can specify
> any
> > > > > boolean field for delete marker and `_hoodie_is_deleted` remains as
> > > > > default.
> > > > >
> > > > > Regards,
> > > > > Raymond
> > > > >
> > > >
> > > --
> > > Regards,
> > > -Sivabalan
> > >
> >
>
Re: [DISCUSS] Make delete marker configurable?
Posted by Shiyan Xu <xu...@gmail.com>.
Thanks for the +1. Filed https://issues.apache.org/jira/browse/HUDI-1058
On Sat, Jun 27, 2020 at 11:34 PM Pratyaksh Sharma <pr...@gmail.com>
wrote:
> The suggestion looks good to me as well.
>
> On Sun, Jun 28, 2020 at 8:17 AM Sivabalan <n....@gmail.com> wrote:
>
> > +1, I just left it as a todo for future patch when I worked on it.
> >
> > On Sat, Jun 27, 2020 at 8:32 PM Bhavani Sudha <bh...@gmail.com>
> > wrote:
> >
> > > Hi Raymond,
> > >
> > > I am trying to understand the use case . Can you please provide more
> > > context on what problem this addresses ?
> > >
> > >
> > > Thanks,
> > > Sudha
> > >
> > > On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu <xu.shiyan.raymond@gmail.com
> >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > A small suggestion: as delta streamer relies on `_hoodie_is_deleted`
> to
> > > do
> > > > hard delete, can we make it configurable? as in users can specify any
> > > > boolean field for delete marker and `_hoodie_is_deleted` remains as
> > > > default.
> > > >
> > > > Regards,
> > > > Raymond
> > > >
> > >
> > --
> > Regards,
> > -Sivabalan
> >
>
Re: [DISCUSS] Make delete marker configurable?
Posted by Pratyaksh Sharma <pr...@gmail.com>.
The suggestion looks good to me as well.
On Sun, Jun 28, 2020 at 8:17 AM Sivabalan <n....@gmail.com> wrote:
> +1, I just left it as a todo for future patch when I worked on it.
>
> On Sat, Jun 27, 2020 at 8:32 PM Bhavani Sudha <bh...@gmail.com>
> wrote:
>
> > Hi Raymond,
> >
> > I am trying to understand the use case . Can you please provide more
> > context on what problem this addresses ?
> >
> >
> > Thanks,
> > Sudha
> >
> > On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu <xu...@gmail.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > A small suggestion: as delta streamer relies on `_hoodie_is_deleted` to
> > do
> > > hard delete, can we make it configurable? as in users can specify any
> > > boolean field for delete marker and `_hoodie_is_deleted` remains as
> > > default.
> > >
> > > Regards,
> > > Raymond
> > >
> >
> --
> Regards,
> -Sivabalan
>
Re: [DISCUSS] Make delete marker configurable?
Posted by Sivabalan <n....@gmail.com>.
+1, I just left it as a todo for future patch when I worked on it.
On Sat, Jun 27, 2020 at 8:32 PM Bhavani Sudha <bh...@gmail.com>
wrote:
> Hi Raymond,
>
> I am trying to understand the use case . Can you please provide more
> context on what problem this addresses ?
>
>
> Thanks,
> Sudha
>
> On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu <xu...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > A small suggestion: as delta streamer relies on `_hoodie_is_deleted` to
> do
> > hard delete, can we make it configurable? as in users can specify any
> > boolean field for delete marker and `_hoodie_is_deleted` remains as
> > default.
> >
> > Regards,
> > Raymond
> >
>
--
Regards,
-Sivabalan
Re: [DISCUSS] Make delete marker configurable?
Posted by Bhavani Sudha <bh...@gmail.com>.
Hi Raymond,
I am trying to understand the use case . Can you please provide more
context on what problem this addresses ?
Thanks,
Sudha
On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu <xu...@gmail.com>
wrote:
> Hi all,
>
> A small suggestion: as delta streamer relies on `_hoodie_is_deleted` to do
> hard delete, can we make it configurable? as in users can specify any
> boolean field for delete marker and `_hoodie_is_deleted` remains as
> default.
>
> Regards,
> Raymond
>