You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Robert Metzger <rm...@apache.org> on 2020/03/12 14:40:46 UTC
[DISCUSS] Drop Bucketing Sink
Hi all,
I'm currently investigating a failing end to end test for the bucketing
sink [1].
The bucketing sink has been deprecated in the 1.9 release [2], because we
have the new StreamingFileSink [3] for quite a while.
Before putting any effort into fixing the end to end test for the sink, I
wanted to propose dropping the bucketing sink from master for the upcoming
1.11 release.
What do you think?
[1] https://issues.apache.org/jira/browse/FLINK-16227
[2] https://issues.apache.org/jira/browse/FLINK-13396
[3] https://issues.apache.org/jira/browse/FLINK-9749
Re: [DISCUSS] Drop Bucketing Sink
Posted by Yu Li <ca...@gmail.com>.
Thanks for bringing up this discussion Robert!
According to the inputs, I suggest we create an umbrella JIRA issue to
track all critical improvements StreamingFileSink should have before we
could completely discard bucketing sink, so we could have a clear view of
the progress and how soon we could achieve the goal. What do you think?
btw, we also need to create sub-tasks to track the known issues, for
example it seems I cannot find any JIRA tracking the ORC support yet.
Best Regards,
Yu
On Fri, 13 Mar 2020 at 10:07, Jingsong Li <ji...@gmail.com> wrote:
> Hi Robert,
>
> +1 to drop it but maybe not 1.11.
>
> ORC has not been supported on StreamingFileSink. I have seen lots of users
> run ORC in the bucketing sink.
>
> Best,
> Jingsong Lee
>
> On Fri, Mar 13, 2020 at 1:11 AM Seth Wiesman <sj...@gmail.com> wrote:
>
> > Sorry, I meant FLIP-46.
> >
> > Seth
> >
> > On Thu, Mar 12, 2020 at 11:52 AM Seth Wiesman <sj...@gmail.com>
> wrote:
> >
> > > I agree with David, I think FLIP-49 needs to be prioritized for 1.11 if
> > we
> > > want to drop the bucketing sink.
> > >
> > > Seth
> > >
> > > On Thu, Mar 12, 2020 at 10:53 AM David Anderson <da...@ververica.com>
> > > wrote:
> > >
> > >> The BucketingSink is still somewhat widely used, I think in part
> because
> > >> of
> > >> shortcomings in the StreamingFileSink.
> > >>
> > >> I would hope that in tandem with removing the bucketing sink we could
> > also
> > >> address some of these issues. I'm thinking in particular of issues
> that
> > >> are
> > >> waiting on FLIP-46 [1].
> > >>
> > >> Removing the bucketing sink will go down better, in my opinion, if
> it's
> > >> coupled with progress on some of the open StreamingFileSink tickets.
> > >>
> > >> Best,
> > >> David
> > >>
> > >> [1]
> > >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-46%3A+Graceful+Shutdown+Handling+by+UDFs
> > >>
> > >>
> > >> On Thu, Mar 12, 2020 at 4:27 PM Zhijiang <wangzhijiang999@aliyun.com
> > >> .invalid>
> > >> wrote:
> > >>
> > >> > Thanks for driving this discussion, Robert!
> > >> >
> > >> > This e2e test really fails frequently. +1 to drop bucketing sink,
> it
> > is
> > >> > not worth paying more efforts since deprecated.
> > >> >
> > >> > Best,
> > >> > Zhijiang
> > >> >
> > >> >
> > >> > ------------------------------------------------------------------
> > >> > From:Jeff Zhang <zj...@gmail.com>
> > >> > Send Time:2020 Mar. 12 (Thu.) 23:17
> > >> > To:dev <de...@flink.apache.org>
> > >> > Subject:Re: [DISCUSS] Drop Bucketing Sink
> > >> >
> > >> > +1， dropping deprecated api is always necessary for a sustainable
> > >> project.
> > >> >
> > >> > Kostas Kloudas <kk...@gmail.com> 于2020年3月12日周四 下午11:06写道：
> > >> >
> > >> > > Hi Robert,
> > >> > >
> > >> > > +1 for dropping the BucketingSink.
> > >> > > In any case, it has not been maintained for quite some time.
> > >> > >
> > >> > > Cheers,
> > >> > > Kostas
> > >> > >
> > >> > > On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger <
> rmetzger@apache.org
> > >
> > >> > > wrote:
> > >> > > >
> > >> > > > Hi all,
> > >> > > >
> > >> > > > I'm currently investigating a failing end to end test for the
> > >> bucketing
> > >> > > > sink [1].
> > >> > > > The bucketing sink has been deprecated in the 1.9 release [2],
> > >> because
> > >> > we
> > >> > > > have the new StreamingFileSink [3] for quite a while.
> > >> > > > Before putting any effort into fixing the end to end test for
> the
> > >> > sink, I
> > >> > > > wanted to propose dropping the bucketing sink from master for
> the
> > >> > > upcoming
> > >> > > > 1.11 release.
> > >> > > >
> > >> > > > What do you think?
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > [1] https://issues.apache.org/jira/browse/FLINK-16227
> > >> > > > [2] https://issues.apache.org/jira/browse/FLINK-13396
> > >> > > > [3] https://issues.apache.org/jira/browse/FLINK-9749
> > >> > >
> > >> >
> > >> >
> > >> > --
> > >> > Best Regards
> > >> >
> > >> > Jeff Zhang
> > >> >
> > >> >
> > >>
> > >
> >
>
>
> --
> Best, Jingsong Lee
>
Re: [DISCUSS] Drop Bucketing Sink
Posted by Robert Metzger <rm...@apache.org>.
By the way: I have disabled the test in the Hadoop 2.4.1 build, so my
original problem is resolved.
I'm not convinced that posting on the user@ ml is the right approach.
The last few questions to the user@ list asking for feedback have not
really been answered. I believe that we have deprecated the sink for long
enough, also we seem to have a common understanding of missing features we
need to resolve first.
Last but not least, we can always port the bucketing sink to another
repository (apache bahir) and keep compatibility with later Flink versions
there if a lot of users complain once it has been removed.
On Tue, Mar 17, 2020 at 10:35 AM Kostas Kloudas <kk...@gmail.com> wrote:
> Thanks Robert for all this,
>
> I think that we should also post a thread in the user ML so that users
> can also comment on the topic.
>
> What do you think?
>
> Kostas
>
> On Mon, Mar 16, 2020 at 12:27 PM Robert Metzger <rm...@apache.org>
> wrote:
> >
> > Thank you all for your feedback.
> >
> > I will try to fix the test then (or disable it).
> >
> > Here's a ticket for dropping the BucketingSink:
> > https://issues.apache.org/jira/browse/FLINK-16616 Please mark whatever
> we
> > consider necessary as a "depends on" ticket.
> > @David / @Seth: Where are the tickets depending on FLIP-46 listed? Can
> one
> > of you add them to FLINK-16616
> >
> >
> > On Fri, Mar 13, 2020 at 11:32 AM Guowei Ma <gu...@gmail.com> wrote:
> >
> > > +1 to drop it.
> > >
> > > To Jingsong :
> > > we are planning to implement the orc StreamingFileSink in 1.11.
> > > I think users also could reference the old BucktSink from the old
> version.
> > >
> > > Best,
> > > Guowei
> > >
> > >
> > > Jingsong Li <ji...@gmail.com> 于2020年3月13日周五 上午10:07写道：
> > >
> > > > Hi Robert,
> > > >
> > > > +1 to drop it but maybe not 1.11.
> > > >
> > > > ORC has not been supported on StreamingFileSink. I have seen lots of
> > > users
> > > > run ORC in the bucketing sink.
> > > >
> > > > Best,
> > > > Jingsong Lee
> > > >
> > > > On Fri, Mar 13, 2020 at 1:11 AM Seth Wiesman <sj...@gmail.com>
> > > wrote:
> > > >
> > > > > Sorry, I meant FLIP-46.
> > > > >
> > > > > Seth
> > > > >
> > > > > On Thu, Mar 12, 2020 at 11:52 AM Seth Wiesman <sjwiesman@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > > I agree with David, I think FLIP-49 needs to be prioritized for
> 1.11
> > > if
> > > > > we
> > > > > > want to drop the bucketing sink.
> > > > > >
> > > > > > Seth
> > > > > >
> > > > > > On Thu, Mar 12, 2020 at 10:53 AM David Anderson <
> david@ververica.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > >> The BucketingSink is still somewhat widely used, I think in part
> > > > because
> > > > > >> of
> > > > > >> shortcomings in the StreamingFileSink.
> > > > > >>
> > > > > >> I would hope that in tandem with removing the bucketing sink we
> > > could
> > > > > also
> > > > > >> address some of these issues. I'm thinking in particular of
> issues
> > > > that
> > > > > >> are
> > > > > >> waiting on FLIP-46 [1].
> > > > > >>
> > > > > >> Removing the bucketing sink will go down better, in my opinion,
> if
> > > > it's
> > > > > >> coupled with progress on some of the open StreamingFileSink
> tickets.
> > > > > >>
> > > > > >> Best,
> > > > > >> David
> > > > > >>
> > > > > >> [1]
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-46%3A+Graceful+Shutdown+Handling+by+UDFs
> > > > > >>
> > > > > >>
> > > > > >> On Thu, Mar 12, 2020 at 4:27 PM Zhijiang <
> > > wangzhijiang999@aliyun.com
> > > > > >> .invalid>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Thanks for driving this discussion, Robert!
> > > > > >> >
> > > > > >> > This e2e test really fails frequently. +1 to drop bucketing
> sink,
> > > > it
> > > > > is
> > > > > >> > not worth paying more efforts since deprecated.
> > > > > >> >
> > > > > >> > Best,
> > > > > >> > Zhijiang
> > > > > >> >
> > > > > >> >
> > > > > >> >
> ------------------------------------------------------------------
> > > > > >> > From:Jeff Zhang <zj...@gmail.com>
> > > > > >> > Send Time:2020 Mar. 12 (Thu.) 23:17
> > > > > >> > To:dev <de...@flink.apache.org>
> > > > > >> > Subject:Re: [DISCUSS] Drop Bucketing Sink
> > > > > >> >
> > > > > >> > +1， dropping deprecated api is always necessary for a
> sustainable
> > > > > >> project.
> > > > > >> >
> > > > > >> > Kostas Kloudas <kk...@gmail.com> 于2020年3月12日周四 下午11:06写道：
> > > > > >> >
> > > > > >> > > Hi Robert,
> > > > > >> > >
> > > > > >> > > +1 for dropping the BucketingSink.
> > > > > >> > > In any case, it has not been maintained for quite some time.
> > > > > >> > >
> > > > > >> > > Cheers,
> > > > > >> > > Kostas
> > > > > >> > >
> > > > > >> > > On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger <
> > > > rmetzger@apache.org
> > > > > >
> > > > > >> > > wrote:
> > > > > >> > > >
> > > > > >> > > > Hi all,
> > > > > >> > > >
> > > > > >> > > > I'm currently investigating a failing end to end test for
> the
> > > > > >> bucketing
> > > > > >> > > > sink [1].
> > > > > >> > > > The bucketing sink has been deprecated in the 1.9 release
> [2],
> > > > > >> because
> > > > > >> > we
> > > > > >> > > > have the new StreamingFileSink [3] for quite a while.
> > > > > >> > > > Before putting any effort into fixing the end to end test
> for
> > > > the
> > > > > >> > sink, I
> > > > > >> > > > wanted to propose dropping the bucketing sink from master
> for
> > > > the
> > > > > >> > > upcoming
> > > > > >> > > > 1.11 release.
> > > > > >> > > >
> > > > > >> > > > What do you think?
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > > [1] https://issues.apache.org/jira/browse/FLINK-16227
> > > > > >> > > > [2] https://issues.apache.org/jira/browse/FLINK-13396
> > > > > >> > > > [3] https://issues.apache.org/jira/browse/FLINK-9749
> > > > > >> > >
> > > > > >> >
> > > > > >> >
> > > > > >> > --
> > > > > >> > Best Regards
> > > > > >> >
> > > > > >> > Jeff Zhang
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best, Jingsong Lee
> > > >
> > >
>
Re: [DISCUSS] Drop Bucketing Sink
Posted by Kostas Kloudas <kk...@gmail.com>.
Thanks Robert for all this,
I think that we should also post a thread in the user ML so that users
can also comment on the topic.
What do you think?
Kostas
On Mon, Mar 16, 2020 at 12:27 PM Robert Metzger <rm...@apache.org> wrote:
>
> Thank you all for your feedback.
>
> I will try to fix the test then (or disable it).
>
> Here's a ticket for dropping the BucketingSink:
> https://issues.apache.org/jira/browse/FLINK-16616 Please mark whatever we
> consider necessary as a "depends on" ticket.
> @David / @Seth: Where are the tickets depending on FLIP-46 listed? Can one
> of you add them to FLINK-16616
>
>
> On Fri, Mar 13, 2020 at 11:32 AM Guowei Ma <gu...@gmail.com> wrote:
>
> > +1 to drop it.
> >
> > To Jingsong :
> > we are planning to implement the orc StreamingFileSink in 1.11.
> > I think users also could reference the old BucktSink from the old version.
> >
> > Best,
> > Guowei
> >
> >
> > Jingsong Li <ji...@gmail.com> 于2020年3月13日周五 上午10:07写道：
> >
> > > Hi Robert,
> > >
> > > +1 to drop it but maybe not 1.11.
> > >
> > > ORC has not been supported on StreamingFileSink. I have seen lots of
> > users
> > > run ORC in the bucketing sink.
> > >
> > > Best,
> > > Jingsong Lee
> > >
> > > On Fri, Mar 13, 2020 at 1:11 AM Seth Wiesman <sj...@gmail.com>
> > wrote:
> > >
> > > > Sorry, I meant FLIP-46.
> > > >
> > > > Seth
> > > >
> > > > On Thu, Mar 12, 2020 at 11:52 AM Seth Wiesman <sj...@gmail.com>
> > > wrote:
> > > >
> > > > > I agree with David, I think FLIP-49 needs to be prioritized for 1.11
> > if
> > > > we
> > > > > want to drop the bucketing sink.
> > > > >
> > > > > Seth
> > > > >
> > > > > On Thu, Mar 12, 2020 at 10:53 AM David Anderson <david@ververica.com
> > >
> > > > > wrote:
> > > > >
> > > > >> The BucketingSink is still somewhat widely used, I think in part
> > > because
> > > > >> of
> > > > >> shortcomings in the StreamingFileSink.
> > > > >>
> > > > >> I would hope that in tandem with removing the bucketing sink we
> > could
> > > > also
> > > > >> address some of these issues. I'm thinking in particular of issues
> > > that
> > > > >> are
> > > > >> waiting on FLIP-46 [1].
> > > > >>
> > > > >> Removing the bucketing sink will go down better, in my opinion, if
> > > it's
> > > > >> coupled with progress on some of the open StreamingFileSink tickets.
> > > > >>
> > > > >> Best,
> > > > >> David
> > > > >>
> > > > >> [1]
> > > > >>
> > > > >>
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-46%3A+Graceful+Shutdown+Handling+by+UDFs
> > > > >>
> > > > >>
> > > > >> On Thu, Mar 12, 2020 at 4:27 PM Zhijiang <
> > wangzhijiang999@aliyun.com
> > > > >> .invalid>
> > > > >> wrote:
> > > > >>
> > > > >> > Thanks for driving this discussion, Robert!
> > > > >> >
> > > > >> > This e2e test really fails frequently. +1 to drop bucketing sink,
> > > it
> > > > is
> > > > >> > not worth paying more efforts since deprecated.
> > > > >> >
> > > > >> > Best,
> > > > >> > Zhijiang
> > > > >> >
> > > > >> >
> > > > >> > ------------------------------------------------------------------
> > > > >> > From:Jeff Zhang <zj...@gmail.com>
> > > > >> > Send Time:2020 Mar. 12 (Thu.) 23:17
> > > > >> > To:dev <de...@flink.apache.org>
> > > > >> > Subject:Re: [DISCUSS] Drop Bucketing Sink
> > > > >> >
> > > > >> > +1， dropping deprecated api is always necessary for a sustainable
> > > > >> project.
> > > > >> >
> > > > >> > Kostas Kloudas <kk...@gmail.com> 于2020年3月12日周四 下午11:06写道：
> > > > >> >
> > > > >> > > Hi Robert,
> > > > >> > >
> > > > >> > > +1 for dropping the BucketingSink.
> > > > >> > > In any case, it has not been maintained for quite some time.
> > > > >> > >
> > > > >> > > Cheers,
> > > > >> > > Kostas
> > > > >> > >
> > > > >> > > On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger <
> > > rmetzger@apache.org
> > > > >
> > > > >> > > wrote:
> > > > >> > > >
> > > > >> > > > Hi all,
> > > > >> > > >
> > > > >> > > > I'm currently investigating a failing end to end test for the
> > > > >> bucketing
> > > > >> > > > sink [1].
> > > > >> > > > The bucketing sink has been deprecated in the 1.9 release [2],
> > > > >> because
> > > > >> > we
> > > > >> > > > have the new StreamingFileSink [3] for quite a while.
> > > > >> > > > Before putting any effort into fixing the end to end test for
> > > the
> > > > >> > sink, I
> > > > >> > > > wanted to propose dropping the bucketing sink from master for
> > > the
> > > > >> > > upcoming
> > > > >> > > > 1.11 release.
> > > > >> > > >
> > > > >> > > > What do you think?
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > [1] https://issues.apache.org/jira/browse/FLINK-16227
> > > > >> > > > [2] https://issues.apache.org/jira/browse/FLINK-13396
> > > > >> > > > [3] https://issues.apache.org/jira/browse/FLINK-9749
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >> > --
> > > > >> > Best Regards
> > > > >> >
> > > > >> > Jeff Zhang
> > > > >> >
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best, Jingsong Lee
> > >
> >
Re: [DISCUSS] Drop Bucketing Sink
Posted by Robert Metzger <rm...@apache.org>.
Thank you all for your feedback.
I will try to fix the test then (or disable it).
Here's a ticket for dropping the BucketingSink:
https://issues.apache.org/jira/browse/FLINK-16616 Please mark whatever we
consider necessary as a "depends on" ticket.
@David / @Seth: Where are the tickets depending on FLIP-46 listed? Can one
of you add them to FLINK-16616
On Fri, Mar 13, 2020 at 11:32 AM Guowei Ma <gu...@gmail.com> wrote:
> +1 to drop it.
>
> To Jingsong :
> we are planning to implement the orc StreamingFileSink in 1.11.
> I think users also could reference the old BucktSink from the old version.
>
> Best,
> Guowei
>
>
> Jingsong Li <ji...@gmail.com> 于2020年3月13日周五 上午10:07写道：
>
> > Hi Robert,
> >
> > +1 to drop it but maybe not 1.11.
> >
> > ORC has not been supported on StreamingFileSink. I have seen lots of
> users
> > run ORC in the bucketing sink.
> >
> > Best,
> > Jingsong Lee
> >
> > On Fri, Mar 13, 2020 at 1:11 AM Seth Wiesman <sj...@gmail.com>
> wrote:
> >
> > > Sorry, I meant FLIP-46.
> > >
> > > Seth
> > >
> > > On Thu, Mar 12, 2020 at 11:52 AM Seth Wiesman <sj...@gmail.com>
> > wrote:
> > >
> > > > I agree with David, I think FLIP-49 needs to be prioritized for 1.11
> if
> > > we
> > > > want to drop the bucketing sink.
> > > >
> > > > Seth
> > > >
> > > > On Thu, Mar 12, 2020 at 10:53 AM David Anderson <david@ververica.com
> >
> > > > wrote:
> > > >
> > > >> The BucketingSink is still somewhat widely used, I think in part
> > because
> > > >> of
> > > >> shortcomings in the StreamingFileSink.
> > > >>
> > > >> I would hope that in tandem with removing the bucketing sink we
> could
> > > also
> > > >> address some of these issues. I'm thinking in particular of issues
> > that
> > > >> are
> > > >> waiting on FLIP-46 [1].
> > > >>
> > > >> Removing the bucketing sink will go down better, in my opinion, if
> > it's
> > > >> coupled with progress on some of the open StreamingFileSink tickets.
> > > >>
> > > >> Best,
> > > >> David
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-46%3A+Graceful+Shutdown+Handling+by+UDFs
> > > >>
> > > >>
> > > >> On Thu, Mar 12, 2020 at 4:27 PM Zhijiang <
> wangzhijiang999@aliyun.com
> > > >> .invalid>
> > > >> wrote:
> > > >>
> > > >> > Thanks for driving this discussion, Robert!
> > > >> >
> > > >> > This e2e test really fails frequently. +1 to drop bucketing sink,
> > it
> > > is
> > > >> > not worth paying more efforts since deprecated.
> > > >> >
> > > >> > Best,
> > > >> > Zhijiang
> > > >> >
> > > >> >
> > > >> > ------------------------------------------------------------------
> > > >> > From:Jeff Zhang <zj...@gmail.com>
> > > >> > Send Time:2020 Mar. 12 (Thu.) 23:17
> > > >> > To:dev <de...@flink.apache.org>
> > > >> > Subject:Re: [DISCUSS] Drop Bucketing Sink
> > > >> >
> > > >> > +1， dropping deprecated api is always necessary for a sustainable
> > > >> project.
> > > >> >
> > > >> > Kostas Kloudas <kk...@gmail.com> 于2020年3月12日周四 下午11:06写道：
> > > >> >
> > > >> > > Hi Robert,
> > > >> > >
> > > >> > > +1 for dropping the BucketingSink.
> > > >> > > In any case, it has not been maintained for quite some time.
> > > >> > >
> > > >> > > Cheers,
> > > >> > > Kostas
> > > >> > >
> > > >> > > On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger <
> > rmetzger@apache.org
> > > >
> > > >> > > wrote:
> > > >> > > >
> > > >> > > > Hi all,
> > > >> > > >
> > > >> > > > I'm currently investigating a failing end to end test for the
> > > >> bucketing
> > > >> > > > sink [1].
> > > >> > > > The bucketing sink has been deprecated in the 1.9 release [2],
> > > >> because
> > > >> > we
> > > >> > > > have the new StreamingFileSink [3] for quite a while.
> > > >> > > > Before putting any effort into fixing the end to end test for
> > the
> > > >> > sink, I
> > > >> > > > wanted to propose dropping the bucketing sink from master for
> > the
> > > >> > > upcoming
> > > >> > > > 1.11 release.
> > > >> > > >
> > > >> > > > What do you think?
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > [1] https://issues.apache.org/jira/browse/FLINK-16227
> > > >> > > > [2] https://issues.apache.org/jira/browse/FLINK-13396
> > > >> > > > [3] https://issues.apache.org/jira/browse/FLINK-9749
> > > >> > >
> > > >> >
> > > >> >
> > > >> > --
> > > >> > Best Regards
> > > >> >
> > > >> > Jeff Zhang
> > > >> >
> > > >> >
> > > >>
> > > >
> > >
> >
> >
> > --
> > Best, Jingsong Lee
> >
>
Re: [DISCUSS] Drop Bucketing Sink
Posted by Guowei Ma <gu...@gmail.com>.
+1 to drop it.
To Jingsong :
we are planning to implement the orc StreamingFileSink in 1.11.
I think users also could reference the old BucktSink from the old version.
Best,
Guowei
Jingsong Li <ji...@gmail.com> 于2020年3月13日周五 上午10:07写道：
> Hi Robert,
>
> +1 to drop it but maybe not 1.11.
>
> ORC has not been supported on StreamingFileSink. I have seen lots of users
> run ORC in the bucketing sink.
>
> Best,
> Jingsong Lee
>
> On Fri, Mar 13, 2020 at 1:11 AM Seth Wiesman <sj...@gmail.com> wrote:
>
> > Sorry, I meant FLIP-46.
> >
> > Seth
> >
> > On Thu, Mar 12, 2020 at 11:52 AM Seth Wiesman <sj...@gmail.com>
> wrote:
> >
> > > I agree with David, I think FLIP-49 needs to be prioritized for 1.11 if
> > we
> > > want to drop the bucketing sink.
> > >
> > > Seth
> > >
> > > On Thu, Mar 12, 2020 at 10:53 AM David Anderson <da...@ververica.com>
> > > wrote:
> > >
> > >> The BucketingSink is still somewhat widely used, I think in part
> because
> > >> of
> > >> shortcomings in the StreamingFileSink.
> > >>
> > >> I would hope that in tandem with removing the bucketing sink we could
> > also
> > >> address some of these issues. I'm thinking in particular of issues
> that
> > >> are
> > >> waiting on FLIP-46 [1].
> > >>
> > >> Removing the bucketing sink will go down better, in my opinion, if
> it's
> > >> coupled with progress on some of the open StreamingFileSink tickets.
> > >>
> > >> Best,
> > >> David
> > >>
> > >> [1]
> > >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-46%3A+Graceful+Shutdown+Handling+by+UDFs
> > >>
> > >>
> > >> On Thu, Mar 12, 2020 at 4:27 PM Zhijiang <wangzhijiang999@aliyun.com
> > >> .invalid>
> > >> wrote:
> > >>
> > >> > Thanks for driving this discussion, Robert!
> > >> >
> > >> > This e2e test really fails frequently. +1 to drop bucketing sink,
> it
> > is
> > >> > not worth paying more efforts since deprecated.
> > >> >
> > >> > Best,
> > >> > Zhijiang
> > >> >
> > >> >
> > >> > ------------------------------------------------------------------
> > >> > From:Jeff Zhang <zj...@gmail.com>
> > >> > Send Time:2020 Mar. 12 (Thu.) 23:17
> > >> > To:dev <de...@flink.apache.org>
> > >> > Subject:Re: [DISCUSS] Drop Bucketing Sink
> > >> >
> > >> > +1， dropping deprecated api is always necessary for a sustainable
> > >> project.
> > >> >
> > >> > Kostas Kloudas <kk...@gmail.com> 于2020年3月12日周四 下午11:06写道：
> > >> >
> > >> > > Hi Robert,
> > >> > >
> > >> > > +1 for dropping the BucketingSink.
> > >> > > In any case, it has not been maintained for quite some time.
> > >> > >
> > >> > > Cheers,
> > >> > > Kostas
> > >> > >
> > >> > > On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger <
> rmetzger@apache.org
> > >
> > >> > > wrote:
> > >> > > >
> > >> > > > Hi all,
> > >> > > >
> > >> > > > I'm currently investigating a failing end to end test for the
> > >> bucketing
> > >> > > > sink [1].
> > >> > > > The bucketing sink has been deprecated in the 1.9 release [2],
> > >> because
> > >> > we
> > >> > > > have the new StreamingFileSink [3] for quite a while.
> > >> > > > Before putting any effort into fixing the end to end test for
> the
> > >> > sink, I
> > >> > > > wanted to propose dropping the bucketing sink from master for
> the
> > >> > > upcoming
> > >> > > > 1.11 release.
> > >> > > >
> > >> > > > What do you think?
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > [1] https://issues.apache.org/jira/browse/FLINK-16227
> > >> > > > [2] https://issues.apache.org/jira/browse/FLINK-13396
> > >> > > > [3] https://issues.apache.org/jira/browse/FLINK-9749
> > >> > >
> > >> >
> > >> >
> > >> > --
> > >> > Best Regards
> > >> >
> > >> > Jeff Zhang
> > >> >
> > >> >
> > >>
> > >
> >
>
>
> --
> Best, Jingsong Lee
>
Re: [DISCUSS] Drop Bucketing Sink
Posted by jinhai wang <ji...@gmail.com>.
Hi
+1 to drop bucketing sink
FLIP-115 also needs to be prioritized for 1.11
在 2020/3/13 上午10:07，“Jingsong Li”<ji...@gmail.com> 写入:
Hi Robert,
+1 to drop it but maybe not 1.11.
ORC has not been supported on StreamingFileSink. I have seen lots of users
run ORC in the bucketing sink.
Best,
Jingsong Lee
On Fri, Mar 13, 2020 at 1:11 AM Seth Wiesman <sj...@gmail.com> wrote:
> Sorry, I meant FLIP-46.
>
> Seth
>
> On Thu, Mar 12, 2020 at 11:52 AM Seth Wiesman <sj...@gmail.com> wrote:
>
> > I agree with David, I think FLIP-49 needs to be prioritized for 1.11 if
> we
> > want to drop the bucketing sink.
> >
> > Seth
> >
> > On Thu, Mar 12, 2020 at 10:53 AM David Anderson <da...@ververica.com>
> > wrote:
> >
> >> The BucketingSink is still somewhat widely used, I think in part because
> >> of
> >> shortcomings in the StreamingFileSink.
> >>
> >> I would hope that in tandem with removing the bucketing sink we could
> also
> >> address some of these issues. I'm thinking in particular of issues that
> >> are
> >> waiting on FLIP-46 [1].
> >>
> >> Removing the bucketing sink will go down better, in my opinion, if it's
> >> coupled with progress on some of the open StreamingFileSink tickets.
> >>
> >> Best,
> >> David
> >>
> >> [1]
> >>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-46%3A+Graceful+Shutdown+Handling+by+UDFs
> >>
> >>
> >> On Thu, Mar 12, 2020 at 4:27 PM Zhijiang <wangzhijiang999@aliyun.com
> >> .invalid>
> >> wrote:
> >>
> >> > Thanks for driving this discussion, Robert!
> >> >
> >> > This e2e test really fails frequently. +1 to drop bucketing sink, it
> is
> >> > not worth paying more efforts since deprecated.
> >> >
> >> > Best,
> >> > Zhijiang
> >> >
> >> >
> >> > ------------------------------------------------------------------
> >> > From:Jeff Zhang <zj...@gmail.com>
> >> > Send Time:2020 Mar. 12 (Thu.) 23:17
> >> > To:dev <de...@flink.apache.org>
> >> > Subject:Re: [DISCUSS] Drop Bucketing Sink
> >> >
> >> > +1， dropping deprecated api is always necessary for a sustainable
> >> project.
> >> >
> >> > Kostas Kloudas <kk...@gmail.com> 于2020年3月12日周四 下午11:06写道：
> >> >
> >> > > Hi Robert,
> >> > >
> >> > > +1 for dropping the BucketingSink.
> >> > > In any case, it has not been maintained for quite some time.
> >> > >
> >> > > Cheers,
> >> > > Kostas
> >> > >
> >> > > On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger <rmetzger@apache.org
> >
> >> > > wrote:
> >> > > >
> >> > > > Hi all,
> >> > > >
> >> > > > I'm currently investigating a failing end to end test for the
> >> bucketing
> >> > > > sink [1].
> >> > > > The bucketing sink has been deprecated in the 1.9 release [2],
> >> because
> >> > we
> >> > > > have the new StreamingFileSink [3] for quite a while.
> >> > > > Before putting any effort into fixing the end to end test for the
> >> > sink, I
> >> > > > wanted to propose dropping the bucketing sink from master for the
> >> > > upcoming
> >> > > > 1.11 release.
> >> > > >
> >> > > > What do you think?
> >> > > >
> >> > > >
> >> > > >
> >> > > > [1] https://issues.apache.org/jira/browse/FLINK-16227
> >> > > > [2] https://issues.apache.org/jira/browse/FLINK-13396
> >> > > > [3] https://issues.apache.org/jira/browse/FLINK-9749
> >> > >
> >> >
> >> >
> >> > --
> >> > Best Regards
> >> >
> >> > Jeff Zhang
> >> >
> >> >
> >>
> >
>
--
Best, Jingsong Lee
Re: [DISCUSS] Drop Bucketing Sink
Posted by Jingsong Li <ji...@gmail.com>.
Hi Robert,
+1 to drop it but maybe not 1.11.
ORC has not been supported on StreamingFileSink. I have seen lots of users
run ORC in the bucketing sink.
Best,
Jingsong Lee
On Fri, Mar 13, 2020 at 1:11 AM Seth Wiesman <sj...@gmail.com> wrote:
> Sorry, I meant FLIP-46.
>
> Seth
>
> On Thu, Mar 12, 2020 at 11:52 AM Seth Wiesman <sj...@gmail.com> wrote:
>
> > I agree with David, I think FLIP-49 needs to be prioritized for 1.11 if
> we
> > want to drop the bucketing sink.
> >
> > Seth
> >
> > On Thu, Mar 12, 2020 at 10:53 AM David Anderson <da...@ververica.com>
> > wrote:
> >
> >> The BucketingSink is still somewhat widely used, I think in part because
> >> of
> >> shortcomings in the StreamingFileSink.
> >>
> >> I would hope that in tandem with removing the bucketing sink we could
> also
> >> address some of these issues. I'm thinking in particular of issues that
> >> are
> >> waiting on FLIP-46 [1].
> >>
> >> Removing the bucketing sink will go down better, in my opinion, if it's
> >> coupled with progress on some of the open StreamingFileSink tickets.
> >>
> >> Best,
> >> David
> >>
> >> [1]
> >>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-46%3A+Graceful+Shutdown+Handling+by+UDFs
> >>
> >>
> >> On Thu, Mar 12, 2020 at 4:27 PM Zhijiang <wangzhijiang999@aliyun.com
> >> .invalid>
> >> wrote:
> >>
> >> > Thanks for driving this discussion, Robert!
> >> >
> >> > This e2e test really fails frequently. +1 to drop bucketing sink, it
> is
> >> > not worth paying more efforts since deprecated.
> >> >
> >> > Best,
> >> > Zhijiang
> >> >
> >> >
> >> > ------------------------------------------------------------------
> >> > From:Jeff Zhang <zj...@gmail.com>
> >> > Send Time:2020 Mar. 12 (Thu.) 23:17
> >> > To:dev <de...@flink.apache.org>
> >> > Subject:Re: [DISCUSS] Drop Bucketing Sink
> >> >
> >> > +1， dropping deprecated api is always necessary for a sustainable
> >> project.
> >> >
> >> > Kostas Kloudas <kk...@gmail.com> 于2020年3月12日周四 下午11:06写道：
> >> >
> >> > > Hi Robert,
> >> > >
> >> > > +1 for dropping the BucketingSink.
> >> > > In any case, it has not been maintained for quite some time.
> >> > >
> >> > > Cheers,
> >> > > Kostas
> >> > >
> >> > > On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger <rmetzger@apache.org
> >
> >> > > wrote:
> >> > > >
> >> > > > Hi all,
> >> > > >
> >> > > > I'm currently investigating a failing end to end test for the
> >> bucketing
> >> > > > sink [1].
> >> > > > The bucketing sink has been deprecated in the 1.9 release [2],
> >> because
> >> > we
> >> > > > have the new StreamingFileSink [3] for quite a while.
> >> > > > Before putting any effort into fixing the end to end test for the
> >> > sink, I
> >> > > > wanted to propose dropping the bucketing sink from master for the
> >> > > upcoming
> >> > > > 1.11 release.
> >> > > >
> >> > > > What do you think?
> >> > > >
> >> > > >
> >> > > >
> >> > > > [1] https://issues.apache.org/jira/browse/FLINK-16227
> >> > > > [2] https://issues.apache.org/jira/browse/FLINK-13396
> >> > > > [3] https://issues.apache.org/jira/browse/FLINK-9749
> >> > >
> >> >
> >> >
> >> > --
> >> > Best Regards
> >> >
> >> > Jeff Zhang
> >> >
> >> >
> >>
> >
>
--
Best, Jingsong Lee
Re: [DISCUSS] Drop Bucketing Sink
Posted by Seth Wiesman <sj...@gmail.com>.
Sorry, I meant FLIP-46.
Seth
On Thu, Mar 12, 2020 at 11:52 AM Seth Wiesman <sj...@gmail.com> wrote:
> I agree with David, I think FLIP-49 needs to be prioritized for 1.11 if we
> want to drop the bucketing sink.
>
> Seth
>
> On Thu, Mar 12, 2020 at 10:53 AM David Anderson <da...@ververica.com>
> wrote:
>
>> The BucketingSink is still somewhat widely used, I think in part because
>> of
>> shortcomings in the StreamingFileSink.
>>
>> I would hope that in tandem with removing the bucketing sink we could also
>> address some of these issues. I'm thinking in particular of issues that
>> are
>> waiting on FLIP-46 [1].
>>
>> Removing the bucketing sink will go down better, in my opinion, if it's
>> coupled with progress on some of the open StreamingFileSink tickets.
>>
>> Best,
>> David
>>
>> [1]
>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-46%3A+Graceful+Shutdown+Handling+by+UDFs
>>
>>
>> On Thu, Mar 12, 2020 at 4:27 PM Zhijiang <wangzhijiang999@aliyun.com
>> .invalid>
>> wrote:
>>
>> > Thanks for driving this discussion, Robert!
>> >
>> > This e2e test really fails frequently. +1 to drop bucketing sink, it is
>> > not worth paying more efforts since deprecated.
>> >
>> > Best,
>> > Zhijiang
>> >
>> >
>> > ------------------------------------------------------------------
>> > From:Jeff Zhang <zj...@gmail.com>
>> > Send Time:2020 Mar. 12 (Thu.) 23:17
>> > To:dev <de...@flink.apache.org>
>> > Subject:Re: [DISCUSS] Drop Bucketing Sink
>> >
>> > +1， dropping deprecated api is always necessary for a sustainable
>> project.
>> >
>> > Kostas Kloudas <kk...@gmail.com> 于2020年3月12日周四 下午11:06写道：
>> >
>> > > Hi Robert,
>> > >
>> > > +1 for dropping the BucketingSink.
>> > > In any case, it has not been maintained for quite some time.
>> > >
>> > > Cheers,
>> > > Kostas
>> > >
>> > > On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger <rm...@apache.org>
>> > > wrote:
>> > > >
>> > > > Hi all,
>> > > >
>> > > > I'm currently investigating a failing end to end test for the
>> bucketing
>> > > > sink [1].
>> > > > The bucketing sink has been deprecated in the 1.9 release [2],
>> because
>> > we
>> > > > have the new StreamingFileSink [3] for quite a while.
>> > > > Before putting any effort into fixing the end to end test for the
>> > sink, I
>> > > > wanted to propose dropping the bucketing sink from master for the
>> > > upcoming
>> > > > 1.11 release.
>> > > >
>> > > > What do you think?
>> > > >
>> > > >
>> > > >
>> > > > [1] https://issues.apache.org/jira/browse/FLINK-16227
>> > > > [2] https://issues.apache.org/jira/browse/FLINK-13396
>> > > > [3] https://issues.apache.org/jira/browse/FLINK-9749
>> > >
>> >
>> >
>> > --
>> > Best Regards
>> >
>> > Jeff Zhang
>> >
>> >
>>
>
Re: [DISCUSS] Drop Bucketing Sink
Posted by Seth Wiesman <sj...@gmail.com>.
I agree with David, I think FLIP-49 needs to be prioritized for 1.11 if we
want to drop the bucketing sink.
Seth
On Thu, Mar 12, 2020 at 10:53 AM David Anderson <da...@ververica.com> wrote:
> The BucketingSink is still somewhat widely used, I think in part because of
> shortcomings in the StreamingFileSink.
>
> I would hope that in tandem with removing the bucketing sink we could also
> address some of these issues. I'm thinking in particular of issues that are
> waiting on FLIP-46 [1].
>
> Removing the bucketing sink will go down better, in my opinion, if it's
> coupled with progress on some of the open StreamingFileSink tickets.
>
> Best,
> David
>
> [1]
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-46%3A+Graceful+Shutdown+Handling+by+UDFs
>
>
> On Thu, Mar 12, 2020 at 4:27 PM Zhijiang <wangzhijiang999@aliyun.com
> .invalid>
> wrote:
>
> > Thanks for driving this discussion, Robert!
> >
> > This e2e test really fails frequently. +1 to drop bucketing sink, it is
> > not worth paying more efforts since deprecated.
> >
> > Best,
> > Zhijiang
> >
> >
> > ------------------------------------------------------------------
> > From:Jeff Zhang <zj...@gmail.com>
> > Send Time:2020 Mar. 12 (Thu.) 23:17
> > To:dev <de...@flink.apache.org>
> > Subject:Re: [DISCUSS] Drop Bucketing Sink
> >
> > +1， dropping deprecated api is always necessary for a sustainable
> project.
> >
> > Kostas Kloudas <kk...@gmail.com> 于2020年3月12日周四 下午11:06写道：
> >
> > > Hi Robert,
> > >
> > > +1 for dropping the BucketingSink.
> > > In any case, it has not been maintained for quite some time.
> > >
> > > Cheers,
> > > Kostas
> > >
> > > On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger <rm...@apache.org>
> > > wrote:
> > > >
> > > > Hi all,
> > > >
> > > > I'm currently investigating a failing end to end test for the
> bucketing
> > > > sink [1].
> > > > The bucketing sink has been deprecated in the 1.9 release [2],
> because
> > we
> > > > have the new StreamingFileSink [3] for quite a while.
> > > > Before putting any effort into fixing the end to end test for the
> > sink, I
> > > > wanted to propose dropping the bucketing sink from master for the
> > > upcoming
> > > > 1.11 release.
> > > >
> > > > What do you think?
> > > >
> > > >
> > > >
> > > > [1] https://issues.apache.org/jira/browse/FLINK-16227
> > > > [2] https://issues.apache.org/jira/browse/FLINK-13396
> > > > [3] https://issues.apache.org/jira/browse/FLINK-9749
> > >
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
> >
> >
>
Re: [DISCUSS] Drop Bucketing Sink
Posted by David Anderson <da...@ververica.com>.
The BucketingSink is still somewhat widely used, I think in part because of
shortcomings in the StreamingFileSink.
I would hope that in tandem with removing the bucketing sink we could also
address some of these issues. I'm thinking in particular of issues that are
waiting on FLIP-46 [1].
Removing the bucketing sink will go down better, in my opinion, if it's
coupled with progress on some of the open StreamingFileSink tickets.
Best,
David
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-46%3A+Graceful+Shutdown+Handling+by+UDFs
On Thu, Mar 12, 2020 at 4:27 PM Zhijiang <wa...@aliyun.com.invalid>
wrote:
> Thanks for driving this discussion, Robert!
>
> This e2e test really fails frequently. +1 to drop bucketing sink, it is
> not worth paying more efforts since deprecated.
>
> Best,
> Zhijiang
>
>
> ------------------------------------------------------------------
> From:Jeff Zhang <zj...@gmail.com>
> Send Time:2020 Mar. 12 (Thu.) 23:17
> To:dev <de...@flink.apache.org>
> Subject:Re: [DISCUSS] Drop Bucketing Sink
>
> +1， dropping deprecated api is always necessary for a sustainable project.
>
> Kostas Kloudas <kk...@gmail.com> 于2020年3月12日周四 下午11:06写道：
>
> > Hi Robert,
> >
> > +1 for dropping the BucketingSink.
> > In any case, it has not been maintained for quite some time.
> >
> > Cheers,
> > Kostas
> >
> > On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger <rm...@apache.org>
> > wrote:
> > >
> > > Hi all,
> > >
> > > I'm currently investigating a failing end to end test for the bucketing
> > > sink [1].
> > > The bucketing sink has been deprecated in the 1.9 release [2], because
> we
> > > have the new StreamingFileSink [3] for quite a while.
> > > Before putting any effort into fixing the end to end test for the
> sink, I
> > > wanted to propose dropping the bucketing sink from master for the
> > upcoming
> > > 1.11 release.
> > >
> > > What do you think?
> > >
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/FLINK-16227
> > > [2] https://issues.apache.org/jira/browse/FLINK-13396
> > > [3] https://issues.apache.org/jira/browse/FLINK-9749
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>
>
Re: [DISCUSS] Drop Bucketing Sink
Posted by Zhijiang <wa...@aliyun.com.INVALID>.
Thanks for driving this discussion, Robert!
This e2e test really fails frequently. +1 to drop bucketing sink, it is not worth paying more efforts since deprecated.
Best,
Zhijiang
------------------------------------------------------------------
From:Jeff Zhang <zj...@gmail.com>
Send Time:2020 Mar. 12 (Thu.) 23:17
To:dev <de...@flink.apache.org>
Subject:Re: [DISCUSS] Drop Bucketing Sink
+1， dropping deprecated api is always necessary for a sustainable project.
Kostas Kloudas <kk...@gmail.com> 于2020年3月12日周四 下午11:06写道：
> Hi Robert,
>
> +1 for dropping the BucketingSink.
> In any case, it has not been maintained for quite some time.
>
> Cheers,
> Kostas
>
> On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger <rm...@apache.org>
> wrote:
> >
> > Hi all,
> >
> > I'm currently investigating a failing end to end test for the bucketing
> > sink [1].
> > The bucketing sink has been deprecated in the 1.9 release [2], because we
> > have the new StreamingFileSink [3] for quite a while.
> > Before putting any effort into fixing the end to end test for the sink, I
> > wanted to propose dropping the bucketing sink from master for the
> upcoming
> > 1.11 release.
> >
> > What do you think?
> >
> >
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-16227
> > [2] https://issues.apache.org/jira/browse/FLINK-13396
> > [3] https://issues.apache.org/jira/browse/FLINK-9749
>
--
Best Regards
Jeff Zhang
Re: [DISCUSS] Drop Bucketing Sink
Posted by Aljoscha Krettek <al...@apache.org>.
+1
Aljoscha
Re: [DISCUSS] Drop Bucketing Sink
Posted by Jeff Zhang <zj...@gmail.com>.
+1， dropping deprecated api is always necessary for a sustainable project.
Kostas Kloudas <kk...@gmail.com> 于2020年3月12日周四 下午11:06写道：
> Hi Robert,
>
> +1 for dropping the BucketingSink.
> In any case, it has not been maintained for quite some time.
>
> Cheers,
> Kostas
>
> On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger <rm...@apache.org>
> wrote:
> >
> > Hi all,
> >
> > I'm currently investigating a failing end to end test for the bucketing
> > sink [1].
> > The bucketing sink has been deprecated in the 1.9 release [2], because we
> > have the new StreamingFileSink [3] for quite a while.
> > Before putting any effort into fixing the end to end test for the sink, I
> > wanted to propose dropping the bucketing sink from master for the
> upcoming
> > 1.11 release.
> >
> > What do you think?
> >
> >
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-16227
> > [2] https://issues.apache.org/jira/browse/FLINK-13396
> > [3] https://issues.apache.org/jira/browse/FLINK-9749
>
--
Best Regards
Jeff Zhang
Re: [DISCUSS] Drop Bucketing Sink
Posted by Kostas Kloudas <kk...@gmail.com>.
Hi Robert,
+1 for dropping the BucketingSink.
In any case, it has not been maintained for quite some time.
Cheers,
Kostas
On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger <rm...@apache.org> wrote:
>
> Hi all,
>
> I'm currently investigating a failing end to end test for the bucketing
> sink [1].
> The bucketing sink has been deprecated in the 1.9 release [2], because we
> have the new StreamingFileSink [3] for quite a while.
> Before putting any effort into fixing the end to end test for the sink, I
> wanted to propose dropping the bucketing sink from master for the upcoming
> 1.11 release.
>
> What do you think?
>
>
>
> [1] https://issues.apache.org/jira/browse/FLINK-16227
> [2] https://issues.apache.org/jira/browse/FLINK-13396
> [3] https://issues.apache.org/jira/browse/FLINK-9749
Re: [DISCUSS] Drop Bucketing Sink
Posted by Hequn Cheng <ch...@gmail.com>.
Good idea! +1 for dropping the BucketingSink.
Best,
Hequn
> On Mar 12, 2020, at 10:40 PM, Robert Metzger <rm...@apache.org> wrote:
>
> Hi all,
>
> I'm currently investigating a failing end to end test for the bucketing
> sink [1].
> The bucketing sink has been deprecated in the 1.9 release [2], because we
> have the new StreamingFileSink [3] for quite a while.
> Before putting any effort into fixing the end to end test for the sink, I
> wanted to propose dropping the bucketing sink from master for the upcoming
> 1.11 release.
>
> What do you think?
>
>
>
> [1] https://issues.apache.org/jira/browse/FLINK-16227
> [2] https://issues.apache.org/jira/browse/FLINK-13396
> [3] https://issues.apache.org/jira/browse/FLINK-9749