You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Dong Lin <li...@gmail.com> on 2022/12/21 07:13:02 UTC
[VOTE] FLIP-208: Add RecordEvaluator to dynamically stop source based on de-serialized records
Hi all,
We would like to start the vote for FLIP-208: Add RecordEvaluator to
dynamically stop source based on de-serialized records [1]. This FLIP was
discussed in this thread [2].
This feature is needed by users who currently depend on
KafkaDeserializationSchema::isEndOfStream() to migrate their Flink job from
FlinkKafkaConsumer to KafkaSource. And we won't be able to
remove FlinkKafkaConsumer and FlinkKafkaProducer before adding this feature.
Thanks,
Dong
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-208%3A+Add+RecordEvaluator+to+dynamically+stop+source+based+on+de-serialized+records
[2] https://lists.apache.org/thread/z87m68ggzkx0s427tmrllswm4l1g7owc
Re: [VOTE] FLIP-208: Add RecordEvaluator to dynamically stop source based on de-serialized records
Posted by Qingsheng Ren <re...@gmail.com>.
Thanks for the update, Dong!
+1 for the new table option.
Best,
Qingsheng
On Mon, Feb 6, 2023 at 2:39 PM Dong Lin <li...@gmail.com> wrote:
> As I am reviewing the FLIP-208 PR here
> <https://github.com/apache/flink/pull/>, I realized that a new public API
> change was added to the Proposed Change section in this
> <https://github.com/apache/flink/pull/> modification after this voting
> thread starts, without being mentioned in this thread.
>
> I have moved this change to the Public Interface section now. The change is
> that "a new connector option 'scan.record.evaluator.class' will be added to
> provide the custom RecordEvaluator class".
>
> Since this change is relatively minor and looks good to me, I will re-use
> this voting thread to confirm this change is OK.
>
> Qingsheng and Leonard: can you help check whether this public interface
> change looks good to you?
>
> I will keep this discussion open for at least 72 hours before merging the
> PR.
>
> Thanks,
> Dong
>
>
> On Tue, Dec 27, 2022 at 3:29 PM Leonard Xu <xb...@gmail.com> wrote:
>
> > +1 (binding)
> >
> > Best,
> > Leonard
> >
> >
> > > On Dec 26, 2022, at 4:22 PM, Qingsheng Ren <re...@gmail.com> wrote:
> > >
> > > +1 (binding)
> > >
> > > Best,
> > > Qingsheng
> > > Ververica (Alibaba)
> > >
> > > On Wed, Dec 21, 2022 at 3:13 PM Dong Lin <li...@gmail.com> wrote:
> > >
> > >> Hi all,
> > >>
> > >> We would like to start the vote for FLIP-208: Add RecordEvaluator to
> > >> dynamically stop source based on de-serialized records [1]. This FLIP
> > was
> > >> discussed in this thread [2].
> > >>
> > >> This feature is needed by users who currently depend on
> > >> KafkaDeserializationSchema::isEndOfStream() to migrate their Flink job
> > from
> > >> FlinkKafkaConsumer to KafkaSource. And we won't be able to
> > >> remove FlinkKafkaConsumer and FlinkKafkaProducer before adding this
> > >> feature.
> > >>
> > >> Thanks,
> > >> Dong
> > >>
> > >> [1]
> > >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-208%3A+Add+RecordEvaluator+to+dynamically+stop+source+based+on+de-serialized+records
> > >> [2] https://lists.apache.org/thread/z87m68ggzkx0s427tmrllswm4l1g7owc
> > >>
> >
> >
>
Re: [VOTE] FLIP-208: Add RecordEvaluator to dynamically stop source based on de-serialized records
Posted by Leonard Xu <xb...@gmail.com>.
> Since this change is relatively minor and looks good to me, I will re-use this voting thread to confirm this change is OK.
>
> Qingsheng and Leonard: can you help check whether this public interface change looks good to you?
>
Thanks Dong and Hang for driving this.
The change looks good to me, and +1 from my side.
Best,
Leonard
Re: [VOTE] FLIP-208: Add RecordEvaluator to dynamically stop source based on de-serialized records
Posted by Dong Lin <li...@gmail.com>.
As I am reviewing the FLIP-208 PR here
<https://github.com/apache/flink/pull/>, I realized that a new public API
change was added to the Proposed Change section in this
<https://github.com/apache/flink/pull/> modification after this voting
thread starts, without being mentioned in this thread.
I have moved this change to the Public Interface section now. The change is
that "a new connector option 'scan.record.evaluator.class' will be added to
provide the custom RecordEvaluator class".
Since this change is relatively minor and looks good to me, I will re-use
this voting thread to confirm this change is OK.
Qingsheng and Leonard: can you help check whether this public interface
change looks good to you?
I will keep this discussion open for at least 72 hours before merging the
PR.
Thanks,
Dong
On Tue, Dec 27, 2022 at 3:29 PM Leonard Xu <xb...@gmail.com> wrote:
> +1 (binding)
>
> Best,
> Leonard
>
>
> > On Dec 26, 2022, at 4:22 PM, Qingsheng Ren <re...@gmail.com> wrote:
> >
> > +1 (binding)
> >
> > Best,
> > Qingsheng
> > Ververica (Alibaba)
> >
> > On Wed, Dec 21, 2022 at 3:13 PM Dong Lin <li...@gmail.com> wrote:
> >
> >> Hi all,
> >>
> >> We would like to start the vote for FLIP-208: Add RecordEvaluator to
> >> dynamically stop source based on de-serialized records [1]. This FLIP
> was
> >> discussed in this thread [2].
> >>
> >> This feature is needed by users who currently depend on
> >> KafkaDeserializationSchema::isEndOfStream() to migrate their Flink job
> from
> >> FlinkKafkaConsumer to KafkaSource. And we won't be able to
> >> remove FlinkKafkaConsumer and FlinkKafkaProducer before adding this
> >> feature.
> >>
> >> Thanks,
> >> Dong
> >>
> >> [1]
> >>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-208%3A+Add+RecordEvaluator+to+dynamically+stop+source+based+on+de-serialized+records
> >> [2] https://lists.apache.org/thread/z87m68ggzkx0s427tmrllswm4l1g7owc
> >>
>
>
Re: [VOTE] FLIP-208: Add RecordEvaluator to dynamically stop source based on de-serialized records
Posted by Leonard Xu <xb...@gmail.com>.
+1 (binding)
Best,
Leonard
> On Dec 26, 2022, at 4:22 PM, Qingsheng Ren <re...@gmail.com> wrote:
>
> +1 (binding)
>
> Best,
> Qingsheng
> Ververica (Alibaba)
>
> On Wed, Dec 21, 2022 at 3:13 PM Dong Lin <li...@gmail.com> wrote:
>
>> Hi all,
>>
>> We would like to start the vote for FLIP-208: Add RecordEvaluator to
>> dynamically stop source based on de-serialized records [1]. This FLIP was
>> discussed in this thread [2].
>>
>> This feature is needed by users who currently depend on
>> KafkaDeserializationSchema::isEndOfStream() to migrate their Flink job from
>> FlinkKafkaConsumer to KafkaSource. And we won't be able to
>> remove FlinkKafkaConsumer and FlinkKafkaProducer before adding this
>> feature.
>>
>> Thanks,
>> Dong
>>
>> [1]
>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-208%3A+Add+RecordEvaluator+to+dynamically+stop+source+based+on+de-serialized+records
>> [2] https://lists.apache.org/thread/z87m68ggzkx0s427tmrllswm4l1g7owc
>>
Re: [VOTE] FLIP-208: Add RecordEvaluator to dynamically stop source based on de-serialized records
Posted by Qingsheng Ren <re...@gmail.com>.
+1 (binding)
Best,
Qingsheng
Ververica (Alibaba)
On Wed, Dec 21, 2022 at 3:13 PM Dong Lin <li...@gmail.com> wrote:
> Hi all,
>
> We would like to start the vote for FLIP-208: Add RecordEvaluator to
> dynamically stop source based on de-serialized records [1]. This FLIP was
> discussed in this thread [2].
>
> This feature is needed by users who currently depend on
> KafkaDeserializationSchema::isEndOfStream() to migrate their Flink job from
> FlinkKafkaConsumer to KafkaSource. And we won't be able to
> remove FlinkKafkaConsumer and FlinkKafkaProducer before adding this
> feature.
>
> Thanks,
> Dong
>
> [1]
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-208%3A+Add+RecordEvaluator+to+dynamically+stop+source+based+on+de-serialized+records
> [2] https://lists.apache.org/thread/z87m68ggzkx0s427tmrllswm4l1g7owc
>
Re: [VOTE] FLIP-208: Add RecordEvaluator to dynamically stop source based on de-serialized records
Posted by Dong Lin <li...@gmail.com>.
The vote will last for at least 72 hours (Dec 26 at 8 am UTC, excluding weekend
days). It needs consensus approval, requiring 3 binding +1 votes and no
binding vetoes.
On Wed, Dec 21, 2022 at 3:13 PM Dong Lin <li...@gmail.com> wrote:
> Hi all,
>
> We would like to start the vote for FLIP-208: Add RecordEvaluator to
> dynamically stop source based on de-serialized records [1]. This FLIP was
> discussed in this thread [2].
>
> This feature is needed by users who currently depend on
> KafkaDeserializationSchema::isEndOfStream() to migrate their Flink job from
> FlinkKafkaConsumer to KafkaSource. And we won't be able to
> remove FlinkKafkaConsumer and FlinkKafkaProducer before adding this feature.
>
> Thanks,
> Dong
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-208%3A+Add+RecordEvaluator+to+dynamically+stop+source+based+on+de-serialized+records
> [2] https://lists.apache.org/thread/z87m68ggzkx0s427tmrllswm4l1g7owc
>
>
>
>
Re: [VOTE] FLIP-208: Add RecordEvaluator to dynamically stop source based on de-serialized records
Posted by Hang Ruan <ru...@gmail.com>.
+1 (non-binding)
Best,
Hang
Dong Lin <li...@gmail.com> 于2022年12月21日周三 15:13写道：
> Hi all,
>
> We would like to start the vote for FLIP-208: Add RecordEvaluator to
> dynamically stop source based on de-serialized records [1]. This FLIP was
> discussed in this thread [2].
>
> This feature is needed by users who currently depend on
> KafkaDeserializationSchema::isEndOfStream() to migrate their Flink job from
> FlinkKafkaConsumer to KafkaSource. And we won't be able to
> remove FlinkKafkaConsumer and FlinkKafkaProducer before adding this
> feature.
>
> Thanks,
> Dong
>
> [1]
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-208%3A+Add+RecordEvaluator+to+dynamically+stop+source+based+on+de-serialized+records
> [2] https://lists.apache.org/thread/z87m68ggzkx0s427tmrllswm4l1g7owc
>