You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Yuval Itzchakov <yu...@gmail.com> on 2020/04/09 17:52:54 UTC
Passing checkpoint lock object to StreamSourceContexts.getSourceContext
after StreamTask.getCheckpointLock deprecation
Hi,
I have an implementation of a custom source, which uses
StreamSourceContexts.getSourceContext (
https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/StreamSourceContexts.java#L46)
which has a checkpointLock argument that's used for watermark emission and
possibly additional work.
Flink 1.10 marks StreamTask.getCheckpointLock as deprecated and points to
using YieldingOperatorFactory instead.
Question is, with the above method now being deprecated, which object
should we pass to getSourceContext for the required checkpointLock?
--
Best Regards,
Yuval Itzchakov.
Re: Passing checkpoint lock object to StreamSourceContexts.getSourceContext
after StreamTask.getCheckpointLock deprecation
Posted by Arvid Heise <ar...@ververica.com>.
Hi Yuval,
sorry for the late response on the mailing list.
I'm assuming your stackoverflow post is referring to the same question and
would just cross-reference it here. [1]
[1]
https://stackoverflow.com/questions/61141240/streamtask-getcheckpointlock-deprecation-and-custom-flink-sources
On Thu, Apr 9, 2020 at 7:56 PM Yuval Itzchakov <yu...@gmail.com> wrote:
> One thing that had just occurred to me is that the context is later used
> to emit watermarks and send elements downstream using ctx.collect. Perhaps
> all these operations should now be switched to the new mailbox executor
> instead?
>
> On Thu, Apr 9, 2020 at 8:52 PM Yuval Itzchakov <yu...@gmail.com> wrote:
>
>> Hi,
>>
>> I have an implementation of a custom source, which uses
>> StreamSourceContexts.getSourceContext (
>> https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/StreamSourceContexts.java#L46)
>> which has a checkpointLock argument that's used for watermark emission and
>> possibly additional work.
>>
>> Flink 1.10 marks StreamTask.getCheckpointLock as deprecated and points to
>> using YieldingOperatorFactory instead.
>>
>> Question is, with the above method now being deprecated, which object
>> should we pass to getSourceContext for the required checkpointLock?
>>
>> --
>> Best Regards,
>> Yuval Itzchakov.
>>
>
>
> --
> Best Regards,
> Yuval Itzchakov.
>
--
Arvid Heise | Senior Java Developer
<https://www.ververica.com/>
Follow us @VervericaData
--
Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Toni) Cheng
Re: Passing checkpoint lock object to StreamSourceContexts.getSourceContext
after StreamTask.getCheckpointLock deprecation
Posted by Yuval Itzchakov <yu...@gmail.com>.
One thing that had just occurred to me is that the context is later used to
emit watermarks and send elements downstream using ctx.collect. Perhaps all
these operations should now be switched to the new mailbox executor instead?
On Thu, Apr 9, 2020 at 8:52 PM Yuval Itzchakov <yu...@gmail.com> wrote:
> Hi,
>
> I have an implementation of a custom source, which uses
> StreamSourceContexts.getSourceContext (
> https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/StreamSourceContexts.java#L46)
> which has a checkpointLock argument that's used for watermark emission and
> possibly additional work.
>
> Flink 1.10 marks StreamTask.getCheckpointLock as deprecated and points to
> using YieldingOperatorFactory instead.
>
> Question is, with the above method now being deprecated, which object
> should we pass to getSourceContext for the required checkpointLock?
>
> --
> Best Regards,
> Yuval Itzchakov.
>
--
Best Regards,
Yuval Itzchakov.