You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "David Anderson (Jira)" <ji...@apache.org> on 2020/09/01 07:34:00 UTC
[jira] [Updated] (FLINK-19109) Split Reader eats chained periodic
watermarks
[ https://issues.apache.org/jira/browse/FLINK-19109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Anderson updated FLINK-19109:
-----------------------------------
Description:
Attempting to generate watermarks chained to the Split Reader / ContinuousFileReaderOperator, as in
{code:java}
SingleOutputStreamOperator<Event> results = env
.readTextFile(...)
.map(...)
.assignTimestampsAndWatermarks(bounded)
.keyBy(...)
.process(...);{code}
leads to the Watermarks failing to be produced. Breaking the chain, via {{disableOperatorChaining()}} or a {{rebalance}}, works around the bug. Using punctuated watermarks also avoids the issue.
Looking at this in the debugger reveals that timer service is being prematurely quiesced.
In many respects this is FLINK-7666 brought back to life.
The problem is not present in 1.9.3.
There's a minimal reproducible example in [https://github.com/alpinegizmo/flink-question-001/tree/bug].
was:
Attempting to generate watermarks chained to the Split Reader / ContinuousFileReaderOperator, as in
{code:java}
SingleOutputStreamOperator<Event> results = env
.readTextFile(...)
.map(...)
.assignTimestampsAndWatermarks(bounded)
.keyBy(...)
.process(...);{code}
leads to the Watermarks failing to be produced. Breaking the chain, via {{disableOperatorChaining()}} or a {{rebalance}}, works around the bug. Using punctuated watermarks also avoids the issue.
Looking at this in the debugger reveals that timer service is being prematurely quiesced.
In many respects this is FLINK-7666 brought back to life.
The problem is not present in 1.9.3.
There's a minimal reproducible example in [https://github.com/alpinegizmo/flink-question-001/tree/bug].
> Split Reader eats chained periodic watermarks
> ---------------------------------------------
>
> Key: FLINK-19109
> URL: https://issues.apache.org/jira/browse/FLINK-19109
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.10.0, 1.10.1, 1.11.0, 1.10.2, 1.11.1
> Reporter: David Anderson
> Priority: Critical
>
> Attempting to generate watermarks chained to the Split Reader / ContinuousFileReaderOperator, as in
> {code:java}
> SingleOutputStreamOperator<Event> results = env
> .readTextFile(...)
> .map(...)
> .assignTimestampsAndWatermarks(bounded)
> .keyBy(...)
> .process(...);{code}
> leads to the Watermarks failing to be produced. Breaking the chain, via {{disableOperatorChaining()}} or a {{rebalance}}, works around the bug. Using punctuated watermarks also avoids the issue.
> Looking at this in the debugger reveals that timer service is being prematurely quiesced.
> In many respects this is FLINK-7666 brought back to life.
> The problem is not present in 1.9.3.
> There's a minimal reproducible example in [https://github.com/alpinegizmo/flink-question-001/tree/bug].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)