You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user-zh@flink.apache.org by haishui <ha...@126.com> on 2023/03/17 02:47:58 UTC

水位线对齐与空闲问题

hi,


我在1.15.x和1.16.1对水位线策略进行测试发现水位线对齐和idleness同时使用会造成变成空闲的source无法再消费kafka数据。这是一个bug吗？
我的水位线策略如下：
           WatermarkStrategy
                .<String>forBoundedOutOfOrderness(Duration.ofMillis(0))
                .withTimestampAssigner((element, recordTimestamp) -> Long.parseLong(element))
                .withWatermarkAlignment("default", Duration.ofMillis(10), Duration.ofSeconds(2))
                .withIdleness(Duration.ofSeconds(30));


在经过30s以后，水位线对齐的日志如下：
10:31:25,717 INFO  org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Distributing maxAllowedWatermark=11 to subTaskIds=[0]
10:31:25,717 INFO  org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Distributing maxAllowedWatermark=11 to subTaskIds=[0]
10:31:27,710 INFO  org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Distributing maxAllowedWatermark=-9223372036854775799 to subTaskIds=[0]
10:31:27,710 INFO  org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Distributing maxAllowedWatermark=-9223372036854775799 to subTaskIds=[0]


此时再向topic里面写数据已经不再消费