You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Jark Wu (Jira)" <ji...@apache.org> on 2021/10/22 16:55:00 UTC
[jira] [Updated] (FLINK-24622) Unified sources do not close
scheduled threads from SplitEnumeratorContext#callAsync()
[ https://issues.apache.org/jira/browse/FLINK-24622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jark Wu updated FLINK-24622:
----------------------------
Component/s: Runtime / Coordination
> Unified sources do not close scheduled threads from SplitEnumeratorContext#callAsync()
> --------------------------------------------------------------------------------------
>
> Key: FLINK-24622
> URL: https://issues.apache.org/jira/browse/FLINK-24622
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Common, Runtime / Coordination
> Affects Versions: 1.13.2
> Reporter: Mason Chen
> Priority: Major
>
> From user mailing list:
> I was wondering how to cancel a task that is enqueued by the callAsync() method, the one that takes in a time interval. For example, the KafkaSource uses this for topic partition discovery. It would be straightforward if the API returned the underlying future so that a process can cancel it.
> For Kafka, the enumerator shutdown seems to be unclean since it only closes the admin client and kafka consumer but not the topic partition discovery task. Furthermore, exceptions from that task will cause job failure and can potentially happen if the task is still running with the admin client closed. How can we address this?
> This seems to be a bug with the current KafkaSource and also the unified Sources in general. Can you open a bug ticket in jira? I think the enumerator should take of first joining all the async threads before closing the enumerator.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)