You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "Tzu-Li (Gordon) Tai (JIRA)" <ji...@apache.org> on 2017/04/18 00:53:42 UTC
[jira] [Created] (FLINK-6316) Kinesis connector docs should mention
that FlinkKinesisConsumer does not provide strong ordering guarantees on
resharding
Tzu-Li (Gordon) Tai created FLINK-6316:
------------------------------------------
Summary: Kinesis connector docs should mention that FlinkKinesisConsumer does not provide strong ordering guarantees on resharding
Key: FLINK-6316
URL: https://issues.apache.org/jira/browse/FLINK-6316
Project: Flink
Issue Type: Improvement
Components: Documentation, Kinesis Connector
Reporter: Tzu-Li (Gordon) Tai
Since the {{FlinkKinesisConsumer}} depends only on local information to determine whether or not a new shard due to resharding should be subscribed by a subtask, there is no coordination wrt parent-child shard relationship across subtasks.
Therefore, {{FlinkKinesisConsumer}} does not provide any strong processing ordering guarantees.
Take for example, if initially the assignment is:
Subtask #1 --> Shard A
Subtask #2 --> Shard B
Assume A & B is merged to create shard C, and subtask #1 locally determines that it should be assigned shard C.
Since Flink generally does not provide coordinating facilities between subtask instances, there is no means of coordinating that shard C is consumed only after shard B is also fully consumed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)