[jira] [Commented] (FLINK-16112) KafkaDeserializationSchema#isEndOfStream is ill-defined

["Dawid Wysakowicz (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-16112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038157#comment-17038157 ] 

Dawid Wysakowicz commented on FLINK-16112:
------------------------------------------

I think a better solution would be to split that apart. I don't think this method fits into the {{DeserializationSchema}} interface. 

Right now the {{KafkaDeserializationSchema}} mixes two different aspects: deserialization with user defined logic. I suggest moving that method to the {{KafkaSource}} itself.

> KafkaDeserializationSchema#isEndOfStream is ill-defined
> -------------------------------------------------------
>
>                 Key: FLINK-16112
>                 URL: https://issues.apache.org/jira/browse/FLINK-16112
>             Project: Flink
>          Issue Type: Improvement
>          Components: Connectors / Kafka
>            Reporter: Tzu-Li (Gordon) Tai
>            Priority: Major
>
> Motivated by this email thread: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/KafkaFetcher-closed-before-end-of-stream-is-received-for-all-partitions-td32827.html
> In general, the {{isEndOfStream}} method is ill-defined. An possible improvement is to redefine the semantics and method signature, to:
> {code}
> boolean isEndOfPartition(T nextElement, String topic, int partition);
> {code}
> so that it indicates end of a Kafka partition. With this, the Kafka consumer is able to properly define end-of-subtask to be when all of its assigned partitions have signaled EOF.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)