You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Daniel Halperin (JIRA)" <ji...@apache.org> on 2016/12/02 20:05:58 UTC

[jira] [Resolved] (BEAM-223) KafkaIO: don't use SerializableCoder

     [ https://issues.apache.org/jira/browse/BEAM-223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Halperin resolved BEAM-223.
----------------------------------
       Resolution: Fixed
    Fix Version/s: 0.4.0-incubating

> KafkaIO: don't use SerializableCoder
> ------------------------------------
>
>                 Key: BEAM-223
>                 URL: https://issues.apache.org/jira/browse/BEAM-223
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-extensions
>            Reporter: Daniel Halperin
>            Assignee: Raghu Angadi
>             Fix For: 0.4.0-incubating
>
>
> Reuven says:
> {quote}
> I noticed that we're using SerializableCoder for the checkpoint mark in KafkaIO. This is generally highly discouraged in streaming pipelines. Partially because it's inefficient, but more importantly because Java serialization is not guaranteed to be stable. If a user updates their pipeline, the new pipeline may not be able to decode the existing checkpoint marks; this will either cause exceptions to be thrown, or data loss.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)