You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Jingsong Lee (JIRA)" <ji...@apache.org> on 2017/06/28 00:57:00 UTC
[jira] [Commented] (BEAM-2393) BoundedSource is not fault-tolerant
in FlinkRunner Streaming mode
[ https://issues.apache.org/jira/browse/BEAM-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16065750#comment-16065750 ]
Jingsong Lee commented on BEAM-2393:
------------------------------------
Now the {{UnboundedSourceWrapper}} has already supported the exit when the watermark exceeds TIMESTAMP_MAX_VALUE.
So can we use {{BoundedToUnboundedSourceAdapter}}?
bq. Checkpoints are created by calling {{BoundedReader#splitAtFraction}} on inner {{BoundedSource}}.
bq. Sources that cannot be split are read entirely into memory, so this transform does not work well with large, unsplittable sources.
But at least we can provide an accurate semantics.
> BoundedSource is not fault-tolerant in FlinkRunner Streaming mode
> -----------------------------------------------------------------
>
> Key: BEAM-2393
> URL: https://issues.apache.org/jira/browse/BEAM-2393
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: Jingsong Lee
>
> {{BoundedSourceWrapper}} does not implement snapshot() and restore(), when the failure to restart, it will send duplicate data.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)