You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Xu Mingmin (JIRA)" <ji...@apache.org> on 2017/03/21 15:55:41 UTC
[jira] [Created] (BEAM-1775) fix issue of
start_from_previous_offset in KafkaIO
Xu Mingmin created BEAM-1775:
--------------------------------
Summary: fix issue of start_from_previous_offset in KafkaIO
Key: BEAM-1775
URL: https://issues.apache.org/jira/browse/BEAM-1775
Project: Beam
Issue Type: Improvement
Components: sdk-java-extensions
Reporter: Xu Mingmin
Assignee: Xu Mingmin
Jins George jins.george@aeris.net via aermail.onmicrosoft.com
5:50 PM (15 hours ago)
to user
Hello,
I am writing a Beam pipeline(streaming) with Flink runner to consume data from Kafka and apply some transformations and persist to Hbase.
If I restart the application ( due to failure/manual restart), consumer does not resume from the offset where it was prior to restart. It always resume from the latest offset.
If I enable Flink checkpionting with hdfs state back-end, system appears to be resuming from the earliest offset
Is there a recommended way to resume from the offset where it was stopped ?
Thanks,
Jins George
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)