You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by 네이버 <k5...@naver.com> on 2018/08/08 06:18:30 UTC
unsubscribtion
> On 7 Aug 2018, at 19:42, Yan Zhou [FDS Science] <yz...@coupang.com> wrote:
>
> Thank you Vino. It is very helpful.
> From: vino yang <ya...@gmail.com>
> Sent: Tuesday, August 7, 2018 7:22:50 PM
> To: Yan Zhou [FDS Science]
> Cc: user
> Subject: Re: checkpoint recovery behavior when kafka source is set to start from timestamp
>
> Hi Yan Zhou:
>
> I think the java doc of the setStartFromTimestamp method has been explained very clearly, posted here:
>
> /**
> * Specify the consumer to start reading partitions from a specified timestamp.
> * The specified timestamp must be before the current timestamp.
> * This lets the consumer ignore any committed group offsets in Zookeeper / Kafka brokers.
> *
> * <p>The consumer will look up the earliest offset whose timestamp is greater than or equal
> * to the specific timestamp from Kafka. If there's no such offset, the consumer will use the
> * latest offset to read data from kafka.
> *
> * <p>This method does not affect where partitions are read from when the consumer is restored
> * from a checkpoint or savepoint. When the consumer is restored from a checkpoint or
> * savepoint, only the offsets in the restored state will be used.
> *
> * @param startupOffsetsTimestamp timestamp for the startup offsets, as milliseconds from epoch.
> *
> * @return The consumer object, to allow function chaining.
> */
>
> Thanks, vino.
>
> Yan Zhou [FDS Science] <yz...@coupang.com> 于2018年8月8日周三 上午9:06写道:
> Hi Experts,
>
> In my application, the kafka source is set to start from a specified timestamp, by calling method FlinkKafkaConsumer010#setStartFromTimestamp(long startupOffsetsTimestamp).
>
> If the application have run a while and then recover from a checkpoint because of failure, what's the offset will the kafka source to read from? I suppose it will read from the offset that has been committed before the failure. Is it right?
>
> I am going to verify it, however some clarification is good in case my test result doesn't meet my assumption.
>
> Best
> Yan
>
Re: unsubscribtion
Posted by Timo Walther <tw...@apache.org>.
Hi,
see https://flink.apache.org/community.html#mailing-lists for unsubscribing:
Use:
user-unsubscribe@flink.apache.org
Regards,
Timo
Am 08.08.18 um 08:18 schrieb 네이버:
>
>
> On 7 Aug 2018, at 19:42, Yan Zhou [FDS Science] <yzhou@coupang.com
> <ma...@coupang.com>> wrote:
>
>> Thank you Vino. It is very helpful.
>>
>> ------------------------------------------------------------------------
>> *From:* vino yang <yanghua1127@gmail.com <ma...@gmail.com>>
>> *Sent:* Tuesday, August 7, 2018 7:22:50 PM
>> *To:* Yan Zhou [FDS Science]
>> *Cc:* user
>> *Subject:* Re: checkpoint recovery behavior when kafka source is set
>> to start from timestamp
>> Hi Yan Zhou:
>>
>> I think the java doc of the setStartFromTimestamp method has been
>> explained very clearly, posted here:
>>
>> /*/***/
>> /** Specify the consumer to start reading partitions from a specified
>> timestamp.*/
>> /** The specified timestamp must be before the current timestamp.*/
>> /** This lets the consumer ignore any committed group offsets in
>> Zookeeper / Kafka brokers.*/
>> /***/
>> /** <p>The consumer will look up the earliest offset whose timestamp
>> is greater than or equal*/
>> /** to the specific timestamp from Kafka. If there's no such offset,
>> the consumer will use the*/
>> /** latest offset to read data from kafka.*/
>> /***/
>> /** <p>This method does not affect where partitions are read from
>> when the consumer is restored*/
>> /** from a checkpoint or savepoint. When the consumer is restored
>> from a checkpoint or*/
>> /** savepoint, only the offsets in the restored state will be used.*/
>> /***/
>> /** @param startupOffsetsTimestamp timestamp for the startup offsets,
>> as milliseconds from epoch.*/
>> /***/
>> /** @return The consumer object, to allow function chaining.*/
>> /**/*/
>>
>> Thanks, vino.
>>
>> Yan Zhou [FDS Science] <yzhou@coupang.com <ma...@coupang.com>>
>> 于2018年8月8日周三 上午9:06写道:
>>
>> Hi Experts,
>>
>>
>> In my application, the kafka source is set to start from a
>> specified timestamp, by calling method
>> FlinkKafkaConsumer010#setStartFromTimestamp(long
>> startupOffsetsTimestamp).
>>
>>
>> If the application have run a while and then recover from a
>> checkpoint because of failure, what's the offset will the kafka
>> source to read from? I suppose it will read from the offset that
>> has been committed before the failure. Is it right?
>>
>>
>> I am going to verify it, however some clarification is good in
>> case my test result doesn't meet my assumption.
>>
>>
>> Best
>>
>> Yan
>>
>>