You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Jerry Peng <je...@gmail.com> on 2015/12/01 23:04:11 UTC
Question about flink message processing guarantee
Hello,
I have a question regarding link streaming. I now if you enable
checkpointing you can have exactly once processing guarantee. If I do
not enable checkpointing what are the semantics of the processing? At
least once?
Best,
Jerry
Re: Question about flink message processing guarantee
Posted by Stephan Ewen <se...@apache.org>.
There is an overview of what guarantees what sources can give you:
https://ci.apache.org/projects/flink/flink-docs-master/apis/fault_tolerance.html#fault-tolerance-guarantees-of-data-sources-and-sinks
On Wed, Dec 2, 2015 at 9:19 AM, Till Rohrmann <ti...@gmail.com>
wrote:
> Just a small addition. Your sources have to be replayable to some extent.
> With replayable I mean that they can continue from some kind of offset.
> Otherwise the check pointing won't help you. The Kafka source supports that
> for example.
>
> Cheers,
> Till
> On Dec 1, 2015 11:55 PM, "Márton Balassi" <ba...@gmail.com>
> wrote:
>
>> Dear Jerry,
>>
>> If you do not enable checkpointing you get the at most once processing
>> guarantee (some might call that no guarantee at all). When you enable
>> checkpointing you can choose between exactly and at least once semantics.
>> The latter provides better latency.
>>
>> Best,
>>
>> Marton
>>
>> On Tue, Dec 1, 2015 at 11:04 PM, Jerry Peng <je...@gmail.com>
>> wrote:
>>
>>> Hello,
>>>
>>> I have a question regarding link streaming. I now if you enable
>>> checkpointing you can have exactly once processing guarantee. If I do
>>> not enable checkpointing what are the semantics of the processing? At
>>> least once?
>>>
>>> Best,
>>>
>>> Jerry
>>>
>>
>>
Re: Question about flink message processing guarantee
Posted by Till Rohrmann <ti...@gmail.com>.
Just a small addition. Your sources have to be replayable to some extent.
With replayable I mean that they can continue from some kind of offset.
Otherwise the check pointing won't help you. The Kafka source supports that
for example.
Cheers,
Till
On Dec 1, 2015 11:55 PM, "Márton Balassi" <ba...@gmail.com> wrote:
> Dear Jerry,
>
> If you do not enable checkpointing you get the at most once processing
> guarantee (some might call that no guarantee at all). When you enable
> checkpointing you can choose between exactly and at least once semantics.
> The latter provides better latency.
>
> Best,
>
> Marton
>
> On Tue, Dec 1, 2015 at 11:04 PM, Jerry Peng <je...@gmail.com>
> wrote:
>
>> Hello,
>>
>> I have a question regarding link streaming. I now if you enable
>> checkpointing you can have exactly once processing guarantee. If I do
>> not enable checkpointing what are the semantics of the processing? At
>> least once?
>>
>> Best,
>>
>> Jerry
>>
>
>
Re: Question about flink message processing guarantee
Posted by Márton Balassi <ba...@gmail.com>.
Dear Jerry,
If you do not enable checkpointing you get the at most once processing
guarantee (some might call that no guarantee at all). When you enable
checkpointing you can choose between exactly and at least once semantics.
The latter provides better latency.
Best,
Marton
On Tue, Dec 1, 2015 at 11:04 PM, Jerry Peng <je...@gmail.com>
wrote:
> Hello,
>
> I have a question regarding link streaming. I now if you enable
> checkpointing you can have exactly once processing guarantee. If I do
> not enable checkpointing what are the semantics of the processing? At
> least once?
>
> Best,
>
> Jerry
>