You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Daniel Li <da...@gmail.com> on 2016/06/29 20:58:29 UTC

Switch to skip the stream alignment during a checkpoint?

I am reading Stream Checkpointing doc below. But somehow couldn't find that
"switch" in any other Apache Flink docs. Has anyone of you tried this
switch?


https://ci.apache.org/projects/flink/flink-docs-master/internals/stream_checkpointing.html

"Flink has a switch to skip the stream alignment during a checkpoint.
Checkpoint snapshots are still drawn as soon as an operator has seen the
checkpoint barrier from each input."


thx
Daniel

Re: Switch to skip the stream alignment during a checkpoint?

Posted by Daniel Li <da...@gmail.com>.

Thanks Ufuk. Really appreciated.

On Thu, Jun 30, 2016 at 2:07 AM, Ufuk Celebi <uc...@apache.org> wrote:

> You are right, this is not very well-documented. You can do it like this:
>
>
> env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.AT_LEAST_ONCE);
>
> With this the operators don't wait for all barriers to align. Example
> for checkpoint mode setting is here:
>
> https://ci.apache.org/projects/flink/flink-docs-master/apis/streaming/fault_tolerance.html
>
>
> On Wed, Jun 29, 2016 at 10:58 PM, Daniel Li <da...@gmail.com> wrote:
> > I am reading Stream Checkpointing doc below. But somehow couldn't find
> that
> > "switch" in any other Apache Flink docs. Has anyone of you tried this
> > switch?
> >
> >
> >
> https://ci.apache.org/projects/flink/flink-docs-master/internals/stream_checkpointing.html
> >
> > "Flink has a switch to skip the stream alignment during a checkpoint.
> > Checkpoint snapshots are still drawn as soon as an operator has seen the
> > checkpoint barrier from each input."
> >
> >
> > thx
> > Daniel
>

Re: Switch to skip the stream alignment during a checkpoint?

Posted by Ufuk Celebi <uc...@apache.org>.

You are right, this is not very well-documented. You can do it like this:

env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.AT_LEAST_ONCE);

With this the operators don't wait for all barriers to align. Example
for checkpoint mode setting is here:
https://ci.apache.org/projects/flink/flink-docs-master/apis/streaming/fault_tolerance.html

On Wed, Jun 29, 2016 at 10:58 PM, Daniel Li <da...@gmail.com> wrote:
> I am reading Stream Checkpointing doc below. But somehow couldn't find that
> "switch" in any other Apache Flink docs. Has anyone of you tried this
> switch?
>
>
> https://ci.apache.org/projects/flink/flink-docs-master/internals/stream_checkpointing.html
>
> "Flink has a switch to skip the stream alignment during a checkpoint.
> Checkpoint snapshots are still drawn as soon as an operator has seen the
> checkpoint barrier from each input."
>
>
> thx
> Daniel