You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Zakelly Lan <za...@gmail.com> on 2025/01/07 11:08:48 UTC
Re: Full Checkpoint information in UI and API
Hi,
I would +1 for this.
As for the name: incremental: true vs. false sounds good to me, which is
aligned with the config name 'execution.checkpointing.incremental'.
Best,
Zakelly
On Tue, Dec 17, 2024 at 12:01 PM Ryan van Huuksloot
<ry...@shopify.com.invalid> wrote:
> Hello,
>
> Would the community be open to adding additional information about the
> SNAPSHOT_TYPE to the Flink UI?
>
> Currently, you can tell if you have an aligned or unaligned checkpoint,
> savepoint or sync savepoint. You can infer incremental vs full checkpoints
> but it would be nice to expose it in the API and UI.
>
> I believe a good candidate to put this would be
> `org.apache.flink.runtime.checkpoint.SnapshotType`. The system should
> already know which type of checkpoint, we just need to store the
> information.
>
> Thoughts?
>
> If it is something of interest, what do you think would be a good name?
> There is already a checkpoint_type for aligned vs unaligned.
>
> Ryan van Huuksloot
> Sr. Production Engineer | Streaming Platform
> [image: Shopify]
> <https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email>
>
Re: Full Checkpoint information in UI and API
Posted by Gabor Somogyi <ga...@gmail.com>.
Hi Ryan,
+1 for the name "incremental". Just had a look at the PR and looks like it
has a compile error [1].
Additionally if it's possible it would be good to add some automated test.
[1]
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=64974&view=logs&j=52b61abe-a3cc-5bde-cc35-1bbe89bb7df5&t=54421a62-0c80-5aad-3319-094ff69180bb&l=7
BR,
G
On Tue, Jan 7, 2025 at 11:33 PM Ryan van Huuksloot
<ry...@shopify.com.invalid> wrote:
> Hi,
>
> I've updated the PR [https://github.com/apache/flink/pull/25899] to follow
> the incremental recommendation.
>
> It is short a test or two but otherwise should be ready to review.
>
> Ryan van Huuksloot
> Sr. Production Engineer | Streaming Platform
> [image: Shopify]
> <https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email>
>
>
> On Tue, Jan 7, 2025 at 6:11 AM Zakelly Lan <za...@gmail.com> wrote:
>
> > Hi,
> >
> > I would +1 for this.
> >
> > As for the name: incremental: true vs. false sounds good to me, which is
> > aligned with the config name 'execution.checkpointing.incremental'.
> >
> >
> > Best,
> > Zakelly
> >
> > On Tue, Dec 17, 2024 at 12:01 PM Ryan van Huuksloot
> > <ry...@shopify.com.invalid> wrote:
> >
> > > Hello,
> > >
> > > Would the community be open to adding additional information about the
> > > SNAPSHOT_TYPE to the Flink UI?
> > >
> > > Currently, you can tell if you have an aligned or unaligned checkpoint,
> > > savepoint or sync savepoint. You can infer incremental vs full
> > checkpoints
> > > but it would be nice to expose it in the API and UI.
> > >
> > > I believe a good candidate to put this would be
> > > `org.apache.flink.runtime.checkpoint.SnapshotType`. The system should
> > > already know which type of checkpoint, we just need to store the
> > > information.
> > >
> > > Thoughts?
> > >
> > > If it is something of interest, what do you think would be a good name?
> > > There is already a checkpoint_type for aligned vs unaligned.
> > >
> > > Ryan van Huuksloot
> > > Sr. Production Engineer | Streaming Platform
> > > [image: Shopify]
> > > <
> https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email
> > >
> > >
> >
>
Re: Full Checkpoint information in UI and API
Posted by Ryan van Huuksloot <ry...@shopify.com.INVALID>.
Hi,
I've updated the PR [https://github.com/apache/flink/pull/25899] to follow
the incremental recommendation.
It is short a test or two but otherwise should be ready to review.
Ryan van Huuksloot
Sr. Production Engineer | Streaming Platform
[image: Shopify]
<https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email>
On Tue, Jan 7, 2025 at 6:11 AM Zakelly Lan <za...@gmail.com> wrote:
> Hi,
>
> I would +1 for this.
>
> As for the name: incremental: true vs. false sounds good to me, which is
> aligned with the config name 'execution.checkpointing.incremental'.
>
>
> Best,
> Zakelly
>
> On Tue, Dec 17, 2024 at 12:01 PM Ryan van Huuksloot
> <ry...@shopify.com.invalid> wrote:
>
> > Hello,
> >
> > Would the community be open to adding additional information about the
> > SNAPSHOT_TYPE to the Flink UI?
> >
> > Currently, you can tell if you have an aligned or unaligned checkpoint,
> > savepoint or sync savepoint. You can infer incremental vs full
> checkpoints
> > but it would be nice to expose it in the API and UI.
> >
> > I believe a good candidate to put this would be
> > `org.apache.flink.runtime.checkpoint.SnapshotType`. The system should
> > already know which type of checkpoint, we just need to store the
> > information.
> >
> > Thoughts?
> >
> > If it is something of interest, what do you think would be a good name?
> > There is already a checkpoint_type for aligned vs unaligned.
> >
> > Ryan van Huuksloot
> > Sr. Production Engineer | Streaming Platform
> > [image: Shopify]
> > <https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email
> >
> >
>