You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by anaray <an...@gmail.com> on 2019/06/06 19:03:50 UTC
Flink 1.7.1 flink-s3-fs-hadoop-1.7.1 doesn't delete older chk-
directories
Hi,
I am using 1.7.1 and we store checkpoints in Ceph and we use
flink-s3-fs-hadoop-1.7.1 to connect to Ceph. I have only 1 checkpoint
retained. Issue I see is that previous/old chk-<id> directories are still
around. I verified that those older doesn't contain any checkpoint data. But
the directories keep accumulating.
Why is that these old stale directories aren't deleted after part checkpoint
deletion step? Please let me know.
I see :
metacheckpoints/00000000000000000000000000000000/chk-175
metacheckpoints/00000000000000000000000000000000/chk-176
metacheckpoints/00000000000000000000000000000000/chk-177 <- latest
Thanks,
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Flink 1.7.1 flink-s3-fs-hadoop-1.7.1 doesn't delete older
chk- directories
Posted by anaray <an...@gmail.com>.
Hi Fabian,
Thank you. Your observation is correct. The stale directories belong to the
failed checkpoints. So it is related to FLINK-10855. I will closely follow
FLINK-10855 and test when fix is available
Thank You,
anaray
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Flink 1.7.1 flink-s3-fs-hadoop-1.7.1 doesn't delete older
chk- directories
Posted by Fabian Hueske <fh...@gmail.com>.
Hi,
I found a few issues in Jira that are related to not deleted checkpoint
directories, but only FLINK-10855 [1] seems to be a possible reason in your
case.
Is it possible that the checkpoints of the remaining directories failed?
If that's not the case, would you mind creating a Jira issue and describe
the bug?
Thank you,
Fabian
[1] https://issues.apache.org/jira/browse/FLINK-10855
Am Do., 6. Juni 2019 um 21:04 Uhr schrieb anaray <an...@gmail.com>:
> Hi,
>
> I am using 1.7.1 and we store checkpoints in Ceph and we use
> flink-s3-fs-hadoop-1.7.1 to connect to Ceph. I have only 1 checkpoint
> retained. Issue I see is that previous/old chk-<id> directories are still
> around. I verified that those older doesn't contain any checkpoint data.
> But
> the directories keep accumulating.
> Why is that these old stale directories aren't deleted after part
> checkpoint
> deletion step? Please let me know.
>
> I see :
> metacheckpoints/00000000000000000000000000000000/chk-175
> metacheckpoints/00000000000000000000000000000000/chk-176
> metacheckpoints/00000000000000000000000000000000/chk-177 <- latest
>
>
> Thanks,
>
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>