You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2019/04/12 08:45:11 UTC
[GitHub] [flink] Myasuka edited a comment on issue #7281:
[FLINK-11107][state] Avoid memory stateBackend to create arbitrary folders
under HA path when no checkpoint path configured
Myasuka edited a comment on issue #7281: [FLINK-11107][state] Avoid memory stateBackend to create arbitrary folders under HA path when no checkpoint path configured
URL: https://github.com/apache/flink/pull/7281#issuecomment-468532245
Since Flink-1.8 is about to release, @StephanEwen @StefanRRichter could anyone take a look at this problem?
I submitted the same job with the same configuration (no checkpoint path but HA configured) with released Flink-1.3.2 (still has no such `MemoryStateBackend` creating random checkpoint path code, which should be treated as `old behavior`) and Flink-1.7.2 (already contained that part of code.)
As you can see `Flink-1.3.2` would have a blob service folder, a completed checkpoint file and a submitted job graph file. I think this is the `old behavior`.
<img width="781" alt="20190301113237" src="https://user-images.githubusercontent.com/1709104/53614879-67353c80-3c16-11e9-8fac-0dee85b676d4.png">
However, `Flink-1.7.2` would have many checkpoint paths created by `MemoryStateBackend` from task-side, as you could guess, `41a7c8b8e62d81225868d2a5a60846f7` is the actual job-id of this job. These created checkpoint path should actually be useless, and might lead to `MaxDirectoryItemsExceededException` under high availability folder.
<img width="1052" alt="20190301113317" src="https://user-images.githubusercontent.com/1709104/53614943-9e0b5280-3c16-11e9-81c4-868c3187a09b.png">
Moreover, as you can see, I don't think this would `keep supporting the old behavior` due to the great directory structure difference.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services