[GitHub] [flink] Myasuka opened a new pull request #7942: [FLINK-11696][checkpoint] Avoid to send mkdir requests to DFS from task side

[GitBox <gi...@apache.org>]

Myasuka opened a new pull request #7942: [FLINK-11696][checkpoint] Avoid to send mkdir requests to DFS from task side
URL: https://github.com/apache/flink/pull/7942
 
 
   ## What is the purpose of the change
   Previously, not only checkpoint coordinator but also tasks would always send those `mkdir` requests out when initialization. This would actually put a lot of pressure to master of distributed file system.
   This pull request avoids to call file system's `mkdir` when creating `CheckpointStorage`, but call it only once when checkpoint coordinator `initializeLocationForCheckpoint`. 
   
   ## Brief change log
   
     - Move the `mkdir` operations lazily to first `initializeLocationForCheckpoint` instead of each time when creating the `CheckpointStorage`.
   
   
   ## Verifying this change
   
   This change added tests and can be verified as follows:
   
     - refactor `testStorageLocationMkdirs` to verify checkpoint directories would be created once `initializeLocationForCheckpoint` instead of each time creating the `CheckpointStorage`.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): **no**
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no**
     - The serializers: **no**
     - The runtime per-record code paths (performance sensitive): **no**
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: **no**
     - The S3 file system connector: **no**
   
   ## Documentation
   
     - Does this pull request introduce a new feature? **no**
     - If yes, how is the feature documented? **not applicable**
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services