[GitHub] [hudi] XuQianJin-Stars commented on a diff in pull request #6284: [HUDI-4526] Improve spillableMapBasePath disk directory is full

[GitBox <gi...@apache.org>]

XuQianJin-Stars commented on code in PR #6284:
URL: https://github.com/apache/hudi/pull/6284#discussion_r994153351


##########
hudi-common/src/main/java/org/apache/hudi/common/util/FileIOUtils.java:
##########
@@ -204,4 +204,40 @@ public static Option<byte[]> readDataFromPath(FileSystem fileSystem, org.apache.
   public static Option<byte[]> readDataFromPath(FileSystem fileSystem, org.apache.hadoop.fs.Path detailPath) {
     return readDataFromPath(fileSystem, detailPath, false);
   }
+
+  /**
+   * Return the configured local directories where hudi can write files. This
+   * method does not create any directories on its own, it only encapsulates the
+   * logic of locating the local directories according to deployment mode.
+   */
+  public static String[] getConfiguredLocalDirs() {
+    if (isRunningInYarnContainer()) {
+      // If we are in yarn mode, systems can have different disk layouts so we must set it

Review Comment:
   > We totally ignore the option through hoodie write config then ?
   
   First, get the local dir from the yarn container. If you can't get it, you can get it from the hoodie write config. In this way, users on yarn basically do not need to actively modify the hoodie write config.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org