You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@uniffle.apache.org by js...@apache.org on 2022/07/01 06:57:53 UTC
[incubator-uniffle] 04/04: [Bugfix] [0.5] Fix MR don't have remote storage information when we use dynamic conf and MEMORY_LOCALE_HDFS storageType (#195) (#196)
This is an automated email from the ASF dual-hosted git repository.
jshao pushed a commit to branch branch-0.5.0
in repository https://gitbox.apache.org/repos/asf/incubator-uniffle.git
commit 55cb16fb6b9f494f657068721ca81c74043a4bb9
Author: roryqi <je...@gmail.com>
AuthorDate: Thu Jun 23 10:52:59 2022 +0800
[Bugfix] [0.5] Fix MR don't have remote storage information when we use dynamic conf and MEMORY_LOCALE_HDFS storageType (#195) (#196)
backport 0.5
### What changes were proposed in this pull request?
We should aquire the storageType from extraConf.
### Why are the changes needed?
If we don't have this patch, MR don't work when we use dynamic conf and MEMORY_LOCALE_HDFS storageType.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Manual test
---
.../main/java/org/apache/hadoop/mapreduce/v2/app/RssMRAppMaster.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/client-mr/src/main/java/org/apache/hadoop/mapreduce/v2/app/RssMRAppMaster.java b/client-mr/src/main/java/org/apache/hadoop/mapreduce/v2/app/RssMRAppMaster.java
index 7511104..976b03c 100644
--- a/client-mr/src/main/java/org/apache/hadoop/mapreduce/v2/app/RssMRAppMaster.java
+++ b/client-mr/src/main/java/org/apache/hadoop/mapreduce/v2/app/RssMRAppMaster.java
@@ -180,7 +180,7 @@ public class RssMRAppMaster extends MRAppMaster {
RssMRUtils.applyDynamicClientConf(extraConf, clusterClientConf);
}
- String storageType = conf.get(RssMRConfig.RSS_STORAGE_TYPE);
+ String storageType = RssMRUtils.getString(extraConf, conf, RssMRConfig.RSS_STORAGE_TYPE);
RemoteStorageInfo defaultRemoteStorage =
new RemoteStorageInfo(conf.get(RssMRConfig.RSS_REMOTE_STORAGE_PATH, ""));
RemoteStorageInfo remoteStorage = ClientUtils.fetchRemoteStorage(