You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "teng wang (Jira)" <ji...@apache.org> on 2022/10/19 07:49:00 UTC
[jira] [Updated] (MAPREDUCE-7424) Document Fix: the dependency between mapreduce.job.sharedcache.mode and yarn.sharedcache.enabled
[ https://issues.apache.org/jira/browse/MAPREDUCE-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
teng wang updated MAPREDUCE-7424:
---------------------------------
Description:
Suggestions to fix the document (description of mapreduce.job.sharedcache.mode in mapred-default.xml):
There is one dependency between mapreduce.job.sharedcache.mode and yarn.sharedcache.enabled in source code. That is, mapreduce.job.sharedcache.mode can work if the shared cache (yarn.sharedcache.enabled) is enabled. However, the document (mapred-default.xml) does not mention it, which could affects the use of this configuration.
The dependency code:
{code:java}
/* /apache/hadoop/mapreduce/SharedCacheConfig.java */
public void init(Configuration conf) {
if(!conf.getBoolean(YarnConfiguration.SHARED_CACHE_ENABLED,
YarnConfiguration.DEFAULT_SHARED_CACHE_ENABLED)) {
return;
}
Collection<String> configs = StringUtils.getTrimmedStringCollection(
conf.get(MRJobConfig.SHARED_CACHE_MODE,
MRJobConfig.SHARED_CACHE_MODE_DEFAULT));
if (configs.contains("files")) {
this.sharedCacheFilesEnabled = true;
}{code}
was:
Suggestions to fix the document (description of mapreduce.job.sharedcache.mode in mapred-default.xml):
There is one dependency between mapreduce.job.sharedcache.mode and yarn.sharedcache.enabled in source code. That is, mapreduce.job.sharedcache.mode can work if the shared cache (yarn.sharedcache.enabled) is enabled. However, the document (mapred-default.xml) does not mention it, which could affects the use of this configuration.
The dependency code:
```
/* /apache/hadoop/mapreduce/SharedCacheConfig.java */
public void init(Configuration conf) {
if(!conf.getBoolean(YarnConfiguration.SHARED_CACHE_ENABLED,
YarnConfiguration.DEFAULT_SHARED_CACHE_ENABLED)) {
return;
}
Collection<String> configs = StringUtils.getTrimmedStringCollection(
conf.get(MRJobConfig.SHARED_CACHE_MODE,
MRJobConfig.SHARED_CACHE_MODE_DEFAULT));
if (configs.contains("files")) {
this.sharedCacheFilesEnabled = true;
}
```
> Document Fix: the dependency between mapreduce.job.sharedcache.mode and yarn.sharedcache.enabled
> ------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-7424
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7424
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
> Affects Versions: 3.3.4
> Reporter: teng wang
> Priority: Major
>
> Suggestions to fix the document (description of mapreduce.job.sharedcache.mode in mapred-default.xml):
> There is one dependency between mapreduce.job.sharedcache.mode and yarn.sharedcache.enabled in source code. That is, mapreduce.job.sharedcache.mode can work if the shared cache (yarn.sharedcache.enabled) is enabled. However, the document (mapred-default.xml) does not mention it, which could affects the use of this configuration.
>
> The dependency code:
>
> {code:java}
> /* /apache/hadoop/mapreduce/SharedCacheConfig.java */
> public void init(Configuration conf) {
> if(!conf.getBoolean(YarnConfiguration.SHARED_CACHE_ENABLED,
> YarnConfiguration.DEFAULT_SHARED_CACHE_ENABLED)) {
> return;
> }
> Collection<String> configs = StringUtils.getTrimmedStringCollection(
> conf.get(MRJobConfig.SHARED_CACHE_MODE,
> MRJobConfig.SHARED_CACHE_MODE_DEFAULT));
>
> if (configs.contains("files")) {
> this.sharedCacheFilesEnabled = true;
> }{code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org