You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "xieguiming (Created) (JIRA)" <ji...@apache.org> on 2012/03/26 08:48:32 UTC
[jira] [Created] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
------------------------------------------------------------------------------
Key: MAPREDUCE-4066
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: job submission, mrv2
Affects Versions: 0.23.1
Environment: client is windows eclipse, server is suse
Reporter: xieguiming
Priority: Minor
when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
{code:title=MRApps.java|borderStyle=solid}
public static Path getStagingAreaDir(Configuration conf, String user) {
return new Path(
conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
}
{code}
should modify to:
{code:title=MRApps.java|borderStyle=solid}
public static Path getStagingAreaDir(Configuration conf, String user) {
return new Path(
conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238413#comment-13238413 ]
Hudson commented on MAPREDUCE-4066:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1939 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1939/])
MAPREDUCE-4066. Use default value when fetching MR_AM_STAGING_DIR. Contributed by xieguiming. (Revision 1305343)
Result = ABORTED
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1305343
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-4066.patch, MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Harsh J (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238157#comment-13238157 ]
Harsh J commented on MAPREDUCE-4066:
------------------------------------
Good catch xieguiming.
Could you post a patch for this as well?
There are other spots that may need defaults too:
{code}
./hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java: doneDirPrefix = conf.get(MRJobConfig.MR_AM_STAGING_DIR)
./hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java: doneDirPrefix = conf.get(MRJobConfig.MR_AM_STAGING_DIR)
./hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java: conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
{code}
In case you are willing to provide a patch, please make sure to address the following:
* Define the default property and value in mapred-default.xml
* Define a constant DEFAULT_MR_AM_STAGING_DIR in MRJobConfig, set to same value as above
* Use the above constant in the code where you'll add the defaults to those three spots above (MRApps and JobHistoryUtils).
Thanks! :)
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238293#comment-13238293 ]
Hadoop QA commented on MAPREDUCE-4066:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12519932/MAPREDUCE-4066.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2098//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2098//console
This message is automatically generated.
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Attachments: MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Harsh J (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238361#comment-13238361 ]
Harsh J commented on MAPREDUCE-4066:
------------------------------------
Patch contained CRLF (Windows) newlines instead of just LF (Unix). I've fixed it manually for now but in future please submit purely Unix-newline diffs (Git settings may be changed to reflect this) :)
{code}
➜ trunk patch -p0 < ~/Desktop/MAPREDUCE-4066.patch.txt
(Stripping trailing CRs from patch.)
patching file hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
(Stripping trailing CRs from patch.)
patching file hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java
(Stripping trailing CRs from patch.)
patching file hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
{code}
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Attachments: MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Harsh J (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-4066:
-------------------------------
Attachment: MAPREDUCE-4066.patch
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Attachments: MAPREDUCE-4066.patch, MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated MAPREDUCE-4066:
----------------------------------------------
Assignee: xieguiming
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Assignee: xieguiming
> Priority: Minor
> Fix For: 2.0.0
>
> Attachments: MAPREDUCE-4066.patch, MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238380#comment-13238380 ]
Hudson commented on MAPREDUCE-4066:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #717 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/717/])
svn merge -c 1305343. Merge MAPREDUCE-4066 into branch-0.23 from trunk. (Revision 1305348)
Result = SUCCESS
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1305348
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/bin
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-4066.patch, MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239476#comment-13239476 ]
Hudson commented on MAPREDUCE-4066:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #1032 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1032/])
MAPREDUCE-4066. Use default value when fetching MR_AM_STAGING_DIR. Contributed by xieguiming. (Revision 1305343)
Result = SUCCESS
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1305343
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-4066.patch, MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238420#comment-13238420 ]
Hudson commented on MAPREDUCE-4066:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #736 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/736/])
svn merge -c 1305343. Merge MAPREDUCE-4066 into branch-0.23 from trunk. (Revision 1305348)
Result = ABORTED
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1305348
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/bin
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-4066.patch, MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "xieguiming (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238202#comment-13238202 ]
xieguiming commented on MAPREDUCE-4066:
---------------------------------------
Sure, I will.
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "xieguiming (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xieguiming updated MAPREDUCE-4066:
----------------------------------
Attachment: MAPREDUCE-4066.patch
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Attachments: MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239429#comment-13239429 ]
Hudson commented on MAPREDUCE-4066:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #997 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/997/])
MAPREDUCE-4066. Use default value when fetching MR_AM_STAGING_DIR. Contributed by xieguiming. (Revision 1305343)
Result = FAILURE
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1305343
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-4066.patch, MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238382#comment-13238382 ]
Hudson commented on MAPREDUCE-4066:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #727 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/727/])
svn merge -c 1305343. Merge MAPREDUCE-4066 into branch-0.23 from trunk. (Revision 1305348)
Result = SUCCESS
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1305348
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/bin
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-4066.patch, MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Harsh J (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-4066:
-------------------------------
Resolution: Fixed
Fix Version/s: 0.23.3
Target Version/s: (was: 0.23.3, 0.24.0)
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk and branch-0.23. Thanks for your contribution xieguiming! Hoping to see more :)
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-4066.patch, MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Harsh J (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-4066:
-------------------------------
Target Version/s: 0.24.0, 0.23.3
+1. Thank you for the patch xieguiming. The default additions look good to me.
Thanks also for pointing out it already exists in mapred-default.xml (So apparently the tool you were using did not load it up? To address that, we may open a new issue. In any case, get-calls should use a default if there is one available.)
Committing shortly.
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Attachments: MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238370#comment-13238370 ]
Hadoop QA commented on MAPREDUCE-4066:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12519955/MAPREDUCE-4066.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2099//console
This message is automatically generated.
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Attachments: MAPREDUCE-4066.patch, MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "xieguiming (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xieguiming updated MAPREDUCE-4066:
----------------------------------
Status: Patch Available (was: Open)
bq. •Define the default property and value in mapred-default.xml
It is already present in mapred-default.xml
{quote}
•Define a constant DEFAULT_MR_AM_STAGING_DIR in MRJobConfig, set to same value as above
•Use the above constant in the code where you'll add the defaults to those three spots above (MRApps and JobHistoryUtils).
{quote}
I updated the patch with these changes. Please review it.
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Attachments: MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238372#comment-13238372 ]
Hudson commented on MAPREDUCE-4066:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2003 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2003/])
MAPREDUCE-4066. Use default value when fetching MR_AM_STAGING_DIR. Contributed by xieguiming. (Revision 1305343)
Result = SUCCESS
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1305343
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Attachments: MAPREDUCE-4066.patch, MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4066) To get
"yarn.app.mapreduce.am.staging-dir" value, should set the default value
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238373#comment-13238373 ]
Hudson commented on MAPREDUCE-4066:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1929 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1929/])
MAPREDUCE-4066. Use default value when fetching MR_AM_STAGING_DIR. Contributed by xieguiming. (Revision 1305343)
Result = SUCCESS
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1305343
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4066
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.23.1
> Environment: client is windows eclipse, server is suse
> Reporter: xieguiming
> Priority: Minor
> Attachments: MAPREDUCE-4066.patch, MAPREDUCE-4066.patch
>
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir value is null.
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR) +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
> public static Path getStagingAreaDir(Configuration conf, String user) {
> return new Path(
> conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") +
> Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira