You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Amar Kamat (JIRA)" <ji...@apache.org> on 2010/07/29 10:32:17 UTC
[jira] Created: (MAPREDUCE-1982) [Rumen] TraceBuilder's output
shows jobname as NULL for jobhistory files with valid jobnames
[Rumen] TraceBuilder's output shows jobname as NULL for jobhistory files with valid jobnames
--------------------------------------------------------------------------------------------
Key: MAPREDUCE-1982
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1982
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: tools/rumen
Reporter: Amar Kamat
{{TraceBuilder}} fails to extract configuration properties (like job-name) from the job-conf if the job-conf has the properties stored using the deprecated keys.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1982) [Rumen] TraceBuilder's output
shows jobname as NULL for jobhistory files with valid jobnames
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1982:
-----------------------------------------------
Fix Version/s: 0.22.0
> [Rumen] TraceBuilder's output shows jobname as NULL for jobhistory files with valid jobnames
> --------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1982
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1982
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tools/rumen
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
> Fix For: 0.22.0
>
> Attachments: 1982.v1.1.patch, 1982.v1.patch
>
>
> {{TraceBuilder}} fails to extract configuration properties (like job-name) from the job-conf if the job-conf has the properties stored using the deprecated keys.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1982) [Rumen] TraceBuilder's output
shows jobname as NULL for jobhistory files with valid jobnames
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894521#action_12894521 ]
Ravi Gummadi commented on MAPREDUCE-1982:
-----------------------------------------
Attaching new patch with minor changes to the comments and assertion failure messages in testcases.
> [Rumen] TraceBuilder's output shows jobname as NULL for jobhistory files with valid jobnames
> --------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1982
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1982
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tools/rumen
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
> Attachments: 1982.v1.1.patch, 1982.v1.patch
>
>
> {{TraceBuilder}} fails to extract configuration properties (like job-name) from the job-conf if the job-conf has the properties stored using the deprecated keys.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1982) [Rumen] TraceBuilder's output
shows jobname as NULL for jobhistory files with valid jobnames
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893560#action_12893560 ]
Amar Kamat commented on MAPREDUCE-1982:
---------------------------------------
Also {{TraceBuilder}} extracts various job configuration properties like
# job-name
# queue-names
# mapred-child task jvm settings etc
from job's conf. So this bug is seen only when there is a jobconf present in the jobhistory folder.
> [Rumen] TraceBuilder's output shows jobname as NULL for jobhistory files with valid jobnames
> --------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1982
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1982
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tools/rumen
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
>
> {{TraceBuilder}} fails to extract configuration properties (like job-name) from the job-conf if the job-conf has the properties stored using the deprecated keys.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1982) [Rumen] TraceBuilder's output
shows jobname as NULL for jobhistory files with valid jobnames
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894527#action_12894527 ]
Ravi Gummadi commented on MAPREDUCE-1982:
-----------------------------------------
All unit tests passed on my local machine.
ant test-patch gave:
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 8 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
> [Rumen] TraceBuilder's output shows jobname as NULL for jobhistory files with valid jobnames
> --------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1982
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1982
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tools/rumen
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
> Attachments: 1982.v1.1.patch, 1982.v1.patch
>
>
> {{TraceBuilder}} fails to extract configuration properties (like job-name) from the job-conf if the job-conf has the properties stored using the deprecated keys.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1982) [Rumen] TraceBuilder's output
shows jobname as NULL for jobhistory files with valid jobnames
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-1982:
-----------------------------------------------
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Resolution: Fixed
I just committed this. Thanks Ravi !
In Long term, I think Rumen should be changed to use Configuration directly instead of parsing conf from Properies.
> [Rumen] TraceBuilder's output shows jobname as NULL for jobhistory files with valid jobnames
> --------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1982
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1982
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tools/rumen
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
> Attachments: 1982.v1.1.patch, 1982.v1.patch
>
>
> {{TraceBuilder}} fails to extract configuration properties (like job-name) from the job-conf if the job-conf has the properties stored using the deprecated keys.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1982) [Rumen] TraceBuilder's output
shows jobname as NULL for jobhistory files with valid jobnames
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12926126#action_12926126 ]
Hudson commented on MAPREDUCE-1982:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/])
> [Rumen] TraceBuilder's output shows jobname as NULL for jobhistory files with valid jobnames
> --------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1982
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1982
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tools/rumen
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
> Fix For: 0.22.0
>
> Attachments: 1982.v1.1.patch, 1982.v1.patch
>
>
> {{TraceBuilder}} fails to extract configuration properties (like job-name) from the job-conf if the job-conf has the properties stored using the deprecated keys.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (MAPREDUCE-1982) [Rumen]
TraceBuilder's output shows jobname as NULL for jobhistory files with valid
jobnames
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893552#action_12893552 ]
Owen O'Malley edited comment on MAPREDUCE-1982 at 7/29/10 12:15 PM:
--------------------------------------------------------------------
An example fragment is:
--------------------------------------- 8< ---------------------------------------
{
"priority" : "NORMAL",
"jobID" : "job_201007280625_0001",
"user" : "gridperf",
"jobName" : null,
"mapTasks" : [ {
"startTime" : 1280298532016,
"taskID" : "task_201007280625_0001_m_000323",
"taskType" : "MAP",
"attempts" : [ {
"location" : {
"layers" : [ "98\\.137\\.102\\.64", "node1\\.yahoo\\.com" ]
},
"hostName" : "/98\\.137\\.102\\.64/node1\\.yahoo\\.com",
"startTime" : 1280298532028,
"finishTime" : 1280298597501,
"result" : "SUCCESS",
--------------------------------------- 8< ---------------------------------------
This is because Rumen directly uses Properties instead of Configuration - the
new key-name is "mapreduce.job.name", while the old key-name was
"mapred.job.name". Using Configuration insulates against such changes.
was (Author: ranjit):
An example fragment is:
--------------------------------------- 8< ---------------------------------------
{
"priority" : "NORMAL",
"jobID" : "job_201007280625_0001",
"user" : "gridperf",
"jobName" : null,
"mapTasks" : [ {
"startTime" : 1280298532016,
"taskID" : "task_201007280625_0001_m_000323",
"taskType" : "MAP",
"attempts" : [ {
"location" : {
"layers" : [ "98\\.137\\.102\\.64", "gsbl91438\\.blue\\.ygrid\\.yahoo\\.com" ]
},
"hostName" : "/98\\.137\\.102\\.64/gsbl91438\\.blue\\.ygrid\\.yahoo\\.com",
"startTime" : 1280298532028,
"finishTime" : 1280298597501,
"result" : "SUCCESS",
--------------------------------------- 8< ---------------------------------------
This is because Rumen directly uses Properties instead of Configuration - the
new key-name is "mapreduce.job.name", while the old key-name was
"mapred.job.name". Using Configuration insulates against such changes.
> [Rumen] TraceBuilder's output shows jobname as NULL for jobhistory files with valid jobnames
> --------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1982
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1982
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tools/rumen
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
> Attachments: 1982.v1.patch
>
>
> {{TraceBuilder}} fails to extract configuration properties (like job-name) from the job-conf if the job-conf has the properties stored using the deprecated keys.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1982) [Rumen] TraceBuilder's output
shows jobname as NULL for jobhistory files with valid jobnames
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894728#action_12894728 ]
Amar Kamat commented on MAPREDUCE-1982:
---------------------------------------
The latest patch looks good.
> [Rumen] TraceBuilder's output shows jobname as NULL for jobhistory files with valid jobnames
> --------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1982
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1982
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tools/rumen
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
> Attachments: 1982.v1.1.patch, 1982.v1.patch
>
>
> {{TraceBuilder}} fails to extract configuration properties (like job-name) from the job-conf if the job-conf has the properties stored using the deprecated keys.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1982) [Rumen] TraceBuilder's output
shows jobname as NULL for jobhistory files with valid jobnames
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated MAPREDUCE-1982:
------------------------------------
Status: Patch Available (was: Open)
> [Rumen] TraceBuilder's output shows jobname as NULL for jobhistory files with valid jobnames
> --------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1982
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1982
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tools/rumen
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
> Attachments: 1982.v1.1.patch, 1982.v1.patch
>
>
> {{TraceBuilder}} fails to extract configuration properties (like job-name) from the job-conf if the job-conf has the properties stored using the deprecated keys.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (MAPREDUCE-1982) [Rumen] TraceBuilder's output
shows jobname as NULL for jobhistory files with valid jobnames
Posted by "Ranjit Mathew (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ranjit Mathew reassigned MAPREDUCE-1982:
----------------------------------------
Assignee: Ravi Gummadi
An example fragment is:
--------------------------------------- 8< ---------------------------------------
{
"priority" : "NORMAL",
"jobID" : "job_201007280625_0001",
"user" : "gridperf",
"jobName" : null,
"mapTasks" : [ {
"startTime" : 1280298532016,
"taskID" : "task_201007280625_0001_m_000323",
"taskType" : "MAP",
"attempts" : [ {
"location" : {
"layers" : [ "98\\.137\\.102\\.64", "gsbl91438\\.blue\\.ygrid\\.yahoo\\.com" ]
},
"hostName" : "/98\\.137\\.102\\.64/gsbl91438\\.blue\\.ygrid\\.yahoo\\.com",
"startTime" : 1280298532028,
"finishTime" : 1280298597501,
"result" : "SUCCESS",
--------------------------------------- 8< ---------------------------------------
This is because Rumen directly uses Properties instead of Configuration - the
new key-name is "mapreduce.job.name", while the old key-name was
"mapred.job.name". Using Configuration insulates against such changes.
> [Rumen] TraceBuilder's output shows jobname as NULL for jobhistory files with valid jobnames
> --------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1982
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1982
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tools/rumen
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
>
> {{TraceBuilder}} fails to extract configuration properties (like job-name) from the job-conf if the job-conf has the properties stored using the deprecated keys.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1982) [Rumen] TraceBuilder's output
shows jobname as NULL for jobhistory files with valid jobnames
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated MAPREDUCE-1982:
------------------------------------
Attachment: 1982.v1.1.patch
> [Rumen] TraceBuilder's output shows jobname as NULL for jobhistory files with valid jobnames
> --------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1982
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1982
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tools/rumen
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
> Attachments: 1982.v1.1.patch, 1982.v1.patch
>
>
> {{TraceBuilder}} fails to extract configuration properties (like job-name) from the job-conf if the job-conf has the properties stored using the deprecated keys.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1982) [Rumen] TraceBuilder's output
shows jobname as NULL for jobhistory files with valid jobnames
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated MAPREDUCE-1982:
------------------------------------
Attachment: 1982.v1.patch
Attaching patch fixing the issue. Added testcase that fails without the fix and passes with the fix.
> [Rumen] TraceBuilder's output shows jobname as NULL for jobhistory files with valid jobnames
> --------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1982
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1982
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tools/rumen
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
> Attachments: 1982.v1.patch
>
>
> {{TraceBuilder}} fails to extract configuration properties (like job-name) from the job-conf if the job-conf has the properties stored using the deprecated keys.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.