You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Owen O'Malley (JIRA)" <ji...@apache.org> on 2009/05/21 01:23:45 UTC
[jira] Issue Comment Edited: (HADOOP-5834) Job History log file
format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/HADOOP-5834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711411#action_12711411 ]
Owen O'Malley edited comment on HADOOP-5834 at 5/20/09 4:23 PM:
----------------------------------------------------------------
It would look like:
{code}
{"KIND":"MapAttempt",
"TASK_TYPE":"MAP",
"TASKID":"task_200904210931_0001_m_180280",
"TASK_ATTEMPT_ID":"attempt_200904210931_0001_m_180280_0",
"TASK_STATUS":"SUCCESS",
"FINISH_TIME":"1240321545820",
"HOSTNAME":"/rack1/node1.purple.ygrid.yahoo.com",
"STATE_STRING":"",
"COUNTERS":[{"ID":"org.apache.hadoop.mapred.Task$Counter", "NAME":"Map-Reduce Framework"
"LIST":[{"ID":"COMBINE_OUTPUT_RECORDS", "NAME":"Combine output records", "VALUE":0},
{"ID":"MAP_INPUT_RECORDS","NAME":"Map input records", "VALUE":3363235}]}]
}
{code}
To support grep, we shouldn't add any newlines.
was (Author: owen.omalley):
It would look like:
{code}
{"KIND":"MapAttempt",
"TASK_TYPE":"MAP",
"TASKID":"task_200904210931_0001_m_180280",
"TASK_ATTEMPT_ID":"attempt_200904210931_0001_m_180280_0",
"TASK_STATUS":"SUCCESS",
"FINISH_TIME":"1240321545820",
"HOSTNAME":"/rack1/node1.purple.ygrid.yahoo.com",
"STATE_STRING":"",
"COUNTERS":[{"ID":"org.apache.hadoop.mapred.Task$Counter", "NAME":"Map-Reduce Framework"
LIST:[{"ID":"COMBINE_OUTPUT_RECORDS", "NAME":"Combine output records", "VALUE":0},
{"ID":"MAP_INPUT_RECORDS","NAME":"Map input records", "VALUE":3363235}]}]
}
{code}
> Job History log file format is not friendly for external tools.
> ---------------------------------------------------------------
>
> Key: HADOOP-5834
> URL: https://issues.apache.org/jira/browse/HADOOP-5834
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Owen O'Malley
>
> Currently, parsing the job history logs with external tools is very difficult because of the format. The most critical problem is that newlines aren't escaped in the strings. That makes using tools like grep, sed, and awk very tricky.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.