[jira] Commented: (MAPREDUCE-2000) Rumen is not able to extract counters for Job history logs from Hadoop 0.20

["Hong Tang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896184#action_12896184 ] 

Hong Tang commented on MAPREDUCE-2000:
--------------------------------------

An example input line that triggers this bug:
{noformat}
MapAttempt TASK_TYPE="MAP" TASKID="task_201005120512_181225_m_000266" TASK_ATTEMPT_ID="attempt_201005120512_181225_m_000266_0" 
TASK_STATUS="SUCCESS" FINISH_TIME="1275354731626" STATE_STRING="qry:\"RFC 3584\" Value: [Simple][timestamp:1275276122][type:c][/Simple]" COUNTERS="
{(FileSystemCounters)(FileSystemCounters)[(FILE_BYTES_READ)(FILE_BYTES_READ)(1609)][(HDFS_BYTES_READ)(HDFS_BYTES_READ)(67412713)]
[(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(5648633)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)
(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(92297)][(SPILLED_RECORDS)(Spilled Records)(74370)][(MAP_OUTPUT_BYTES)(Map output 
bytes)(18737847)][(MAP_INPUT_BYTES)(Map input bytes)(67211804)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output 
records)(74370)]}" .
{noformat}

It outputs the following error messages:
{noformat}
10/08/06 23:14:40 WARN rumen.HistoryEventEmitter: HistoryEventEmitters: null counter detected:
10/08/06 23:14:40 WARN rumen.HistoryEventEmitter: HistoryEventEmitters: null counter detected:
...
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
        at java.lang.String.substring(String.java:1938)
        at org.apache.hadoop.tools.rumen.ParsedLine.<init>(ParsedLine.java:100)
        at org.apache.hadoop.tools.rumen.Hadoop20JHParser.nextEvent(Hadoop20JHParser.java:131)
        at org.apache.hadoop.tools.rumen.TraceBuilder.processJobHistory(TraceBuilder.java:287)
        at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:242)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:120)
{noformat}

> Rumen is not able to extract counters for Job history logs from Hadoop 0.20
> ---------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2000
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2000
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Hong Tang
>
> Rumen tries to match the end of a value string through indexOf("\""). It does not take into account the case when an escaped '"' in the value string. This leads to the incorrect parsing the remaining key=value properties in the same line.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.