[jira] Commented: (HADOOP-3970) Counters written to the job history cannot be recovered back

["Hemanth Yamijala (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12628439#action_12628439 ] 

Hemanth Yamijala commented on HADOOP-3970:
------------------------------------------

StringUtils

- I think the {{escapeString(String str, char escapeChar, char charToEscape)}} API is still useful by itself, and is easier to use in cases where the user has only one char to escape. So, do we really need to deprecate this ?
- charHasPreEscape is not needed. It is always going to be the escape char. This is a bug in the previous code too.

Counters

- The token delimiters can be declared final.

Counter

- equals and hashCode implementation: Since the counter's value can change over time, I am thinking that these methods should not depend on the value of the counter. Needs to be thought out a bit more.
- makeEscapedCompactString - use StringBuffer, instead of appending spaces.
- makeEscapedCompactString - do we need the spaces ? I think the expression is compact without them. If we need them, then a space is missing when the counter closes.

Group
- equals: Code is incorrectly returning true if classes don't match. 
- hashCode: Are there standard ways in which we can combine hashCodes of various implementations. Also, the hashCode should not use the toString method to get the hashCode.
makeCompactEscapedString - names are different between similar API in Counter. Can we have them uniform ? Also, you can use StringBuffer without any String concatanation at all. Same comment on blank spaces.

Counters:
- getBlock: We expect a start and end. If there's no start or no end, should we fail fast ? I guess today if the end does not come as expected, it returns the remaining String as the value.
- Same comments here as with the Group's implementation for overriding equals and hashcode.

TestCounters:

- testCounter need not check the mainCounter and copyCounter's equality before getting them from the strings. In fact, why do we need copyCounter at all ?


> Counters written to the job history cannot be recovered back
> ------------------------------------------------------------
>
>                 Key: HADOOP-3970
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3970
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Amar Kamat
>         Attachments: HADOOP-3970-v1.patch, HADOOP-3970-v2.patch, HADOOP-3970-v3.patch
>
>
> Counters that are written to the JobHistory are stringified using {{Counters.makeCompactString()}}. The format in which this api converts the counter into a string is _groupname.countername:value_. The problem is that _groupname_ and _countername_ can contain a '.' and hence recovering the counter becomes difficult. Since JobHistory can be used for various purposes, reconstructing the counter object back might be useful. One such usecase is HADOOP-3245. There should be some way to recover the counter object back from its string representation and also to keep the string version readable.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.