[jira] Commented: (HADOOP-3970) Counters written to the job history cannot be recovered back

["Amar Kamat (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12623776#action_12623776 ] 

Amar Kamat commented on HADOOP-3970:
------------------------------------

Currently the way counters are converted to string (in jobhistory) is as follows
_groupname.countername:value_
So for a counter which has the following contents
||groupname||countername||value||
|g1|c1|v1|
|g1|c2|v2|
|g1|c3|v3|
|g2|c4|v4|
|g2|c5|v5|
We get 
{{g1.c1:v1, g1.c2:v2, g1.c3:v3, g2.c4:v4, g2.c5:v5}}

One way to overcome the problem stated above is to use the length of the names present in the counter
So the above  counter might look like
{{[|g1|] g1 {[|c1|] c1:v1, [|c2|] c2:v2, [|c3|] c3:v3 }, [|g2|] g2 {[|c4|] c4:v4, [|c5|] c5:v5 }}}
Here |s| means length of s.
Hence the length of the name in the counter helps to correctly identify the name and helps in counter recovery.
----
Thoughts?

> Counters written to the job history cannot be recovered back
> ------------------------------------------------------------
>
>                 Key: HADOOP-3970
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3970
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Amar Kamat
>
> Counters that are written to the JobHistory are stringified using {{Counters.makeCompactString()}}. The format in which this api converts the counter into a string is _groupname.countername:value_. The problem is that _groupname_ and _countername_ can contain a '.' and hence recovering the counter becomes difficult. Since JobHistory can be used for various purposes, reconstructing the counter object back might be useful. One such usecase is HADOOP-3245. There should be some way to recover the counter object back from its string representation and also to keep the string version readable.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.