You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Suhas Gogate (JIRA)" <ji...@apache.org> on 2009/03/26 17:20:49 UTC
[jira] Created: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
---------------------------------------------------------------------------------------------------------------------------------------
Key: HADOOP-5582
URL: https://issues.apache.org/jira/browse/HADOOP-5582
Project: Hadoop Core
Issue Type: Bug
Affects Versions: 0.20.0
Reporter: Suhas Gogate
Fix For: 0.20.0, 0.21.0
Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709937#action_12709937 ]
Amar Kamat commented on HADOOP-5582:
------------------------------------
Looks fine. +1.
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Assignee: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-0.21.0-5582-5764.patch, vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708162#action_12708162 ]
Suhas Gogate commented on HADOOP-5582:
--------------------------------------
Can someone pl. review and commit this patch. I need to add some more diagnostic rules which require this patch.
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Assignee: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-0.21.0-5582-5764.patch, vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suhas Gogate updated HADOOP-5582:
---------------------------------
Attachment: vaidya-0.21.0-5582-5764.patch
Submitting new patch to also incorporate the fix for HADOOP-5764, which depends on previous patch submitted for this issue. After committing this patch issue in HADOOP-5764 would also be resolved.
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Assignee: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-0.21.0-5582-5764.patch, vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708401#action_12708401 ]
Amar Kamat commented on HADOOP-5582:
------------------------------------
Checked the code change related to {{Counters}}. Looks fine to me and seems like a right direction (i.e counter string obtained from jobhistory should be decoded using {{Counters}}).
+1.
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Assignee: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-0.21.0-5582-5764.patch, vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-5582:
--------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Suhas!
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Assignee: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-0.21.0-5582-5764.patch, vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705035#action_12705035 ]
Suhas Gogate commented on HADOOP-5582:
--------------------------------------
Can someone pl. commit this patch?
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suhas Gogate updated HADOOP-5582:
---------------------------------
Attachment: vaidya_patch_21.patch
This is a patch for both hadoop-0.20.0 and trunk (0.21.0)
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.20.0, 0.21.0
>
> Attachments: vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709482#action_12709482 ]
Suhas Gogate commented on HADOOP-5582:
--------------------------------------
Amar, hope this clarifies your questions. Let me know if you need any further clarification. If not, then can you pl. commit the fix. Thanks & Regards, Suhas
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Assignee: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-0.21.0-5582-5764.patch, vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suhas Gogate updated HADOOP-5582:
---------------------------------
Status: Patch Available (was: Open)
Submitting a new patch. See comment above.
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Assignee: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-0.21.0-5582-5764.patch, vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suhas Gogate updated HADOOP-5582:
---------------------------------
Fix Version/s: (was: 0.20.0)
Release Note: This patch fixes the number format exception causes due to change in job history counters string format.
Status: Patch Available (was: Open)
This patch is applicable for both hadoop 0.20.0 and trunk (0.21.0).
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705464#action_12705464 ]
Hadoop QA commented on HADOOP-5582:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12407120/vaidya-0.21.0-5582-5764.patch
against trunk revision 770685.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/283/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/283/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/283/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/283/console
This message is automatically generated.
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Assignee: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-0.21.0-5582-5764.patch, vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suhas Gogate updated HADOOP-5582:
---------------------------------
Status: Open (was: Patch Available)
Adding new patch to also incorporate fix for HADOOP-5764, which depends on this one.
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Assignee: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708493#action_12708493 ]
Suhas Gogate commented on HADOOP-5582:
--------------------------------------
1. About converting to compact string and then using existing code is definitely a less change, but may not be a right way to go. What if tomorrow the compact format gets deprecated? Also as you mentioned we don't need to do extra conversion just to use existing code.
2. Vaidya is a offline tool and uses the default job history parser to read and parse the job history file. Although it presents all the counters as a key/value pairs in the format or interface exposed by Vaidya tool for its rule writers. Thus it makes rules agnostic to any changes in hadoop counters and their representation.
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Assignee: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-0.21.0-5582-5764.patch, vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12694127#action_12694127 ]
Hadoop QA commented on HADOOP-5582:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12404195/vaidya_patch_21.patch
against trunk revision 760376.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/82/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/82/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/82/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/82/console
This message is automatically generated.
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-5582:
--------------------------------
Assignee: Suhas Gogate
Amar, can you review this please?
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Assignee: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708404#action_12708404 ]
Amar Kamat commented on HADOOP-5582:
------------------------------------
>From what I understand, you could something like
{code}
Counters c = Counters.fromEscapedCompactString(counterString);
String compactString = c.toCompactStrihng();
// do what you do currently .. but with compactString and not counterString
Matcher m = _pattern.matcher(counters);
while(m.find()){
String ctuple = m.group(0);
//String ctuple = c1tuple.substring(0, c1tuple.length()-1);
....
...
.
{code}
This might not be efficient but will will reduce code change and testing time. So basically what you will be doing here is you will convert the new format to old format and pass to the code.
----
Few questions :
# Seems like there is a parser written here for job history. Why cant you use the existing job history parser?
# Seems like counters are maintained as key val pair. Why can a global counter be used and all new counter can simply be added to it?
I have not seen the whole code. These questions are based on the impression that I am getting looking at the patch.
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Assignee: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya-0.21.0-5582-5764.patch, vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12698453#action_12698453 ]
Suhas Gogate commented on HADOOP-5582:
--------------------------------------
does not have a formal automated tests suite available for it yet.
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5582) Hadoop Vaidya throws number format
exception due to changes in the job history counters string format (escaped
compact representation).
Posted by "Suhas Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12698448#action_12698448 ]
Suhas Gogate commented on HADOOP-5582:
--------------------------------------
1. This is a standalone contrib tool and does not have a formal automated tests suite build for it yet.
2. Failed tests are unrelated.
So this patch should go in.
> Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5582
> URL: https://issues.apache.org/jira/browse/HADOOP-5582
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.20.0
> Reporter: Suhas Gogate
> Fix For: 0.21.0
>
> Attachments: vaidya_patch_21.patch
>
>
> Hadoop Vaidya (contrib/vaidya) tool throws number format exception while parsing the job history log files due to change in the format of counters string in 0.20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.