You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Amar Kamat (JIRA)" <ji...@apache.org> on 2008/09/16 16:58:44 UTC
[jira] Created: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Changes to JobHistory makes it backward incompatible
----------------------------------------------------
Key: HADOOP-4190
URL: https://issues.apache.org/jira/browse/HADOOP-4190
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Reporter: Amar Kamat
With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4190:
-------------------------------
Attachment: HADOOP-4190-v6.1.patch
Attaching an updated patch the incorporates Amareshwari's comment.
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch, HADOOP-4190-v6.1.patch, HADOOP-4190-v6.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12634748#action_12634748 ]
Amareshwari Sriramadasu commented on HADOOP-4190:
-------------------------------------------------
one minor nit - I think the newly added enums should be added at the end.
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch, HADOOP-4190-v6.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-4190:
------------------------------------
Release Note: Changed job history format to add a dot at end of each line.
Hadoop Flags: [Incompatible change, Reviewed] (was: [Reviewed])
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch, HADOOP-4190-v6.1.patch, HADOOP-4190-v6.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645771#action_12645771 ]
Hudson commented on HADOOP-4190:
--------------------------------
Integrated in Hadoop-trunk #654 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/654/])
. Adding testcase that I forgot to add earlier.
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch, HADOOP-4190-v6.1.patch, HADOOP-4190-v6.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635289#action_12635289 ]
Amareshwari Sriramadasu commented on HADOOP-4190:
-------------------------------------------------
+1 Latest patch looks good
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch, HADOOP-4190-v6.1.patch, HADOOP-4190-v6.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4190:
-------------------------------
Attachment: HADOOP-4190-v4.patch
Attaching a patch that takes care of the escaping/un-escaping business.
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4190:
-------------------------------
Status: Patch Available (was: Open)
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4190:
-------------------------------
Status: Patch Available (was: Open)
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632867#action_12632867 ]
Arun C Murthy commented on HADOOP-4190:
---------------------------------------
Forgot to add that this testcase does not fail *without* this patch...
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12633671#action_12633671 ]
Amar Kamat commented on HADOOP-4190:
------------------------------------
Result of test-patch on branch-19
{noformat}
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 6 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
{noformat}
Running ant-test.
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12634581#action_12634581 ]
Hadoop QA commented on HADOOP-4190:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12390907/HADOOP-4190-v6.patch
against trunk revision 698721.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3370/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3370/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3370/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3370/console
This message is automatically generated.
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch, HADOOP-4190-v6.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4190:
-------------------------------
Attachment: HADOOP-4190-v3.patch
Attaching a patch that
- adds version to job history which is logged to the history file as the 1st line
- switches to old/default regex for line-end matching if the version info is missing else uses the new regex
- adds a test case that tests the job history parser with old and new file formats
Manually tested the patch with old and new history files and works fine. More testing in progress. Will try and make it more cleaner and optimized.
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632819#action_12632819 ]
Arun C Murthy commented on HADOOP-4190:
---------------------------------------
Did this patch pass 'ant test-patch test' ?
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated HADOOP-4190:
-------------------------------------
Fix Version/s: 0.19.0
Assignee: Amar Kamat
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Fix For: 0.19.0
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635412#action_12635412 ]
Hudson commented on HADOOP-4190:
--------------------------------
Integrated in Hadoop-trunk #618 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/618/])
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch, HADOOP-4190-v6.1.patch, HADOOP-4190-v6.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4190:
-------------------------------
Attachment: HADOOP-4190-v4.patch
Thanks Arun for the ant-test. The test failed because of missing version line.
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v4.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4190:
-------------------------------
Status: Open (was: Patch Available)
Will resubmit the patch that applies to trunk.
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632320#action_12632320 ]
Arun C Murthy commented on HADOOP-4190:
---------------------------------------
+1 to the concept.
As an aside: please ensure this patch works post HADOOP-2165 too. Thanks!
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12633882#action_12633882 ]
Hadoop QA commented on HADOOP-4190:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12390563/HADOOP-4190-v5.patch
against trunk revision 698187.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3354/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3354/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3354/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3354/console
This message is automatically generated.
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631429#action_12631429 ]
Amar Kamat commented on HADOOP-4190:
------------------------------------
Ideally there should be some kind of a version attached to JobHistory. A parser can parse a history file only if their version matches. Thoughts?
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4190:
-------------------------------
Attachment: HADOOP-4190-v5.patch
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631472#action_12631472 ]
Doug Cutting commented on HADOOP-4190:
--------------------------------------
+1 Adding a format version to the file is definitely a good idea.
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4190:
-------------------------------
Status: Patch Available (was: Open)
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-4190:
----------------------------------
Status: Open (was: Patch Available)
I ran 'ant test' and TestJobHistoryParsing failed with:
{noformat}
estsuite: org.apache.hadoop.mapred.TestJobHistoryParsing
Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 0.307 sec
Testcase: testHistoryParsing took 0.305 sec
FAILED
expected:<Value has equal=to, "quotes" and spaces in it> but was:<>
junit.framework.ComparisonFailure: expected:<Value has equal=to, "quotes" and spaces in it> but was:<>
at org.apache.hadoop.mapred.TestJobHistoryParsing.testHistoryParsing(TestJobHistoryParsing.java:93)
{noformat}
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12634739#action_12634739 ]
Amar Kamat commented on HADOOP-4190:
------------------------------------
The findbugs warning is because of the {{IOException}} which the parser ignores while it tries to close the stream. Not something that I have introduced. Here is a section from the patch that explains the warning
{code}
- }
+ } while ((line = reader.readLine())!= null);
} finally {
try { reader.close(); } catch (IOException ex) {}
{code}
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch, HADOOP-4190-v6.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4190:
-------------------------------
Attachment: HADOOP-4190-v6.patch
Had an off-line discussion with Devaraj. Devaraj suggested that the line delimiter would not change across versions. Hence later versions should also use '.' as the delim. Attaching a patch that incorporates these comments.
_ant test-patch_ fails on findbugs. Didnt see any thing related to my patch. _ant test_ passed on my box.
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch, HADOOP-4190-v6.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4190:
-------------------------------
Status: Open (was: Patch Available)
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch, HADOOP-4190-v6.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4190:
-------------------------------
Priority: Blocker (was: Major)
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-4190:
--------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Amar!
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch, HADOOP-4190-v6.1.patch, HADOOP-4190-v6.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4190:
-------------------------------
Attachment: (was: HADOOP-4190-v4.patch)
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4190) Changes to JobHistory makes it
backward incompatible
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4190:
-------------------------------
Status: Patch Available (was: Open)
> Changes to JobHistory makes it backward incompatible
> ----------------------------------------------------
>
> Key: HADOOP-4190
> URL: https://issues.apache.org/jira/browse/HADOOP-4190
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4190-v3.patch, HADOOP-4190-v4.patch, HADOOP-4190-v5.patch, HADOOP-4190-v6.patch
>
>
> With HADOOP-3245, the job history format has changed. Every log line now ends with a '.' (dot). JobHistory's parser will consider a history line valid only if it ends in a dot
> Hence the new parser wont be able to parse old history files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.