You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ramya Sunil (JIRA)" <ji...@apache.org> on 2011/08/11 20:28:27 UTC
[jira] [Created] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
[MR-279] Missing fields in job summary logs
--------------------------------------------
Key: MAPREDUCE-2821
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Priority: Minor
Fix For: 0.23.0
The following fields are missing in the job summary logs in mrv2:
- numSlotsPerMap
- numSlotsPerReduce
- clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Arun C Murthy (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy reassigned MAPREDUCE-2821:
----------------------------------------
Assignee: Mahadev konar (was: Harsh J)
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133807#comment-13133807 ]
Mahadev konar commented on MAPREDUCE-2821:
------------------------------------------
@Harsh,
Mind if I take this over?
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Harsh J
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13109186#comment-13109186 ]
Arun C Murthy commented on MAPREDUCE-2821:
------------------------------------------
We can add 'resourcePerMap'/'resourcePerReduce' instead...
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Harsh J
> Priority: Blocker
> Fix For: 0.23.0
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134884#comment-13134884 ]
Hudson commented on MAPREDUCE-2821:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1150 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1150/])
MAPREDUCE-2821. Added missing fields (resourcePerMap & resourcePerReduce) to JobSummary logs. Contributed by Mahadev Konar.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188528
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobSummary.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobUnsuccessfulCompletionEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/NormalizedResourceEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskStartedEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEvents.java
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-2821:
-------------------------------------
Attachment: MAPREDUCE-2821.patch
WIP. This adds the normalized map/reduce resource requirements to the job summary log. Pending some testing/unit tests.
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Harsh J
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2821:
-------------------------------------
Status: Open (was: Patch Available)
Needs to be rebased, sorry!
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135051#comment-13135051 ]
Hudson commented on MAPREDUCE-2821:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #842 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/842/])
MAPREDUCE-2821. Added missing fields (resourcePerMap & resourcePerReduce) to JobSummary logs. Contributed by Mahadev Konar.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188528
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobSummary.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobUnsuccessfulCompletionEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/NormalizedResourceEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskStartedEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEvents.java
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-2821:
-------------------------------------
Attachment: MAPREDUCE-2821.patch
Forgot to clean up some minor nits in the patch.
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127960#comment-13127960 ]
Arun C Murthy commented on MAPREDUCE-2821:
------------------------------------------
Just use the configs mapreduce.job.map.memory.mb ...
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Harsh J
> Priority: Blocker
> Fix For: 0.23.0
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134864#comment-13134864 ]
Hudson commented on MAPREDUCE-2821:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #48 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/48/])
Merge -c 1188528 from trunk to branch-0.23 to complete fix for MAPREDUCE-2821.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188529
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobSummary.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobUnsuccessfulCompletionEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/NormalizedResourceEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskStartedEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEvents.java
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-2821:
-------------------------------------
Attachment: MAPREDUCE-2821.patch
Updated patch.
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135027#comment-13135027 ]
Hudson commented on MAPREDUCE-2821:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #871 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/871/])
MAPREDUCE-2821. Added missing fields (resourcePerMap & resourcePerReduce) to JobSummary logs. Contributed by Mahadev Konar.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188528
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobSummary.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobUnsuccessfulCompletionEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/NormalizedResourceEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskStartedEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEvents.java
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114167#comment-13114167 ]
Harsh J commented on MAPREDUCE-2821:
------------------------------------
I did get started on this, but my activity died down during finding where the JIP:getNumSlotsPerMap(…) form of logic has moved to. Any help on finding that? I somehow do not see it being computed inside of JobImpl at least.
Sorry, first patch over MR2. Feel free to reassign if there's a great urgency, else I'll trudge on with your help.
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Harsh J
> Priority: Blocker
> Fix For: 0.23.0
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096752#comment-13096752 ]
Harsh J commented on MAPREDUCE-2821:
------------------------------------
Per code comments, these appear to have been disabled for a reason:
{code}
// private int numSlotsPerMap; | Doesn't make sense with potentially different
// resource models
// private int numSlotsPerReduce; | Doesn't make sense with potentially
// different resource models
{code}
So is your reasoning atop these?
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Harsh J
> Priority: Minor
> Fix For: 0.23.0
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2821:
-------------------------------------
Priority: Blocker (was: Minor)
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Harsh J
> Priority: Blocker
> Fix For: 0.23.0
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114045#comment-13114045 ]
Arun C Murthy commented on MAPREDUCE-2821:
------------------------------------------
Harsh - will you have time to work on this anytime soon? Tx
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Harsh J
> Priority: Blocker
> Fix For: 0.23.0
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-2821:
-------------------------------------
Attachment: MAPREDUCE-2821.patch
Tested patch with added test. Includes some fixing of javac warning/unwanted imports and some indentation. I know should have filed another jira but couldnt keep myself from not doing it. Also, the new history event is not serialized (didnt want to break rumen/others). This should be fine except wehn the AM restarts when all the tasks are run and it just has to write the summary logs. In that case the resources logged would be 0.
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-2821:
-------------------------------------
Status: Patch Available (was: Open)
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134483#comment-13134483 ]
Hadoop QA commented on MAPREDUCE-2821:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12500534/MAPREDUCE-2821.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 160 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1124//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1124//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1124//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1124//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1124//console
This message is automatically generated.
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134874#comment-13134874 ]
Hudson commented on MAPREDUCE-2821:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #48 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/48/])
Merge -c 1188528 from trunk to branch-0.23 to complete fix for MAPREDUCE-2821.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188529
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobSummary.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobUnsuccessfulCompletionEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/NormalizedResourceEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskStartedEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEvents.java
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J reassigned MAPREDUCE-2821:
----------------------------------
Assignee: Harsh J
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Harsh J
> Priority: Minor
> Fix For: 0.23.0
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129989#comment-13129989 ]
Mahadev konar commented on MAPREDUCE-2821:
------------------------------------------
Harsh,
any update on this?
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Harsh J
> Priority: Blocker
> Fix For: 0.23.0
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2821:
-------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks Mahadev!
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-2821:
-------------------------------------
Status: Patch Available (was: Open)
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134888#comment-13134888 ]
Hudson commented on MAPREDUCE-2821:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #49 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/49/])
Merge -c 1188528 from trunk to branch-0.23 to complete fix for MAPREDUCE-2821.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188529
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobSummary.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobUnsuccessfulCompletionEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/NormalizedResourceEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskStartedEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEvents.java
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134896#comment-13134896 ]
Hudson commented on MAPREDUCE-2821:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1164 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1164/])
MAPREDUCE-2821. Added missing fields (resourcePerMap & resourcePerReduce) to JobSummary logs. Contributed by Mahadev Konar.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188528
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobSummary.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobUnsuccessfulCompletionEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/NormalizedResourceEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskStartedEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEvents.java
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134879#comment-13134879 ]
Hudson commented on MAPREDUCE-2821:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1228 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1228/])
MAPREDUCE-2821. Added missing fields (resourcePerMap & resourcePerReduce) to JobSummary logs. Contributed by Mahadev Konar.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188528
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobSummary.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobUnsuccessfulCompletionEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/NormalizedResourceEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskStartedEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEvents.java
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134995#comment-13134995 ]
Hudson commented on MAPREDUCE-2821:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #50 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/50/])
Merge -c 1188528 from trunk to branch-0.23 to complete fix for MAPREDUCE-2821.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188529
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobSummary.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobUnsuccessfulCompletionEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/NormalizedResourceEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskStartedEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEvents.java
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2821) [MR-279] Missing fields in job
summary logs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135033#comment-13135033 ]
Hudson commented on MAPREDUCE-2821:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #62 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/62/])
Merge -c 1188528 from trunk to branch-0.23 to complete fix for MAPREDUCE-2821.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188529
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobSummary.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobUnsuccessfulCompletionEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/NormalizedResourceEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskStartedEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEvents.java
> [MR-279] Missing fields in job summary logs
> --------------------------------------------
>
> Key: MAPREDUCE-2821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch, MAPREDUCE-2821.patch
>
>
> The following fields are missing in the job summary logs in mrv2:
> - numSlotsPerMap
> - numSlotsPerReduce
> - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x)
> The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira