You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Adam Kramer (JIRA)" <ji...@apache.org> on 2011/01/14 03:28:46 UTC
[jira] Created: (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Job status exceeds 100% in some cases
--------------------------------------
Key: MAPREDUCE-2264
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Reporter: Adam Kramer
I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13010017#comment-13010017 ]
Ravi Gummadi commented on MAPREDUCE-2264:
-----------------------------------------
Hmm. Then that may not be considered/handled properly when adding up sizes of segments to get totalBytes ? The size of segment could be compressed size. But the totalBytesProcessed seems to be uncompressed size.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
> Attachments: more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2264:
---------------------------------
Attachment: more than 100%.bmp
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
> Attachments: more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2264:
-------------------------------------
Status: Open (was: Patch Available)
Cancelling since Devaraj is still adding unit tests.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.20.205.0, 0.22.0
>
> Attachments: MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036974#comment-13036974 ]
Hadoop QA commented on MAPREDUCE-2264:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12479914/MAPREDUCE-2264-trunk.patch
against trunk revision 1125428.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these core unit tests:
org.apache.hadoop.mapred.TestReduceFetch
org.apache.hadoop.mapreduce.TestMapReduceLocal
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/286//testReport/
Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/286//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/286//console
This message is automatically generated.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.20.4, 0.23.0
>
> Attachments: MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2264:
---------------------------------
Affects Version/s: 0.20.205.0
0.20.2
Fix Version/s: (was: 0.23.0)
0.22.0
0.20.205.0
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.20.205.0, 0.22.0
>
> Attachments: MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2264:
---------------------------------
Fix Version/s: 0.23.0
0.20.4
Status: Patch Available (was: Open)
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.20.4, 0.23.0
>
> Attachments: MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K reassigned MAPREDUCE-2264:
------------------------------------
Assignee: Devaraj K
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Attachments: MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496647#comment-13496647 ]
Hadoop QA commented on MAPREDUCE-2264:
--------------------------------------
{color:green}+1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12553396/MAPREDUCE-2264-trunk-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3028//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3028//console
This message is automatically generated.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Labels: critical-0.22.0
> Attachments: MAPREDUCE-2264-0.20.205-1.patch, MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk-1.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2264:
---------------------------------
Status: Patch Available (was: Open)
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.20.205.0, 0.22.0
>
> Attachments: MAPREDUCE-2264-0.20.205-1.patch, MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2264:
---------------------------------
Attachment: MAPREDUCE-2264-0.20.205-1.patch
Sorry for late response.
Updated the patch with unit test.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.20.205.0, 0.22.0
>
> Attachments: MAPREDUCE-2264-0.20.205-1.patch, MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100175#comment-13100175 ]
Devaraj K commented on MAPREDUCE-2264:
--------------------------------------
Updated patch is intended for 0.20.205 version.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.20.205.0, 0.22.0
>
> Attachments: MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12981630#action_12981630 ]
Ravi Gummadi commented on MAPREDUCE-2264:
-----------------------------------------
Is this an issue at job level progress only ? Are task progress percentages for all tasks under 100% ? If a particular task's progress is shown as > 100%, please provide the info of which phase that task is in when you see progress > 100%.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Updated] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2264:
-------------------------------------
Status: Open (was: Patch Available)
Sorry, patch has gone stale. Can you pls rebase it? Tx
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Matt Foley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Foley updated MAPREDUCE-2264:
----------------------------------
Target Version/s: 0.22.0, 1.2.0 (was: 0.22.0, 1.1.1)
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Labels: critical-0.22.0
> Attachments: MAPREDUCE-2264-0.20.205-1.patch, MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk-1.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2264:
---------------------------------
Attachment: MAPREDUCE-2264-0.20.205.patch
It is not applicable for mrv2. Provided patch for 0.20.205 version.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.20.205.0, 0.22.0
>
> Attachments: MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2264:
---------------------------------
Status: Patch Available (was: Open)
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.20.205.0, 0.22.0
>
> Attachments: MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036887#comment-13036887 ]
Devaraj K commented on MAPREDUCE-2264:
--------------------------------------
It was considering compressed bytes size for calculating the totalBytes because of this it is showing the progress as more than 100%.
Provided patch consider the uncompressed size of the data for calculating the progress.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.20.4, 0.23.0
>
> Attachments: MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13010008#comment-13010008 ]
Devaraj K commented on MAPREDUCE-2264:
--------------------------------------
yes, "mapred.compress.map.output" is true.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
> Attachments: more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100162#comment-13100162 ]
Hadoop QA commented on MAPREDUCE-2264:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12493349/MAPREDUCE-2264-0.20.205.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/640//console
This message is automatically generated.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.20.205.0, 0.22.0
>
> Attachments: MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009708#comment-13009708 ]
Devaraj K commented on MAPREDUCE-2264:
--------------------------------------
Hi Ravi,
I am also facing the same problem. Reduce tasks progress is going more than 100% when the reduce input size is high.
{code:title=Merger.java|borderStyle=solid}
private void adjustPriorityQueue(Segment<K, V> reader) throws IOException{
long startPos = reader.getPosition();
boolean hasNext = reader.next();
long endPos = reader.getPosition();
totalBytesProcessed += endPos - startPos;
mergeProgress.set(totalBytesProcessed * progPerByte);
if (hasNext) {
adjustTop();
} else {
pop();
reader.close();
}{code}
Here the totalBytesProcessed is becoming more than the totalBytes and (totalBytesProcessed * progPerByte) is coming more than 1.
I am using the 0.20.2 version and I have HADOOP-5210 ,HADOOP-5572 patches also in the code. Please check the screenshot attached.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13010003#comment-13010003 ]
Ravi Gummadi commented on MAPREDUCE-2264:
-----------------------------------------
Is map output compression turned ON ?
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
> Attachments: more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Sandy Ryza (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sandy Ryza updated MAPREDUCE-2264:
----------------------------------
Attachment: MAPREDUCE-2264-trunk-1.patch
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Labels: critical-0.22.0
> Attachments: MAPREDUCE-2264-0.20.205-1.patch, MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk-1.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106658#comment-13106658 ]
Hadoop QA commented on MAPREDUCE-2264:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12494838/MAPREDUCE-2264-0.20.205-1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/768//console
This message is automatically generated.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.20.205.0, 0.22.0
>
> Attachments: MAPREDUCE-2264-0.20.205-1.patch, MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Matt Foley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Foley updated MAPREDUCE-2264:
----------------------------------
Target Version/s: 0.22.0, 1.1.1 (was: 0.22.0, 1.1.0)
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Labels: critical-0.22.0
> Attachments: MAPREDUCE-2264-0.20.205-1.patch, MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Liyin Liang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12981623#action_12981623 ]
Liyin Liang commented on MAPREDUCE-2264:
----------------------------------------
I think [HADOOP-5210|https://issues.apache.org/jira/browse/HADOOP-5210] has fixed this bug.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100247#comment-13100247 ]
Devaraj K commented on MAPREDUCE-2264:
--------------------------------------
Thanks Ravi for looking into the patch.
I tested this patch with the job which was going beyond 100%.
For the job with two reducers and input data as 150 GB, before this patch it was going around 300% and after applying this patch it worked fine.
I will try to add unit tests for this.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.20.205.0, 0.22.0
>
> Attachments: MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Matt Foley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501525#comment-13501525 ]
Matt Foley commented on MAPREDUCE-2264:
---------------------------------------
moved target version to 1.2.0 upon publishing 1.1.1 RC.
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Labels: critical-0.22.0
> Attachments: MAPREDUCE-2264-0.20.205-1.patch, MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk-1.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Matt Foley (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Foley updated MAPREDUCE-2264:
----------------------------------
Target Version/s: 0.20.206.0, 0.22.0
Fix Version/s: (was: 0.20.205.0)
(was: 0.22.0)
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Attachments: MAPREDUCE-2264-0.20.205-1.patch, MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100219#comment-13100219 ]
Ravi Gummadi commented on MAPREDUCE-2264:
-----------------------------------------
Had a quick look at the patch. Code changes look fine to me.
Devaraj, Did you test this patch with your job(which was giving >100% reduceTaskProgress earlier) ?
Would it be possible for you to add a unit test for this fix ?
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Adam Kramer
> Assignee: Devaraj K
> Fix For: 0.20.205.0, 0.22.0
>
> Attachments: MAPREDUCE-2264-0.20.205.patch, MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2264:
---------------------------------
Attachment: MAPREDUCE-2264-trunk.patch
MAPREDUCE-2264-0.20.3.patch
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
> Attachments: MAPREDUCE-2264-0.20.3.patch, MAPREDUCE-2264-trunk.patch, more than 100%.bmp
>
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2264) Job status exceeds 100% in some
cases
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12981631#action_12981631 ]
Ravi Gummadi commented on MAPREDUCE-2264:
-----------------------------------------
BTW, Which version of hadoop are you using ? Does it already include the fixes/patches of HADOOP-5210 and HADOOP-5572 ?
> Job status exceeds 100% in some cases
> --------------------------------------
>
> Key: MAPREDUCE-2264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
>
> I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.
> I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.
> I suggest that whatever logic is computing these set 99.99% as a hard maximum.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.