You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ramya Sunil (JIRA)" <ji...@apache.org> on 2011/08/10 02:23:27 UTC
[jira] [Created] (MAPREDUCE-2794) [MR-279] Incorrect metrics value
for AvailableGB per queue per user
[MR-279] Incorrect metrics value for AvailableGB per queue per user
-------------------------------------------------------------------
Key: MAPREDUCE-2794
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Fix For: 0.23.0
AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics
value for AvailableGB per queue per user
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121136#comment-13121136 ]
Hadoop QA commented on MAPREDUCE-2794:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12497806/MAPREDUCE-2794.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/952//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/952//console
This message is automatically generated.
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics
value for AvailableGB per queue per user
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122536#comment-13122536 ]
Mahadev konar commented on MAPREDUCE-2794:
------------------------------------------
+1 the patch looks good. Will go ahead and commit.
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics
value for AvailableGB per queue per user
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122783#comment-13122783 ]
Hudson commented on MAPREDUCE-2794:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #853 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/853/])
MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179936
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics
value for AvailableGB per queue per user
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122727#comment-13122727 ]
Hudson commented on MAPREDUCE-2794:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #823 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/823/])
MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179936
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics
value for AvailableGB per queue per user
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122765#comment-13122765 ]
Hudson commented on MAPREDUCE-2794:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #39 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/39/])
MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev) - Merging r1179936 from trunk.
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179937
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics
value for AvailableGB per queue per user
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122543#comment-13122543 ]
Hudson commented on MAPREDUCE-2794:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1031 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1031/])
MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179936
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2794) [MR-279] Incorrect metrics value
for AvailableGB per queue per user
Posted by "John George (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John George reassigned MAPREDUCE-2794:
--------------------------------------
Assignee: John George
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2794) [MR-279] Incorrect metrics value
for AvailableGB per queue per user
Posted by "John George (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John George updated MAPREDUCE-2794:
-----------------------------------
Status: Patch Available (was: Open)
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics
value for AvailableGB per queue per user
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122547#comment-13122547 ]
Hudson commented on MAPREDUCE-2794:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1109 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1109/])
MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179936
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2794) [MR-279] Incorrect metrics value
for AvailableGB per queue per user
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-2794:
-------------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Just committed this. Thanks John!
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics
value for AvailableGB per queue per user
Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119900#comment-13119900 ]
Arun C Murthy commented on MAPREDUCE-2794:
------------------------------------------
John, the patch seems reasonable - can you please add a test case? Thanks.
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics
value for AvailableGB per queue per user
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122553#comment-13122553 ]
Hudson commented on MAPREDUCE-2794:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1049 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1049/])
MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179936
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2794) [MR-279] Incorrect metrics value
for AvailableGB per queue per user
Posted by "John George (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John George updated MAPREDUCE-2794:
-----------------------------------
Attachment: MAPREDUCE-2794.patch
Uploading a patch with unit tests
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics
value for AvailableGB per queue per user
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122747#comment-13122747 ]
Hudson commented on MAPREDUCE-2794:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #32 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/32/])
MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev) - Merging r1179936 from trunk.
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179937
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2794) [MR-279] Incorrect metrics value
for AvailableGB per queue per user
Posted by "John George (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John George updated MAPREDUCE-2794:
-----------------------------------
Attachment: MAPREDUCE-2794.patch
There are two issues mentioned in this JIRA:
1. AvailableGB metrics for user queue is higher than what it is for the default queue
2. AvailableGB does not reflect the "real time" AvailableGB in the case of user queue.
Issue (1) is because user queue rounds up to the next "minimum allocation". This patch tries to round up all the leaf queues (not just the user leaf queue) to the next minimum allocation.
Issue (2) is because AvailableGB is not updated every time an allocation is made.
The uploaded patch is to check if the approach is acceptable or if there are any other suggestions to solve this...
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2794) [MR-279] Incorrect metrics value
for AvailableGB per queue per user
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2794:
-------------------------------------
Priority: Blocker (was: Major)
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2794
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Priority: Blocker
> Fix For: 0.23.0
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1)
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira