You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ramya Sunil (JIRA)" <ji...@apache.org> on 2011/08/10 02:23:27 UTC

[jira] [Created] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

[MR-279] Incorrect metrics value for AvailableGB per queue per user
-------------------------------------------------------------------

                 Key: MAPREDUCE-2794
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv2
    Affects Versions: 0.23.0
            Reporter: Ramya Sunil
             Fix For: 0.23.0


AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 

Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121136#comment-13121136 ] 

Hadoop QA commented on MAPREDUCE-2794:
--------------------------------------

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12497806/MAPREDUCE-2794.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 6 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    +1 core tests.  The patch passed unit tests in .

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/952//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/952//console

This message is automatically generated.
                
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122536#comment-13122536 ] 

Mahadev konar commented on MAPREDUCE-2794:
------------------------------------------

+1 the patch looks good. Will go ahead and commit.
                
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122783#comment-13122783 ] 

Hudson commented on MAPREDUCE-2794:
-----------------------------------

Integrated in Hadoop-Mapreduce-trunk #853 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/853/])
    MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev)

mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179936
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java

                
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122727#comment-13122727 ] 

Hudson commented on MAPREDUCE-2794:
-----------------------------------

Integrated in Hadoop-Hdfs-trunk #823 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/823/])
    MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev)

mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179936
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java

                
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122765#comment-13122765 ] 

Hudson commented on MAPREDUCE-2794:
-----------------------------------

Integrated in Hadoop-Mapreduce-0.23-Build #39 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/39/])
    MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev) - Merging r1179936 from trunk.

mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179937
Files : 
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java

                
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122543#comment-13122543 ] 

Hudson commented on MAPREDUCE-2794:
-----------------------------------

Integrated in Hadoop-Common-trunk-Commit #1031 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1031/])
    MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev)

mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179936
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java

                
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Assigned] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "John George (Assigned) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

John George reassigned MAPREDUCE-2794:
--------------------------------------

    Assignee: John George
    
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "John George (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

John George updated MAPREDUCE-2794:
-----------------------------------

    Status: Patch Available  (was: Open)
    
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122547#comment-13122547 ] 

Hudson commented on MAPREDUCE-2794:
-----------------------------------

Integrated in Hadoop-Hdfs-trunk-Commit #1109 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1109/])
    MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev)

mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179936
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java

                
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mahadev konar updated MAPREDUCE-2794:
-------------------------------------

      Resolution: Fixed
    Hadoop Flags: Reviewed
          Status: Resolved  (was: Patch Available)

Just committed this. Thanks John!
                
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119900#comment-13119900 ] 

Arun C Murthy commented on MAPREDUCE-2794:
------------------------------------------

John, the patch seems reasonable - can you please add a test case? Thanks.
                
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122553#comment-13122553 ] 

Hudson commented on MAPREDUCE-2794:
-----------------------------------

Integrated in Hadoop-Mapreduce-trunk-Commit #1049 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1049/])
    MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev)

mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179936
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java

                
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "John George (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

John George updated MAPREDUCE-2794:
-----------------------------------

    Attachment: MAPREDUCE-2794.patch

Uploading a patch with unit tests
                
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122747#comment-13122747 ] 

Hudson commented on MAPREDUCE-2794:
-----------------------------------

Integrated in Hadoop-Hdfs-0.23-Build #32 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/32/])
    MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per queue per user. (John George via mahadev) - Merging r1179936 from trunk.

mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179937
Files : 
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java

                
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2794.patch, MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "John George (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

John George updated MAPREDUCE-2794:
-----------------------------------

    Attachment: MAPREDUCE-2794.patch

There are two issues mentioned in this JIRA:
1. AvailableGB metrics for user queue is higher than what it is for the default queue

2. AvailableGB does not reflect the "real time" AvailableGB in the case of user queue.

Issue (1) is because user queue rounds up to the next "minimum allocation". This patch tries to round up all the leaf queues (not just the user leaf queue) to the next minimum allocation. 

Issue (2) is because AvailableGB is not updated every time an allocation is made.

The uploaded patch is to check if the approach is acceptable or if there are any other suggestions to solve this...
                
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2794.patch
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user

Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arun C Murthy updated MAPREDUCE-2794:
-------------------------------------

    Priority: Blocker  (was: Major)

> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Priority: Blocker
>             Fix For: 0.23.0
>
>
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the "default" queue.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira