[jira] Commented: (HADOOP-4658) User limit is not expanding back properly.

["Karam Singh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647544#action_12647544 ] 

Karam Singh commented on HADOOP-4658:
-------------------------------------

Submitted 4 sleep jobs with different user -:

Job J1, User1, maps=208, reduces=0, sleeptime=120000 milliseconds.
Job J2, User2, maps=208, reduces=0, sleeptime=120000 milliseconds.
Job J3, User3, maps=104, reduces=0, sleeptime=124000 milliseconds.
Job J4, User4, maps=104, reduces=0, sleeptime=124000 milliseconds.

When all job starts running, the status from Jobtacker UI is -:
Cluster Summary: Nodes=104, Maps=208, Reduces=0, Map_Capacity=208, Reduce_Capacity=208, Avg. Task/Node=4.00
Queue: Name=default, running_maps=207, waiting_maps=423
J1 having pending_maps=159, running_maps=52
J2 having pending_maps=159, running-maps=52
J3 having pending_maps=52,  running_maps=52
J4 having pending_maps=52,  running_maps=52


When J3 and J4 finish, status from Jobtacker UI is -:
Cluster Summary: Nodes=104, Maps=208, Reduces=0, Map_Capacity=208, Reduce_Capacity=208, Avg. Task/Node=4.00
Queues: Name=default, running_maps=207, waiting_maps=34

J1 having pending_maps=0,  running_maps=104
J2 having pending_maps=33, running_maps=71
 


Observation -:
1. J2 should also be using 104 as its user-limit should expand back to 50% like J1 whereas it is using 71 slots.
2. Actually running_maps=J1+j2=175, pending_maps=J1+j2=33, where as cluster status and queue information shows maps=208 (running), where slots 33 went.

Note -: speculative execution was true.


> User limit is not expanding back properly.
> ------------------------------------------
>
>                 Key: HADOOP-4658
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4658
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/capacity-sched
>    Affects Versions: 0.19.0
>         Environment: GC=100% nodes=104, map_capacity=208, reduce_capacity=208, user-limit=25%;
>            Reporter: Karam Singh
>
> User limit is not expanding back properly.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.