[jira] Commented: (HADOOP-4211) Capacity Scheduler does not divide queue resources properly among users, when jobs are submitted one after other.

["Karam Singh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-4211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632510#action_12632510 ] 

Karam Singh commented on HADOOP-4211:
-------------------------------------

Assumption: User are submitting the jobs one after another and not at the same time.

Here is test scenario:
User1 submits job J1 with 612 maps and 408 reduces.
When J1 starts running it uses all resources i.e 204 maps and 204 reducers, user2 submits job J2 with 204 maps and 204 reduces.
J2 will be queued until  running maps of  J1 starts getting completed.

The Observation is -:
When both J1 and J2 are running -:
J1 is running 153 maps equal to 75% of resources
J2 is running 51 maps equal to 25% of resources.

Similarly User3 submit J3 then:
J1 runs 102 maps equal to 50% of resources
J2 runs 51 maps equal to 25% of resources
J3 runs 51 maps equal to 25% of resources

According to documentation -: When there two users running jobs then resources will be shared 50% by both users.
And Similarly When there are three users running jobs then resources will be shared 33% by users.

Note -: If all users submit jobs at the same time, the resources are shared equally 50% (each for 2 user jobs) and 33% (each for 3 user jobs). 



> Capacity Scheduler does not divide queue resources properly among users, when jobs are submitted one after other.
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-4211
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4211
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/capacity-sched
>    Affects Versions: 0.19.0
>         Environment: Mapred Cluster capacity with 204 Maps and 204 Reduces. User limit =25% and only one queue.
>            Reporter: Karam Singh
>
> Capacity Scheduler does not divide queue resources  properly among users, when job are submitted one after other. E.g. user limit =25. Say User1's job is running. Then user2 submits a job. Then user1's job uses 75% and user2's job 25%=user limit.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.