[jira] [Commented] (MAPREDUCE-5870) Support for passing Job priority through Application Submission Context in Mapreduce Side

["Eric Payne (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662064#comment-14662064 ] 

Eric Payne commented on MAPREDUCE-5870:
---------------------------------------

[~sunilg], for what it's worth, I have downloaded the latest patch (version 003) and tested and verified it in conjunction with the changes that were made for YARN-2003.

I performed the following sleep jobs with 10 tasks each. My one-node cluster can run 5 containers at once.
- I submit sleep job1 to the default queue, setting {{-Dmapreduce.job.priority=LOW}}
- Job1 starts running 5 containers and has 5 tasks pending.
- I submit sleep job2 to the default queue, setting {{-Dmapreduce.job.priority=HIGH}}
- All 10 job2 tasks are pending.
- Once tasks from job1 complete, job2 gets the containers. Although job1 has 5 tasks pending, the number of running tasks for job1 remains 0 until job2 has no more pending tasks and job2's running tasks begin to complete.
- At that point, job1's tasks begin again to receive containers.

I also verified that you can specify {{-Dmapreduce.job.priority=_number_}}, and the container allocations go to the higher numbered jobs.
Finally, I verified that if you make the priority higher than the cluster max, it silently sets the job priority to cluster max.

So, the bottom line is LGTM :-)
+1

> Support for passing Job priority through Application Submission Context in Mapreduce Side
> -----------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5870
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5870
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: client
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: 0001-MAPREDUCE-5870.patch, 0002-MAPREDUCE-5870.patch, 0003-MAPREDUCE-5870.patch, Yarn-2002.1.patch
>
>
> Job Prioirty can be set from client side as below [Configuration and api].
> 			a.	JobConf.getJobPriority() and Job.setPriority(JobPriority priority) 
> 			b.	We can also use configuration "mapreduce.job.priority".
> 		Now this Job priority can be passed in Application Submission context from Client side.
> 		Here we can reuse the MRJobConfig.PRIORITY configuration. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)