You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Konstantinos Karanasos (JIRA)" <ji...@apache.org> on 2016/08/04 23:58:20 UTC
[jira] [Comment Edited] (YARN-4902) [Umbrella] Generalized and
unified scheduling-strategies in YARN
[ https://issues.apache.org/jira/browse/YARN-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408657#comment-15408657 ]
Konstantinos Karanasos edited comment on YARN-4902 at 8/4/16 11:57 PM:
-----------------------------------------------------------------------
I am uploading a design document that describes our vision for scheduling long-running applications (LRA).
It is a very initial version, but I am sharing it, so that it helps drive the discussion.
There are overlapping bits with this JIRA (after all, up to a point, it targets the same problem), but there are clearly new points, especially when it comes to LRA planning.
As I had explained to [~leftnoteasy] offline during the Hadoop Summit, our focus is not on the scheduling given affinity/anti-affinity constraints, but on the LRA *planning*.
We did a first implementation of affinity, anti-affinity and *cardinality* constraints, because it was required for us to proceed with the LRA planning and nothing was available at that time.
[That said, we have already added support for cardinality and I think we have a different support for tags (but I need to take a closer look on YARN-1042) -- let's continue the discussion at that JIRA.]
Given that Wangda marked YARN-5468 as duplicate, do you believe that the LRA planing belongs to this or another existing JIRA?
As far as I can tell, it does not.
Let me know what you think, so that we can use the proper JIRAs and avoid duplicate effort going forward.
Thanks.
was (Author: kkaranasos):
I am uploading a design document that describes our vision for scheduling long-running applications (LRA).
It is a very initial version, but I am sharing it, so that it helps drive the discussion.
There are overlapping bits with this JIRA (after all, up to a point, it targets the same problem), but there are clearly new points, especially when it comes to LRA planning.
As I had explained to [~leftnoteasy] offline during the Hadoop Summit, our focus is not on the scheduling given affinity/anti-affinity constraints, but on the LRA *planning*.
We did a first implementation of affinity, anti-affinity and *cardinality* constraints, because it was required for us to proceed with the LRA planning and nothing was available at that time.
[That said, we have already added support for cardinality and I think we have a different support for tags (but I need to take a closer look on YARN-1042) -- let's continue the discussion at that JIRA.]
Given that Wangda marked YARN-5048 as duplicate, do you believe that the LRA planing belongs to this or another existing JIRA?
As far as I can tell, it does not.
Let me know what you think, so that we can use the proper JIRAs and avoid duplicate effort going forward.
Thanks.
> [Umbrella] Generalized and unified scheduling-strategies in YARN
> ----------------------------------------------------------------
>
> Key: YARN-4902
> URL: https://issues.apache.org/jira/browse/YARN-4902
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Wangda Tan
> Attachments: Generalized and unified scheduling-strategies in YARN -v0.pdf, LRA-scheduling-design.v0.pdf, YARN-5468.prototype.patch
>
>
> Apache Hadoop YARN's ResourceRequest mechanism is the core part of the YARN's scheduling API for applications to use. The ResourceRequest mechanism is a powerful API for applications (specifically ApplicationMasters) to indicate to YARN what size of containers are needed, and where in the cluster etc.
> However a host of new feature requirements are making the API increasingly more and more complex and difficult to understand by users and making it very complicated to implement within the code-base.
> This JIRA aims to generalize and unify all such scheduling-strategies in YARN.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org