You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Arun Suresh (JIRA)" <ji...@apache.org> on 2017/12/29 07:00:16 UTC
[jira] [Comment Edited] (YARN-7666) Introduce scheduler specific
environment variable support in ASC for better scheduling placement
configurations
[ https://issues.apache.org/jira/browse/YARN-7666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306046#comment-16306046 ]
Arun Suresh edited comment on YARN-7666 at 12/29/17 7:00 AM:
-------------------------------------------------------------
Hey [~sunilg], thanks for the patch.
I think having an String -> String mapping provided by the Client is definitely useful, but rather than hooking it all the way to the RM, I would like to somehow send this mapping to the AM. And then let the AM use this mapping to negotiate some policy with the RM (possibly via the registerAM call). This I feel has a couple of benefits:
# ensure that Client <-> RM commmunication is restricted to just negotiating the start of the AM and nothing more. Everything else should be handled by the AMRMProtocol.
# We can leverage the fact that failovers require AM to re-register with the RM - Since there is no such requirement for the Client-RM protocol, we have to explicitly deal with persisting any user provided information, while in the former case - we don't, since we know the AM will re-register (and any required state will be resent and thus auto-recovered)
# We can reuse this for YARN-6592 - essentially, the Client can use some tool to serialize and push to the AM some placementConstraints and when the AM starts up, it can deserialize this PlacementConstraints and use it to register with the RM.
was (Author: asuresh):
Hey [~sunilg], thanks for the patch.
I think having an String -> String mapping provided by the Client is definitely useful, but rather than hooking it all the way to the RM, lets somehow send this mapping to the AM. And then let the AM use this mapping to negotiate some policy with the RM (possibly via the registerAM call). This I feel has a couple of benefits:
# ensure that Client <-> RM commmunication is restricted to just negotiating the start of the AM and nothing more.
# We can leverage the fact that failovers require AM to re-register with the RM - Since there is no such requirement for the Client-RM protocol, we have to explicitly deal with persisting any user provided information, while in the former case - we don't, since we know the AM will re-register (and any required state will be resent and thus auto-recovered)
# We can reuse this for YARN-6592 - essentially, the Client can use some tool to serialize and push to the AM some placementConstraints and when the AM starts up, it can deserialize this PlacementConstraints and use it to register with the RM.
> Introduce scheduler specific environment variable support in ASC for better scheduling placement configurations
> ---------------------------------------------------------------------------------------------------------------
>
> Key: YARN-7666
> URL: https://issues.apache.org/jira/browse/YARN-7666
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Sunil G
> Assignee: Sunil G
> Attachments: YARN-7666.001.patch, YARN-7666.002.patch
>
>
> Introduce a scheduler specific key-value map to hold env variables in ASC.
> And also convert AppPlacementAllocator initialization to each app based on policy configured at each app.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org