You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Juan Rodríguez Hortalá (JIRA)" <ji...@apache.org> on 2017/05/04 17:23:04 UTC
[jira] [Commented] (YARN-6483) Add nodes transitioning to
DECOMMISSIONING state to the list of updated nodes returned by the Resource
Manager as a response to the Application Master heartbeat
[ https://issues.apache.org/jira/browse/YARN-6483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997111#comment-15997111 ]
Juan Rodríguez Hortalá commented on YARN-6483:
----------------------------------------------
Hi [~Naganarasimha], sorry for the late answer. I have attached a patch for commit 103dfae7b9bbcec673dd7c53d38df034b78693a2 that is the current head of branch-2.8 in https://github.com/apache/hadoop. However since I opened this issue I had the opportunity to study the umbrella YARN-914 for decommissioning and I see it already has a sub-task YARN-3224 for notifying the AM of the node decommission. The design proposed on YARN-914 uses the preemption framework described in the umbrella YARN-45. There has been not much activity lately in YARN-3224, and it seems to be blocked on the sub-task YARN-3784 of the preemption umbrella YARN-45. However my patch here is not able to send the decommission timeout, as it is based on `NodeState` that is an enum that doesn't have any field. I don't think it would be a good idea to add a timeout field to `NodeState` because it doesn't make sense for all enum values. So I think the options here are:
# Either modify `NodeState` to replace the enum by a class hierarchy, so the subclass for decommissioning might have a timeout. This looks like a significant modification, that anyway diverges from the design of YARN-914
# Or implement the missing issues for the preemption framework and the decommissioning umbrella, in particular YARN-3784 and then YARN-3224, that would replace this issue
Which one of those do you think it would be the best option?
> Add nodes transitioning to DECOMMISSIONING state to the list of updated nodes returned by the Resource Manager as a response to the Application Master heartbeat
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-6483
> URL: https://issues.apache.org/jira/browse/YARN-6483
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
> Affects Versions: 2.8.0
> Reporter: Juan Rodríguez Hortalá
> Attachments: YARN-6483-v1.patch
>
>
> The DECOMMISSIONING node state is currently used as part of the graceful decommissioning mechanism to give time for tasks to complete in a node that is scheduled for decommission, and for reducer tasks to read the shuffle blocks in that node. Also, YARN effectively blacklists nodes in DECOMMISSIONING state by assigning them a capacity of 0, to prevent additional containers to be launched in those nodes, so no more shuffle blocks are written to the node. This blacklisting is not effective for applications like Spark, because a Spark executor running in a YARN container will keep receiving more tasks after the corresponding node has been blacklisted at the YARN level. We would like to propose a modification of the YARN heartbeat mechanism so nodes transitioning to DECOMMISSIONING are added to the list of updated nodes returned by the Resource Manager as a response to the Application Master heartbeat. This way a Spark application master would be able to blacklist a DECOMMISSIONING at the Spark level.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org