You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2018/05/08 03:09:00 UTC
[jira] [Comment Edited] (YARN-8255) Allow option to disable flex
for a service component
[ https://issues.apache.org/jira/browse/YARN-8255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466800#comment-16466800 ]
Eric Yang edited comment on YARN-8255 at 5/8/18 3:08 AM:
---------------------------------------------------------
[~leftnoteasy] Recompute and expandable are intertwined. They are not the same thing. At conceptual level, teragen has no dependency of input format. You can add more partitions to get more data generated. Hadoop's own implementation limited this from happening, but this does not mean docker containers should be imposed by the same initialization time limitation. On the other hand, we must optimize the framework for general purpose usage and prevent ourselves from giving too many untested and unsupported options. I think it make sense to reduce the flex options to 2 main types instead of giving all 6 options.
was (Author: eyang):
[~leftnoteasy] Recompute and expandable are intertwined. They are the same thing. At conceptual level, teragen has no dependency of input format. You can add more partitions to get more data generated. Hadoop's own implementation limited this from happening, but this does not mean docker containers should be imposed by the same initialization time limitation. On the other hand, we must optimize the framework for general purpose usage and prevent ourselves from giving too many untested and unsupported options. I think it make sense to reduce the flex options to 2 main types instead of giving all 6 options.
> Allow option to disable flex for a service component
> -----------------------------------------------------
>
> Key: YARN-8255
> URL: https://issues.apache.org/jira/browse/YARN-8255
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn-native-services
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Priority: Major
>
> YARN-8080 implements restart capabilities for service component instances. YARN service components should add an option to disallow flexing to support workloads which are essentially batch/iterative jobs which terminate with restart_policy=NEVER/ON_FAILURE. This could be disabled by default for components where restart_policy=NEVER/ON_FAILURE and enabled by default when restart_policy=ALWAYS(which is the default restart_policy) unless explicitly set at the service spec.
> The option could be exposed as part of the component spec as "allow_flexing".
> cc [~billie.rinaldi] [~gsaha] [~eyang] [~csingh] [~wangda]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org