You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Wangda Tan (JIRA)" <ji...@apache.org> on 2017/10/16 18:26:00 UTC
[jira] [Comment Edited] (YARN-7307) Revisit resource-types.xml
loading behaviors
[ https://issues.apache.org/jira/browse/YARN-7307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16206347#comment-16206347 ]
Wangda Tan edited comment on YARN-7307 at 10/16/17 6:25 PM:
------------------------------------------------------------
Thanks [~sunilg] for updating the patch,
Few comments:
1) Could we add a new static method to ResourceUtils to initialize from ResourceTypeInfo directly, and move logics below to the method:
{code}
225 List<ResourceTypeInfo> resourceTypes = getResourceTypeInfo();
226 Map<String, ResourceInformation> resourceInformationMap = new HashMap<>();
227
228 for (ResourceTypeInfo resourceType : resourceTypes) {
229 resourceInformationMap.put(resourceType.getName(),
230 ResourceInformation.newInstance(resourceType.getName(),
231 resourceType.getDefaultUnit(), resourceType.getResourceType()));
232 }
{code}
2) This patch updated YarnClientImpl service start to set resource types directly. I'm not sure if we should move this to a separate method so client can opt-out initialize resource types. I can see the biggest advantage of your approach is client can get resource types updated without changing any code if YarnClient initialized and started before any resource object created. However since applications need to update their code in order to use this feature anyway, I'm not sure if it is the best option to add this call inside serviceStart.
Another (safer) option is to add a boolean flag to getResourceTypeInfo to indicate that if client want to update ResourceUtils with the method call or not.
The same comment applies to AM.
3) Do you want to add changes to AMRMClient for now or move it to a separate ticket?
(I prefer to move it to a separate ticket.)
was (Author: leftnoteasy):
Thanks [~sunilg] for updating the patch,
Few comments:
1) Could we add a new static method to ResourceUtils to initialize from ResourceTypeInfo directly, and move logics below to the method:
{code}
225 List<ResourceTypeInfo> resourceTypes = getResourceTypeInfo();
226 Map<String, ResourceInformation> resourceInformationMap = new HashMap<>();
227
228 for (ResourceTypeInfo resourceType : resourceTypes) {
229 resourceInformationMap.put(resourceType.getName(),
230 ResourceInformation.newInstance(resourceType.getName(),
231 resourceType.getDefaultUnit(), resourceType.getResourceType()));
232 }
{code}
2) This patch updated YarnClientImpl service start to set resource types directly. I'm not sure if we should move this to a separate method so client can opt-out initialize resource types. I can see the biggest advantage of your approach is client can get resource types updated without changing any code if YarnClient initialized and started before any resource object created. However since applications need to update their code in order to use this feature anyway, I'm not sure if it is the best option to add this call inside serviceStart.
Another (safer) option is to add a boolean flag to getResourceTypeInfo to indicate that if client want to update ResourceUtils with the method call or not.
The same comment applies to AM.
3) Do you want to add changes to AMRMClient for now or move it to a separate ticket?
> Revisit resource-types.xml loading behaviors
> --------------------------------------------
>
> Key: YARN-7307
> URL: https://issues.apache.org/jira/browse/YARN-7307
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Reporter: Wangda Tan
> Assignee: Sunil G
> Priority: Blocker
> Attachments: YARN-7307.001.patch, YARN-7307.002.patch
>
>
> Existing feature requires every client has a resource-types.xml in order to use multiple resource types, should we allow client/AM update supported resource types via Yarn APIs?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org