You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Zhankun Tang (JIRA)" <ji...@apache.org> on 2018/11/03 09:01:00 UTC
[jira] [Commented] (YARN-8851) [Umbrella] A new pluggable device
plugin framework to ease vendor plugin development
[ https://issues.apache.org/jira/browse/YARN-8851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673972#comment-16673972 ]
Zhankun Tang commented on YARN-8851:
------------------------------------
[~leftnoteasy] [~csingh] . Thanks for the review. After offline discussion with wangda, we'll prefer to give below one interface for the plugin to prepare device and reply DeviceRuntimeSpec after device allocated.
Also, we can improve it as time goes by.
{code:java}
/**
* Asking how these devices should be prepared/used
* before/when container launch. A plugin can do some tasks in its own or
* define it in DeviceRuntimeSpec to let the framework do it.
* For instance, define {@code VolumeSpec} to let the
* framework to create volume before running container.
*
* @param allocatedDevices A set of allocated {@link Device}.
* @param yarnRuntime Indicate which runtime YARN will use
* Could be {@code docker} or {@code default}
* in {@link DeviceRuntimeSpec} constants
* @return a {@link DeviceRuntimeSpec} description about environment,
* {@link VolumeSpec}, {@link MountVolumeSpec}. etc
* */
DeviceRuntimeSpec onDeviceAllocated(Set<Device> allocatedDevices,
String yarnRuntime);{code}
> [Umbrella] A new pluggable device plugin framework to ease vendor plugin development
> ------------------------------------------------------------------------------------
>
> Key: YARN-8851
> URL: https://issues.apache.org/jira/browse/YARN-8851
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: yarn
> Reporter: Zhankun Tang
> Assignee: Zhankun Tang
> Priority: Major
> Attachments: YARN-8851-WIP2-trunk.001.patch, YARN-8851-WIP3-trunk.001.patch, YARN-8851-WIP4-trunk.001.patch, YARN-8851-WIP5-trunk.001.patch, YARN-8851-WIP6-trunk.001.patch, YARN-8851-WIP7-trunk.001.patch, YARN-8851-WIP8-trunk.001.patch, YARN-8851-WIP9-trunk.001.patch, [YARN-8851] YARN_New_Device_Plugin_Framework_Design_Proposal-3.pdf, [YARN-8851] YARN_New_Device_Plugin_Framework_Design_Proposal.pdf
>
>
> At present, we support GPU/FPGA device in YARN through a native, coupling way. But it's difficult for a vendor to implement such a device plugin because the developer needs much knowledge of YARN internals. And this brings burden to the community to maintain both YARN core and vendor-specific code.
> Here we propose a new device plugin framework to ease vendor device plugin development and provide a more flexible way to integrate with YARN NM.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org