You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Zhankun Tang (JIRA)" <ji...@apache.org> on 2018/11/26 13:18:00 UTC
[jira] [Updated] (YARN-8820) [Umbrella] GPU support on YARN - Phase
2
[ https://issues.apache.org/jira/browse/YARN-8820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhankun Tang updated YARN-8820:
-------------------------------
Description:
In YARN-6223, we've done a basic support for Nvidia GPU on YARN including resource discovery, allocation, cgroups isolation as well as docker support (Nvidia-docker v1). But there's still room for us to improve.
For instance, multiple GPU cards in one host bring the requirements of GPU hierarchy scheduling. The Nvidia-docker v2 emerged and v1 has been deprecated. And we're planning a new device plugin framework in YARN which has relation to GPU support too. (maybe in the long term)
So here we converge threads related to the above and open an umbrella here to track the next stage tasks for convenience.
One thing to note is that a pluggable device framework is in progress (YARN-8851), once that framework is mature, we should prefer to utilize the ability of the framework to achieve these phase 2 support.
was:
In YARN-6223, we've done a basic support for Nvidia GPU on YARN including resource discovery, allocation, cgroups isolation as well as docker support (Nvidia-docker v1). But there's still room for us to improve.
For instance, multiple GPU cards in one host bring the requirements of GPU hierarchy scheduling. The Nvidia-docker v2 emerged and v1 has been deprecated. And we're planning a new device plugin framework in YARN which has relation to GPU support too. (maybe in the long term)
So here we converge threads related to the above and open an umbrella here to track the next stage tasks for convenience.
> [Umbrella] GPU support on YARN - Phase 2
> ----------------------------------------
>
> Key: YARN-8820
> URL: https://issues.apache.org/jira/browse/YARN-8820
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: yarn
> Reporter: Zhankun Tang
> Priority: Major
>
> In YARN-6223, we've done a basic support for Nvidia GPU on YARN including resource discovery, allocation, cgroups isolation as well as docker support (Nvidia-docker v1). But there's still room for us to improve.
> For instance, multiple GPU cards in one host bring the requirements of GPU hierarchy scheduling. The Nvidia-docker v2 emerged and v1 has been deprecated. And we're planning a new device plugin framework in YARN which has relation to GPU support too. (maybe in the long term)
> So here we converge threads related to the above and open an umbrella here to track the next stage tasks for convenience.
> One thing to note is that a pluggable device framework is in progress (YARN-8851), once that framework is mature, we should prefer to utilize the ability of the framework to achieve these phase 2 support.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org