You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Bibin A Chundatt (JIRA)" <ji...@apache.org> on 2018/06/17 08:48:00 UTC
[jira] [Comment Edited] (YARN-8423) GPU does not get released even
though the application gets killed.
[ https://issues.apache.org/jira/browse/YARN-8423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16515026#comment-16515026 ]
Bibin A Chundatt edited comment on YARN-8423 at 6/17/18 8:47 AM:
-----------------------------------------------------------------
[~leftnoteasy]
I had mentioned the same as part of design comments.
https://issues.apache.org/jira/browse/YARN-8320?focusedCommentId=16499938&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16499938
Current patch seems handling only GPU can we have a common fix as part of this jira?
was (Author: bibinchundatt):
[~leftnoteasy]
I had mentioned the same as part of design comments.
https://issues.apache.org/jira/browse/YARN-8320?focusedCommentId=16499938&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16499938
Current patch seems not handling only GPU can we have a common fix as part of this jira?
> GPU does not get released even though the application gets killed.
> ------------------------------------------------------------------
>
> Key: YARN-8423
> URL: https://issues.apache.org/jira/browse/YARN-8423
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Reporter: Sumana Sathish
> Assignee: Sunil Govindan
> Priority: Critical
> Attachments: YARN-8423.001.patch, kill-container-nm.log
>
>
> Run an Tensor flow app requesting one GPU.
> Kill the application once the GPU is allocated
> Query the nodemanger once the application is killed.We see that GPU is not being released.
> {code}
> curl -i <NM>/ws/v1/node/resources/yarn.io%2Fgpu
> {"gpuDeviceInformation":{"gpus":[{"productName":"<productName>","uuid":"GPU-<UID>","minorNumber":0,"gpuUtilizations":{"overallGpuUtilization":0.0},"gpuMemoryUsage":{"usedMemoryMiB":73,"availMemoryMiB":12125,"totalMemoryMiB":12198},"temperature":{"currentGpuTemp":28.0,"maxGpuTemp":85.0,"slowThresholdGpuTemp":82.0}},{"productName":"<productName>","uuid":"GPU-<UID>","minorNumber":1,"gpuUtilizations":{"overallGpuUtilization":0.0},"gpuMemoryUsage":{"usedMemoryMiB":73,"availMemoryMiB":12125,"totalMemoryMiB":12198},"temperature":{"currentGpuTemp":28.0,"maxGpuTemp":85.0,"slowThresholdGpuTemp":82.0}}],"driverVersion":"<version>"},"totalGpuDevices":[{"index":0,"minorNumber":0},{"index":1,"minorNumber":1}],"assignedGpuDevices":[{"index":0,"minorNumber":0,"containerId":"container_<containerID>"}]}
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org