You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Gera Shegalov (Jira)" <ji...@apache.org> on 2026/10/01 03:30:00 UTC
[jira] [Assigned] (YARN-11995) ATS container history drops custom resource allocations such as `yarn.io/gpu`
[ https://issues.apache.org/jira/browse/YARN-11995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gera Shegalov reassigned YARN-11995:
------------------------------------
Assignee: Gera Shegalov
> ATS container history drops custom resource allocations such as `yarn.io/gpu`
> -----------------------------------------------------------------------------
>
> Key: YARN-11995
> URL: https://issues.apache.org/jira/browse/YARN-11995
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineservice
> Affects Versions: 3.4.3
> Reporter: Gera Shegalov
> Assignee: Gera Shegalov
> Priority: Major
>
> When a YARN container is allocated a custom resource such as {{yarn.io/gpu}}, the ResourceManager reports the correct allocation, but the Timeline Service v1/v1.5 container history reports zero for that resource. The raw {{YARN_CONTAINER}} Timeline entity does not contain the custom allocation, so this is not solely a REST serialization problem. Memory and vcores remain correct.
> In Hadoop 3.4.2, {{TimelineServiceV1Publisher.containerCreated}} puts only {{getMemorySize()}} and {{getVirtualCores()}} into the Timeline entity. {{ApplicationHistoryManagerOnTimelineStore.convertToContainerReport}} reads only those fields and builds {{Resource.newInstance(allocatedMem, allocatedVcore)}}. These v1 publisher and reader code paths are still present in Hadoop trunk as of 2026-09-28. The same two-field publishing pattern appears in {{TimelineServiceV2Publisher.containerCreated}}; ATS v2 behavior has not been reproduced here.
> * [TimelineServiceV1Publisher.java, lines 374-390|https://github.com/apache/hadoop/blob/rel/release-3.4.2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TimelineServiceV1Publisher.java#L374-L390]
> * [ApplicationHistoryManagerOnTimelineStore.java, lines 565-673|https://github.com/apache/hadoop/blob/rel/release-3.4.2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java#L565-L673]
> * [TimelineServiceV2Publisher.java, lines 390-420|https://github.com/apache/hadoop/blob/rel/release-3.4.2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TimelineServiceV2Publisher.java#L390-L420]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org