You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@yunikorn.apache.org by "Weiwei Yang (Jira)" <ji...@apache.org> on 2021/10/13 17:07:00 UTC
[jira] [Commented] (YUNIKORN-876) Optimize scheduler memory
consumption
[ https://issues.apache.org/jira/browse/YUNIKORN-876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17428353#comment-17428353 ]
Weiwei Yang commented on YUNIKORN-876:
--------------------------------------
Top:
{code}
(pprof) top
Showing nodes accounting for 4897.57MB, 85.09% of 5755.63MB total
Dropped 247 nodes (cum <= 28.78MB)
Showing top 10 nodes out of 102
flat flat% sum% cum cum%
1078.54MB 18.74% 18.74% 1101.10MB 19.13% github.com/json-iterator/go.(*Iterator).ReadString
1028.96MB 17.88% 36.62% 1028.96MB 17.88% k8s.io/apimachinery/pkg/apis/meta/v1.(*FieldsV1).DeepCopyInto (inline)
518.25MB 9.00% 45.62% 1604.23MB 27.87% k8s.io/api/core/v1.(*PodSpec).DeepCopyInto
488.78MB 8.49% 54.11% 488.78MB 8.49% k8s.io/api/core/v1.(*ResourceRequirements).DeepCopyInto
448.14MB 7.79% 61.90% 448.14MB 7.79% github.com/apache/incubator-yunikorn-k8shim/pkg/common.CreateTagsForTask
406.15MB 7.06% 68.96% 1454.11MB 25.26% k8s.io/apimachinery/pkg/apis/meta/v1.(*ObjectMeta).DeepCopyInto
367.69MB 6.39% 75.34% 902.47MB 15.68% k8s.io/api/core/v1.(*Container).DeepCopyInto
367.36MB 6.38% 81.73% 3425.70MB 59.52% k8s.io/api/core/v1.(*Pod).DeepCopy
104.02MB 1.81% 83.53% 104.02MB 1.81% k8s.io/apimachinery/pkg/apis/meta/v1.(*FieldsV1).UnmarshalJSON
89.68MB 1.56% 85.09% 89.68MB 1.56% reflect.unsafe_NewArray
{code}
> Optimize scheduler memory consumption
> --------------------------------------
>
> Key: YUNIKORN-876
> URL: https://issues.apache.org/jira/browse/YUNIKORN-876
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: core - cache, shim - kubernetes
> Reporter: Weiwei Yang
> Priority: Critical
> Labels: performance
> Fix For: 1.0.0
>
> Attachments: profile001.pdf
>
>
> On a cluster with about 500 nodes, we've seen the memory consumption goes over 12GB. This is not optimal. Using http://yunikorn.apache.org/docs/next/performance/profiling#memory-profiling, we got the memory footprint dump as attached in the JIRA. We need to optimize resource consumption and reduce the memory usage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org