You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@yunikorn.apache.org by "Craig Condit (Jira)" <ji...@apache.org> on 2024/04/04 03:46:00 UTC
[jira] [Comment Edited] (YUNIKORN-2532) Resource usage report has an incompatible format change
[ https://issues.apache.org/jira/browse/YUNIKORN-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17833810#comment-17833810 ]
Craig Condit edited comment on YUNIKORN-2532 at 4/4/24 3:45 AM:
----------------------------------------------------------------
Log messages are not considered stable, therefore this is not a bug. For example, scoped logging was added in a previous release, and now there is more data to be shown.
was (Author: ccondit):
This is not a bug, but an intentional format change to allow for scoped logging. Log messages are not guaranteed not to change. If you are attempting to parse them, you should be looking specifically for the structure of the message, not the surrounding text.
> Resource usage report has an incompatible format change
> -------------------------------------------------------
>
> Key: YUNIKORN-2532
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2532
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: core - scheduler
> Reporter: Yongjun Zhang
> Priority: Major
>
> There is some recent change that caused the application resource usage report to have a new format:
> Prior the change, the format was:
> {code:java}
> YK_APP_SUMMARY: {"appID": "adf53ee0-experiment-organicad-94520240-1-1", "submissionTime": 1712169262131, "startTime": 1712169264134, "finishTime": 1712173619983, "user": "system:serviceaccount:spark-operator-02:spark-operator", "queue": "root.queue-large", "state": "Completed", "rmID": "test-cluster", "resourceUsage": {"abc":{"memory":139178200478515200,"pods":1729129,"vcore":5183062000},"def":{"memory":113789789798400,"pods":1413,"vcore":4239000}}, "preemptedResource": {}}
> {code}
> with the change, the new format is:
> {code:java}
> 2024-04-04T00:33:08.532Z INFO core.scheduler.application.usage objects/application_summary.go:60 YK_APP_SUMMARY: {ApplicationID: afa303d0-test-trino-sparksql--20240404-2-1, SubmissionTime: 1712190615461, StartTime: 1712190617496, FinishTime: 1712190788532, User: system:serviceaccount:spark-operator-01:spark-operator, Queue: root.queue-large, State: Completed, RmID: test-cluster, ResourceUsage: TrackedResource{UNKNOWN:pods=177,UNKNOWN:vcore=354000,UNKNOWN:memory=1431454089216}, PreemptedResource: TrackedResource{}, PlaceholderResource: TrackedResource{}}{code}
> There are several incompatibilities:
> 1. the class name TrackedResource was not there before, now it is.
> 2. the instance type was outside the resource part before, not it's embedded
> 3. the instance type was reported correctly before the change, now it's UNKNOWN
> #3 may be a different issue, but it's observed by us at the same time.
> I think what should change the format back to the original one, as this is an incompatible change. What do you think [~wilfreds] , [~pbacsko] ,[~ccondit] ?
> Thanks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org