You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@yunikorn.apache.org by "Craig Condit (Jira)" <ji...@apache.org> on 2023/04/11 17:05:00 UTC
[jira] [Commented] (YUNIKORN-1684) Configured queues got removed automatically after startup
[ https://issues.apache.org/jira/browse/YUNIKORN-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17711063#comment-17711063 ]
Craig Condit commented on YUNIKORN-1684:
----------------------------------------
I have seen reports of this in the community as well - I'm not sure it's limited to test environments. I wrote the original hot-reload code for 1.2, I'll take a look.
> Configured queues got removed automatically after startup
> ---------------------------------------------------------
>
> Key: YUNIKORN-1684
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1684
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: core - scheduler
> Reporter: Peter Bacsko
> Assignee: Craig Condit
> Priority: Major
>
> We observed a problem on Minikube and KIND related to configuration and event handling. Sometimes, when Yunikorn starts and there is a configmap with a queue hierarchy, occasionally the configured queues got removed 10 seconds later by the partition manager:
> {noformat}
> 2023-03-14T07:34:26.186Z INFO conf/schedulerconf.go:472 Set data {"key": "queues.yaml", "value": "partitions:\n- name: default\n statedumpfilepath: /etc/xxx\n queuedeletepolicy: preserve\n queues:\n - name: root\n parent: true\n queues:\n - name: users\n parent: true\n queues:\n - name: dev\n parent: false\n submitacl: \"*\"\n"}
> 2023-03-14T07:34:26.186Z INFO cache/context.go:420 Merged configmap {"confmap": {"queues.yaml":"partitions:\n- name: default\n statedumpfilepath: /etc/xxx\n queuedeletepolicy: preserve\n queues:\n - name: root\n parent: true\n queues:\n - name: users\n parent: true\n queues:\n - name: dev\n parent: false\n submitacl: \"*\"\n"}}
> 2023-03-14T07:34:26.186Z INFO cache/context.go:422 reloading scheduler configuration
> 2023-03-14T07:34:26.186Z INFO utils/utils.go:314 found yaml in configmap
> 2023-03-14T07:34:26.186Z INFO scheduler/context.go:214 Update {"extraConfig": {}}
> 2023-03-14T07:34:26.186Z INFO scheduler/context.go:218 Update {"Config": "partitions:\n- name: default\n statedumpfilepath: /etc/xxx\n queuedeletepolicy: preserve\n queues:\n - name: root\n parent: true\n queues:\n - name: users\n parent: true\n queues:\n - name: dev\n parent: false\n submitacl: \"*\"\n"}
> 2023-03-14T07:34:26.186Z WARN configs/configvalidator.go:405 Ignoring deprecated partition setting 'statedumpfilepath'. This parameter will be removed in a future release.
> 2023-03-14T07:34:26.186Z INFO objects/queue.go:146 configured queue added to scheduler {"queueName": "root"}
> 2023-03-14T07:34:26.186Z INFO objects/queue.go:146 configured queue added to scheduler {"queueName": "root.users"}
> 2023-03-14T07:34:26.186Z INFO security/acl.go:63 user list is wildcard, allowing all access
> 2023-03-14T07:34:26.186Z INFO objects/queue.go:146 configured queue added to scheduler {"queueName": "root.users.dev"}
> 2023-03-14T07:34:26.186Z INFO scheduler/partition.go:120 root queue added {"partitionName": "[mycluster]default", "rmID": "mycluster"}
> 2023-03-14T07:34:26.186Z INFO scheduler/partition.go:147 NodeSorting policy set from config {"policyName": "fair"}
> 2023-03-14T07:34:26.186Z INFO scheduler/context.go:384 updating partitions {"partitionName": "[mycluster]default"}
> 2023-03-14T07:34:26.186Z INFO scheduler/partition.go:161 Updating placement manager rules on config reload
> 2023-03-14T07:34:26.186Z INFO placement/placement.go:73 Placement manager rules removed on config reload
> 2023-03-14T07:34:26.186Z INFO scheduler/partition.go:147 NodeSorting policy set from config {"policyName": "fair"}
> 2023-03-14T07:34:26.187Z INFO security/acl.go:63 user list is wildcard, allowing all access
> 2023-03-14T07:34:35.171Z INFO objects/queue.go:911 removing queue {"queue": "root.users.dev"}
> 2023-03-14T07:34:35.171Z INFO objects/queue.go:911 removing queue {"queue": "root.users"}
> {noformat}
> The issue cannot be reproduced consistently, but it does happen regularly. It seems to affect only Minikube and KIND and not real environments.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org