You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@yunikorn.apache.org by GitBox <gi...@apache.org> on 2022/10/14 21:39:58 UTC
[GitHub] [yunikorn-release] DerekTBrown opened a new pull request, #111: [Feature] Bring-your-own ConfigMap
DerekTBrown opened a new pull request, #111:
URL: https://github.com/apache/yunikorn-release/pull/111
This PR adds a simple feature; if `Values.configuration` is empty, the chart will not create the `ConfigMap`. This enables users to create the ConfigMap in their own way.
## Test Plan
```bash
> helm template ./helm-charts/yunikorn | gh gist create
- Creating gist...
✓ Created gist
https://gist.github.com/02127740f78187d4abf7a32716f73aa5
> helm template --set configuration=null ./helm-charts/yunikorn | gh gist create
- Creating gist...
✓ Created gist
https://gist.github.com/1ab6e9b97952e6becd60b44ef39b70e6
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
[GitHub] [yunikorn-release] craigcondit commented on pull request #111: [YUNIKORN-1359] Bring-your-own ConfigMap
Posted by GitBox <gi...@apache.org>.
craigcondit commented on PR #111:
URL: https://github.com/apache/yunikorn-release/pull/111#issuecomment-1302530920
This is looking better, but I think we should wait until at least https://issues.apache.org/jira/browse/YUNIKORN-1365 has landed, as this will allow the scheduler to start without a configmap present.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
[GitHub] [yunikorn-release] DerekTBrown commented on pull request #111: [YUNIKORN-1359] Bring-your-own ConfigMap
Posted by GitBox <gi...@apache.org>.
DerekTBrown commented on PR #111:
URL: https://github.com/apache/yunikorn-release/pull/111#issuecomment-1283043050
To describe the usecase a bit more; we need to be able to perform separate deployments for the infrastructure pieces (the YuniKorn scheduler) and the configuration (the queue ConfigMap in particular). There are a few reasons for this:
1. We want to release changes to the configuration at a different cadence / through a different process than the rest of the infrastructure. For instance, we want to automate the configuration of queues. In our particular case, we do this through multiple Argo applications.
2. We want to control the permissions to touch configuration and the infrastructure separately.
I imagine that other users will have similar stories; they run YuniKorn as a piece of platform infrastructure, and want to be able to periodically re-deploy configuration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
[GitHub] [yunikorn-release] DerekTBrown commented on pull request #111: [YUNIKORN-1359] Bring-your-own ConfigMap
Posted by GitBox <gi...@apache.org>.
DerekTBrown commented on PR #111:
URL: https://github.com/apache/yunikorn-release/pull/111#issuecomment-1299418145
@wilfred-s could you take a look at this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
[GitHub] [yunikorn-release] craigcondit commented on pull request #111: [YUNIKORN-1359] Bring-your-own ConfigMap
Posted by GitBox <gi...@apache.org>.
craigcondit commented on PR #111:
URL: https://github.com/apache/yunikorn-release/pull/111#issuecomment-1299455530
@DerekTBrown, this is going to be incompatible with the configuration redesign that is in progress. I've updated the original JIR A you've opened, and have added my comments here for clarity.
Please see the design doc in [YUNIKORN-1221](https://issues.apache.org/jira/browse/YUNIKORN-1221), which is our updated configuration design. Since we will be using a configmap for all configuration moving forward (and not just the core config), we need an alternative mechanism to implement disabling configmap generation.
I would recommend adding a new Helm attribute:
- enableConfigMap: true | false (default true for backwards compatibility)
-- When true, ConfigMap will be generated as part of the Helm deployment (as it is currently).
-- When false, no ConfigMap will be generated at all as part of Helm deployment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
[GitHub] [yunikorn-release] DerekTBrown commented on pull request #111: [YUNIKORN-1359] Bring-your-own ConfigMap
Posted by GitBox <gi...@apache.org>.
DerekTBrown commented on PR #111:
URL: https://github.com/apache/yunikorn-release/pull/111#issuecomment-1281724817
> How are you handling the fact that we cannot start without a configuration? We fail and exit if we cannot read the config. It would fail the helm deployment. That means the config map must be there before you deploy.
In our case the `ConfigMap` will already be present when we deploy YuniKorn.
> We also already have the option to bring your [own config map](https://yunikorn.apache.org/docs/next/user_guide/queue_config#configuration) if you want.
I am not quite sure what you are referring to? This section of the documentation describes the ability to use a different `.yaml` file for configuration. This doesn't solve the "bring your own" configuration problem, because the user would still need to mount the `ConfigMap` to the `Deployment` somehow, which requires modifying the official chart.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
[GitHub] [yunikorn-release] DerekTBrown commented on pull request #111: [YUNIKORN-1359] Bring-your-own ConfigMap
Posted by GitBox <gi...@apache.org>.
DerekTBrown commented on PR #111:
URL: https://github.com/apache/yunikorn-release/pull/111#issuecomment-1302518514
@craigcondit I have made the requested changes. Can you take another look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
[GitHub] [yunikorn-release] DerekTBrown commented on pull request #111: [YUNIKORN-1359] Bring-your-own ConfigMap
Posted by GitBox <gi...@apache.org>.
DerekTBrown commented on PR #111:
URL: https://github.com/apache/yunikorn-release/pull/111#issuecomment-1283049530
> Wouldn't the real solution then be to allow changing the config map in the deployment? Combined with the current possibility of changing the yaml file you would have full control over the deployment. Both from a config map and from a yaml perspective.
To summarize my understanding:
As proposed in this PR:
- User will create their own `ConfigMap` named `yunikorn-configs`
- User will set `configuration=null` so that default `ConfigMap` is not created.
As I believe you are proposing:
- User will create their own `ConfigMap` named whatever they want.
- User will then set something so that the YuniKorn Deployment will pull in that `ConfigMap`.
- User will set something so that an extraneous `ConfigMap` is not created.
My preference would be to stick with the approach proposed in this PR, as (1) it is strictly simpler than the other approach, and (2) I don't believe users need the ability to customize the `ConfigMap` name (since it is essentially irrelevant).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
[GitHub] [yunikorn-release] craigcondit commented on pull request #111: [YUNIKORN-1359] Bring-your-own ConfigMap
Posted by GitBox <gi...@apache.org>.
craigcondit commented on PR #111:
URL: https://github.com/apache/yunikorn-release/pull/111#issuecomment-1319332412
Closing in favor of the implementation done as part of https://issues.apache.org/jira/browse/YUNIKORN-1221.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
[GitHub] [yunikorn-release] craigcondit closed pull request #111: [YUNIKORN-1359] Bring-your-own ConfigMap
Posted by GitBox <gi...@apache.org>.
craigcondit closed pull request #111: [YUNIKORN-1359] Bring-your-own ConfigMap
URL: https://github.com/apache/yunikorn-release/pull/111
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
[GitHub] [yunikorn-release] wilfred-s commented on pull request #111: [Feature] Bring-your-own ConfigMap
Posted by GitBox <gi...@apache.org>.
wilfred-s commented on PR #111:
URL: https://github.com/apache/yunikorn-release/pull/111#issuecomment-1279796851
Please first open a jira for this. We track everything via jira. Check the [how to contribute](https://yunikorn.apache.org/community/how_to_contribute) document on the website.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
[GitHub] [yunikorn-release] wilfred-s commented on pull request #111: [Feature] Bring-your-own ConfigMap
Posted by GitBox <gi...@apache.org>.
wilfred-s commented on PR #111:
URL: https://github.com/apache/yunikorn-release/pull/111#issuecomment-1281209375
How are you handling the fact that we cannot start without a configuration? We fail and exit if we cannot read the config. It would fail the helm deployment. That means the config map must be there before you deploy.
We also already have the option to bring your [own config map](https://yunikorn.apache.org/docs/next/user_guide/queue_config#configuration) if you want. We could easily expose that but it does not change the fact that it must exist before we start.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
[GitHub] [yunikorn-release] wilfred-s commented on pull request #111: [YUNIKORN-1359] Bring-your-own ConfigMap
Posted by GitBox <gi...@apache.org>.
wilfred-s commented on PR #111:
URL: https://github.com/apache/yunikorn-release/pull/111#issuecomment-1282937854
Wouldn't the real solution then be to allow changing the config map in the deployment? Combined with the current possibility of changing the yaml file you would have full control over the deployment. Both from a config map and from a yaml perspective. This solution feels like it is only a partial one.
BTW: a start was made to move all the configs to config maps and move away from a lot of the command line settings. Which means we might have multiple in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org