You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@openwhisk.apache.org by GitBox <gi...@apache.org> on 2022/07/21 06:57:25 UTC
[GitHub] [openwhisk] style95 opened a new pull request, #5289: Add tags to invokers.
style95 opened a new pull request, #5289:
URL: https://github.com/apache/openwhisk/pull/5289
## Description
This is to enable tag-based scheduling.
We can add tags to invokers like below.
```
[invokers]
invoker0 ansible_host=172.17.0.1 ansible_connection=local tags="['v2', 'gpu']"
```
## Related issue and scope
<!--- Please include a link to a related issue if there is one. -->
- [ ] I opened an issue to propose and discuss this change (#????)
## My changes affect the following components
<!--- Select below all system components are affected by your change. -->
<!--- Enter an `x` in all applicable boxes. -->
- [ ] API
- [ ] Controller
- [ ] Message Bus (e.g., Kafka)
- [ ] Loadbalancer
- [x] Scheduler
- [x] Invoker
- [ ] Intrinsic actions (e.g., sequences, conductors)
- [ ] Data stores (e.g., CouchDB)
- [ ] Tests
- [ ] Deployment
- [ ] CLI
- [ ] General tooling
- [ ] Documentation
## Types of changes
<!--- What types of changes does your code introduce? Use `x` in all the boxes that apply: -->
- [ ] Bug fix (generally a non-breaking change which closes an issue).
- [x] Enhancement or new feature (adds new functionality).
- [ ] Breaking change (a bug fix or enhancement which changes existing behavior).
## Checklist:
<!--- Please review the points below which help you make sure you've covered all aspects of the change you're making. -->
- [x] I signed an [Apache CLA](https://github.com/apache/openwhisk/blob/master/CONTRIBUTING.md).
- [x] I reviewed the [style guides](https://github.com/apache/openwhisk/blob/master/CONTRIBUTING.md#coding-standards) and followed the recommendations (Travis CI will check :).
- [ ] I added tests to cover my changes.
- [ ] My changes require further changes to the documentation.
- [ ] I updated the documentation where necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] style95 merged pull request #5289: Add tags to invokers.
Posted by GitBox <gi...@apache.org>.
style95 merged PR #5289:
URL: https://github.com/apache/openwhisk/pull/5289
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] style95 commented on pull request #5289: Add tags to invokers.
Posted by GitBox <gi...@apache.org>.
style95 commented on PR #5289:
URL: https://github.com/apache/openwhisk/pull/5289#issuecomment-1192113544
It is required for the subsequent change.
Let me merge this for now.
Please feel free to leave any comments. I would apply them in the subsequent PR if anyone has.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on pull request #5289: Add tags to invokers.
Posted by GitBox <gi...@apache.org>.
ningyougang commented on PR #5289:
URL: https://github.com/apache/openwhisk/pull/5289#issuecomment-1191133994
LGTM with some small questions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] codecov-commenter commented on pull request #5289: Add tags to invokers.
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #5289:
URL: https://github.com/apache/openwhisk/pull/5289#issuecomment-1191166984
# [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5289?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#5289](https://codecov.io/gh/apache/openwhisk/pull/5289?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (15f5799) into [master](https://codecov.io/gh/apache/openwhisk/commit/a03507ceeb2c4648c875ed68273de1448ec0d074?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a03507c) will **decrease** coverage by `75.32%`.
> The diff coverage is `9.09%`.
```diff
@@ Coverage Diff @@
## master #5289 +/- ##
==========================================
- Coverage 79.90% 4.57% -75.33%
==========================================
Files 238 238
Lines 14161 14171 +10
Branches 603 622 +19
==========================================
- Hits 11316 649 -10667
- Misses 2845 13522 +10677
```
| [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/5289?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...la/org/apache/openwhisk/core/invoker/Invoker.scala](https://codecov.io/gh/apache/openwhisk/pull/5289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvaW52b2tlci9JbnZva2VyLnNjYWxh) | `0.00% <0.00%> (-70.77%)` | :arrow_down: |
| [.../scala/org/apache/openwhisk/core/WhiskConfig.scala](https://codecov.io/gh/apache/openwhisk/pull/5289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvV2hpc2tDb25maWcuc2NhbGE=) | `68.39% <100.00%> (-27.57%)` | :arrow_down: |
| [.../main/scala/org/apache/openwhisk/core/WarmUp.scala](https://codecov.io/gh/apache/openwhisk/pull/5289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvV2FybVVwLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ain/scala/org/apache/openwhisk/spi/SpiLoader.scala](https://codecov.io/gh/apache/openwhisk/pull/5289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL3NwaS9TcGlMb2FkZXIuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...n/scala/org/apache/openwhisk/utils/JsHelpers.scala](https://codecov.io/gh/apache/openwhisk/pull/5289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL3V0aWxzL0pzSGVscGVycy5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...scala/org/apache/openwhisk/common/Prometheus.scala](https://codecov.io/gh/apache/openwhisk/pull/5289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Qcm9tZXRoZXVzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...scala/org/apache/openwhisk/core/FeatureFlags.scala](https://codecov.io/gh/apache/openwhisk/pull/5289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvRmVhdHVyZUZsYWdzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ala/org/apache/openwhisk/common/ConfigMXBean.scala](https://codecov.io/gh/apache/openwhisk/pull/5289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNWEJlYW4uc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...la/org/apache/openwhisk/http/BasicRasService.scala](https://codecov.io/gh/apache/openwhisk/pull/5289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNSYXNTZXJ2aWNlLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...a/org/apache/openwhisk/common/ConfigMapValue.scala](https://codecov.io/gh/apache/openwhisk/pull/5289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNYXBWYWx1ZS5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [204 more](https://codecov.io/gh/apache/openwhisk/pull/5289/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on pull request #5289: Add tags to invokers.
Posted by GitBox <gi...@apache.org>.
ningyougang commented on PR #5289:
URL: https://github.com/apache/openwhisk/pull/5289#issuecomment-1191124859
* Can add some example how to user it? e.g. If my action wants to use special invokers(e.g. include TAG: cpu in ansible), my action should meet what condition?
* Can use in kubernetes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] style95 commented on pull request #5289: Add tags to invokers.
Posted by GitBox <gi...@apache.org>.
style95 commented on PR #5289:
URL: https://github.com/apache/openwhisk/pull/5289#issuecomment-1191333269
I confirmed it's working as expected.
I wl add documentation about this as well.
OW operators can configure tags for invokers like this.
```
invoker0 ansible_host=${INVOKER0} tags="['v1', 'gpu']"
invoker1 ansible_host=${INVOKER1} tags="['v1', 'cpu']"
invoker2 ansible_host=${INVOKER2} tags="['v2', 'gpu']"
invoker3 ansible_host=${INVOKER3} tags="['v2', 'cpu']"
invoker4 ansible_host=${INVOKER4} tags="['v1', 'mem']"
invoker5 ansible_host=${INVOKER5} tags="['v2', 'mem']"
invoker6 ansible_host=${INVOKER6} tags="['v2']"
invoker7 ansible_host=${INVOKER7} tags="['v2']"
invoker8 ansible_host=${INVOKER8}
invoker9 ansible_host=${INVOKER9}
```
Users can add the following annotations to their actions.
```
wsk action update params tests/dat/actions/params.js -i -a invoker-resources '["v2", "gpu"]'
```
Activationtivation for this action will be delivered to `invoker2`.
The annotations and the correspondinginvokersinvkoers are as follows.
* `["v1", "gpu"]` -> `invoker0`
* `["v2", "gpu"]` -> `invoker2`
* `["v1", "cpu"]` -> `invoker1`
* `["v2"]` -> One of `invoker2`, `invoker3`, `invoker5`, `invoker6`, and `invoker7`
* `["v1"]` -> One of `invoker0`, `invoker1`, `invoker4`
* `No annotation` -> One of `invoker8` and `invoker9` is chosen first. if they have no resource, choose one of the invokers with tags.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] style95 commented on a diff in pull request #5289: Add tags to invokers.
Posted by GitBox <gi...@apache.org>.
style95 commented on code in PR #5289:
URL: https://github.com/apache/openwhisk/pull/5289#discussion_r926551978
##########
core/invoker/src/main/scala/org/apache/openwhisk/core/invoker/Invoker.scala:
##########
@@ -109,7 +109,13 @@ object Invoker {
implicit val logger = new AkkaLogging(akka.event.Logging.getLogger(actorSystem, this))
val poolConfig: ContainerPoolConfig = loadConfigOrThrow[ContainerPoolConfig](ConfigKeys.containerPool)
val limitConfig: ConcurrencyLimitConfig = loadConfigOrThrow[ConcurrencyLimitConfig](ConfigKeys.concurrencyLimit)
+ val tags: Seq[String] = Some(loadConfigOrThrow[String](ConfigKeys.invokerResourceTags))
+ .map(_.trim())
+ .filter(_ != "")
+ .map(_.split(",").toSeq)
+ .getOrElse(Seq.empty[String])
+ logger.info(this, s"invoker tags: (${tags.mkString(", ")})")
Review Comment:
It looks like this.
```
[2022-07-21T11:00:02.599Z] [INFO] [#tid_sid_unknown] [Invoker] invoker tags: (v1, cpu)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org