You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@openwhisk.apache.org by GitBox <gi...@apache.org> on 2021/01/20 10:47:30 UTC
[GitHub] [openwhisk] ningyougang opened a new pull request #5048: Don't create prewarm container when used memory reaches the limit
ningyougang opened a new pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048
- [x] Don't create prewarm container when used memory reaches the limit
- [x] have no need to judge whether memory when take container from busy/free pool due to `don't create new container`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r561561397
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -437,8 +439,10 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
* @param memory The amount of memory to check.
* @return true, if there is enough space for the given amount of memory.
*/
- def hasPoolSpaceFor[A](pool: Map[A, ContainerData], memory: ByteSize): Boolean = {
- memoryConsumptionOf(pool) + memory.toMB <= poolConfig.userMemory.toMB
+ def hasPoolSpaceFor[A](pool: Map[A, ContainerData],
+ prewarmStartingPool: Map[A, (String, ByteSize)],
+ memory: ByteSize): Boolean = {
+ memoryConsumptionOf(pool) + +prewarmStartingPool.map(_._2._2.toMB).sum + memory.toMB <= poolConfig.userMemory.toMB
Review comment:
When cal whether has idle space, it is better to calculate prewarmStartingPool's memory as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] style95 commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
style95 commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r566528602
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -439,8 +441,10 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
* @param memory The amount of memory to check.
* @return true, if there is enough space for the given amount of memory.
*/
- def hasPoolSpaceFor[A](pool: Map[A, ContainerData], memory: ByteSize): Boolean = {
- memoryConsumptionOf(pool) + memory.toMB <= poolConfig.userMemory.toMB
+ def hasPoolSpaceFor[A](pool: Map[A, ContainerData],
+ prewarmStartingPool: Map[A, (String, ByteSize)],
+ memory: ByteSize): Boolean = {
+ memoryConsumptionOf(pool) + prewarmStartingPool.map(_._2._2.toMB).sum + memory.toMB <= poolConfig.userMemory.toMB
Review comment:
BTW, when the prewarm pool are backfilled again?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r561421295
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -369,9 +365,15 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
/** Creates a new prewarmed container */
def prewarmContainer(exec: CodeExec[_], memoryLimit: ByteSize, ttl: Option[FiniteDuration]): Unit = {
- val newContainer = childFactory(context)
- prewarmStartingPool = prewarmStartingPool + (newContainer -> (exec.kind, memoryLimit))
- newContainer ! Start(exec, memoryLimit, ttl)
+ if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, memoryLimit)) {
Review comment:
When create prewarm container, judge whether exist enough memory.
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -437,8 +439,10 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
* @param memory The amount of memory to check.
* @return true, if there is enough space for the given amount of memory.
*/
- def hasPoolSpaceFor[A](pool: Map[A, ContainerData], memory: ByteSize): Boolean = {
- memoryConsumptionOf(pool) + memory.toMB <= poolConfig.userMemory.toMB
+ def hasPoolSpaceFor[A](pool: Map[A, ContainerData],
+ prewarmStartingPool: Map[A, (String, ByteSize)],
+ memory: ByteSize): Boolean = {
+ memoryConsumptionOf(pool) + +prewarmStartingPool.map(_._2._2.toMB).sum + memory.toMB <= poolConfig.userMemory.toMB
Review comment:
When cal whether has idle space, it is better to calculate prewarmStartingPool's memory as well.
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -437,8 +439,10 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
* @param memory The amount of memory to check.
* @return true, if there is enough space for the given amount of memory.
*/
- def hasPoolSpaceFor[A](pool: Map[A, ContainerData], memory: ByteSize): Boolean = {
- memoryConsumptionOf(pool) + memory.toMB <= poolConfig.userMemory.toMB
+ def hasPoolSpaceFor[A](pool: Map[A, ContainerData],
+ prewarmStartingPool: Map[A, (String, ByteSize)],
+ memory: ByteSize): Boolean = {
+ memoryConsumptionOf(pool) + +prewarmStartingPool.map(_._2._2.toMB).sum + memory.toMB <= poolConfig.userMemory.toMB
Review comment:
When calculate whether has idle space, it is better to calculate prewarmStartingPool's memory as well.
##########
File path: tests/src/test/scala/org/apache/openwhisk/core/containerpool/test/ContainerPoolTests.scala
##########
@@ -829,7 +842,7 @@ class ContainerPoolTests
stream.reset()
val prewarmExpirationCheckIntervel = 2.seconds
val poolConfig =
- ContainerPoolConfig(MemoryLimit.STD_MEMORY * 8, 0.5, false, prewarmExpirationCheckIntervel, None, 100)
+ ContainerPoolConfig(MemoryLimit.STD_MEMORY * 12, 0.5, false, prewarmExpirationCheckIntervel, None, 100)
Review comment:
Due to calculate prewarmStartingPool's memory, so need to adjust the user memory here.
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
- // Schedule a job to a warm container
- ContainerPool
- .schedule(r.action, r.msg.user.namespace.name, freePool)
- .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
- .orElse(
- // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
-
- // Is there enough space to create a new container or do other containers have to be removed?
- if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, memory)) {
+ // Schedule a job to a warm container
+ ContainerPool
Review comment:
have no need to judge whether exist memory when take container from busy/free pool due to don't create new container
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
Review comment:
Due didn't create any container here, so it is better to remove below logic here.
```
// Is there enough space on the invoker for this action to be executed.
if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory))
```
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
Review comment:
Due to `didn't create any container here`, so it is better to remove below logic here.
```
// Is there enough space on the invoker for this action to be executed.
if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory))
```
##########
File path: tests/src/test/scala/org/apache/openwhisk/core/containerpool/test/ContainerPoolTests.scala
##########
@@ -829,7 +842,7 @@ class ContainerPoolTests
stream.reset()
val prewarmExpirationCheckIntervel = 2.seconds
val poolConfig =
- ContainerPoolConfig(MemoryLimit.STD_MEMORY * 8, 0.5, false, prewarmExpirationCheckIntervel, None, 100)
+ ContainerPoolConfig(MemoryLimit.STD_MEMORY * 12, 0.5, false, prewarmExpirationCheckIntervel, None, 100)
Review comment:
Due to `add logic of whether exist enough memory`, so need to adjust the user memory here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang closed pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang closed pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#issuecomment-766515780
@style95 ,AFAIK, the travis ci problem is fixed by you, but currently, it seems some new problems happened

Have any idea? or guide me to solve it as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] style95 commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
style95 commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r565866140
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
- // Schedule a job to a warm container
- ContainerPool
- .schedule(r.action, r.msg.user.namespace.name, freePool)
- .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
- .orElse(
- // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
-
- // Is there enough space to create a new container or do other containers have to be removed?
- if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, memory)) {
+ // Schedule a job to a warm container
+ ContainerPool
+ .schedule(r.action, r.msg.user.namespace.name, freePool)
+ .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
+ .orElse(
+ // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
+
+ // Is there enough space to create a new container or do other containers have to be removed?
+ if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, prewarmStartingPool, memory)) {
Review comment:
If we take a container from the `prewarmPool` as there is a matched container, do we still need to concern about the total memory?
Since one container is moved from `prewarmPool` to `busyPool` and there would be no difference in the sum of memory consumption.
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -437,8 +439,10 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
* @param memory The amount of memory to check.
* @return true, if there is enough space for the given amount of memory.
*/
- def hasPoolSpaceFor[A](pool: Map[A, ContainerData], memory: ByteSize): Boolean = {
- memoryConsumptionOf(pool) + memory.toMB <= poolConfig.userMemory.toMB
+ def hasPoolSpaceFor[A](pool: Map[A, ContainerData],
+ prewarmStartingPool: Map[A, (String, ByteSize)],
+ memory: ByteSize): Boolean = {
+ memoryConsumptionOf(pool) + +prewarmStartingPool.map(_._2._2.toMB).sum + memory.toMB <= poolConfig.userMemory.toMB
Review comment:
Is this a typo?
`+ +`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] codecov-io edited a comment on pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#issuecomment-763785825
# [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=h1) Report
> Merging [#5048](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=desc) (c9cba12) into [master](https://codecov.io/gh/apache/openwhisk/commit/a6ad9e418e605894c5e96e5601c3e1b8ded4166a?el=desc) (a6ad9e4) will **decrease** coverage by `11.26%`.
> The diff coverage is `95.65%`.
[](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #5048 +/- ##
===========================================
- Coverage 82.51% 71.25% -11.27%
===========================================
Files 199 199
Lines 9735 9736 +1
Branches 435 442 +7
===========================================
- Hits 8033 6937 -1096
- Misses 1702 2799 +1097
```
| [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...e/openwhisk/core/containerpool/ContainerPool.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udGFpbmVycG9vbC9Db250YWluZXJQb29sLnNjYWxh) | `88.04% <95.65%> (-9.16%)` | :arrow_down: |
| [...che/openwhisk/core/invoker/LogStoreCollector.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvaW52b2tlci9Mb2dTdG9yZUNvbGxlY3Rvci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...core/database/cosmosdb/RxObservableImplicits.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvUnhPYnNlcnZhYmxlSW1wbGljaXRzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ore/database/cosmosdb/cache/CacheInvalidator.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NhY2hlSW52YWxpZGF0b3Iuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...e/database/cosmosdb/cache/ChangeFeedConsumer.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NoYW5nZUZlZWRDb25zdW1lci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...core/database/cosmosdb/CosmosDBArtifactStore.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlLnNjYWxh) | `0.00% <0.00%> (-95.85%)` | :arrow_down: |
| [...sk/core/database/cosmosdb/CosmosDBViewMapper.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJWaWV3TWFwcGVyLnNjYWxh) | `0.00% <0.00%> (-93.90%)` | :arrow_down: |
| [...tabase/cosmosdb/cache/CacheInvalidatorConfig.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NhY2hlSW52YWxpZGF0b3JDb25maWcuc2NhbGE=) | `0.00% <0.00%> (-92.31%)` | :arrow_down: |
| [...a/org/apache/openwhisk/http/BasicHttpService.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNIdHRwU2VydmljZS5zY2FsYQ==) | `3.33% <0.00%> (-86.67%)` | :arrow_down: |
| [...la/org/apache/openwhisk/http/BasicRasService.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNSYXNTZXJ2aWNlLnNjYWxh) | `16.66% <0.00%> (-83.34%)` | :arrow_down: |
| ... and [43 more](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=footer). Last update [a6ad9e4...c9cba12](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang edited a comment on pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang edited a comment on pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#issuecomment-766515780
@style95 ,AFAIK, the travis ci problem is fixed by you, but currently, it seems some new problems happened

Have any idea? or can guide me to solve it as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r565877511
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
- // Schedule a job to a warm container
- ContainerPool
- .schedule(r.action, r.msg.user.namespace.name, freePool)
- .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
- .orElse(
- // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
-
- // Is there enough space to create a new container or do other containers have to be removed?
- if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, memory)) {
+ // Schedule a job to a warm container
+ ContainerPool
+ .schedule(r.action, r.msg.user.namespace.name, freePool)
+ .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
+ .orElse(
+ // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
+
+ // Is there enough space to create a new container or do other containers have to be removed?
+ if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, prewarmStartingPool, memory)) {
Review comment:
hm..i think we need to concern about the total memory when take a prewarm conainer from `prewarmPool`,
e.g. let's assume invoker's user memory is `1024 MB`, currently, busyPool is `512MB`, prewarmPool is `512MB`,
if user's activation's corresponding action can match the warmedPool, have no need to add condition: `if (hasPoolSpaceFor(busyPool...)` due to `didn't create new container`
but if doesn't match warmedPool and matches a prewarm container(256MB), then, busyPool will be `512MB+256MB`, prewarmedPool is `512MB-256MB+256MB`, so the total used memory will be `1024MB +256MB`, so it is better to add judge condition before take a prewarmed container.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] codecov-io edited a comment on pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#issuecomment-763785825
# [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=h1) Report
> Merging [#5048](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=desc) (f07708f) into [master](https://codecov.io/gh/apache/openwhisk/commit/a6ad9e418e605894c5e96e5601c3e1b8ded4166a?el=desc) (a6ad9e4) will **decrease** coverage by `10.81%`.
> The diff coverage is `86.36%`.
[](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #5048 +/- ##
===========================================
- Coverage 82.51% 71.70% -10.82%
===========================================
Files 199 206 +7
Lines 9735 10001 +266
Branches 435 443 +8
===========================================
- Hits 8033 7171 -862
- Misses 1702 2830 +1128
```
| [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...e/openwhisk/core/containerpool/ContainerPool.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udGFpbmVycG9vbC9Db250YWluZXJQb29sLnNjYWxh) | `87.64% <86.36%> (-9.56%)` | :arrow_down: |
| [...che/openwhisk/core/invoker/LogStoreCollector.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvaW52b2tlci9Mb2dTdG9yZUNvbGxlY3Rvci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...core/database/cosmosdb/RxObservableImplicits.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvUnhPYnNlcnZhYmxlSW1wbGljaXRzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ore/database/cosmosdb/cache/CacheInvalidator.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NhY2hlSW52YWxpZGF0b3Iuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...e/database/cosmosdb/cache/ChangeFeedConsumer.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NoYW5nZUZlZWRDb25zdW1lci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...core/database/cosmosdb/CosmosDBArtifactStore.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlLnNjYWxh) | `0.00% <0.00%> (-95.85%)` | :arrow_down: |
| [...sk/core/database/cosmosdb/CosmosDBViewMapper.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJWaWV3TWFwcGVyLnNjYWxh) | `0.00% <0.00%> (-93.90%)` | :arrow_down: |
| [...tabase/cosmosdb/cache/CacheInvalidatorConfig.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NhY2hlSW52YWxpZGF0b3JDb25maWcuc2NhbGE=) | `0.00% <0.00%> (-92.31%)` | :arrow_down: |
| [...a/org/apache/openwhisk/http/BasicHttpService.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNIdHRwU2VydmljZS5zY2FsYQ==) | `3.33% <0.00%> (-86.67%)` | :arrow_down: |
| [...la/org/apache/openwhisk/http/BasicRasService.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNSYXNTZXJ2aWNlLnNjYWxh) | `16.66% <0.00%> (-83.34%)` | :arrow_down: |
| ... and [51 more](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=footer). Last update [a6ad9e4...f07708f](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang removed a comment on pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang removed a comment on pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#issuecomment-766515780
@style95 ,AFAIK, the travis ci problem is fixed by you, but currently, it seems some new problems happened

Have any idea? or can guide me to solve it as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r561561397
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -437,8 +439,10 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
* @param memory The amount of memory to check.
* @return true, if there is enough space for the given amount of memory.
*/
- def hasPoolSpaceFor[A](pool: Map[A, ContainerData], memory: ByteSize): Boolean = {
- memoryConsumptionOf(pool) + memory.toMB <= poolConfig.userMemory.toMB
+ def hasPoolSpaceFor[A](pool: Map[A, ContainerData],
+ prewarmStartingPool: Map[A, (String, ByteSize)],
+ memory: ByteSize): Boolean = {
+ memoryConsumptionOf(pool) + +prewarmStartingPool.map(_._2._2.toMB).sum + memory.toMB <= poolConfig.userMemory.toMB
Review comment:
When calculate whether has idle space, it is better to calculate prewarmStartingPool's memory as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r566507836
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
- // Schedule a job to a warm container
- ContainerPool
- .schedule(r.action, r.msg.user.namespace.name, freePool)
- .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
- .orElse(
- // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
-
- // Is there enough space to create a new container or do other containers have to be removed?
- if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, memory)) {
+ // Schedule a job to a warm container
+ ContainerPool
+ .schedule(r.action, r.msg.user.namespace.name, freePool)
+ .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
+ .orElse(
+ // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
+ // When take prewarm container, has no need to judge whether user memory is enough
+ takePrewarmContainer(r.action)
+ .map(container => (container, "prewarmed"))
+ .orElse {
+ // Is there enough space to create a new container or do other containers have to be removed?
+ if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, prewarmStartingPool, memory)) {
+ val container = Some(createContainer(memory), "cold")
+ incrementColdStartCount(kind, memory)
+ container
+ } else None
+ })
+ .orElse(
+ // Remove a container and create a new one for the given job
+ ContainerPool
+ // Only free up the amount, that is really needed to free up
+ .remove(freePool, Math.min(r.action.limits.memory.megabytes, memoryConsumptionOf(freePool)).MB)
+ .map(removeContainer)
+ // If the list had at least one entry, enough containers were removed to start the new container. After
+ // removing the containers, we are not interested anymore in the containers that have been removed.
+ .headOption
+ .map(_ =>
takePrewarmContainer(r.action)
- .map(container => (container, "prewarmed"))
- .orElse {
- val container = Some(createContainer(memory), "cold")
+ .map(container => (container, "recreatedPrewarm"))
+ .getOrElse {
+ val container = (createContainer(memory), "recreated")
Review comment:
For here's coldStart, i think has no need to do memory check. because it will delete some container in advance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#issuecomment-768798893
Have any comment?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r566507279
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
- // Schedule a job to a warm container
- ContainerPool
- .schedule(r.action, r.msg.user.namespace.name, freePool)
- .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
- .orElse(
- // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
-
- // Is there enough space to create a new container or do other containers have to be removed?
- if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, memory)) {
+ // Schedule a job to a warm container
+ ContainerPool
+ .schedule(r.action, r.msg.user.namespace.name, freePool)
+ .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
+ .orElse(
+ // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
+ // When take prewarm container, has no need to judge whether user memory is enough
+ takePrewarmContainer(r.action)
+ .map(container => (container, "prewarmed"))
+ .orElse {
+ // Is there enough space to create a new container or do other containers have to be removed?
+ if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, prewarmStartingPool, memory)) {
Review comment:
Add memory check condition for coldStart
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] codecov-io edited a comment on pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#issuecomment-763785825
# [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=h1) Report
> Merging [#5048](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=desc) (0c91e52) into [master](https://codecov.io/gh/apache/openwhisk/commit/66868205b52ee65f28756038c44d8df5b96d2bcc?el=desc) (6686820) will **decrease** coverage by `76.51%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #5048 +/- ##
==========================================
- Coverage 82.52% 6.00% -76.52%
==========================================
Files 206 206
Lines 10006 10007 +1
Branches 445 451 +6
==========================================
- Hits 8257 601 -7656
- Misses 1749 9406 +7657
```
| [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...e/openwhisk/core/containerpool/ContainerPool.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udGFpbmVycG9vbC9Db250YWluZXJQb29sLnNjYWxh) | `0.00% <0.00%> (-97.21%)` | :arrow_down: |
| [...ain/scala/org/apache/openwhisk/spi/SpiLoader.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL3NwaS9TcGlMb2FkZXIuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...n/scala/org/apache/openwhisk/utils/JsHelpers.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL3V0aWxzL0pzSGVscGVycy5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...scala/org/apache/openwhisk/common/Prometheus.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Qcm9tZXRoZXVzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...scala/org/apache/openwhisk/core/FeatureFlags.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvRmVhdHVyZUZsYWdzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ala/org/apache/openwhisk/common/ConfigMXBean.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNWEJlYW4uc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...la/org/apache/openwhisk/http/BasicRasService.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNSYXNTZXJ2aWNlLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...a/org/apache/openwhisk/common/ConfigMapValue.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNYXBWYWx1ZS5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...a/org/apache/openwhisk/core/database/Batcher.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvQmF0Y2hlci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...a/org/apache/openwhisk/core/entity/WhiskAuth.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L1doaXNrQXV0aC5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [171 more](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=footer). Last update [6686820...0c91e52](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] codecov-io edited a comment on pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#issuecomment-763785825
# [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=h1) Report
> Merging [#5048](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=desc) (c9cba12) into [master](https://codecov.io/gh/apache/openwhisk/commit/a6ad9e418e605894c5e96e5601c3e1b8ded4166a?el=desc) (a6ad9e4) will **decrease** coverage by `11.26%`.
> The diff coverage is `95.65%`.
[](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #5048 +/- ##
===========================================
- Coverage 82.51% 71.25% -11.27%
===========================================
Files 199 199
Lines 9735 9736 +1
Branches 435 442 +7
===========================================
- Hits 8033 6937 -1096
- Misses 1702 2799 +1097
```
| [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...e/openwhisk/core/containerpool/ContainerPool.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udGFpbmVycG9vbC9Db250YWluZXJQb29sLnNjYWxh) | `88.04% <95.65%> (-9.16%)` | :arrow_down: |
| [...che/openwhisk/core/invoker/LogStoreCollector.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvaW52b2tlci9Mb2dTdG9yZUNvbGxlY3Rvci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...core/database/cosmosdb/RxObservableImplicits.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvUnhPYnNlcnZhYmxlSW1wbGljaXRzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ore/database/cosmosdb/cache/CacheInvalidator.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NhY2hlSW52YWxpZGF0b3Iuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...e/database/cosmosdb/cache/ChangeFeedConsumer.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NoYW5nZUZlZWRDb25zdW1lci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...core/database/cosmosdb/CosmosDBArtifactStore.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlLnNjYWxh) | `0.00% <0.00%> (-95.85%)` | :arrow_down: |
| [...sk/core/database/cosmosdb/CosmosDBViewMapper.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJWaWV3TWFwcGVyLnNjYWxh) | `0.00% <0.00%> (-93.90%)` | :arrow_down: |
| [...tabase/cosmosdb/cache/CacheInvalidatorConfig.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NhY2hlSW52YWxpZGF0b3JDb25maWcuc2NhbGE=) | `0.00% <0.00%> (-92.31%)` | :arrow_down: |
| [...a/org/apache/openwhisk/http/BasicHttpService.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNIdHRwU2VydmljZS5zY2FsYQ==) | `3.33% <0.00%> (-86.67%)` | :arrow_down: |
| [...la/org/apache/openwhisk/http/BasicRasService.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNSYXNTZXJ2aWNlLnNjYWxh) | `16.66% <0.00%> (-83.34%)` | :arrow_down: |
| ... and [43 more](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=footer). Last update [a6ad9e4...c9cba12](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r566506478
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
- // Schedule a job to a warm container
- ContainerPool
- .schedule(r.action, r.msg.user.namespace.name, freePool)
- .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
- .orElse(
- // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
-
- // Is there enough space to create a new container or do other containers have to be removed?
- if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, memory)) {
+ // Schedule a job to a warm container
+ ContainerPool
+ .schedule(r.action, r.msg.user.namespace.name, freePool)
+ .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
+ .orElse(
+ // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
+
+ // Is there enough space to create a new container or do other containers have to be removed?
+ if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, prewarmStartingPool, memory)) {
Review comment:
Updated accordingly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r565868948
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -437,8 +439,10 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
* @param memory The amount of memory to check.
* @return true, if there is enough space for the given amount of memory.
*/
- def hasPoolSpaceFor[A](pool: Map[A, ContainerData], memory: ByteSize): Boolean = {
- memoryConsumptionOf(pool) + memory.toMB <= poolConfig.userMemory.toMB
+ def hasPoolSpaceFor[A](pool: Map[A, ContainerData],
+ prewarmStartingPool: Map[A, (String, ByteSize)],
+ memory: ByteSize): Boolean = {
+ memoryConsumptionOf(pool) + +prewarmStartingPool.map(_._2._2.toMB).sum + memory.toMB <= poolConfig.userMemory.toMB
Review comment:
Ah, yes, you are right.
Already fixed to `memoryConsumptionOf(pool) + prewarmStartingPool.map(_._2._2.toMB).sum + memory.toMB <= poolConfig.userMemory.toMB`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang merged pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang merged pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] codecov-io commented on pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#issuecomment-763785825
# [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=h1) Report
> Merging [#5048](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=desc) (f07708f) into [master](https://codecov.io/gh/apache/openwhisk/commit/a6ad9e418e605894c5e96e5601c3e1b8ded4166a?el=desc) (a6ad9e4) will **decrease** coverage by `76.50%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #5048 +/- ##
==========================================
- Coverage 82.51% 6.00% -76.51%
==========================================
Files 199 206 +7
Lines 9735 10001 +266
Branches 435 443 +8
==========================================
- Hits 8033 601 -7432
- Misses 1702 9400 +7698
```
| [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...e/openwhisk/core/containerpool/ContainerPool.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udGFpbmVycG9vbC9Db250YWluZXJQb29sLnNjYWxh) | `0.00% <0.00%> (-97.21%)` | :arrow_down: |
| [...ain/scala/org/apache/openwhisk/spi/SpiLoader.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL3NwaS9TcGlMb2FkZXIuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...n/scala/org/apache/openwhisk/utils/JsHelpers.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL3V0aWxzL0pzSGVscGVycy5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...scala/org/apache/openwhisk/common/Prometheus.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Qcm9tZXRoZXVzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...scala/org/apache/openwhisk/core/FeatureFlags.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvRmVhdHVyZUZsYWdzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ala/org/apache/openwhisk/common/ConfigMXBean.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNWEJlYW4uc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...la/org/apache/openwhisk/http/BasicRasService.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNSYXNTZXJ2aWNlLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...a/org/apache/openwhisk/common/ConfigMapValue.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNYXBWYWx1ZS5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...a/org/apache/openwhisk/core/database/Batcher.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvQmF0Y2hlci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...a/org/apache/openwhisk/core/entity/WhiskAuth.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L1doaXNrQXV0aC5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [177 more](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=footer). Last update [a6ad9e4...f07708f](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r561564023
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
Review comment:
Due to `didn't create any container here`, so it is better to remove below logic here.
```
// Is there enough space on the invoker for this action to be executed.
if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory))
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r565877511
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
- // Schedule a job to a warm container
- ContainerPool
- .schedule(r.action, r.msg.user.namespace.name, freePool)
- .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
- .orElse(
- // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
-
- // Is there enough space to create a new container or do other containers have to be removed?
- if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, memory)) {
+ // Schedule a job to a warm container
+ ContainerPool
+ .schedule(r.action, r.msg.user.namespace.name, freePool)
+ .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
+ .orElse(
+ // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
+
+ // Is there enough space to create a new container or do other containers have to be removed?
+ if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, prewarmStartingPool, memory)) {
Review comment:
hm..i think we need to concern about the total memory when take a prewarm conainer from `prewarmPool`,
e.g. let's assume invoker's user memory is `1024 MB`, currently, busyPool is `512MB`, prewarmPool is `512MB`,
if user's activation's corresponding action can match the warmedPool, have no need to add condition: `if (hasPoolSpaceFor(busyPool...)`.
but if matched a prewarm container(256MB), then, busyPool will be `512MB+256MB`, prewarmedPool is `512MB-256MB+256MB`, so the total used memory will be `1024MB +256MB`, so it is better to add judge condition.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r560864143
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
- // Schedule a job to a warm container
- ContainerPool
- .schedule(r.action, r.msg.user.namespace.name, freePool)
- .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
- .orElse(
- // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
-
- // Is there enough space to create a new container or do other containers have to be removed?
- if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, memory)) {
+ // Schedule a job to a warm container
+ ContainerPool
Review comment:
have no need to judge whether exist memory when take container from busy/free pool due to don't create new container
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r561562887
##########
File path: tests/src/test/scala/org/apache/openwhisk/core/containerpool/test/ContainerPoolTests.scala
##########
@@ -829,7 +842,7 @@ class ContainerPoolTests
stream.reset()
val prewarmExpirationCheckIntervel = 2.seconds
val poolConfig =
- ContainerPoolConfig(MemoryLimit.STD_MEMORY * 8, 0.5, false, prewarmExpirationCheckIntervel, None, 100)
+ ContainerPoolConfig(MemoryLimit.STD_MEMORY * 12, 0.5, false, prewarmExpirationCheckIntervel, None, 100)
Review comment:
Due to `add logic of whether exist enough memory`, so need to adjust the user memory here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang edited a comment on pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang edited a comment on pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#issuecomment-766515780
@style95 ,AFAIK, the travis ci problem is fixed by you, but currently, it seems some new problems happened

Have any idea? or can guide me to solve it as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r561421295
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -369,9 +365,15 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
/** Creates a new prewarmed container */
def prewarmContainer(exec: CodeExec[_], memoryLimit: ByteSize, ttl: Option[FiniteDuration]): Unit = {
- val newContainer = childFactory(context)
- prewarmStartingPool = prewarmStartingPool + (newContainer -> (exec.kind, memoryLimit))
- newContainer ! Start(exec, memoryLimit, ttl)
+ if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, memoryLimit)) {
Review comment:
When create prewarm container, judge whether exist enough memory.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r566507836
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
- // Schedule a job to a warm container
- ContainerPool
- .schedule(r.action, r.msg.user.namespace.name, freePool)
- .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
- .orElse(
- // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
-
- // Is there enough space to create a new container or do other containers have to be removed?
- if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, memory)) {
+ // Schedule a job to a warm container
+ ContainerPool
+ .schedule(r.action, r.msg.user.namespace.name, freePool)
+ .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
+ .orElse(
+ // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
+ // When take prewarm container, has no need to judge whether user memory is enough
+ takePrewarmContainer(r.action)
+ .map(container => (container, "prewarmed"))
+ .orElse {
+ // Is there enough space to create a new container or do other containers have to be removed?
+ if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, prewarmStartingPool, memory)) {
+ val container = Some(createContainer(memory), "cold")
+ incrementColdStartCount(kind, memory)
+ container
+ } else None
+ })
+ .orElse(
+ // Remove a container and create a new one for the given job
+ ContainerPool
+ // Only free up the amount, that is really needed to free up
+ .remove(freePool, Math.min(r.action.limits.memory.megabytes, memoryConsumptionOf(freePool)).MB)
+ .map(removeContainer)
+ // If the list had at least one entry, enough containers were removed to start the new container. After
+ // removing the containers, we are not interested anymore in the containers that have been removed.
+ .headOption
+ .map(_ =>
takePrewarmContainer(r.action)
- .map(container => (container, "prewarmed"))
- .orElse {
- val container = Some(createContainer(memory), "cold")
+ .map(container => (container, "recreatedPrewarm"))
+ .getOrElse {
+ val container = (createContainer(memory), "recreated")
Review comment:
For here's coldStart, i think has no need to do memory check. because it will delete a container from freePool in advance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang closed pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang closed pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r561562887
##########
File path: tests/src/test/scala/org/apache/openwhisk/core/containerpool/test/ContainerPoolTests.scala
##########
@@ -829,7 +842,7 @@ class ContainerPoolTests
stream.reset()
val prewarmExpirationCheckIntervel = 2.seconds
val poolConfig =
- ContainerPoolConfig(MemoryLimit.STD_MEMORY * 8, 0.5, false, prewarmExpirationCheckIntervel, None, 100)
+ ContainerPoolConfig(MemoryLimit.STD_MEMORY * 12, 0.5, false, prewarmExpirationCheckIntervel, None, 100)
Review comment:
Due to calculate prewarmStartingPool's memory, so need to adjust the user memory here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] style95 commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
style95 commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r565987618
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
- // Schedule a job to a warm container
- ContainerPool
- .schedule(r.action, r.msg.user.namespace.name, freePool)
- .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
- .orElse(
- // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
-
- // Is there enough space to create a new container or do other containers have to be removed?
- if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, memory)) {
+ // Schedule a job to a warm container
+ ContainerPool
+ .schedule(r.action, r.msg.user.namespace.name, freePool)
+ .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
+ .orElse(
+ // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
+
+ // Is there enough space to create a new container or do other containers have to be removed?
+ if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, prewarmStartingPool, memory)) {
Review comment:
But if we take the current approach, even under the condition that there are 512MB of prewarm containers, they can't be used.
I'd rather let it take prewarm containers but prevent the subsequent creation of a new prewarm container.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#issuecomment-766515780
@style95 ,AFAIK, the travis ci problem is fixed by you, but currently, it seems some new problems happened

Have any idea? or guide me to solve it as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r566539842
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -439,8 +441,10 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
* @param memory The amount of memory to check.
* @return true, if there is enough space for the given amount of memory.
*/
- def hasPoolSpaceFor[A](pool: Map[A, ContainerData], memory: ByteSize): Boolean = {
- memoryConsumptionOf(pool) + memory.toMB <= poolConfig.userMemory.toMB
+ def hasPoolSpaceFor[A](pool: Map[A, ContainerData],
+ prewarmStartingPool: Map[A, (String, ByteSize)],
+ memory: ByteSize): Boolean = {
+ memoryConsumptionOf(pool) + prewarmStartingPool.map(_._2._2.toMB).sum + memory.toMB <= poolConfig.userMemory.toMB
Review comment:
https://github.com/apache/openwhisk/blob/master/core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala#L88
every 1 minute by default
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r566507159
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
- // Schedule a job to a warm container
- ContainerPool
- .schedule(r.action, r.msg.user.namespace.name, freePool)
- .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
- .orElse(
- // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
-
- // Is there enough space to create a new container or do other containers have to be removed?
- if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, memory)) {
+ // Schedule a job to a warm container
+ ContainerPool
+ .schedule(r.action, r.msg.user.namespace.name, freePool)
+ .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
+ .orElse(
+ // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
+ // When take prewarm container, has no need to judge whether user memory is enough
+ takePrewarmContainer(r.action)
+ .map(container => (container, "prewarmed"))
+ .orElse {
+ // Is there enough space to create a new container or do other containers have to be removed?
+ if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, prewarmStartingPool, memory)) {
+ val container = Some(createContainer(memory), "cold")
+ incrementColdStartCount(kind, memory)
+ container
Review comment:
Add memory check condition for coldStart
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] ningyougang commented on a change in pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#discussion_r560864143
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
- // Schedule a job to a warm container
- ContainerPool
- .schedule(r.action, r.msg.user.namespace.name, freePool)
- .map(container => (container, container._2.initingState)) //warmed, warming, and warmingCold always know their state
- .orElse(
- // There was no warm/warming/warmingCold container. Try to take a prewarm container or a cold container.
-
- // Is there enough space to create a new container or do other containers have to be removed?
- if (hasPoolSpaceFor(busyPool ++ freePool ++ prewarmedPool, memory)) {
+ // Schedule a job to a warm container
+ ContainerPool
Review comment:
have no need to judge whether exist memory when take container from busy/free pool due to don't create new container
##########
File path: core/invoker/src/main/scala/org/apache/openwhisk/core/containerpool/ContainerPool.scala
##########
@@ -129,44 +129,40 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
val memory = r.action.limits.memory.megabytes.MB
val createdContainer =
- // Is there enough space on the invoker for this action to be executed.
- if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory)) {
Review comment:
Due didn't create any container here, so it is better to remove below logic here.
```
// Is there enough space on the invoker for this action to be executed.
if (hasPoolSpaceFor(busyPool ++ prewarmedPool, memory))
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk] codecov-io edited a comment on pull request #5048: Don't create prewarm container when used memory reaches the limit
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #5048:
URL: https://github.com/apache/openwhisk/pull/5048#issuecomment-763785825
# [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=h1) Report
> Merging [#5048](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=desc) (0c91e52) into [master](https://codecov.io/gh/apache/openwhisk/commit/66868205b52ee65f28756038c44d8df5b96d2bcc?el=desc) (6686820) will **decrease** coverage by `10.75%`.
> The diff coverage is `95.65%`.
[](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #5048 +/- ##
===========================================
- Coverage 82.52% 71.76% -10.76%
===========================================
Files 206 206
Lines 10006 10007 +1
Branches 445 451 +6
===========================================
- Hits 8257 7182 -1075
- Misses 1749 2825 +1076
```
| [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...e/openwhisk/core/containerpool/ContainerPool.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udGFpbmVycG9vbC9Db250YWluZXJQb29sLnNjYWxh) | `88.44% <95.65%> (-8.76%)` | :arrow_down: |
| [...che/openwhisk/core/invoker/LogStoreCollector.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvaW52b2tlci9Mb2dTdG9yZUNvbGxlY3Rvci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...core/database/cosmosdb/RxObservableImplicits.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvUnhPYnNlcnZhYmxlSW1wbGljaXRzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ore/database/cosmosdb/cache/CacheInvalidator.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NhY2hlSW52YWxpZGF0b3Iuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...e/database/cosmosdb/cache/ChangeFeedConsumer.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NoYW5nZUZlZWRDb25zdW1lci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...core/database/cosmosdb/CosmosDBArtifactStore.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlLnNjYWxh) | `0.00% <0.00%> (-95.85%)` | :arrow_down: |
| [...sk/core/database/cosmosdb/CosmosDBViewMapper.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJWaWV3TWFwcGVyLnNjYWxh) | `0.00% <0.00%> (-93.90%)` | :arrow_down: |
| [...tabase/cosmosdb/cache/CacheInvalidatorConfig.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NhY2hlSW52YWxpZGF0b3JDb25maWcuc2NhbGE=) | `0.00% <0.00%> (-92.31%)` | :arrow_down: |
| [...a/org/apache/openwhisk/http/BasicHttpService.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNIdHRwU2VydmljZS5zY2FsYQ==) | `3.33% <0.00%> (-86.67%)` | :arrow_down: |
| [...la/org/apache/openwhisk/http/BasicRasService.scala](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNSYXNTZXJ2aWNlLnNjYWxh) | `16.66% <0.00%> (-83.34%)` | :arrow_down: |
| ... and [44 more](https://codecov.io/gh/apache/openwhisk/pull/5048/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=footer). Last update [6686820...0c91e52](https://codecov.io/gh/apache/openwhisk/pull/5048?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org