You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@openwhisk.apache.org by GitBox <gi...@apache.org> on 2018/07/10 09:51:12 UTC
[GitHub] chetanmeh commented on a change in pull request #3747: Memory based
loadbalancing
chetanmeh commented on a change in pull request #3747: Memory based loadbalancing
URL: https://github.com/apache/incubator-openwhisk/pull/3747#discussion_r201282300
##########
File path: core/invoker/src/main/scala/whisk/core/containerpool/ContainerPool.scala
##########
@@ -91,62 +98,87 @@ class ContainerPool(childFactory: ActorRefFactory => ActorRef,
// their requests and send them back to the pool for rescheduling (this may happen if "docker" operations
// fail for example, or a container has aged and was destroying itself when a new request was assigned)
case r: Run =>
- val createdContainer = if (busyPool.size < poolConfig.maxActiveContainers) {
+ // Check if the message is resent from the buffer. Only the first message on the buffer can be resent.
+ val isResentFromBuffer = runBuffer.nonEmpty && runBuffer.dequeueOption.exists(_._1.msg == r.msg)
- // Schedule a job to a warm container
- ContainerPool
- .schedule(r.action, r.msg.user.namespace.name, freePool)
- .map(container => {
- (container, "warm")
- })
- .orElse {
- if (busyPool.size + freePool.size < poolConfig.maxActiveContainers) {
- takePrewarmContainer(r.action)
- .map(container => {
- (container, "prewarmed")
- })
- .orElse {
- Some(createContainer(), "cold")
- }
- } else None
- }
- .orElse {
- // Remove a container and create a new one for the given job
- ContainerPool.remove(freePool).map { toDelete =>
- removeContainer(toDelete)
- takePrewarmContainer(r.action)
- .map(container => {
- (container, "recreated")
- })
- .getOrElse {
- (createContainer(), "recreated")
- }
- }
- }
- } else None
+ // Only process request, if there are no other requests waiting for free slots, or if the current request is the
+ // next request to process
+ // It is guaranteed, that only the first message on the buffer is resent.
+ if (runBuffer.isEmpty || isResentFromBuffer) {
+ val createdContainer =
+ // Is there enough space on the invoker for this action to be executed.
+ if (hasPoolSpaceFor(busyPool, r.action.limits.memory.megabytes.MB)) {
+ // Schedule a job to a warm container
+ ContainerPool
+ .schedule(r.action, r.msg.user.namespace.name, freePool)
+ .map(container => (container, "warm"))
+ .orElse(
+ // There was no warm container. Try to take a prewarm container or a cold container.
+
+ // Is there enough space to create a new container or do other containers have to be removed?
+ if (hasPoolSpaceFor(busyPool ++ freePool, r.action.limits.memory.megabytes.MB)) {
+ takePrewarmContainer(r.action)
+ .map(container => (container, "prewarmed"))
+ .orElse(Some(createContainer(r.action.limits.memory.megabytes.MB), "cold"))
+ } else None)
+ .orElse(
+ // Remove a container and create a new one for the given job
+ ContainerPool
+ // Only free up the amount, that is really needed to free up
+ .remove(freePool, Math.min(r.action.limits.memory.megabytes, memoryConsumptionOf(freePool)).MB)
+ .map(removeContainer)
+ // If the list had at least one entry, enough containers were removed to start the new container. After
+ // removing the containers, we are not interested anymore in the containers that have been removed.
+ .headOption
+ .map(_ =>
+ takePrewarmContainer(r.action)
+ .map(container => (container, "recreated prewarm"))
+ .getOrElse(createContainer(r.action.limits.memory.megabytes.MB), "recreated")))
Review comment:
Better to use non space separator like `recreated_prewarm` as the state gets used for Metric names. Currently following metric names get generated
* `openwhisk.counter.invoker_containerStart.warm_count`
* `openwhisk.counter.invoker_containerStart.cold_count`
* `openwhisk.counter.invoker_containerStart.precreated_count`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services