You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@openwhisk.apache.org by GitBox <gi...@apache.org> on 2021/02/19 07:31:09 UTC

[GitHub] [openwhisk] upgle opened a new pull request #5070: [New Scheduler] Add ActivationService

upgle opened a new pull request #5070:
URL: https://github.com/apache/openwhisk/pull/5070


   <!--- Provide a concise summary of your changes in the Title -->
   
   ## Description
   <!--- Provide a detailed description of your changes. -->
   <!--- Include details of what problem you are solving and how your changes are tested. -->
   
   ActivationService is a GRPC service implementation for the scheduler that communicates with the container proxy.
   
   Please refer to this document:
   https://cwiki.apache.org/confluence/display/OPENWHISK/ActivationServiceImpl
   
   ## Related issue and scope
   <!--- Please include a link to a related issue if there is one. -->
   - [ ] I opened an issue to propose and discuss this change (#????)
   
   ## My changes affect the following components
   <!--- Select below all system components are affected by your change. -->
   <!--- Enter an `x` in all applicable boxes. -->
   - [ ] API
   - [ ] Controller
   - [ ] Message Bus (e.g., Kafka)
   - [ ] Loadbalancer
   - [x] Scheduler
   - [ ] Invoker
   - [ ] Intrinsic actions (e.g., sequences, conductors)
   - [ ] Data stores (e.g., CouchDB)
   - [ ] Tests
   - [ ] Deployment
   - [ ] CLI
   - [ ] General tooling
   - [ ] Documentation
   
   ## Types of changes
   <!--- What types of changes does your code introduce? Use `x` in all the boxes that apply: -->
   - [ ] Bug fix (generally a non-breaking change which closes an issue).
   - [ ] Enhancement or new feature (adds new functionality).
   - [ ] Breaking change (a bug fix or enhancement which changes existing behavior).
   
   ## Checklist:
   <!--- Please review the points below which help you make sure you've covered all aspects of the change you're making. -->
   
   - [x] I signed an [Apache CLA](https://github.com/apache/openwhisk/blob/master/CONTRIBUTING.md).
   - [x] I reviewed the [style guides](https://github.com/apache/openwhisk/wiki/Contributing:-Git-guidelines#code-readiness) and followed the recommendations (Travis CI will check :).
   - [x] I added tests to cover my changes.
   - [ ] My changes require further changes to the documentation.
   - [ ] I updated the documentation where necessary.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [openwhisk] codecov-io edited a comment on pull request #5070: [New Scheduler] Add ActivationService

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #5070:
URL: https://github.com/apache/openwhisk/pull/5070#issuecomment-790453439


   # [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=h1) Report
   > Merging [#5070](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=desc) (4d9c7c6) into [master](https://codecov.io/gh/apache/openwhisk/commit/4a13303fae4d9750da6662bb39b3ec92d6ccf56c?el=desc) (4a13303) will **decrease** coverage by `43.54%`.
   > The diff coverage is `75.82%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/openwhisk/pull/5070/graphs/tree.svg?width=650&height=150&src=pr&token=l0YmsiSAso)](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #5070       +/-   ##
   ===========================================
   - Coverage   81.95%   38.41%   -43.55%     
   ===========================================
     Files         210      218        +8     
     Lines       10167    10700      +533     
     Branches      440      450       +10     
   ===========================================
   - Hits         8332     4110     -4222     
   - Misses       1835     6590     +4755     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [.../openwhisk/core/scheduler/queue/QueueManager.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvUXVldWVNYW5hZ2VyLnNjYWxh) | `56.75% <56.75%> (ø)` | |
   | [...sk/core/scheduler/grpc/ActivationServiceImpl.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvZ3JwYy9BY3RpdmF0aW9uU2VydmljZUltcGwuc2NhbGE=) | `88.00% <88.00%> (ø)` | |
   | [...ala/org/apache/openwhisk/core/entity/DocInfo.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L0RvY0luZm8uc2NhbGE=) | `81.81% <100.00%> (-5.28%)` | :arrow_down: |
   | [...enwhisk/core/entity/FullyQualifiedEntityName.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L0Z1bGx5UXVhbGlmaWVkRW50aXR5TmFtZS5zY2FsYQ==) | `72.97% <100.00%> (-18.46%)` | :arrow_down: |
   | [...a/org/apache/openwhisk/common/ConfigMapValue.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNYXBWYWx1ZS5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [.../apache/openwhisk/core/controller/Namespaces.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udHJvbGxlci9OYW1lc3BhY2VzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...pache/openwhisk/core/controller/CorsSettings.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udHJvbGxlci9Db3JzU2V0dGluZ3Muc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...che/openwhisk/core/entitlement/RateThrottler.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXRsZW1lbnQvUmF0ZVRocm90dGxlci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...he/openwhisk/core/entitlement/KindRestrictor.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXRsZW1lbnQvS2luZFJlc3RyaWN0b3Iuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...penwhisk/core/database/cosmosdb/CosmosDBUtil.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJVdGlsLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | ... and [149 more](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=footer). Last update [4a13303...4d9c7c6](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [openwhisk] codecov-io edited a comment on pull request #5070: [New Scheduler] Add ActivationService

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #5070:
URL: https://github.com/apache/openwhisk/pull/5070#issuecomment-790453439


   # [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=h1) Report
   > Merging [#5070](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=desc) (83f8313) into [master](https://codecov.io/gh/apache/openwhisk/commit/4a13303fae4d9750da6662bb39b3ec92d6ccf56c?el=desc) (4a13303) will **decrease** coverage by `49.23%`.
   > The diff coverage is `75.82%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/openwhisk/pull/5070/graphs/tree.svg?width=650&height=150&src=pr&token=l0YmsiSAso)](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #5070       +/-   ##
   ===========================================
   - Coverage   81.95%   32.71%   -49.24%     
   ===========================================
     Files         210      213        +3     
     Lines       10167    10306      +139     
     Branches      440      459       +19     
   ===========================================
   - Hits         8332     3372     -4960     
   - Misses       1835     6934     +5099     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [.../openwhisk/core/scheduler/queue/QueueManager.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvUXVldWVNYW5hZ2VyLnNjYWxh) | `56.75% <56.75%> (ø)` | |
   | [...sk/core/scheduler/grpc/ActivationServiceImpl.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvZ3JwYy9BY3RpdmF0aW9uU2VydmljZUltcGwuc2NhbGE=) | `88.00% <88.00%> (ø)` | |
   | [...ala/org/apache/openwhisk/core/entity/DocInfo.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L0RvY0luZm8uc2NhbGE=) | `78.78% <100.00%> (-8.31%)` | :arrow_down: |
   | [...enwhisk/core/entity/FullyQualifiedEntityName.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L0Z1bGx5UXVhbGlmaWVkRW50aXR5TmFtZS5zY2FsYQ==) | `45.94% <100.00%> (-45.49%)` | :arrow_down: |
   | [...n/scala/org/apache/openwhisk/utils/JsHelpers.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL3V0aWxzL0pzSGVscGVycy5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...a/org/apache/openwhisk/common/ConfigMapValue.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNYXBWYWx1ZS5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...g/apache/openwhisk/common/ResizableSemaphore.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9SZXNpemFibGVTZW1hcGhvcmUuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...pache/openwhisk/http/LenientSprayJsonSupport.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvTGVuaWVudFNwcmF5SnNvblN1cHBvcnQuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [.../apache/openwhisk/core/controller/Namespaces.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udHJvbGxlci9OYW1lc3BhY2VzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...pache/openwhisk/core/controller/CorsSettings.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udHJvbGxlci9Db3JzU2V0dGluZ3Muc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | ... and [150 more](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=footer). Last update [4a13303...83f8313](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [openwhisk] upgle commented on a change in pull request #5070: [New Scheduler] Add ActivationService

Posted by GitBox <gi...@apache.org>.
upgle commented on a change in pull request #5070:
URL: https://github.com/apache/openwhisk/pull/5070#discussion_r583405527



##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/grpc/ActivationServiceImpl.scala
##########
@@ -0,0 +1,118 @@
+package org.apache.openwhisk.core.scheduler.grpc
+
+import akka.actor.ActorSystem
+import akka.pattern.ask
+import akka.util.Timeout
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.connector.{ActivationMessage, Message}
+import org.apache.openwhisk.core.entity.{DocRevision, FullyQualifiedEntityName}
+import org.apache.openwhisk.core.scheduler.queue._
+import org.apache.openwhisk.grpc.{ActivationService, FetchRequest, FetchResponse, RescheduleRequest, RescheduleResponse}
+import spray.json._
+
+import scala.concurrent.duration._
+import scala.concurrent.{ExecutionContextExecutor, Future}
+import scala.util.Try
+
+class ActivationServiceImpl()(implicit actorSystem: ActorSystem, logging: Logging) extends ActivationService {
+  implicit val requestTimeout: Timeout = Timeout(50.seconds)
+  implicit val ec: ExecutionContextExecutor = actorSystem.dispatcher
+
+  override def rescheduleActivation(request: RescheduleRequest): Future[RescheduleResponse] = {
+    logging.info(this, s"Try to reschedule activation ${request.invocationNamespace} ${request.fqn} ${request.rev}")

Review comment:
       Rescheduling is a special case that happens very occasionally. I think we can keep this log with the `info` level.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [openwhisk] style95 merged pull request #5070: [New Scheduler] Add ActivationService

Posted by GitBox <gi...@apache.org>.
style95 merged pull request #5070:
URL: https://github.com/apache/openwhisk/pull/5070


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [openwhisk] bdoyle0182 commented on a change in pull request #5070: [New Scheduler] Add ActivationService

Posted by GitBox <gi...@apache.org>.
bdoyle0182 commented on a change in pull request #5070:
URL: https://github.com/apache/openwhisk/pull/5070#discussion_r580666115



##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/grpc/ActivationServiceImpl.scala
##########
@@ -0,0 +1,118 @@
+package org.apache.openwhisk.core.scheduler.grpc
+
+import akka.actor.ActorSystem
+import akka.pattern.ask
+import akka.util.Timeout
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.connector.{ActivationMessage, Message}
+import org.apache.openwhisk.core.entity.{DocRevision, FullyQualifiedEntityName}
+import org.apache.openwhisk.core.scheduler.queue._
+import org.apache.openwhisk.grpc.{ActivationService, FetchRequest, FetchResponse, RescheduleRequest, RescheduleResponse}
+import spray.json._
+
+import scala.concurrent.duration._
+import scala.concurrent.{ExecutionContextExecutor, Future}
+import scala.util.Try
+
+class ActivationServiceImpl()(implicit actorSystem: ActorSystem, logging: Logging) extends ActivationService {
+  implicit val requestTimeout: Timeout = Timeout(50.seconds)

Review comment:
       should this not be hardcoded?

##########
File path: core/scheduler/src/main/java/Empty.java
##########
@@ -0,0 +1,5 @@
+public class Empty {
+    // Workaround for this issue https://github.com/akka/akka-grpc/issues/289
+    // Gradle complains about no java sources.
+    // Note. Openwhisk is using a lower gradle version, so the latest akka-grpc version cannot be used.

Review comment:
       Can we fix this? The issue has been fixed

##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/grpc/ActivationServiceImpl.scala
##########
@@ -0,0 +1,118 @@
+package org.apache.openwhisk.core.scheduler.grpc
+
+import akka.actor.ActorSystem
+import akka.pattern.ask
+import akka.util.Timeout
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.connector.{ActivationMessage, Message}
+import org.apache.openwhisk.core.entity.{DocRevision, FullyQualifiedEntityName}
+import org.apache.openwhisk.core.scheduler.queue._
+import org.apache.openwhisk.grpc.{ActivationService, FetchRequest, FetchResponse, RescheduleRequest, RescheduleResponse}
+import spray.json._
+
+import scala.concurrent.duration._
+import scala.concurrent.{ExecutionContextExecutor, Future}
+import scala.util.Try
+
+class ActivationServiceImpl()(implicit actorSystem: ActorSystem, logging: Logging) extends ActivationService {
+  implicit val requestTimeout: Timeout = Timeout(50.seconds)
+  implicit val ec: ExecutionContextExecutor = actorSystem.dispatcher
+
+  override def rescheduleActivation(request: RescheduleRequest): Future[RescheduleResponse] = {
+    logging.info(this, s"Try to reschedule activation ${request.invocationNamespace} ${request.fqn} ${request.rev}")
+    Future(for {
+      fqn <- FullyQualifiedEntityName.parse(request.fqn)
+      rev <- DocRevision.parse(request.rev)
+      msg <- ActivationMessage.parse(request.activationMessage)
+    } yield (fqn, rev, msg)).flatMap(Future.fromTry) flatMap { res =>
+      {
+        val key = res._1.toDocId.asDocInfo(res._2)
+        QueuePool.get(MemoryQueueKey(request.invocationNamespace, key)) match {
+          case Some(queueValue) =>
+            // enqueue activation message to reschedule
+            logging.info(

Review comment:
       Should this be `debug`

##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/QueueManager.scala
##########
@@ -0,0 +1,102 @@
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorRef
+import org.apache.openwhisk.core.connector._
+import org.apache.openwhisk.core.entity._
+import spray.json.{DefaultJsonProtocol, _}
+import scala.collection.concurrent.TrieMap
+import scala.util.Try
+
+object QueueSize
+case class MemoryQueueKey(invocationNamespace: String, docInfo: DocInfo)
+case class MemoryQueueValue(queue: ActorRef, isLeader: Boolean)
+
+sealed trait MemoryQueueError extends Product {
+  val causedBy: String
+}
+
+object MemoryQueueErrorSerdes {
+
+  private implicit val noMessageSerdes = NoActivationMessage.serdes
+  private implicit val noQueueSerdes = NoMemoryQueue.serdes
+  private implicit val mismatchSerdes = ActionMismatch.serdes
+
+  // format that discriminates based on an additional
+  // field "type" that can either be "Cat" or "Dog"
+  implicit val memoryQueueErrorFormat = new RootJsonFormat[MemoryQueueError] {
+    def write(obj: MemoryQueueError): JsValue =
+      JsObject((obj match {
+        case msg: NoActivationMessage => msg.toJson
+        case msg: NoMemoryQueue       => msg.toJson
+        case msg: ActionMismatch      => msg.toJson
+      }).asJsObject.fields + ("type" -> JsString(obj.productPrefix)))
+
+    def read(json: JsValue): MemoryQueueError =
+      json.asJsObject.getFields("type") match {
+        case Seq(JsString("NoActivationMessage")) => json.convertTo[NoActivationMessage]
+        case Seq(JsString("NoMemoryQueue"))       => json.convertTo[NoMemoryQueue]
+        case Seq(JsString("ActionMismatch"))      => json.convertTo[ActionMismatch]
+      }
+  }
+}
+
+case class NoActivationMessage(noActivationMessage: String = NoActivationMessage.asString)
+    extends MemoryQueueError
+    with Message {
+  override val causedBy: String = noActivationMessage
+  override def serialize = NoActivationMessage.serdes.write(this).compactPrint
+}
+
+object NoActivationMessage extends DefaultJsonProtocol {
+  val asString: String = "no activation message exist"
+  def parse(msg: String) = Try(serdes.read(msg.parseJson))
+  implicit val serdes = jsonFormat(NoActivationMessage.apply _, "noActivationMessage")
+}
+
+case class NoMemoryQueue(noMemoryQueue: String = NoMemoryQueue.asString) extends MemoryQueueError with Message {
+  override val causedBy: String = noMemoryQueue
+  override def serialize = NoMemoryQueue.serdes.write(this).compactPrint
+}
+
+object NoMemoryQueue extends DefaultJsonProtocol {
+  val asString: String = "no memory queue exist"
+  def parse(msg: String) = Try(serdes.read(msg.parseJson))
+  implicit val serdes = jsonFormat(NoMemoryQueue.apply _, "noMemoryQueue")
+}
+
+case class ActionMismatch(actionMisMatch: String = ActionMismatch.asString) extends MemoryQueueError with Message {
+  override val causedBy: String = actionMisMatch
+  override def serialize = ActionMismatch.serdes.write(this).compactPrint
+}
+
+object ActionMismatch extends DefaultJsonProtocol {
+  val asString: String = "action version does not match"
+  def parse(msg: String) = Try(serdes.read(msg.parseJson))
+  implicit val serdes = jsonFormat(ActionMismatch.apply _, "actionMisMatch")
+}
+
+object QueuePool {

Review comment:
       Is this safe as a global static object being accessed from futures?

##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/grpc/ActivationServiceImpl.scala
##########
@@ -0,0 +1,118 @@
+package org.apache.openwhisk.core.scheduler.grpc
+
+import akka.actor.ActorSystem
+import akka.pattern.ask
+import akka.util.Timeout
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.connector.{ActivationMessage, Message}
+import org.apache.openwhisk.core.entity.{DocRevision, FullyQualifiedEntityName}
+import org.apache.openwhisk.core.scheduler.queue._
+import org.apache.openwhisk.grpc.{ActivationService, FetchRequest, FetchResponse, RescheduleRequest, RescheduleResponse}
+import spray.json._
+
+import scala.concurrent.duration._
+import scala.concurrent.{ExecutionContextExecutor, Future}
+import scala.util.Try
+
+class ActivationServiceImpl()(implicit actorSystem: ActorSystem, logging: Logging) extends ActivationService {
+  implicit val requestTimeout: Timeout = Timeout(50.seconds)
+  implicit val ec: ExecutionContextExecutor = actorSystem.dispatcher
+
+  override def rescheduleActivation(request: RescheduleRequest): Future[RescheduleResponse] = {
+    logging.info(this, s"Try to reschedule activation ${request.invocationNamespace} ${request.fqn} ${request.rev}")

Review comment:
       I think this should be `debug` statement

##########
File path: core/scheduler/src/main/protobuf/activation.proto
##########
@@ -0,0 +1,52 @@
+syntax = "proto3";
+import "google/protobuf/wrappers.proto";
+
+//#options
+option java_multiple_files = true;
+option java_package = "org.apache.openwhisk.grpc";
+option java_outer_classname = "ActivationProto";
+
+package activation;
+//#options
+
+//#services
+service ActivationService {
+

Review comment:
       nit: remove extra lines

##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/grpc/ActivationServiceImpl.scala
##########
@@ -0,0 +1,118 @@
+package org.apache.openwhisk.core.scheduler.grpc
+
+import akka.actor.ActorSystem
+import akka.pattern.ask
+import akka.util.Timeout
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.connector.{ActivationMessage, Message}
+import org.apache.openwhisk.core.entity.{DocRevision, FullyQualifiedEntityName}
+import org.apache.openwhisk.core.scheduler.queue._
+import org.apache.openwhisk.grpc.{ActivationService, FetchRequest, FetchResponse, RescheduleRequest, RescheduleResponse}
+import spray.json._
+
+import scala.concurrent.duration._
+import scala.concurrent.{ExecutionContextExecutor, Future}
+import scala.util.Try
+
+class ActivationServiceImpl()(implicit actorSystem: ActorSystem, logging: Logging) extends ActivationService {
+  implicit val requestTimeout: Timeout = Timeout(50.seconds)

Review comment:
       should this not be hardcoded?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [openwhisk] codecov-io edited a comment on pull request #5070: [New Scheduler] Add ActivationService

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #5070:
URL: https://github.com/apache/openwhisk/pull/5070#issuecomment-790453439


   # [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=h1) Report
   > Merging [#5070](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=desc) (2a4c69e) into [master](https://codecov.io/gh/apache/openwhisk/commit/4a13303fae4d9750da6662bb39b3ec92d6ccf56c?el=desc) (4a13303) will **decrease** coverage by `47.79%`.
   > The diff coverage is `75.82%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/openwhisk/pull/5070/graphs/tree.svg?width=650&height=150&src=pr&token=l0YmsiSAso)](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #5070       +/-   ##
   ===========================================
   - Coverage   81.95%   34.15%   -47.80%     
   ===========================================
     Files         210      218        +8     
     Lines       10167    10700      +533     
     Branches      440      450       +10     
   ===========================================
   - Hits         8332     3655     -4677     
   - Misses       1835     7045     +5210     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [.../openwhisk/core/scheduler/queue/QueueManager.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvUXVldWVNYW5hZ2VyLnNjYWxh) | `56.75% <56.75%> (ø)` | |
   | [...sk/core/scheduler/grpc/ActivationServiceImpl.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvZ3JwYy9BY3RpdmF0aW9uU2VydmljZUltcGwuc2NhbGE=) | `88.00% <88.00%> (ø)` | |
   | [...ala/org/apache/openwhisk/core/entity/DocInfo.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L0RvY0luZm8uc2NhbGE=) | `78.78% <100.00%> (-8.31%)` | :arrow_down: |
   | [...enwhisk/core/entity/FullyQualifiedEntityName.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L0Z1bGx5UXVhbGlmaWVkRW50aXR5TmFtZS5zY2FsYQ==) | `45.94% <100.00%> (-45.49%)` | :arrow_down: |
   | [...n/scala/org/apache/openwhisk/utils/JsHelpers.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL3V0aWxzL0pzSGVscGVycy5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...a/org/apache/openwhisk/common/ConfigMapValue.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNYXBWYWx1ZS5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...g/apache/openwhisk/common/ResizableSemaphore.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9SZXNpemFibGVTZW1hcGhvcmUuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...pache/openwhisk/http/LenientSprayJsonSupport.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvTGVuaWVudFNwcmF5SnNvblN1cHBvcnQuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [.../apache/openwhisk/core/controller/Namespaces.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udHJvbGxlci9OYW1lc3BhY2VzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...pache/openwhisk/core/controller/CorsSettings.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udHJvbGxlci9Db3JzU2V0dGluZ3Muc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | ... and [159 more](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=footer). Last update [4a13303...2a4c69e](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [openwhisk] style95 commented on pull request #5070: [New Scheduler] Add ActivationService

Posted by GitBox <gi...@apache.org>.
style95 commented on pull request #5070:
URL: https://github.com/apache/openwhisk/pull/5070#issuecomment-802382008


   Seems there are some formatting errors.
   ```
   [/home/travis/build/apache/openwhisk/tools/travis/../../core/scheduler/build/generated/source/proto/main/akkaGrpc/org/apache/openwhisk/grpc/ActivationService.scala]:
          1: file does not include required license header.
          9: line has trailing whitespace.
         10: line has trailing whitespace.
         12: line has trailing whitespace.
         13: line has trailing whitespace.
         15: line has trailing whitespace.
         23: line has trailing whitespace.
         25: line has trailing whitespace.
         27: line has trailing whitespace.
         29: line has trailing whitespace.
         31: line has trailing whitespace.
     [/home/travis/build/apache/openwhisk/tools/travis/../../core/scheduler/build/generated/source/proto/main/akkaGrpc/org/apache/openwhisk/grpc/ActivationServiceClient.scala]:
          1: file does not include required license header.
         37: line has trailing whitespace.
         39: line has trailing whitespace.
         41: line has trailing whitespace.
         43: line has trailing whitespace.
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [openwhisk] codecov-io edited a comment on pull request #5070: [New Scheduler] Add ActivationService

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #5070:
URL: https://github.com/apache/openwhisk/pull/5070#issuecomment-790453439


   # [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=h1) Report
   > Merging [#5070](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=desc) (4d9c7c6) into [master](https://codecov.io/gh/apache/openwhisk/commit/4a13303fae4d9750da6662bb39b3ec92d6ccf56c?el=desc) (4a13303) will **decrease** coverage by `43.52%`.
   > The diff coverage is `75.82%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/openwhisk/pull/5070/graphs/tree.svg?width=650&height=150&src=pr&token=l0YmsiSAso)](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #5070       +/-   ##
   ===========================================
   - Coverage   81.95%   38.42%   -43.53%     
   ===========================================
     Files         210      218        +8     
     Lines       10167    10700      +533     
     Branches      440      450       +10     
   ===========================================
   - Hits         8332     4112     -4220     
   - Misses       1835     6588     +4753     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [.../openwhisk/core/scheduler/queue/QueueManager.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvUXVldWVNYW5hZ2VyLnNjYWxh) | `56.75% <56.75%> (ø)` | |
   | [...sk/core/scheduler/grpc/ActivationServiceImpl.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvZ3JwYy9BY3RpdmF0aW9uU2VydmljZUltcGwuc2NhbGE=) | `88.00% <88.00%> (ø)` | |
   | [...ala/org/apache/openwhisk/core/entity/DocInfo.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L0RvY0luZm8uc2NhbGE=) | `81.81% <100.00%> (-5.28%)` | :arrow_down: |
   | [...enwhisk/core/entity/FullyQualifiedEntityName.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L0Z1bGx5UXVhbGlmaWVkRW50aXR5TmFtZS5zY2FsYQ==) | `72.97% <100.00%> (-18.46%)` | :arrow_down: |
   | [...a/org/apache/openwhisk/common/ConfigMapValue.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNYXBWYWx1ZS5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [.../apache/openwhisk/core/controller/Namespaces.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udHJvbGxlci9OYW1lc3BhY2VzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...pache/openwhisk/core/controller/CorsSettings.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udHJvbGxlci9Db3JzU2V0dGluZ3Muc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...che/openwhisk/core/entitlement/RateThrottler.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXRsZW1lbnQvUmF0ZVRocm90dGxlci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...he/openwhisk/core/entitlement/KindRestrictor.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXRsZW1lbnQvS2luZFJlc3RyaWN0b3Iuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...penwhisk/core/database/cosmosdb/CosmosDBUtil.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJVdGlsLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | ... and [149 more](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=footer). Last update [4a13303...4d9c7c6](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [openwhisk] codecov-io commented on pull request #5070: [New Scheduler] Add ActivationService

Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #5070:
URL: https://github.com/apache/openwhisk/pull/5070#issuecomment-790453439


   # [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=h1) Report
   > Merging [#5070](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=desc) (83f8313) into [master](https://codecov.io/gh/apache/openwhisk/commit/4a13303fae4d9750da6662bb39b3ec92d6ccf56c?el=desc) (4a13303) will **decrease** coverage by `50.95%`.
   > The diff coverage is `75.82%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/openwhisk/pull/5070/graphs/tree.svg?width=650&height=150&src=pr&token=l0YmsiSAso)](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #5070       +/-   ##
   ===========================================
   - Coverage   81.95%   30.99%   -50.96%     
   ===========================================
     Files         210      206        -4     
     Lines       10167    10041      -126     
     Branches      440      456       +16     
   ===========================================
   - Hits         8332     3112     -5220     
   - Misses       1835     6929     +5094     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [.../openwhisk/core/scheduler/queue/QueueManager.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvUXVldWVNYW5hZ2VyLnNjYWxh) | `56.75% <56.75%> (ø)` | |
   | [...sk/core/scheduler/grpc/ActivationServiceImpl.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvZ3JwYy9BY3RpdmF0aW9uU2VydmljZUltcGwuc2NhbGE=) | `88.00% <88.00%> (ø)` | |
   | [...ala/org/apache/openwhisk/core/entity/DocInfo.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L0RvY0luZm8uc2NhbGE=) | `78.78% <100.00%> (-8.31%)` | :arrow_down: |
   | [...enwhisk/core/entity/FullyQualifiedEntityName.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L0Z1bGx5UXVhbGlmaWVkRW50aXR5TmFtZS5zY2FsYQ==) | `45.94% <100.00%> (-45.49%)` | :arrow_down: |
   | [...n/scala/org/apache/openwhisk/utils/JsHelpers.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL3V0aWxzL0pzSGVscGVycy5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...a/org/apache/openwhisk/common/ConfigMapValue.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNYXBWYWx1ZS5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...g/apache/openwhisk/common/ResizableSemaphore.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9SZXNpemFibGVTZW1hcGhvcmUuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...pache/openwhisk/http/LenientSprayJsonSupport.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvTGVuaWVudFNwcmF5SnNvblN1cHBvcnQuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [.../apache/openwhisk/core/controller/Namespaces.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udHJvbGxlci9OYW1lc3BhY2VzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...pache/openwhisk/core/controller/CorsSettings.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udHJvbGxlci9Db3JzU2V0dGluZ3Muc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | ... and [159 more](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=footer). Last update [4a13303...83f8313](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [openwhisk] codecov-io edited a comment on pull request #5070: [New Scheduler] Add ActivationService

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #5070:
URL: https://github.com/apache/openwhisk/pull/5070#issuecomment-790453439


   # [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=h1) Report
   > Merging [#5070](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=desc) (2a4c69e) into [master](https://codecov.io/gh/apache/openwhisk/commit/4a13303fae4d9750da6662bb39b3ec92d6ccf56c?el=desc) (4a13303) will **decrease** coverage by `43.52%`.
   > The diff coverage is `75.82%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/openwhisk/pull/5070/graphs/tree.svg?width=650&height=150&src=pr&token=l0YmsiSAso)](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #5070       +/-   ##
   ===========================================
   - Coverage   81.95%   38.42%   -43.53%     
   ===========================================
     Files         210      218        +8     
     Lines       10167    10700      +533     
     Branches      440      450       +10     
   ===========================================
   - Hits         8332     4112     -4220     
   - Misses       1835     6588     +4753     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [.../openwhisk/core/scheduler/queue/QueueManager.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvUXVldWVNYW5hZ2VyLnNjYWxh) | `56.75% <56.75%> (ø)` | |
   | [...sk/core/scheduler/grpc/ActivationServiceImpl.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvZ3JwYy9BY3RpdmF0aW9uU2VydmljZUltcGwuc2NhbGE=) | `88.00% <88.00%> (ø)` | |
   | [...ala/org/apache/openwhisk/core/entity/DocInfo.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L0RvY0luZm8uc2NhbGE=) | `81.81% <100.00%> (-5.28%)` | :arrow_down: |
   | [...enwhisk/core/entity/FullyQualifiedEntityName.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L0Z1bGx5UXVhbGlmaWVkRW50aXR5TmFtZS5zY2FsYQ==) | `72.97% <100.00%> (-18.46%)` | :arrow_down: |
   | [...a/org/apache/openwhisk/common/ConfigMapValue.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNYXBWYWx1ZS5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [.../apache/openwhisk/core/controller/Namespaces.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udHJvbGxlci9OYW1lc3BhY2VzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...pache/openwhisk/core/controller/CorsSettings.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udHJvbGxlci9Db3JzU2V0dGluZ3Muc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...che/openwhisk/core/entitlement/RateThrottler.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXRsZW1lbnQvUmF0ZVRocm90dGxlci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...he/openwhisk/core/entitlement/KindRestrictor.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29yZS9jb250cm9sbGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXRsZW1lbnQvS2luZFJlc3RyaWN0b3Iuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...penwhisk/core/database/cosmosdb/CosmosDBUtil.scala](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJVdGlsLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | ... and [149 more](https://codecov.io/gh/apache/openwhisk/pull/5070/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=footer). Last update [4a13303...2a4c69e](https://codecov.io/gh/apache/openwhisk/pull/5070?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [openwhisk] upgle commented on a change in pull request #5070: [New Scheduler] Add ActivationService

Posted by GitBox <gi...@apache.org>.
upgle commented on a change in pull request #5070:
URL: https://github.com/apache/openwhisk/pull/5070#discussion_r587365317



##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/QueueManager.scala
##########
@@ -0,0 +1,102 @@
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorRef
+import org.apache.openwhisk.core.connector._
+import org.apache.openwhisk.core.entity._
+import spray.json.{DefaultJsonProtocol, _}
+import scala.collection.concurrent.TrieMap
+import scala.util.Try
+
+object QueueSize
+case class MemoryQueueKey(invocationNamespace: String, docInfo: DocInfo)
+case class MemoryQueueValue(queue: ActorRef, isLeader: Boolean)
+
+sealed trait MemoryQueueError extends Product {
+  val causedBy: String
+}
+
+object MemoryQueueErrorSerdes {
+
+  private implicit val noMessageSerdes = NoActivationMessage.serdes
+  private implicit val noQueueSerdes = NoMemoryQueue.serdes
+  private implicit val mismatchSerdes = ActionMismatch.serdes
+
+  // format that discriminates based on an additional
+  // field "type" that can either be "Cat" or "Dog"
+  implicit val memoryQueueErrorFormat = new RootJsonFormat[MemoryQueueError] {
+    def write(obj: MemoryQueueError): JsValue =
+      JsObject((obj match {
+        case msg: NoActivationMessage => msg.toJson
+        case msg: NoMemoryQueue       => msg.toJson
+        case msg: ActionMismatch      => msg.toJson
+      }).asJsObject.fields + ("type" -> JsString(obj.productPrefix)))
+
+    def read(json: JsValue): MemoryQueueError =
+      json.asJsObject.getFields("type") match {
+        case Seq(JsString("NoActivationMessage")) => json.convertTo[NoActivationMessage]
+        case Seq(JsString("NoMemoryQueue"))       => json.convertTo[NoMemoryQueue]
+        case Seq(JsString("ActionMismatch"))      => json.convertTo[ActionMismatch]
+      }
+  }
+}
+
+case class NoActivationMessage(noActivationMessage: String = NoActivationMessage.asString)
+    extends MemoryQueueError
+    with Message {
+  override val causedBy: String = noActivationMessage
+  override def serialize = NoActivationMessage.serdes.write(this).compactPrint
+}
+
+object NoActivationMessage extends DefaultJsonProtocol {
+  val asString: String = "no activation message exist"
+  def parse(msg: String) = Try(serdes.read(msg.parseJson))
+  implicit val serdes = jsonFormat(NoActivationMessage.apply _, "noActivationMessage")
+}
+
+case class NoMemoryQueue(noMemoryQueue: String = NoMemoryQueue.asString) extends MemoryQueueError with Message {
+  override val causedBy: String = noMemoryQueue
+  override def serialize = NoMemoryQueue.serdes.write(this).compactPrint
+}
+
+object NoMemoryQueue extends DefaultJsonProtocol {
+  val asString: String = "no memory queue exist"
+  def parse(msg: String) = Try(serdes.read(msg.parseJson))
+  implicit val serdes = jsonFormat(NoMemoryQueue.apply _, "noMemoryQueue")
+}
+
+case class ActionMismatch(actionMisMatch: String = ActionMismatch.asString) extends MemoryQueueError with Message {
+  override val causedBy: String = actionMisMatch
+  override def serialize = ActionMismatch.serdes.write(this).compactPrint
+}
+
+object ActionMismatch extends DefaultJsonProtocol {
+  val asString: String = "action version does not match"
+  def parse(msg: String) = Try(serdes.read(msg.parseJson))
+  implicit val serdes = jsonFormat(ActionMismatch.apply _, "actionMisMatch")
+}
+
+object QueuePool {

Review comment:
       The only field _queuePool(TrieMap[MemoryQueueKey, MemoryQueueValue]) which is included in this object is a thread-safe.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [openwhisk] ningyougang commented on a change in pull request #5070: [New Scheduler] Add ActivationService

Posted by GitBox <gi...@apache.org>.
ningyougang commented on a change in pull request #5070:
URL: https://github.com/apache/openwhisk/pull/5070#discussion_r583408560



##########
File path: core/scheduler/src/main/protobuf/activation.proto
##########
@@ -0,0 +1,52 @@
+syntax = "proto3";
+import "google/protobuf/wrappers.proto";
+
+//#options
+option java_multiple_files = true;
+option java_package = "org.apache.openwhisk.grpc";
+option java_outer_classname = "ActivationProto";
+
+package activation;
+//#options
+
+//#services
+service ActivationService {
+
+    rpc FetchActivation (FetchRequest) returns (FetchResponse) {}
+
+    rpc RescheduleActivation (RescheduleRequest) returns (RescheduleResponse) {}

Review comment:
       For the design document, `RescheduleActivation rpc is for rescheduling when the container proxy cannot process messages`, can give some example here? i think just describe it is enough.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [openwhisk] upgle commented on a change in pull request #5070: [New Scheduler] Add ActivationService

Posted by GitBox <gi...@apache.org>.
upgle commented on a change in pull request #5070:
URL: https://github.com/apache/openwhisk/pull/5070#discussion_r583404821



##########
File path: core/scheduler/src/main/java/Empty.java
##########
@@ -0,0 +1,5 @@
+public class Empty {
+    // Workaround for this issue https://github.com/akka/akka-grpc/issues/289
+    // Gradle complains about no java sources.
+    // Note. Openwhisk is using a lower gradle version, so the latest akka-grpc version cannot be used.

Review comment:
       I've already tried it, but It requires a higher Gradle version to use the latest akka-grpc.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org