You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@openwhisk.apache.org by GitBox <gi...@apache.org> on 2020/10/08 04:45:36 UTC

[GitHub] [openwhisk] style95 opened a new pull request #4984: [New Scheduler] Add duration checker

style95 opened a new pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984


   <!--- Provide a concise summary of your changes in the Title -->
   
   ## Description
   This is a subsequent PR of #4983, once #4983 is merged, I would rebase this again.
   
   Major changes are:
   * [ElasticSearchDurationChecker.scala](https://github.com/apache/openwhisk/pull/4984/files#diff-6e93666ed1317e3048297733a0cea522)
   * [NoopDurationChecker.scala](https://github.com/apache/openwhisk/pull/4984/files#diff-814fc54df6fb0f34304af446cdaccab8)
   * [tests/build.gradle](https://github.com/apache/openwhisk/pull/4984/files#diff-f38475a309b117418ec21c944e4d60ae)
   * [ElasticSearchDurationCheckResultFormatTest.scala](https://github.com/apache/openwhisk/pull/4984/files#diff-d7f19a3180155e71cc196f0557a0405b)
   * [ElasticSearchDurationCheckerTests.scala](https://github.com/apache/openwhisk/pull/4984/files#diff-4b71c45765346587308385e8e8227405)
   
   
   This is to add a duration checker for ElasticSearch.
   With a new scheduler, it is important to decide when and how many containers to add.
   The scheduler will calculate the average duration for the recent N activations and compute the processing power of one container, e.g. how many activations can be handled by one container in a given time. Factoring in the average duration, the number of incoming activations, and the number of activations in a queue, the scheduler can add more containers to handle the given activations.
   
   
   
   ## Related issue and scope
   <!--- Please include a link to a related issue if there is one. -->
   - [x] I opened an issue to propose and discuss this change (#4922)
   
   ## My changes affect the following components
   <!--- Select below all system components are affected by your change. -->
   <!--- Enter an `x` in all applicable boxes. -->
   - [ ] API
   - [ ] Controller
   - [ ] Message Bus (e.g., Kafka)
   - [ ] Loadbalancer
   - [ ] Invoker
   - [ ] Intrinsic actions (e.g., sequences, conductors)
   - [ ] Data stores (e.g., CouchDB)
   - [ ] Tests
   - [ ] Deployment
   - [ ] CLI
   - [ ] General tooling
   - [ ] Documentation
   
   ## Types of changes
   <!--- What types of changes does your code introduce? Use `x` in all the boxes that apply: -->
   - [ ] Bug fix (generally a non-breaking change which closes an issue).
   - [x] Enhancement or new feature (adds new functionality).
   - [ ] Breaking change (a bug fix or enhancement which changes existing behavior).
   
   ## Checklist:
   <!--- Please review the points below which help you make sure you've covered all aspects of the change you're making. -->
   
   - [x] I signed an [Apache CLA](https://github.com/apache/openwhisk/blob/master/CONTRIBUTING.md).
   - [x] I reviewed the [style guides](https://github.com/apache/openwhisk/wiki/Contributing:-Git-guidelines#code-readiness) and followed the recommendations (Travis CI will check :).
   - [x] I added tests to cover my changes.
   - [ ] My changes require further changes to the documentation.
   - [ ] I updated the documentation where necessary.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] style95 commented on pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
style95 commented on pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#issuecomment-762061992


   Finally, it has passed all tests cases :)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] bdoyle0182 commented on a change in pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
bdoyle0182 commented on a change in pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#discussion_r494696173



##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/NoopDurationChecker.scala
##########
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorSystem
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.entity.WhiskActionMetaData
+
+import scala.concurrent.Future
+
+object NoopDurationCheckerProvider extends DurationCheckerProvider {

Review comment:
       so this means you operate the schedule without using the average activation duration for an action heuristic. How important is using the heuristic for the performance of the scheduler?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] style95 commented on pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
style95 commented on pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#issuecomment-731589843


   It seems sometimes the build is failed even if it passed all tests because it failed to upload the file.
   
   ```
   All checks passed.
   Compressing logs dir...
   tar: Removing leading `/' from member names
   .
   .
   .
   Uploading to Box...
   Posting result <Response [500]>
   {
     "error": "File upload failed."
   }
   ```
   
   Is this to upload the log files?
   If yes, where can I refer to?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] codecov-io edited a comment on pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#issuecomment-730427203


   # [Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=h1) Report
   > Merging [#4984](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=desc) (307527e) into [master](https://codecov.io/gh/apache/openwhisk/commit/2d0c8a72711cf20da4aedb8ada68d62774c0eca9?el=desc) (2d0c8a7) will **increase** coverage by `46.80%`.
   > The diff coverage is `77.27%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/openwhisk/pull/4984/graphs/tree.svg?width=650&height=150&src=pr&token=l0YmsiSAso)](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #4984       +/-   ##
   ===========================================
   + Coverage   29.09%   75.89%   +46.80%     
   ===========================================
     Files         195      206       +11     
     Lines        9553    10122      +569     
     Branches      413      450       +37     
   ===========================================
   + Hits         2779     7682     +4903     
   + Misses       6774     2440     -4334     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...isk/core/scheduler/queue/NoopDurationChecker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvTm9vcER1cmF0aW9uQ2hlY2tlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
   | [...scheduler/queue/ElasticSearchDurationChecker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvRWxhc3RpY1NlYXJjaER1cmF0aW9uQ2hlY2tlci5zY2FsYQ==) | `78.84% <78.84%> (ø)` | |
   | [.../scala/org/apache/openwhisk/core/WhiskConfig.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvV2hpc2tDb25maWcuc2NhbGE=) | `95.30% <100.00%> (+8.13%)` | :arrow_up: |
   | [...e/elasticsearch/ElasticSearchActivationStore.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvZWxhc3RpY3NlYXJjaC9FbGFzdGljU2VhcmNoQWN0aXZhdGlvblN0b3JlLnNjYWxh) | `84.66% <100.00%> (+84.66%)` | :arrow_up: |
   | [...e/openwhisk/core/containerpool/ContainerPool.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvY29udGFpbmVycG9vbC9Db250YWluZXJQb29sLnNjYWxh) | `89.51% <0.00%> (-2.02%)` | :arrow_down: |
   | [.../core/monitoring/metrics/PrometheusEventsApi.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9tb25pdG9yaW5nL3VzZXItZXZlbnRzL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvbW9uaXRvcmluZy9tZXRyaWNzL1Byb21ldGhldXNFdmVudHNBcGkuc2NhbGE=) | `90.90% <0.00%> (ø)` | |
   | [...nwhisk/core/monitoring/metrics/KamonRecorder.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9tb25pdG9yaW5nL3VzZXItZXZlbnRzL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvbW9uaXRvcmluZy9tZXRyaWNzL0thbW9uUmVjb3JkZXIuc2NhbGE=) | `82.45% <0.00%> (ø)` | |
   | [...nwhisk/core/monitoring/metrics/EventConsumer.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9tb25pdG9yaW5nL3VzZXItZXZlbnRzL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvbW9uaXRvcmluZy9tZXRyaWNzL0V2ZW50Q29uc3VtZXIuc2NhbGE=) | `89.23% <0.00%> (ø)` | |
   | [...rg/apache/openwhisk/core/scheduler/Scheduler.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvU2NoZWR1bGVyLnNjYWxh) | `0.00% <0.00%> (ø)` | |
   | [...penwhisk/core/monitoring/metrics/MetricNames.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9tb25pdG9yaW5nL3VzZXItZXZlbnRzL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvbW9uaXRvcmluZy9tZXRyaWNzL01ldHJpY05hbWVzLnNjYWxh) | `100.00% <0.00%> (ø)` | |
   | ... and [150 more](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=footer). Last update [2d0c8a7...307527e](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] style95 commented on a change in pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
style95 commented on a change in pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#discussion_r496058658



##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/ElasticSearchDurationChecker.scala
##########
@@ -0,0 +1,252 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorSystem
+import com.sksamuel.elastic4s.http.ElasticDsl._
+import com.sksamuel.elastic4s.http.{ElasticClient, ElasticProperties, NoOpRequestConfigCallback}
+import com.sksamuel.elastic4s.{ElasticDate, ElasticDateMath, Seconds}
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.ConfigKeys
+import org.apache.openwhisk.core.entity.WhiskActionMetaData
+import org.apache.openwhisk.spi.Spi
+import pureconfig.loadConfigOrThrow
+import spray.json.{JsArray, JsNumber, JsValue, RootJsonFormat, deserializationError, _}
+
+import scala.concurrent.Future
+import scala.concurrent.duration.FiniteDuration
+import scala.language.implicitConversions
+import scala.util.{Failure, Try}
+
+trait DurationChecker {
+  def checkAverageDuration(invocationNamespace: String, actionMetaData: WhiskActionMetaData)(
+    callback: DurationCheckResult => DurationCheckResult): Future[DurationCheckResult]
+}
+
+case class DurationCheckResult(averageDuration: Option[Double], hitCount: Long, took: Long)
+
+object ElasticSearchDurationChecker {
+  val FilterAggregationName = "filterAggregation"
+  val AverageAggregationName = "averageAggregation"
+
+  implicit val serde = new ElasticSearchDurationCheckResultFormat()
+
+  def getFromDate(timeWindow: FiniteDuration): ElasticDateMath =
+    ElasticDate.now minus (timeWindow.toSeconds.toInt, Seconds)
+}
+
+class ElasticSearchDurationChecker(private val client: ElasticClient, val timeWindow: FiniteDuration)(
+  implicit val actorSystem: ActorSystem,
+  implicit val logging: Logging)
+    extends DurationChecker {
+  import ElasticSearchDurationChecker._
+  import org.apache.openwhisk.core.database.elasticsearch.ElasticSearchActivationStore.generateIndex
+
+  implicit val ec = actorSystem.getDispatcher
+
+  override def checkAverageDuration(invocationNamespace: String, actionMetaData: WhiskActionMetaData)(
+    callback: DurationCheckResult => DurationCheckResult): Future[DurationCheckResult] = {
+    val index = generateIndex(invocationNamespace)
+    val fqn = actionMetaData.fullyQualifiedName(false)
+    val fromDate = getFromDate(timeWindow)
+
+    logging.info(this, s"check average duration for $fqn in $index for last $timeWindow")
+
+    actionMetaData.binding match {
+      case Some(binding) =>
+        client

Review comment:
       updated accordingly




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] style95 commented on a change in pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
style95 commented on a change in pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#discussion_r494717183



##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/NoopDurationChecker.scala
##########
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorSystem
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.entity.WhiskActionMetaData
+
+import scala.concurrent.Future
+
+object NoopDurationCheckerProvider extends DurationCheckerProvider {

Review comment:
       This is just for other DBs such as CouchDB or CosmosDB in case the scheduler is used with other than ES.
   
   Regarding the average duration, it is important to improve the accuracy of calculation but the queue can still work without it. When an action is newly created, there is no activation accordingly no average duration.
   In such a case, it assumes one container can handle one activation for the given time.
   So even if one container can handle multiple activations for a given period, it assumes a container can handle only one activation, so schedulers would tend to overprovision containers.
   
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] bdoyle0182 commented on a change in pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
bdoyle0182 commented on a change in pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#discussion_r496307275



##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/NoopDurationChecker.scala
##########
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorSystem
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.entity.WhiskActionMetaData
+
+import scala.concurrent.Future
+
+object NoopDurationCheckerProvider extends DurationCheckerProvider {

Review comment:
       ah okay cool so if I understand what you just said correctly once the invoker is running, we still will get to use the average activation duration heuristic since we track it in memory. The elasticsearch spi is just for startup. That's good to know the optimization you described sounds more important we'll still get the benefits of tracking activation duration




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] codecov-io edited a comment on pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#issuecomment-730427203


   # [Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=h1) Report
   > Merging [#4984](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=desc) (307527e) into [master](https://codecov.io/gh/apache/openwhisk/commit/2d0c8a72711cf20da4aedb8ada68d62774c0eca9?el=desc) (2d0c8a7) will **increase** coverage by `2.03%`.
   > The diff coverage is `15.15%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/openwhisk/pull/4984/graphs/tree.svg?width=650&height=150&src=pr&token=l0YmsiSAso)](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #4984      +/-   ##
   ==========================================
   + Coverage   29.09%   31.12%   +2.03%     
   ==========================================
     Files         195      206      +11     
     Lines        9553     9998     +445     
     Branches      413      438      +25     
   ==========================================
   + Hits         2779     3112     +333     
   - Misses       6774     6886     +112     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...scheduler/queue/ElasticSearchDurationChecker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvRWxhc3RpY1NlYXJjaER1cmF0aW9uQ2hlY2tlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
   | [...isk/core/scheduler/queue/NoopDurationChecker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvTm9vcER1cmF0aW9uQ2hlY2tlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
   | [.../scala/org/apache/openwhisk/core/WhiskConfig.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvV2hpc2tDb25maWcuc2NhbGE=) | `87.24% <100.00%> (+0.08%)` | :arrow_up: |
   | [...e/elasticsearch/ElasticSearchActivationStore.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvZWxhc3RpY3NlYXJjaC9FbGFzdGljU2VhcmNoQWN0aXZhdGlvblN0b3JlLnNjYWxh) | `50.92% <100.00%> (+50.92%)` | :arrow_up: |
   | [...nwhisk/core/database/ArtifactActivationStore.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvQXJ0aWZhY3RBY3RpdmF0aW9uU3RvcmUuc2NhbGE=) | `0.00% <0.00%> (-57.15%)` | :arrow_down: |
   | [...che/openwhisk/core/database/CouchDbRestStore.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvQ291Y2hEYlJlc3RTdG9yZS5zY2FsYQ==) | `44.94% <0.00%> (-5.56%)` | :arrow_down: |
   | [...he/openwhisk/core/database/CouchDbRestClient.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvQ291Y2hEYlJlc3RDbGllbnQuc2NhbGE=) | `68.75% <0.00%> (-4.17%)` | :arrow_down: |
   | [...ala/org/apache/openwhisk/core/entity/DocInfo.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L0RvY0luZm8uc2NhbGE=) | `70.96% <0.00%> (-3.23%)` | :arrow_down: |
   | [...org/apache/openwhisk/core/entity/WhiskEntity.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L1doaXNrRW50aXR5LnNjYWxh) | `49.23% <0.00%> (-3.08%)` | :arrow_down: |
   | [.../org/apache/openwhisk/core/entity/WhiskStore.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZW50aXR5L1doaXNrU3RvcmUuc2NhbGE=) | `48.48% <0.00%> (-3.04%)` | :arrow_down: |
   | ... and [21 more](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=footer). Last update [2d0c8a7...307527e](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] codecov-io edited a comment on pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#issuecomment-730427203


   # [Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=h1) Report
   > Merging [#4984](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=desc) (307527e) into [master](https://codecov.io/gh/apache/openwhisk/commit/a2025382fa4dbd8ce448b037b14a54d818a224ca?el=desc) (a202538) will **decrease** coverage by `77.67%`.
   > The diff coverage is `1.51%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/openwhisk/pull/4984/graphs/tree.svg?width=650&height=150&src=pr&token=l0YmsiSAso)](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #4984       +/-   ##
   ==========================================
   - Coverage   83.68%   6.01%   -77.68%     
   ==========================================
     Files         202     206        +4     
     Lines        9818    9998      +180     
     Branches      416     438       +22     
   ==========================================
   - Hits         8216     601     -7615     
   - Misses       1602    9397     +7795     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/elasticsearch/ElasticSearchActivationStore.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvZWxhc3RpY3NlYXJjaC9FbGFzdGljU2VhcmNoQWN0aXZhdGlvblN0b3JlLnNjYWxh) | `0.00% <0.00%> (-83.96%)` | :arrow_down: |
   | [...scheduler/queue/ElasticSearchDurationChecker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvRWxhc3RpY1NlYXJjaER1cmF0aW9uQ2hlY2tlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
   | [...isk/core/scheduler/queue/NoopDurationChecker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvTm9vcER1cmF0aW9uQ2hlY2tlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
   | [.../scala/org/apache/openwhisk/core/WhiskConfig.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvV2hpc2tDb25maWcuc2NhbGE=) | `63.08% <100.00%> (-32.19%)` | :arrow_down: |
   | [...ain/scala/org/apache/openwhisk/spi/SpiLoader.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL3NwaS9TcGlMb2FkZXIuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...n/scala/org/apache/openwhisk/utils/JsHelpers.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL3V0aWxzL0pzSGVscGVycy5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...scala/org/apache/openwhisk/common/Prometheus.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Qcm9tZXRoZXVzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...scala/org/apache/openwhisk/core/FeatureFlags.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvRmVhdHVyZUZsYWdzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...ala/org/apache/openwhisk/common/ConfigMXBean.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvbW1vbi9Db25maWdNWEJlYW4uc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...la/org/apache/openwhisk/http/BasicRasService.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNSYXNTZXJ2aWNlLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | ... and [176 more](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=footer). Last update [2d0c8a7...307527e](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] style95 closed pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
style95 closed pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] style95 merged pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
style95 merged pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] codecov-io commented on pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#issuecomment-730427203


   # [Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=h1) Report
   > Merging [#4984](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=desc) (8e698ae) into [master](https://codecov.io/gh/apache/openwhisk/commit/12ca4e307e3b095a266352b7b12cf5ddaeb44577?el=desc) (12ca4e3) will **decrease** coverage by `7.77%`.
   > The diff coverage is `77.27%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/openwhisk/pull/4984/graphs/tree.svg?width=650&height=150&src=pr&token=l0YmsiSAso)](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #4984      +/-   ##
   ==========================================
   - Coverage   83.69%   75.92%   -7.78%     
   ==========================================
     Files         202      206       +4     
     Lines        9802     9997     +195     
     Branches      424      437      +13     
   ==========================================
   - Hits         8204     7590     -614     
   - Misses       1598     2407     +809     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...isk/core/scheduler/queue/NoopDurationChecker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvTm9vcER1cmF0aW9uQ2hlY2tlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
   | [...scheduler/queue/ElasticSearchDurationChecker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvRWxhc3RpY1NlYXJjaER1cmF0aW9uQ2hlY2tlci5zY2FsYQ==) | `78.84% <78.84%> (ø)` | |
   | [.../scala/org/apache/openwhisk/core/WhiskConfig.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvV2hpc2tDb25maWcuc2NhbGE=) | `95.30% <100.00%> (+0.03%)` | :arrow_up: |
   | [...e/elasticsearch/ElasticSearchActivationStore.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvZWxhc3RpY3NlYXJjaC9FbGFzdGljU2VhcmNoQWN0aXZhdGlvblN0b3JlLnNjYWxh) | `84.66% <100.00%> (+0.71%)` | :arrow_up: |
   | [...core/database/cosmosdb/RxObservableImplicits.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvUnhPYnNlcnZhYmxlSW1wbGljaXRzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...ore/database/cosmosdb/cache/CacheInvalidator.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NhY2hlSW52YWxpZGF0b3Iuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...e/database/cosmosdb/cache/ChangeFeedConsumer.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NoYW5nZUZlZWRDb25zdW1lci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...core/database/cosmosdb/CosmosDBArtifactStore.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJBcnRpZmFjdFN0b3JlLnNjYWxh) | `0.00% <0.00%> (-95.85%)` | :arrow_down: |
   | [...sk/core/database/cosmosdb/CosmosDBViewMapper.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvY29zbW9zZGIvQ29zbW9zREJWaWV3TWFwcGVyLnNjYWxh) | `0.00% <0.00%> (-93.90%)` | :arrow_down: |
   | [...tabase/cosmosdb/cache/CacheInvalidatorConfig.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9jb3Ntb3NkYi9jYWNoZS1pbnZhbGlkYXRvci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL29wZW53aGlzay9jb3JlL2RhdGFiYXNlL2Nvc21vc2RiL2NhY2hlL0NhY2hlSW52YWxpZGF0b3JDb25maWcuc2NhbGE=) | `0.00% <0.00%> (-92.31%)` | :arrow_down: |
   | ... and [19 more](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=footer). Last update [12ca4e3...8e698ae](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] codecov-io edited a comment on pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#issuecomment-730427203


   # [Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=h1) Report
   > Merging [#4984](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=desc) (307527e) into [master](https://codecov.io/gh/apache/openwhisk/commit/2d0c8a72711cf20da4aedb8ada68d62774c0eca9?el=desc) (2d0c8a7) will **increase** coverage by `45.82%`.
   > The diff coverage is `15.15%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/openwhisk/pull/4984/graphs/tree.svg?width=650&height=150&src=pr&token=l0YmsiSAso)](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #4984       +/-   ##
   ===========================================
   + Coverage   29.09%   74.91%   +45.82%     
   ===========================================
     Files         195      206       +11     
     Lines        9553     9998      +445     
     Branches      413      438       +25     
   ===========================================
   + Hits         2779     7490     +4711     
   + Misses       6774     2508     -4266     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...scheduler/queue/ElasticSearchDurationChecker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvRWxhc3RpY1NlYXJjaER1cmF0aW9uQ2hlY2tlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
   | [...isk/core/scheduler/queue/NoopDurationChecker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvTm9vcER1cmF0aW9uQ2hlY2tlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
   | [.../scala/org/apache/openwhisk/core/WhiskConfig.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvV2hpc2tDb25maWcuc2NhbGE=) | `95.30% <100.00%> (+8.13%)` | :arrow_up: |
   | [...e/elasticsearch/ElasticSearchActivationStore.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvZWxhc3RpY3NlYXJjaC9FbGFzdGljU2VhcmNoQWN0aXZhdGlvblN0b3JlLnNjYWxh) | `84.66% <100.00%> (+84.66%)` | :arrow_up: |
   | [...hisk/core/monitoring/metrics/OpenWhiskEvents.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9tb25pdG9yaW5nL3VzZXItZXZlbnRzL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvbW9uaXRvcmluZy9tZXRyaWNzL09wZW5XaGlza0V2ZW50cy5zY2FsYQ==) | `93.75% <0.00%> (ø)` | |
   | [...penwhisk/core/monitoring/metrics/MetricNames.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9tb25pdG9yaW5nL3VzZXItZXZlbnRzL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvbW9uaXRvcmluZy9tZXRyaWNzL01ldHJpY05hbWVzLnNjYWxh) | `100.00% <0.00%> (ø)` | |
   | [...pache/openwhisk/core/monitoring/metrics/Main.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9tb25pdG9yaW5nL3VzZXItZXZlbnRzL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvbW9uaXRvcmluZy9tZXRyaWNzL01haW4uc2NhbGE=) | `0.00% <0.00%> (ø)` | |
   | [...che/openwhisk/core/scheduler/SchedulerServer.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvU2NoZWR1bGVyU2VydmVyLnNjYWxh) | `0.00% <0.00%> (ø)` | |
   | [...nwhisk/core/monitoring/metrics/KamonRecorder.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9tb25pdG9yaW5nL3VzZXItZXZlbnRzL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvbW9uaXRvcmluZy9tZXRyaWNzL0thbW9uUmVjb3JkZXIuc2NhbGE=) | `82.45% <0.00%> (ø)` | |
   | ... and [146 more](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=footer). Last update [2d0c8a7...307527e](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] style95 commented on a change in pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
style95 commented on a change in pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#discussion_r494674214



##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/ElasticSearchDurationChecker.scala
##########
@@ -0,0 +1,252 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorSystem
+import com.sksamuel.elastic4s.http.ElasticDsl._
+import com.sksamuel.elastic4s.http.{ElasticClient, ElasticProperties, NoOpRequestConfigCallback}
+import com.sksamuel.elastic4s.{ElasticDate, ElasticDateMath, Seconds}
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.ConfigKeys
+import org.apache.openwhisk.core.entity.WhiskActionMetaData
+import org.apache.openwhisk.spi.Spi
+import pureconfig.loadConfigOrThrow
+import spray.json.{JsArray, JsNumber, JsValue, RootJsonFormat, deserializationError, _}
+
+import scala.concurrent.Future
+import scala.concurrent.duration.FiniteDuration
+import scala.language.implicitConversions
+import scala.util.{Failure, Try}
+
+trait DurationChecker {
+  def checkAverageDuration(invocationNamespace: String, actionMetaData: WhiskActionMetaData)(
+    callback: DurationCheckResult => DurationCheckResult): Future[DurationCheckResult]
+}
+
+case class DurationCheckResult(averageDuration: Option[Double], hitCount: Long, took: Long)
+
+object ElasticSearchDurationChecker {
+  val FilterAggregationName = "filterAggregation"
+  val AverageAggregationName = "averageAggregation"
+
+  implicit val serde = new ElasticSearchDurationCheckResultFormat()
+
+  def getFromDate(timeWindow: FiniteDuration): ElasticDateMath =
+    ElasticDate.now minus (timeWindow.toSeconds.toInt, Seconds)
+}
+
+class ElasticSearchDurationChecker(private val client: ElasticClient, val timeWindow: FiniteDuration)(
+  implicit val actorSystem: ActorSystem,
+  implicit val logging: Logging)
+    extends DurationChecker {
+  import ElasticSearchDurationChecker._
+  import org.apache.openwhisk.core.database.elasticsearch.ElasticSearchActivationStore.generateIndex
+
+  implicit val ec = actorSystem.getDispatcher
+
+  override def checkAverageDuration(invocationNamespace: String, actionMetaData: WhiskActionMetaData)(
+    callback: DurationCheckResult => DurationCheckResult): Future[DurationCheckResult] = {
+    val index = generateIndex(invocationNamespace)
+    val fqn = actionMetaData.fullyQualifiedName(false)
+    val fromDate = getFromDate(timeWindow)
+
+    logging.info(this, s"check average duration for $fqn in $index for last $timeWindow")
+
+    actionMetaData.binding match {
+      case Some(binding) =>
+        client
+          .execute {
+            (search(index) query {
+              boolQuery must {
+                List(
+                  matchQuery("annotations.binding", s"$binding"),
+                  matchQuery("name", actionMetaData.name),
+                  rangeQuery("@timestamp").gte(fromDate))
+              }
+            } aggregations
+              avgAgg(AverageAggregationName, "duration")).size(0)
+          }
+          .map { res =>
+            logging.debug(this, s"ElasticSearch query results: $res")
+            Try(serde.read(res.body.getOrElse("").parseJson))
+          }
+          .flatMap(Future.fromTry)
+          .map(callback(_))
+          .andThen {
+            case Failure(t) =>
+              logging.error(this, s"failed to check the average duration: ${t}")
+          }
+
+      case None =>
+        client
+          .execute {
+            (search(index) query {
+              boolQuery must {
+                List(matchQuery("path.keyword", fqn.toString), rangeQuery("@timestamp").gte(fromDate))
+              }
+            } aggregations
+              avgAgg(AverageAggregationName, "duration")).size(0)
+          }
+          .map { res =>
+            logging.debug(this, s"ElasticSearch query results: $res")
+            Try(serde.read(res.body.getOrElse("").parseJson))
+          }
+          .flatMap(Future.fromTry)
+          .map(callback(_))
+          .andThen {
+            case Failure(t) =>
+              logging.error(this, s"failed to check the average duration: ${t}")
+          }
+    }
+  }
+}
+
+object ElasticSearchDurationCheckerProvider extends DurationCheckerProvider {
+  import org.apache.openwhisk.core.database.elasticsearch.ElasticSearchActivationStore._
+
+  override def instance(actorSystem: ActorSystem, log: Logging): ElasticSearchDurationChecker = {
+    implicit val as: ActorSystem = actorSystem
+    implicit val logging: Logging = log
+
+    val elasticClient =
+      ElasticClient(
+        ElasticProperties(s"${elasticSearchConfig.protocol}://${elasticSearchConfig.hosts}"),
+        NoOpRequestConfigCallback,
+        httpClientCallback)
+
+    new ElasticSearchDurationChecker(elasticClient, durationCheckerConfig.timeWindow)
+  }
+}
+
+trait DurationCheckerProvider extends Spi {

Review comment:
       It is based on the SPI.
   

##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/NoopDurationChecker.scala
##########
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorSystem
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.entity.WhiskActionMetaData
+
+import scala.concurrent.Future
+
+object NoopDurationCheckerProvider extends DurationCheckerProvider {

Review comment:
       This is just for other DBs such as CouchDB or CosmosDB in case the scheduler is used with other than ES.
   
   Regarding the average duration, it is important to improve the accuracy of calculation but the queue can still work without it. When an action is newly created, there is no activation accordingly no average duration.
   In such a case, it assumes one container can handle one activation for the given time.
   So even if one container can handle multiple activations for a given period, it assumes a container can handle only one activation, so schedulers would tend to overprovision containers.
   
   

##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/NoopDurationChecker.scala
##########
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorSystem
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.entity.WhiskActionMetaData
+
+import scala.concurrent.Future
+
+object NoopDurationCheckerProvider extends DurationCheckerProvider {

Review comment:
       This is just for other DBs such as CouchDB or CosmosDB in case the scheduler is used with other than ES.
   (Even if it is highly recommended to use with ES.)
   
   Regarding the average duration, it is important to improve the accuracy of calculation but the queue can still work without it. When an action is newly created, there is no activation accordingly no average duration.
   In such a case, it assumes one container can handle one activation for the given time.
   So even if one container can handle multiple activations for a given period, it assumes a container can handle only one activation, so schedulers would tend to overprovision containers.
   
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] dgrove-oss commented on pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
dgrove-oss commented on pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#issuecomment-731842802


   We failed to upload the logs to Box.  This happens because we don't have a way to automatically remove old logs and the Box folder fills up. However, the upload to Box is optional (will not cause the travisci job to fail).  There is a real test failure earlier in the log:   https://travis-ci.org/github/apache/openwhisk/jobs/744844186#L7457


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] style95 commented on pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
style95 commented on pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#issuecomment-731733096


   It keeps failing.
   
   ```
   Uploading to Box...
   Posting result <Response [500]>
   {
     "error": "File upload failed."
   }
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] style95 commented on a change in pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
style95 commented on a change in pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#discussion_r494674214



##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/ElasticSearchDurationChecker.scala
##########
@@ -0,0 +1,252 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorSystem
+import com.sksamuel.elastic4s.http.ElasticDsl._
+import com.sksamuel.elastic4s.http.{ElasticClient, ElasticProperties, NoOpRequestConfigCallback}
+import com.sksamuel.elastic4s.{ElasticDate, ElasticDateMath, Seconds}
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.ConfigKeys
+import org.apache.openwhisk.core.entity.WhiskActionMetaData
+import org.apache.openwhisk.spi.Spi
+import pureconfig.loadConfigOrThrow
+import spray.json.{JsArray, JsNumber, JsValue, RootJsonFormat, deserializationError, _}
+
+import scala.concurrent.Future
+import scala.concurrent.duration.FiniteDuration
+import scala.language.implicitConversions
+import scala.util.{Failure, Try}
+
+trait DurationChecker {
+  def checkAverageDuration(invocationNamespace: String, actionMetaData: WhiskActionMetaData)(
+    callback: DurationCheckResult => DurationCheckResult): Future[DurationCheckResult]
+}
+
+case class DurationCheckResult(averageDuration: Option[Double], hitCount: Long, took: Long)
+
+object ElasticSearchDurationChecker {
+  val FilterAggregationName = "filterAggregation"
+  val AverageAggregationName = "averageAggregation"
+
+  implicit val serde = new ElasticSearchDurationCheckResultFormat()
+
+  def getFromDate(timeWindow: FiniteDuration): ElasticDateMath =
+    ElasticDate.now minus (timeWindow.toSeconds.toInt, Seconds)
+}
+
+class ElasticSearchDurationChecker(private val client: ElasticClient, val timeWindow: FiniteDuration)(
+  implicit val actorSystem: ActorSystem,
+  implicit val logging: Logging)
+    extends DurationChecker {
+  import ElasticSearchDurationChecker._
+  import org.apache.openwhisk.core.database.elasticsearch.ElasticSearchActivationStore.generateIndex
+
+  implicit val ec = actorSystem.getDispatcher
+
+  override def checkAverageDuration(invocationNamespace: String, actionMetaData: WhiskActionMetaData)(
+    callback: DurationCheckResult => DurationCheckResult): Future[DurationCheckResult] = {
+    val index = generateIndex(invocationNamespace)
+    val fqn = actionMetaData.fullyQualifiedName(false)
+    val fromDate = getFromDate(timeWindow)
+
+    logging.info(this, s"check average duration for $fqn in $index for last $timeWindow")
+
+    actionMetaData.binding match {
+      case Some(binding) =>
+        client
+          .execute {
+            (search(index) query {
+              boolQuery must {
+                List(
+                  matchQuery("annotations.binding", s"$binding"),
+                  matchQuery("name", actionMetaData.name),
+                  rangeQuery("@timestamp").gte(fromDate))
+              }
+            } aggregations
+              avgAgg(AverageAggregationName, "duration")).size(0)
+          }
+          .map { res =>
+            logging.debug(this, s"ElasticSearch query results: $res")
+            Try(serde.read(res.body.getOrElse("").parseJson))
+          }
+          .flatMap(Future.fromTry)
+          .map(callback(_))
+          .andThen {
+            case Failure(t) =>
+              logging.error(this, s"failed to check the average duration: ${t}")
+          }
+
+      case None =>
+        client
+          .execute {
+            (search(index) query {
+              boolQuery must {
+                List(matchQuery("path.keyword", fqn.toString), rangeQuery("@timestamp").gte(fromDate))
+              }
+            } aggregations
+              avgAgg(AverageAggregationName, "duration")).size(0)
+          }
+          .map { res =>
+            logging.debug(this, s"ElasticSearch query results: $res")
+            Try(serde.read(res.body.getOrElse("").parseJson))
+          }
+          .flatMap(Future.fromTry)
+          .map(callback(_))
+          .andThen {
+            case Failure(t) =>
+              logging.error(this, s"failed to check the average duration: ${t}")
+          }
+    }
+  }
+}
+
+object ElasticSearchDurationCheckerProvider extends DurationCheckerProvider {
+  import org.apache.openwhisk.core.database.elasticsearch.ElasticSearchActivationStore._
+
+  override def instance(actorSystem: ActorSystem, log: Logging): ElasticSearchDurationChecker = {
+    implicit val as: ActorSystem = actorSystem
+    implicit val logging: Logging = log
+
+    val elasticClient =
+      ElasticClient(
+        ElasticProperties(s"${elasticSearchConfig.protocol}://${elasticSearchConfig.hosts}"),
+        NoOpRequestConfigCallback,
+        httpClientCallback)
+
+    new ElasticSearchDurationChecker(elasticClient, durationCheckerConfig.timeWindow)
+  }
+}
+
+trait DurationCheckerProvider extends Spi {

Review comment:
       It is based on the SPI.
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] style95 commented on a change in pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
style95 commented on a change in pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#discussion_r496305598



##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/NoopDurationChecker.scala
##########
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorSystem
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.entity.WhiskActionMetaData
+
+import scala.concurrent.Future
+
+object NoopDurationCheckerProvider extends DurationCheckerProvider {

Review comment:
       Yes.
   If required, anyone can create it as it is based on SPI. 
   
   And one thing I forgot to tell you is, after this duration checker is landed, we introduced one more optimization.
   Initially, the average duration was always calculated based on this module, but now(in our downstream), this is only used when a queue is newly created. After then, the queue uses the duration passed from containers.
   As per POEM2, each container autonomously pulls an activation by sending a fetch-request. So when they send the fetch-request, we added one more field `lastDuration`. So the queue can keep the recent `N` duration in the circular queue and calculate the average duration without any external API call.
   
   But when a new queue is created or an action is newly created, there is no data in the circular queue and the duration checker is used in such cases.
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] bdoyle0182 commented on a change in pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
bdoyle0182 commented on a change in pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#discussion_r496259624



##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/NoopDurationChecker.scala
##########
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorSystem
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.entity.WhiskActionMetaData
+
+import scala.concurrent.Future
+
+object NoopDurationCheckerProvider extends DurationCheckerProvider {

Review comment:
       In the case of couchdb or cosmosdb, there is no average activation duration calculation since it uses this correct?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] style95 commented on pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
style95 commented on pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#issuecomment-731743432


   It seems this endpoint no longer accepts any file.
   
   ```
   $ curl -XPOST http://DamCYhF8.mybluemix.net/upload?name=system -H 'Content-Type: application/gzip' -F "file1=@log-upload-test.zip" -v
   Note: Unnecessary use of -X or --request, POST is already inferred.
   *   Trying 169.62.254.79...
   * TCP_NODELAY set
   * Connected to DamCYhF8.mybluemix.net (169.62.254.79) port 80 (#0)
   > POST /upload?name=system HTTP/1.1
   > Host: DamCYhF8.mybluemix.net
   > User-Agent: curl/7.54.0
   > Accept: */*
   > Content-Length: 538
   > Expect: 100-continue
   > Content-Type: application/gzip; boundary=------------------------5207d7f4ef7c0247
   >
   < HTTP/1.1 100 Continue
   < X-Note: Gateway Ack
   < HTTP/1.1 500 Internal Server Error
   < X-Backside-Transport: FAIL FAIL
   < Connection: Keep-Alive
   < Transfer-Encoding: chunked
   < Content-Type: application/json
   < Date: Sun, 22 Nov 2020 12:48:34 GMT
   < X-Global-Transaction-ID: cb47d0745fba5e1fd6a964c3
   * HTTP error before end of send, stop sending
   <
   {
     "error": "File upload failed."
   }
   ```
   
   Could this be because the disk space is fully used?
   
   @dgrove-oss @rabbah 
   Do you have any idea about this?
   I have no idea to whom the endpoint belongs.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] codecov-io edited a comment on pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#issuecomment-730427203


   # [Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=h1) Report
   > Merging [#4984](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=desc) (0032850) into [master](https://codecov.io/gh/apache/openwhisk/commit/6254477d5f95ee8d693e16daf52e9b1938f87b59?el=desc) (6254477) will **decrease** coverage by `23.04%`.
   > The diff coverage is `1.51%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/openwhisk/pull/4984/graphs/tree.svg?width=650&height=150&src=pr&token=l0YmsiSAso)](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #4984       +/-   ##
   ==========================================
   - Coverage   29.05%   6.01%   -23.05%     
   ==========================================
     Files         195     206       +11     
     Lines        9553    9998      +445     
     Branches      413     438       +25     
   ==========================================
   - Hits         2776     601     -2175     
   - Misses       6777    9397     +2620     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/elasticsearch/ElasticSearchActivationStore.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvZWxhc3RpY3NlYXJjaC9FbGFzdGljU2VhcmNoQWN0aXZhdGlvblN0b3JlLnNjYWxh) | `0.00% <0.00%> (ø)` | |
   | [...scheduler/queue/ElasticSearchDurationChecker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvRWxhc3RpY1NlYXJjaER1cmF0aW9uQ2hlY2tlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
   | [...isk/core/scheduler/queue/NoopDurationChecker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvTm9vcER1cmF0aW9uQ2hlY2tlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
   | [.../scala/org/apache/openwhisk/core/WhiskConfig.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvV2hpc2tDb25maWcuc2NhbGE=) | `63.08% <100.00%> (-24.08%)` | :arrow_down: |
   | [...ain/scala/org/apache/openwhisk/spi/SpiLoader.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL3NwaS9TcGlMb2FkZXIuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...scala/org/apache/openwhisk/core/FeatureFlags.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvRmVhdHVyZUZsYWdzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...la/org/apache/openwhisk/http/BasicRasService.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNSYXNTZXJ2aWNlLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...ache/openwhisk/core/database/DocumentFactory.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvRG9jdW1lbnRGYWN0b3J5LnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...che/openwhisk/core/invoker/LogStoreCollector.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvaW52b2tlci9Mb2dTdG9yZUNvbGxlY3Rvci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...he/openwhisk/core/invoker/NamespaceBlacklist.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvaW52b2tlci9OYW1lc3BhY2VCbGFja2xpc3Quc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | ... and [99 more](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=footer). Last update [a202538...0032850](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] bdoyle0182 commented on a change in pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
bdoyle0182 commented on a change in pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#discussion_r494695271



##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/ElasticSearchDurationChecker.scala
##########
@@ -0,0 +1,252 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorSystem
+import com.sksamuel.elastic4s.http.ElasticDsl._
+import com.sksamuel.elastic4s.http.{ElasticClient, ElasticProperties, NoOpRequestConfigCallback}
+import com.sksamuel.elastic4s.{ElasticDate, ElasticDateMath, Seconds}
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.ConfigKeys
+import org.apache.openwhisk.core.entity.WhiskActionMetaData
+import org.apache.openwhisk.spi.Spi
+import pureconfig.loadConfigOrThrow
+import spray.json.{JsArray, JsNumber, JsValue, RootJsonFormat, deserializationError, _}
+
+import scala.concurrent.Future
+import scala.concurrent.duration.FiniteDuration
+import scala.language.implicitConversions
+import scala.util.{Failure, Try}
+
+trait DurationChecker {
+  def checkAverageDuration(invocationNamespace: String, actionMetaData: WhiskActionMetaData)(
+    callback: DurationCheckResult => DurationCheckResult): Future[DurationCheckResult]
+}
+
+case class DurationCheckResult(averageDuration: Option[Double], hitCount: Long, took: Long)
+
+object ElasticSearchDurationChecker {
+  val FilterAggregationName = "filterAggregation"
+  val AverageAggregationName = "averageAggregation"
+
+  implicit val serde = new ElasticSearchDurationCheckResultFormat()
+
+  def getFromDate(timeWindow: FiniteDuration): ElasticDateMath =
+    ElasticDate.now minus (timeWindow.toSeconds.toInt, Seconds)
+}
+
+class ElasticSearchDurationChecker(private val client: ElasticClient, val timeWindow: FiniteDuration)(
+  implicit val actorSystem: ActorSystem,
+  implicit val logging: Logging)
+    extends DurationChecker {
+  import ElasticSearchDurationChecker._
+  import org.apache.openwhisk.core.database.elasticsearch.ElasticSearchActivationStore.generateIndex
+
+  implicit val ec = actorSystem.getDispatcher
+
+  override def checkAverageDuration(invocationNamespace: String, actionMetaData: WhiskActionMetaData)(
+    callback: DurationCheckResult => DurationCheckResult): Future[DurationCheckResult] = {
+    val index = generateIndex(invocationNamespace)
+    val fqn = actionMetaData.fullyQualifiedName(false)
+    val fromDate = getFromDate(timeWindow)
+
+    logging.info(this, s"check average duration for $fqn in $index for last $timeWindow")
+
+    actionMetaData.binding match {
+      case Some(binding) =>
+        client

Review comment:
       the Some and None cases can call a helper function since the only difference in the query is the `List` to match on and pass that `List` as a param. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] style95 closed pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
style95 closed pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] bdoyle0182 commented on a change in pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
bdoyle0182 commented on a change in pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#discussion_r494695271



##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/ElasticSearchDurationChecker.scala
##########
@@ -0,0 +1,252 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorSystem
+import com.sksamuel.elastic4s.http.ElasticDsl._
+import com.sksamuel.elastic4s.http.{ElasticClient, ElasticProperties, NoOpRequestConfigCallback}
+import com.sksamuel.elastic4s.{ElasticDate, ElasticDateMath, Seconds}
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.ConfigKeys
+import org.apache.openwhisk.core.entity.WhiskActionMetaData
+import org.apache.openwhisk.spi.Spi
+import pureconfig.loadConfigOrThrow
+import spray.json.{JsArray, JsNumber, JsValue, RootJsonFormat, deserializationError, _}
+
+import scala.concurrent.Future
+import scala.concurrent.duration.FiniteDuration
+import scala.language.implicitConversions
+import scala.util.{Failure, Try}
+
+trait DurationChecker {
+  def checkAverageDuration(invocationNamespace: String, actionMetaData: WhiskActionMetaData)(
+    callback: DurationCheckResult => DurationCheckResult): Future[DurationCheckResult]
+}
+
+case class DurationCheckResult(averageDuration: Option[Double], hitCount: Long, took: Long)
+
+object ElasticSearchDurationChecker {
+  val FilterAggregationName = "filterAggregation"
+  val AverageAggregationName = "averageAggregation"
+
+  implicit val serde = new ElasticSearchDurationCheckResultFormat()
+
+  def getFromDate(timeWindow: FiniteDuration): ElasticDateMath =
+    ElasticDate.now minus (timeWindow.toSeconds.toInt, Seconds)
+}
+
+class ElasticSearchDurationChecker(private val client: ElasticClient, val timeWindow: FiniteDuration)(
+  implicit val actorSystem: ActorSystem,
+  implicit val logging: Logging)
+    extends DurationChecker {
+  import ElasticSearchDurationChecker._
+  import org.apache.openwhisk.core.database.elasticsearch.ElasticSearchActivationStore.generateIndex
+
+  implicit val ec = actorSystem.getDispatcher
+
+  override def checkAverageDuration(invocationNamespace: String, actionMetaData: WhiskActionMetaData)(
+    callback: DurationCheckResult => DurationCheckResult): Future[DurationCheckResult] = {
+    val index = generateIndex(invocationNamespace)
+    val fqn = actionMetaData.fullyQualifiedName(false)
+    val fromDate = getFromDate(timeWindow)
+
+    logging.info(this, s"check average duration for $fqn in $index for last $timeWindow")
+
+    actionMetaData.binding match {
+      case Some(binding) =>
+        client

Review comment:
       the Some and None cases can call a helper function since the only difference in the query is the `List` to match on and pass that `List` as a param. 

##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/NoopDurationChecker.scala
##########
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorSystem
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.entity.WhiskActionMetaData
+
+import scala.concurrent.Future
+
+object NoopDurationCheckerProvider extends DurationCheckerProvider {

Review comment:
       so this means you operate the schedule without using the average activation duration for an action heuristic. How important is using the heuristic for the performance of the scheduler?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] style95 commented on a change in pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
style95 commented on a change in pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#discussion_r494717183



##########
File path: core/scheduler/src/main/scala/org/apache/openwhisk/core/scheduler/queue/NoopDurationChecker.scala
##########
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.openwhisk.core.scheduler.queue
+
+import akka.actor.ActorSystem
+import org.apache.openwhisk.common.Logging
+import org.apache.openwhisk.core.entity.WhiskActionMetaData
+
+import scala.concurrent.Future
+
+object NoopDurationCheckerProvider extends DurationCheckerProvider {

Review comment:
       This is just for other DBs such as CouchDB or CosmosDB in case the scheduler is used with other than ES.
   (Even if it is highly recommended to use with ES.)
   
   Regarding the average duration, it is important to improve the accuracy of calculation but the queue can still work without it. When an action is newly created, there is no activation accordingly no average duration.
   In such a case, it assumes one container can handle one activation for the given time.
   So even if one container can handle multiple activations for a given period, it assumes a container can handle only one activation, so schedulers would tend to overprovision containers.
   
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] rabbah commented on pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
rabbah commented on pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#issuecomment-734033840


   I will change this upload to use a different object store - will give it a
   try soon.
   
   On Sun, Nov 22, 2020 at 3:32 PM David Grove <no...@github.com>
   wrote:
   
   > We failed to upload the logs to Box. This happens because we don't have a
   > way to automatically remove old logs and the Box folder fills up. However,
   > the upload to Box is optional (will not cause the travisci job to fail).
   > There is a real test failure earlier in the log:
   > https://travis-ci.org/github/apache/openwhisk/jobs/744844186#L7457
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > <https://github.com/apache/openwhisk/pull/4984#issuecomment-731842802>,
   > or unsubscribe
   > <https://github.com/notifications/unsubscribe-auth/ABF25MXSECTY3YXBH6YOU5DSRFYPPANCNFSM4RY4XZXQ>
   > .
   >
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] bdoyle0182 commented on pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
bdoyle0182 commented on pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#issuecomment-700489466


   LGTM


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [openwhisk] codecov-io edited a comment on pull request #4984: [New Scheduler] Add duration checker

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #4984:
URL: https://github.com/apache/openwhisk/pull/4984#issuecomment-730427203


   # [Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=h1) Report
   > Merging [#4984](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=desc) (0032850) into [master](https://codecov.io/gh/apache/openwhisk/commit/6254477d5f95ee8d693e16daf52e9b1938f87b59?el=desc) (6254477) will **increase** coverage by `42.66%`.
   > The diff coverage is `13.63%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/openwhisk/pull/4984/graphs/tree.svg?width=650&height=150&src=pr&token=l0YmsiSAso)](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #4984       +/-   ##
   ===========================================
   + Coverage   29.05%   71.72%   +42.66%     
   ===========================================
     Files         195      206       +11     
     Lines        9553     9998      +445     
     Branches      413      438       +25     
   ===========================================
   + Hits         2776     7171     +4395     
   + Misses       6777     2827     -3950     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...scheduler/queue/ElasticSearchDurationChecker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvRWxhc3RpY1NlYXJjaER1cmF0aW9uQ2hlY2tlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
   | [...isk/core/scheduler/queue/NoopDurationChecker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9zY2hlZHVsZXIvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9vcGVud2hpc2svY29yZS9zY2hlZHVsZXIvcXVldWUvTm9vcER1cmF0aW9uQ2hlY2tlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
   | [...e/elasticsearch/ElasticSearchActivationStore.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvZGF0YWJhc2UvZWxhc3RpY3NlYXJjaC9FbGFzdGljU2VhcmNoQWN0aXZhdGlvblN0b3JlLnNjYWxh) | `84.04% <88.88%> (+84.04%)` | :arrow_up: |
   | [.../scala/org/apache/openwhisk/core/WhiskConfig.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvV2hpc2tDb25maWcuc2NhbGE=) | `93.28% <100.00%> (+6.12%)` | :arrow_up: |
   | [...che/openwhisk/core/invoker/LogStoreCollector.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvaW52b2tlci9Mb2dTdG9yZUNvbGxlY3Rvci5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...la/org/apache/openwhisk/http/BasicRasService.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNSYXNTZXJ2aWNlLnNjYWxh) | `16.66% <0.00%> (-83.34%)` | :arrow_down: |
   | [...a/org/apache/openwhisk/http/BasicHttpService.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2h0dHAvQmFzaWNIdHRwU2VydmljZS5zY2FsYQ==) | `3.33% <0.00%> (-75.00%)` | :arrow_down: |
   | [...la/org/apache/openwhisk/core/invoker/Invoker.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvaW52b2tlci9JbnZva2VyLnNjYWxh) | `1.58% <0.00%> (-68.26%)` | :arrow_down: |
   | [...pache/openwhisk/core/invoker/InvokerReactive.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29yZS9pbnZva2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvaW52b2tlci9JbnZva2VyUmVhY3RpdmUuc2NhbGE=) | `0.00% <0.00%> (-59.14%)` | :arrow_down: |
   | [...apache/openwhisk/core/ack/MessagingActiveAck.scala](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree#diff-Y29tbW9uL3NjYWxhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvb3BlbndoaXNrL2NvcmUvYWNrL01lc3NhZ2luZ0FjdGl2ZUFjay5zY2FsYQ==) | `0.00% <0.00%> (-54.55%)` | :arrow_down: |
   | ... and [156 more](https://codecov.io/gh/apache/openwhisk/pull/4984/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=footer). Last update [a202538...0032850](https://codecov.io/gh/apache/openwhisk/pull/4984?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org