You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/06/14 21:40:14 UTC

[GitHub] [spark] HeartSaVioR opened a new pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

HeartSaVioR opened a new pull request #28828:
URL: https://github.com/apache/spark/pull/28828


   ### What changes were proposed in this pull request?
   
   This PR renames the variable from "numLateInputs" to "numDropppedRowsByWatermark" so that it becomes self-explanation.
   
   ### Why are the changes needed?
   
   This is originated from post-review, see https://github.com/apache/spark/pull/28607#discussion_r439853232
   
   ### Does this PR introduce _any_ user-facing change?
   
   No, as SPARK-24634 is not introduced in any release yet.
   
   ### How was this patch tested?
   
   Existing UTs.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on a change in pull request #28828:
URL: https://github.com/apache/spark/pull/28828#discussion_r439963800



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala
##########
@@ -43,7 +43,7 @@ class StateOperatorProgress private[sql](
     val numRowsTotal: Long,
     val numRowsUpdated: Long,
     val memoryUsedBytes: Long,
-    val numLateInputs: Long,
+    val numDroppedRowsByWatermark: Long,

Review comment:
       Yeah that sounds better in consistency (numRows + blabla explaining how the number is measured) and provides same meaning. Thanks for the suggestion.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643830992






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643826206


   Merged build finished. Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643826119






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643869596


   **[Test build #124017 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124017/testReport)** for PR 28828 at commit [`a712b4d`](https://github.com/apache/spark/commit/a712b4dff959f55633ae543a4a70224c218a61eb).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643827509


   **[Test build #124015 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124015/testReport)** for PR 28828 at commit [`ca3b3de`](https://github.com/apache/spark/commit/ca3b3de653a92090db33ca8282eea18b75ff2420).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644374629






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644477975






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643965873






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on a change in pull request #28828:
URL: https://github.com/apache/spark/pull/28828#discussion_r440429038



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala
##########
@@ -43,7 +43,7 @@ class StateOperatorProgress private[sql](
     val numRowsTotal: Long,

Review comment:
       Looks like "match case" missed lots of spots. I fixed all the spots.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon closed pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #28828:
URL: https://github.com/apache/spark/pull/28828


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643965274


   **[Test build #124040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124040/testReport)** for PR 28828 at commit [`75d12d3`](https://github.com/apache/spark/commit/75d12d31814cf774f35368bb487cf0f1ecea904d).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644373853


   **[Test build #124074 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124074/testReport)** for PR 28828 at commit [`c496eb5`](https://github.com/apache/spark/commit/c496eb5d62187cae5b8d41b57c89f47ad79c7c7c).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644477975






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643826208


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124013/
   Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643826206






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644153619






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644355363






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] jainshashank24 edited a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
jainshashank24 edited a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-686435584


   Hi i have used this MR and the other one https://github.com/apache/spark/pull/28607/files
   even after that i cant see counter value increasing
   Though i can see the counter under "stateOperators"
   
   "stateOperators" : [ {
       "numRowsTotal" : 0,
       "numRowsUpdated" : 0,
       "memoryUsedBytes" : 1344,
       "numRowsDroppedByWatermark" : 0,
       "customMetrics" : {
         "loadedMapCacheHitCount" : 0,
         "loadedMapCacheMissCount" : 0,
         "stateOnCurrentVersionSizeBytes" : 480
       }
     } ],
   
   @HeartSaVioR if you can help out


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644354166


   **[Test build #124073 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124073/testReport)** for PR 28828 at commit [`2e984f3`](https://github.com/apache/spark/commit/2e984f3789c845edf95ba606b8627890700c61ba).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644355370


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124073/
   Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643826003


   **[Test build #124013 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124013/testReport)** for PR 28828 at commit [`cf147aa`](https://github.com/apache/spark/commit/cf147aadf964d01fa1f664a1d9053057ffd2c9a9).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644355363


   Merged build finished. Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643826119






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643873268






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644152195


   **[Test build #124040 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124040/testReport)** for PR 28828 at commit [`75d12d3`](https://github.com/apache/spark/commit/75d12d31814cf774f35368bb487cf0f1ecea904d).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643965873






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644355343


   **[Test build #124073 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124073/testReport)** for PR 28828 at commit [`2e984f3`](https://github.com/apache/spark/commit/2e984f3789c845edf95ba606b8627890700c61ba).
    * This patch **fails Scala style tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643830855


   **[Test build #124017 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124017/testReport)** for PR 28828 at commit [`a712b4d`](https://github.com/apache/spark/commit/a712b4dff959f55633ae543a4a70224c218a61eb).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644354830






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643869852






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643826204


   **[Test build #124013 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124013/testReport)** for PR 28828 at commit [`cf147aa`](https://github.com/apache/spark/commit/cf147aadf964d01fa1f664a1d9053057ffd2c9a9).
    * This patch **fails Scala style tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644591847


   Merged to master.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644374629






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643872713


   **[Test build #124015 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124015/testReport)** for PR 28828 at commit [`ca3b3de`](https://github.com/apache/spark/commit/ca3b3de653a92090db33ca8282eea18b75ff2420).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643869857


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124017/
   Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #28828:
URL: https://github.com/apache/spark/pull/28828#discussion_r439949163



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala
##########
@@ -43,7 +43,7 @@ class StateOperatorProgress private[sql](
     val numRowsTotal: Long,
     val numRowsUpdated: Long,
     val memoryUsedBytes: Long,
-    val numLateInputs: Long,
+    val numDroppedRowsByWatermark: Long,

Review comment:
       nit. Just a question. For the variable naming, `numRowsDroppedByWatermark` seems more consistent with `numRowsTotal` and `numRowsUpdates` within this `StateOperatorProgress` class, doesn't it? (cc @zsxwing since this came from his comment.)




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] jainshashank24 removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
jainshashank24 removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-686435584


   Hi i have used this MR and the other one https://github.com/apache/spark/pull/28607/files
   even after that i cant see counter value increasing
   Though i can see the counter under "stateOperators"
   
   "stateOperators" : [ {
       "numRowsTotal" : 0,
       "numRowsUpdated" : 0,
       "memoryUsedBytes" : 1344,
       "numRowsDroppedByWatermark" : 0,
       "customMetrics" : {
         "loadedMapCacheHitCount" : 0,
         "loadedMapCacheMissCount" : 0,
         "stateOnCurrentVersionSizeBytes" : 480
       }
     } ],
   
   @HeartSaVioR if you can help out


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643906549


   @xuanyuanking too FYI


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644153619






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HeartSaVioR commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643825765


   cc. @zsxwing 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643965274


   **[Test build #124040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124040/testReport)** for PR 28828 at commit [`75d12d3`](https://github.com/apache/spark/commit/75d12d31814cf774f35368bb487cf0f1ecea904d).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644373853


   **[Test build #124074 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124074/testReport)** for PR 28828 at commit [`c496eb5`](https://github.com/apache/spark/commit/c496eb5d62187cae5b8d41b57c89f47ad79c7c7c).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HeartSaVioR commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644609604


   Thanks all for reviewing and merging!


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643827609






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643830992






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643826003


   **[Test build #124013 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124013/testReport)** for PR 28828 at commit [`cf147aa`](https://github.com/apache/spark/commit/cf147aadf964d01fa1f664a1d9053057ffd2c9a9).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643827509


   **[Test build #124015 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124015/testReport)** for PR 28828 at commit [`ca3b3de`](https://github.com/apache/spark/commit/ca3b3de653a92090db33ca8282eea18b75ff2420).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #28828:
URL: https://github.com/apache/spark/pull/28828#discussion_r439949163



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala
##########
@@ -43,7 +43,7 @@ class StateOperatorProgress private[sql](
     val numRowsTotal: Long,
     val numRowsUpdated: Long,
     val memoryUsedBytes: Long,
-    val numLateInputs: Long,
+    val numDroppedRowsByWatermark: Long,

Review comment:
       nit. Just a question. For the variable naming, `numRowsDroppedByWatermark` seems more consistent with `numRowsTotal` and `numRowsUpdates` within this `StateOperatorProgress` class, doesn't it? (cc @zsxwing since this acmes from his comment.)




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643873268






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zsxwing commented on a change in pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
zsxwing commented on a change in pull request #28828:
URL: https://github.com/apache/spark/pull/28828#discussion_r440272044



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala
##########
@@ -43,7 +43,7 @@ class StateOperatorProgress private[sql](
     val numRowsTotal: Long,

Review comment:
       Could you also fix `newNumLateInputs` in this file?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on a change in pull request #28828:
URL: https://github.com/apache/spark/pull/28828#discussion_r440414350



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala
##########
@@ -43,7 +43,7 @@ class StateOperatorProgress private[sql](
     val numRowsTotal: Long,

Review comment:
       Sigh seems like I did find with "match case" and missed it. Will find and fix. Thanks!




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643830855


   **[Test build #124017 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124017/testReport)** for PR 28828 at commit [`a712b4d`](https://github.com/apache/spark/commit/a712b4dff959f55633ae543a4a70224c218a61eb).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643827609






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] jainshashank24 commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
jainshashank24 commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-686435584


   Hi i have used this MR and the other one https://github.com/apache/spark/pull/28607/files
   even after that i cant see counter value increasing
   Though i can see the counter under "stateOperators"
   
   "stateOperators" : [ {
       "numRowsTotal" : 0,
       "numRowsUpdated" : 0,
       "memoryUsedBytes" : 1344,
       "numRowsDroppedByWatermark" : 0,
       "customMetrics" : {
         "loadedMapCacheHitCount" : 0,
         "loadedMapCacheMissCount" : 0,
         "stateOnCurrentVersionSizeBytes" : 480
       }
     } ],


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numDropppedRowsByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-643869852


   Merged build finished. Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644354166


   **[Test build #124073 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124073/testReport)** for PR 28828 at commit [`2e984f3`](https://github.com/apache/spark/commit/2e984f3789c845edf95ba606b8627890700c61ba).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644354830






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28828:
URL: https://github.com/apache/spark/pull/28828#issuecomment-644477261


   **[Test build #124074 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124074/testReport)** for PR 28828 at commit [`c496eb5`](https://github.com/apache/spark/commit/c496eb5d62187cae5b8d41b57c89f47ad79c7c7c).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org