[GitHub] spark pull request #21063: [SPARK-23886][Structured Streaming] Update query ...

[HeartSaVioR <gi...@git.apache.org>]

Github user HeartSaVioR commented on a diff in the pull request:

    https://github.com/apache/spark/pull/21063#discussion_r184241625
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala ---
    @@ -111,7 +112,12 @@ trait ProgressReporter extends Logging {
         logDebug("Starting Trigger Calculation")
         lastTriggerStartTimestamp = currentTriggerStartTimestamp
         currentTriggerStartTimestamp = triggerClock.getTimeMillis()
    -    currentStatus = currentStatus.copy(isTriggerActive = true)
    +    // isTriggerActive field is kept false for ContinuousExecution
    +    // since it is tied to MicroBatchExecution
    +    this match {
    --- End diff --
    
    nit: someone may have a concern that a trait needs to be aware of actual implementation. 
    
    There looks like two options: 
    1. extract method to only update currentStatus for starting trigger defaulting to `isTriggerActive = true`, and let `ContinuousExecution` overrides the method. 
    2. just override `startTrigger()` in `ContinuousExecution`, and call `super.startTrigger()` and update currentStatus once again. It might open very small window for other threads to read invalid status information (isTriggerActive = true), but will require less change if it is acceptable.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org