[GitHub] spark pull request #21188: [SPARK-24046][SS] Fix rate source rowsPerSecond <...

[xuanyuanking <gi...@git.apache.org>]

Github user xuanyuanking commented on a diff in the pull request:

    https://github.com/apache/spark/pull/21188#discussion_r185852663
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamProvider.scala ---
    @@ -107,14 +107,25 @@ object RateStreamProvider {
         // seconds   = 0 1 2  3  4  5  6
         // speed     = 0 2 4  6  8 10 10 (speedDeltaPerSecond * seconds)
         // end value = 0 2 6 12 20 30 40 (0 + speedDeltaPerSecond * seconds) * (seconds + 1) / 2
    -    val speedDeltaPerSecond = rowsPerSecond / (rampUpTimeSeconds + 1)
    +    val speedDeltaPerSecond = math.max(1, rowsPerSecond / (rampUpTimeSeconds + 1))
    --- End diff --
    
    Keep at-least 1 per second and leave other seconds to 0 is ok IMOP. 


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org