[GitHub] zsxwing opened a new pull request #23718: [SPARK-26806][SS]EventTimeStats.merge should handle zeros correctly

[GitBox <gi...@apache.org>]

zsxwing opened a new pull request #23718: [SPARK-26806][SS]EventTimeStats.merge should handle zeros correctly
URL: https://github.com/apache/spark/pull/23718
 
 
   ## What changes were proposed in this pull request?
   
   Right now, EventTimeStats.merge doesn't handle `zero.merge(zero)` correctly. This will make `avg` become `NaN`. And whatever gets merged with the result of `zero.merge(zero)`, `avg` will still be `NaN`. Then finally, we call `NaN.toLong` and get `0`, and the user will see the following incorrect report:
   
   ```
   "eventTime" : {
       "avg" : "1970-01-01T00:00:00.000Z",
       "max" : "2019-01-31T12:57:00.000Z",
       "min" : "2019-01-30T18:44:04.000Z",
       "watermark" : "1970-01-01T00:00:00.000Z"
     }
   ```
   
   This PR fixes the above issue.
   
   ## How was this patch tested?
   
   The new unit tests.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org