[GitHub] [incubator-pinot] siddharthteotia opened a new pull request #4698: Fix simple date time column segment generation test

[GitBox <gi...@apache.org>]

siddharthteotia opened a new pull request #4698: Fix simple date time column segment generation test
URL: https://github.com/apache/incubator-pinot/pull/4698
 
 
   TimeUtils (Pinot code) is using UTC time zone to specify 
   allowed segment minTime and maxTime for time column values.
   
   For the particular test testSimpleDateSegmentGeneration that sometimes
   failed, we were generating random value of milliseconds
   between TimeUtils.getValidMinTimeMillis() and System.currentTimeInMillis.
   
   This random value was then used to build a DateTime with
   format yyyymmdd since the test is for simple date format.
   
   The problem happens when the random generated value is in fact
   equal to TimeUtils.getValidMinTimeMillis() = 31536000000ms.
   Converting this to DateTime would lead to 19701231 date since
   we didn't specify the time zone and it uses the default (JDK's)
   which happens to be PST (behind UTC)
   
   Subsequently during segment generation we get this simple date
   (19701231) as the minDate and we convert it back to millis
   before we do the validation against allowed min time.
   
   The converted millis will be slightly less than 31536000000ms
   since we didn't take care of the timezone originally when
   we generated the random value and it defaulted to PST. Thus
   the validity check against min/max allowed values will fail.
   
   The fix is to get the DateTime from millis in UTC in the test code.
   This would ensure that if we happen to get the random value as
   the minimum allowed value 31536000000ms, it would imply 19710101
   in DateTime for yyyymmdd format and not 19701231
    
   When this date is later used during segment generation code,
   the converted millis are within the range and validity check passes
   
   cc @mcvsubbu 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org