[GitHub] gianm opened a new issue #6177: TimeBoundary, DataSourceMetadata filterSegments flawed with overlapping segments

[GitBox <gi...@apache.org>]

gianm opened a new issue #6177: TimeBoundary, DataSourceMetadata filterSegments flawed with overlapping segments
URL: https://github.com/apache/incubator-druid/issues/6177
 
 
   The "interval" on a LogicalSegment corresponds to the timeline object holder interval, meaning that with overlapping segments, it is only the queryable part. This means that the code in filterSegments of both timeBoundary and dataSourceMetadata is flawed. It looks like this in both of them:
   
   ```java
       final T min = query.isMaxTime() ? null : segments.get(0);
       final T max = query.isMinTime() ? null : segments.get(segments.size() - 1);
   
       return Lists.newArrayList(
           Iterables.filter(
               segments,
               input -> (min != null && input.getInterval().overlaps(min.getInterval())) ||
                      (max != null && input.getInterval().overlaps(max.getInterval()))
           )
       );
   ```
   
   But because of how "interval" works on the LogicalSegments, `input.getInterval()` for two different LogicalSegments will never overlap. This causes erroneous results in cases like this:
   
   - Segment A is 2017/2018 (YEAR granularity) and has data up through May 2017.
   - Segment B is 2017-08-01/2017-08-02 (DAY granularity) and has data for August 1 2017.
   
   In this case, three LogicalSegments are returned:
   
   - 2017/2017-08-01
   - 2017-08-01/2017-08-02
   - 2017-08-02/2018
   
   And the filterSegments methods on these two query types will only inspect the last LogicalSegment, meaning they'll miss the _actual_ max time, which is in the 2017-08-01/2017-08-02 holder.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org