You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/03/24 01:56:21 UTC
[GitHub] [druid] maytasm commented on a change in pull request #11019: Auto-compaction with segment granularity retrieve incomplete segments from timeline when interval overlap
maytasm commented on a change in pull request #11019:
URL: https://github.com/apache/druid/pull/11019#discussion_r600098544
##########
File path: integration-tests/src/test/java/org/apache/druid/tests/coordinator/duty/ITAutoCompactionTest.java
##########
@@ -438,6 +438,67 @@ public void testAutoCompactionDutyWithSegmentGranularityAndExistingCompactedSegm
}
}
+ @Test
+ public void testAutoCompactionDutyWithSegmentGranularityAndSmallerSegmentGranularityCoveringMultipleSegmentsInTimeline() throws Exception
+ {
+ loadData(INDEX_TASK);
+ try (final Closeable ignored = unloader(fullDatasourceName)) {
+ final List<String> intervalsBeforeCompaction = coordinator.getSegmentIntervals(fullDatasourceName);
+ intervalsBeforeCompaction.sort(null);
+ // 4 segments across 2 days (4 total)...
+ verifySegmentsCount(4);
+ verifyQuery(INDEX_QUERIES_RESOURCE);
+
+ Granularity newGranularity = Granularities.YEAR;
+ submitCompactionConfig(MAX_ROWS_PER_SEGMENT_COMPACTED, NO_SKIP_OFFSET, new UserCompactionTaskGranularityConfig(newGranularity, null));
+
+ List<String> expectedIntervalAfterCompaction = new ArrayList<>();
+ // We wil have one segment with interval of 2013-01-01/2014-01-01 (compacted with YEAR)
+ for (String interval : intervalsBeforeCompaction) {
+ for (Interval newinterval : newGranularity.getIterable(new Interval(interval, ISOChronology.getInstanceUTC()))) {
+ expectedIntervalAfterCompaction.add(newinterval.toString());
+ }
+ }
+ forceTriggerAutoCompaction(1);
+ verifyQuery(INDEX_QUERIES_RESOURCE);
+ verifySegmentsCompacted(1, MAX_ROWS_PER_SEGMENT_COMPACTED);
+ checkCompactionIntervals(expectedIntervalAfterCompaction);
+
+ loadData(INDEX_TASK);
+ verifySegmentsCount(5);
+ verifyQuery(INDEX_QUERIES_RESOURCE);
+ // 5 segments. 1 compacted YEAR segment and 4 newly ingested DAY segments across 2 days
+ // We wil have one segment with interval of 2013-01-01/2014-01-01 (compacted with YEAR) from the compaction
+ // two segments with interval of 2013-09-01/2013-10-01 (newly ingested with DAY)
Review comment:
The interval in the comment is incorrect. It should be one day interval
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org