You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "clintropolis (via GitHub)" <gi...@apache.org> on 2026/02/21 00:51:31 UTC
[PR] introduce SegmentPruner interface to abstract segment pruning operations (druid)
clintropolis opened a new pull request, #19042:
URL: https://github.com/apache/druid/pull/19042
changes:
* add new `SegmentPruner` interface as a replacement for `DimFilterUtils.filterShards` static method
* `SegmentPruner` interface has a single 'prune' method that takes an `Iterable<T>` and `Function` to extract a `DataSegment` from the iterable objects, returning a `Collection<T>` of the filtered objects
* `DimFilterUtils.filterShards` has been removed, its logic has been migrated to `FilterSegmentPruner` implementation of the new `SegmentPruner` interface
* `TableInputSpec` now has a `SegmentPruner` field instead of separate `filter` and `filterFields` (these fields still remain for compat purposes)
* transition `DartTableInputSpecSlicer`, `IndexerTableInputSpecSlicer, and `CachingClusteredClient` to use `SegmentPruner` (the former 2 via the `TableInputSpec`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] introduce SegmentPruner interface to abstract segment pruning operations (druid)
Posted by "github-advanced-security[bot] (via GitHub)" <gi...@apache.org>.
github-advanced-security[bot] commented on code in PR #19042:
URL: https://github.com/apache/druid/pull/19042#discussion_r2844871072
##########
multi-stage-query/src/main/java/org/apache/druid/msq/exec/StageProcessor.java:
##########
@@ -77,4 +79,7 @@
*/
@SuppressWarnings("rawtypes")
ExtraInfoHolder makeExtraInfoHolder(@Nullable ExtraInfoType extra);
+
+ @Nullable
+ SegmentPruner getPruner(InputSpec inputSpec, int inputNumber);
Review Comment:
## Useless parameter
The parameter 'inputNumber' is never used.
[Show more details](https://github.com/apache/druid/security/code-scanning/10827)
##########
multi-stage-query/src/main/java/org/apache/druid/msq/exec/StageProcessor.java:
##########
@@ -77,4 +79,7 @@
*/
@SuppressWarnings("rawtypes")
ExtraInfoHolder makeExtraInfoHolder(@Nullable ExtraInfoType extra);
+
+ @Nullable
+ SegmentPruner getPruner(InputSpec inputSpec, int inputNumber);
Review Comment:
## Useless parameter
The parameter 'inputSpec' is never used.
[Show more details](https://github.com/apache/druid/security/code-scanning/10826)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] introduce SegmentPruner interface to abstract segment pruning operations (druid)
Posted by "gianm (via GitHub)" <gi...@apache.org>.
gianm merged PR #19042:
URL: https://github.com/apache/druid/pull/19042
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] introduce SegmentPruner interface to abstract segment pruning operations (druid)
Posted by "github-advanced-security[bot] (via GitHub)" <gi...@apache.org>.
github-advanced-security[bot] commented on code in PR #19042:
URL: https://github.com/apache/druid/pull/19042#discussion_r2845152285
##########
multi-stage-query/src/main/java/org/apache/druid/msq/exec/StageProcessor.java:
##########
@@ -77,4 +79,14 @@
*/
@SuppressWarnings("rawtypes")
ExtraInfoHolder makeExtraInfoHolder(@Nullable ExtraInfoType extra);
+
+ /**
+ * Produces an optional {@link SegmentPruner}, which can used for best-effort pruning of {@link DataSegment} objects
+ * by processors which deal with them
+ */
+ @Nullable
+ default SegmentPruner getPruner(InputSpec inputSpec, int inputNumber)
Review Comment:
## Useless parameter
The parameter 'inputNumber' is never used.
[Show more details](https://github.com/apache/druid/security/code-scanning/10829)
##########
multi-stage-query/src/main/java/org/apache/druid/msq/exec/StageProcessor.java:
##########
@@ -77,4 +79,14 @@
*/
@SuppressWarnings("rawtypes")
ExtraInfoHolder makeExtraInfoHolder(@Nullable ExtraInfoType extra);
+
+ /**
+ * Produces an optional {@link SegmentPruner}, which can used for best-effort pruning of {@link DataSegment} objects
+ * by processors which deal with them
+ */
+ @Nullable
+ default SegmentPruner getPruner(InputSpec inputSpec, int inputNumber)
Review Comment:
## Useless parameter
The parameter 'inputSpec' is never used.
[Show more details](https://github.com/apache/druid/security/code-scanning/10828)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] introduce SegmentPruner interface to abstract segment pruning operations (druid)
Posted by "clintropolis (via GitHub)" <gi...@apache.org>.
clintropolis commented on code in PR #19042:
URL: https://github.com/apache/druid/pull/19042#discussion_r2848378250
##########
multi-stage-query/src/main/java/org/apache/druid/msq/exec/std/BasicStageProcessor.java:
##########
@@ -54,4 +56,11 @@ public ExtraInfoHolder makeExtraInfoHolder(@Nullable Object extra)
return NilExtraInfoHolder.instance();
}
+
+ @Nullable
+ @Override
+ public SegmentPruner getPruner(InputSpec inputSpec, int inputNumber)
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] introduce SegmentPruner interface to abstract segment pruning operations (druid)
Posted by "gianm (via GitHub)" <gi...@apache.org>.
gianm commented on code in PR #19042:
URL: https://github.com/apache/druid/pull/19042#discussion_r2844847887
##########
multi-stage-query/src/main/java/org/apache/druid/msq/exec/std/BasicStageProcessor.java:
##########
@@ -54,4 +56,11 @@ public ExtraInfoHolder makeExtraInfoHolder(@Nullable Object extra)
return NilExtraInfoHolder.instance();
}
+
+ @Nullable
+ @Override
+ public SegmentPruner getPruner(InputSpec inputSpec, int inputNumber)
Review Comment:
IMO this would be better as a default impl on `StageProcessor`.
##########
multi-stage-query/src/main/java/org/apache/druid/msq/indexing/processor/SegmentGeneratorStageProcessor.java:
##########
@@ -271,6 +273,13 @@ public ExtraInfoHolder makeExtraInfoHolder(final List<SegmentIdWithShardSpec> se
return new SegmentGeneratorExtraInfoHolder(segmentIdsWithShardSpecs);
}
+ @Nullable
+ @Override
+ public SegmentPruner getPruner(InputSpec inputSpec, int inputNumber)
Review Comment:
Shouldn't be needed once the default impl is moved up to the interface.
##########
multi-stage-query/src/main/java/org/apache/druid/msq/exec/StageProcessor.java:
##########
@@ -77,4 +79,7 @@ public interface StageProcessor<R, ExtraInfoType>
*/
@SuppressWarnings("rawtypes")
ExtraInfoHolder makeExtraInfoHolder(@Nullable ExtraInfoType extra);
+
+ @Nullable
+ SegmentPruner getPruner(InputSpec inputSpec, int inputNumber);
Review Comment:
Javadocs please.
##########
multi-stage-query/src/main/java/org/apache/druid/msq/kernel/StageDefinition.java:
##########
@@ -231,6 +232,11 @@ public boolean doesSortDuringShuffle()
}
}
+ public SegmentPruner getSegmentPruner(int inputNumber)
Review Comment:
Javadoc please.
##########
multi-stage-query/src/main/java/org/apache/druid/msq/querykit/results/ExportResultsStageProcessor.java:
##########
@@ -201,6 +203,13 @@ public ExtraInfoHolder makeExtraInfoHolder(@Nullable Object extra)
return NilExtraInfoHolder.instance();
}
+ @Nullable
+ @Override
+ public SegmentPruner getPruner(InputSpec inputSpec, int inputNumber)
Review Comment:
Should be removable once the default impl moves up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org