You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2021/06/14 08:00:27 UTC

[GitHub] [lucene] gsmiller opened a new pull request #181: LUCENE-10001: Make CollectionTerminatedException handling in MultiCollector configurable

gsmiller opened a new pull request #181:
URL: https://github.com/apache/lucene/pull/181


   # Description
   
   There could be use-cases where a user wants to terminate all wrapped collectors in a MultiCollector when one terminates. Today, other collectors continue collecting. For example, a user collecting matching docs while also collecting into a FacetsCollector may want to terminate both collectors when the "main" matching docs collector terminates.
   
   # Solution
   
   Added the ability to configure the behavior through an enum specified to `wrap`
   
   # Tests
   
   Updated the existing tests to check behavior under both options.
   
   # Checklist
   
   Please review the following and check all that apply:
   
   - [x] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/lucene/HowToContribute) and my code conforms to the standards described there to the best of my ability.
   - [x] I have created a Jira issue and added the issue ID to my pull request title.
   - [x] I have given Lucene maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended)
   - [x] I have developed this patch against the `main` branch.
   - [x] I have run `./gradlew check`.
   - [x] I have added tests for my changes.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


[GitHub] [lucene] mikemccand commented on a change in pull request #181: LUCENE-10001: Make CollectionTerminatedException handling in MultiCollector configurable

Posted by GitBox <gi...@apache.org>.
mikemccand commented on a change in pull request #181:
URL: https://github.com/apache/lucene/pull/181#discussion_r659245670



##########
File path: lucene/core/src/java/org/apache/lucene/search/MultiCollector.java
##########
@@ -78,14 +107,17 @@ public static Collector wrap(Iterable<? extends Collector> collectors) {
           colls[n++] = c;
         }
       }
-      return new MultiCollector(colls);
+      return new MultiCollector(earlyTerminationBehavior, colls);
     }
   }
 
+  private final EarlyTerminationBehavior earlyTerminationBehavior;
   private final boolean cacheScores;
   private final Collector[] collectors;
 
-  private MultiCollector(Collector... collectors) {
+  private MultiCollector(
+      EarlyTerminationBehavior earlyTerminationBehavior, Collector... collectors) {

Review comment:
       This might be a little too much over-designing for now ("design for today").  And the `FacetCollector` example is not a normal use case I think (`FacetCollector` does not normally early terminate?).  I think most use-cases for `MultiCollector` would want all collectors to collect the same set of hits.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


[GitHub] [lucene] gsmiller commented on a change in pull request #181: LUCENE-10001: Make CollectionTerminatedException handling in MultiCollector configurable

Posted by GitBox <gi...@apache.org>.
gsmiller commented on a change in pull request #181:
URL: https://github.com/apache/lucene/pull/181#discussion_r652200243



##########
File path: lucene/core/src/java/org/apache/lucene/search/MultiCollector.java
##########
@@ -25,13 +25,39 @@
 /**
  * A {@link Collector} which allows running a search with several {@link Collector}s. It offers a
  * static {@link #wrap} method which accepts a list of collectors and wraps them with {@link
- * MultiCollector}, while filtering out the <code>null</code> null ones.
+ * MultiCollector}, while filtering out the <code>null</code> ones.
  */
 public class MultiCollector implements Collector {
 
+  /**
+   * Defines the behavior to take when a wrapped {@link Collector} signals early termination by
+   * throwing a {@link CollectionTerminatedException}.
+   */
+  public enum EarlyTerminationBehavior {

Review comment:
       Good suggestion, thanks!




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


[GitHub] [lucene] gautamworah96 commented on a change in pull request #181: LUCENE-10001: Make CollectionTerminatedException handling in MultiCollector configurable

Posted by GitBox <gi...@apache.org>.
gautamworah96 commented on a change in pull request #181:
URL: https://github.com/apache/lucene/pull/181#discussion_r651419099



##########
File path: lucene/CHANGES.txt
##########
@@ -127,6 +127,10 @@ API Changes
 
 * LUCENE-8143: SpanBoostQuery has been removed. (Alan Woodward)
 
+* LUCENE-10001: MultiCollector allows users to specify whether to early terminate all collecting

Review comment:
       +1 on the overall idea. Why was this not there before? :) 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


[GitHub] [lucene] gsmiller commented on pull request #181: LUCENE-10001: Make CollectionTerminatedException handling in MultiCollector configurable

Posted by GitBox <gi...@apache.org>.
gsmiller commented on pull request #181:
URL: https://github.com/apache/lucene/pull/181#issuecomment-861881544


   @gautamworah96 thanks for the feedback! Adrien and I have been having some conversation around whether-or-not this change makes sense over in the Jira issue. As a result, I'm going to try out a little bit of a different approach next, but I appreciate your comments here and will work them in as possible in my next revision. Thanks again!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


[GitHub] [lucene] gsmiller commented on pull request #181: LUCENE-10001: Make CollectionTerminatedException handling in MultiCollector configurable

Posted by GitBox <gi...@apache.org>.
gsmiller commented on pull request #181:
URL: https://github.com/apache/lucene/pull/181#issuecomment-870083122


   I've put this in "Draft" state for now after a suggestion over on the Jira issue to approach this differently. I'll update here (and take out of "Draft") once I've had a chance to explore a different approach. Thanks everyone for the comments so far!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


[GitHub] [lucene] gautamworah96 commented on a change in pull request #181: LUCENE-10001: Make CollectionTerminatedException handling in MultiCollector configurable

Posted by GitBox <gi...@apache.org>.
gautamworah96 commented on a change in pull request #181:
URL: https://github.com/apache/lucene/pull/181#discussion_r651418529



##########
File path: lucene/core/src/java/org/apache/lucene/search/MultiCollector.java
##########
@@ -25,13 +25,39 @@
 /**
  * A {@link Collector} which allows running a search with several {@link Collector}s. It offers a
  * static {@link #wrap} method which accepts a list of collectors and wraps them with {@link
- * MultiCollector}, while filtering out the <code>null</code> null ones.
+ * MultiCollector}, while filtering out the <code>null</code> ones.
  */
 public class MultiCollector implements Collector {
 
+  /**
+   * Defines the behavior to take when a wrapped {@link Collector} signals early termination by
+   * throwing a {@link CollectionTerminatedException}.
+   */
+  public enum EarlyTerminationBehavior {

Review comment:
       Can we name this `EarlyTerminationCollectorBehavior` everywhere? It just makes the name a bit more explicit

##########
File path: lucene/core/src/java/org/apache/lucene/search/MultiCollector.java
##########
@@ -78,14 +107,17 @@ public static Collector wrap(Iterable<? extends Collector> collectors) {
           colls[n++] = c;
         }
       }
-      return new MultiCollector(colls);
+      return new MultiCollector(earlyTerminationBehavior, colls);
     }
   }
 
+  private final EarlyTerminationBehavior earlyTerminationBehavior;
   private final boolean cacheScores;
   private final Collector[] collectors;
 
-  private MultiCollector(Collector... collectors) {
+  private MultiCollector(
+      EarlyTerminationBehavior earlyTerminationBehavior, Collector... collectors) {

Review comment:
       How about giving different levels of `EarlyTerminationBehavior` privileges to each collector? For example, a normal matching docs collector could terminate all collectors but a `FacetCollector` can only terminate itself. 
   
   This would mean this ctor would take in a `EarlyTerminationBehavior... earlyTerminationBehavior` array I think? Then hopefully while catching the exception we can check the collector privilege and accordingly throw the error or just continue

##########
File path: lucene/core/src/test/org/apache/lucene/search/TestMultiCollector.java
##########
@@ -105,14 +106,38 @@ public void testCollectionTerminatedExceptionHandling() throws IOException {
       final int numCollectors = TestUtil.nextInt(random(), 1, 5);
       for (int i = 0; i < numCollectors; ++i) {
         final int terminateAfter = random().nextInt(numDocs + 10);
-        final int expectedCount = terminateAfter > numDocs ? numDocs : terminateAfter;
+        final int expectedCount = Math.min(terminateAfter, numDocs);

Review comment:
       Neat!




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


[GitHub] [lucene] gsmiller commented on pull request #181: LUCENE-10001: Make CollectionTerminatedException handling in MultiCollector configurable

Posted by GitBox <gi...@apache.org>.
gsmiller commented on pull request #181:
URL: https://github.com/apache/lucene/pull/181#issuecomment-870083122


   I've put this in "Draft" state for now after a suggestion over on the Jira issue to approach this differently. I'll update here (and take out of "Draft") once I've had a chance to explore a different approach. Thanks everyone for the comments so far!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org