You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Tim Smith (JIRA)" <ji...@apache.org> on 2009/08/24 16:35:59 UTC

[jira] Created: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Add OutOfOrderCollector and InOrderCollector subclasses of Collector
--------------------------------------------------------------------

                 Key: LUCENE-1849
                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
             Project: Lucene - Java
          Issue Type: Wish
          Components: Search
    Affects Versions: 2.9
            Reporter: Tim Smith
            Priority: Minor
             Fix For: 2.9


I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)

would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747851#action_12747851 ] 

Michael McCandless commented on LUCENE-1849:
--------------------------------------------

We do need to index your memory cells!

Except: that entry is showing the [sizable] perf gains of disabling scoring when sorting by field (I think?). We were instead looking for the comparison of BooleanScorer vs BoolenScorer2.

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12746912#action_12746912 ] 

Shai Erera commented on LUCENE-1849:
------------------------------------

bq. (and bloats amount of required code by about 5 lines)

You can do it in 2 (or even 1) :)

{code}
@Override
public boolean acceptsDocsOutOfOrder() { return true/false; }
{code}

I don't mind if those two will be added. I still don't think they're that useful, but they don't do any harm either.

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747637#action_12747637 ] 

Michael McCandless commented on LUCENE-1849:
--------------------------------------------

bq. Any stats on the performance gains here available?

I think somewhere in LUCENE-1483 is the answer to this question, but I can't find it now.  (We need a whole new search index just for that issue alone!).  But the optimization currently only applies if the query contains just SHOULD terms plus up to 32 MUST_NOT terms.  More testing here would be great... the specialized search in LUCENE-1594 also uses out-of-order collection for certain AND queries but I haven't confirmed if/when that's a win.

{quote}
Also, are the implications of out of order docids coming back from nextDoc() well documented (javadoc?, wiki?)?
I guess out of order docids really screw up advance(int), so you should never call advance(int) if you allowed out of order collection for a Scorer?
{quote}

Out-of-order scoring is only used for top-scorers today in Lucene, so nextDoc/advance are not used.  (Though: BooleanScorer, which provides docs out-of-order, is I think able to nextDoc() through the out-of-order docs, but I don't think we take advantage of that anywhere, though we could do so to improve performance of BooleanQuery as a sub-clause of BooleanQuery).

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Tim Smith (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747046#action_12747046 ] 

Tim Smith commented on LUCENE-1849:
-----------------------------------

bq. we force them to think a little bit and then do what's best for them

the more you force people to think, the more likely they will come to the wrong solution (in my experience)

i love the power of the new Collector API, and i know how to take advantage of it to eek out the utmost performance where it matters or is possible.  But with some cases, i just want that AbstractCollector because it reduces my code complexity for subclasses and does everything i need without me introducing duplicated code

Also, the AbstractCollector makes it much easier to create anonymous subclasses of Collector (just one method to override) (i hate anonymous subclasses myself, but i see them used a lot inside lucene). I know in 2.4 there were tons of anonymous HitCollectors 

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12746926#action_12746926 ] 

Uwe Schindler commented on LUCENE-1849:
---------------------------------------

How about making acceptsDocsOutOfOrder()  no abstract and provide a default implementation that just returns a field that can only be set by a protected method? E.g. protected setAcceptsDocsOutOfOrder()  can be initialized by a ctor? Or make the field private and initialize it by a ctor in the abstract calss? By that cou can create a subclass of Collector like the following:
{code}
Collector c = new Collector(true/false) {
  public void collect(...)...
  ...
}
{code}
I would like this, as it is much simplier to use instead of always implementing this silly one-liner.

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747206#action_12747206 ] 

Shai Erera commented on LUCENE-1849:
------------------------------------

bq. I think we should simply do nothing

+1. So many use cases brought up in this issue, and so many "helper" classes. I just can't help imagining the future of this "I want a non scoring collector, which accepts docs out of order, and keeps its next reader ..." and we'll try very hard to create a helper class which does that because now we cannot inherit from two 'abstract' Collectors, each achieving part of it.

I also feel that when I'll upgrade all of HitCollectors to Collector, the decision on what to do where will not be that difficult - it's pretty straightforward I think. At least it's the same decision as "which collector should I subclass" ...

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Tim Smith (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747645#action_12747645 ] 

Tim Smith commented on LUCENE-1849:
-----------------------------------

bq. Out-of-order scoring is only used for top-scorers today in Lucene

I see that FilteredQuery passes "scoreDocsInOrder" down to its sub query
Is this incorrect?
seems like this could cause problems as FilteredQuery does call nextDoc/advance on its sub query (which could be out of order because of this)

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747805#action_12747805 ] 

Shai Erera commented on LUCENE-1849:
------------------------------------

bq. I think somewhere in LUCENE-1483 is the answer to this question

I tracked it down to LUCENE-1575 which was the huge refactoring to HitCollector issue: https://issues.apache.org/jira/browse/LUCENE-1575?focusedCommentId=12695784&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12695784
(perhaps we should index my memory cells? :) ).

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Resolved: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless resolved LUCENE-1849.
----------------------------------------

    Resolution: Won't Fix

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747029#action_12747029 ] 

Shai Erera commented on LUCENE-1849:
------------------------------------

How would this flag reduce any "code bloat" in subclasses? They now have to call super() in their ctors.

And TSDC and TFC will need to add a parameter to their private ctors, so that they can pass this parameter on to TopDocsCollector, which will need to pass it on to Collector ...

I think this will get pretty messy and complicated.

What exactly are we trying to solve here? 5 lines of code that every Collector developer (how many are out there anyway?) need to invest 3 minutes thinking about them before he codes that method?

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747584#action_12747584 ] 

Michael McCandless commented on LUCENE-1849:
--------------------------------------------

I would prefer not to make a default here, ie, force an explicit
choice, because it is an expert API.

BooleanQuery gets sizable gains in performance if you let it return
docs out of order.  If we default it (to false), most users won't
realize how much performance they're accidentally losing.


> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Tim Smith (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12746920#action_12746920 ] 

Tim Smith commented on LUCENE-1849:
-----------------------------------

People tend to always reformat single line functions like that to use at lest 2 more lines (i think checkstyle/eclipse formatting will often screw up my compact code if someone else ever touches it))
also, you need the extra line for javadoc, so thats always 5 lines :(

I can always add these to classes to my class hierarchy (and i probably will if it doesn't get added to lucene's search package)
but i think these are in general useful to anyone implementing collectors

a typical person porting to 2.9 can switch their HitCollector to subclass InOrderCollector instead (in order to keep getting docs in order like lucene 2.4)
This then means they don't need to even think about acceptDocsOutOfOrder() semantics unless they really want to
Also one less method to implement incorrectly for us application developers :)

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747427#action_12747427 ] 

Yonik Seeley commented on LUCENE-1849:
--------------------------------------

bq. I think we should simply do nothing. This is an expert API.

+1

The reality is that order will matter for some custom collectors, and the abstract method makes that clear.
The API is fine as it is, but if we wanted to provide a default in Collector, that would be fine too.
*If* we were to provide a default in Collector, it should be a simple constant, not a variable.  Subclasses changing would simply override the method as normal.  We should strive to treat our abstract base classes as interfaces, and adding fields and final methods should be avoided.

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Tim Smith (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12746928#action_12746928 ] 

Tim Smith commented on LUCENE-1849:
-----------------------------------

I like the idea of this flag being private final and initialized via a Collector constructor

Collector.acceptDocsOutOfOrder() should then be made final though? (otherwise each collector has a boolean flag that may never be used if a subclass implements acceptDocsOutOfOrder() its own way)

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Issue Comment Edited: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747054#action_12747054 ] 

Uwe Schindler edited comment on LUCENE-1849 at 8/24/09 1:41 PM:
----------------------------------------------------------------

There should also be not only the variation that simply stores docBase in a field. I normally never used that, I always store the reader in a field and then directly operate on the subreader (e.g. for retriving values from the field cache). For me global docIds are unimportant (at least in my collectors). Because of this problem, I had created exactly such a abstract helper class handling this.

For the special case FieldCache, the proposed abstract class simply storing all set values into protected fields does not make sense. This FieldCache-using collectors normally override setNextReader(), where they retrieve the needed arrays from the field cache (whenever a new reader occurs, you can switch your value arrays to the next reader instantly). Collector then only operates on the field cache array, the reader is unimportant. And even the docBase is unimportant. I have such an collector, that uses numerical document ids from the relational database out of the field cache.

Lot's of use-cases... Maybe we should put these helper classes into contrib/queries?

      was (Author: thetaphi):
    There should also be not only the variation that simply stores docBase in a field. I normally never used that, I always store the reader in a field and then directly operate on the subreader (e.g. for retriving values from the field cache). For me global docIds are unimportant (at least in my collectors).

Because of this problem, I had created exactly such a abstract helper class handling this.
  
> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Tim Smith (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747441#action_12747441 ] 

Tim Smith commented on LUCENE-1849:
-----------------------------------

bq. If we were to provide a default in Collector, it should be a simple constant, not a variable.

in that case, it may be useful to have this method return "false" by default (expecting docs in order, as this is the "default" in 2.4)


> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Tim Smith (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747028#action_12747028 ] 

Tim Smith commented on LUCENE-1849:
-----------------------------------

will do

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747026#action_12747026 ] 

Michael McCandless commented on LUCENE-1849:
--------------------------------------------

bq. protected instead please, 

OK.  Can you cons up a patch here?

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Tim Smith (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747021#action_12747021 ] 

Tim Smith commented on LUCENE-1849:
-----------------------------------

bq. Or just make it package private? This flag is only used by oal.search.* to mate the right scorer to the collector.

protected instead please, 
Collector subclasses should be able to inspect this value if they want/need to



> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747874#action_12747874 ] 

Shai Erera commented on LUCENE-1849:
------------------------------------

Yes I know that - I remember that you once ran w/ BS vs. BS2 and thought the results were reported in that issue. But I've scanned it and I don't find it. Perhaps it was in an email, but I seem to remember you reported ~20-30% improvement in favor of BS. I'll try to digg it up from the bottom of my memory pit.

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12746904#action_12746904 ] 

Shai Erera commented on LUCENE-1849:
------------------------------------

But then what will classes like TopFieldCollector and TopScoreDocCollector will do? They won't be able to extend from any of them because a Collector may support both, one impl more efficient than the other.

Of course it doesn't mean that every Collector needs to extend either of these, but I wonder if it's such a big deal to decide on your own, once you've decided to write a Collector, if you support it or not.

Those two do not do any harm, but we'll just need to maintain them going forward, and I wonder if they are really that useful ...

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747050#action_12747050 ] 

Shai Erera commented on LUCENE-1849:
------------------------------------

If we want to help people impl Collector, then let's at least make a useful subclass, like ScoringCollector which impls setScorer and setNextReader by keeping Scorer and docBase. It will leave you to impl collect() however you want.

But just have an AbstractCollector which stores everything that the setter methods passed does not look a good idea to me. What do you think?

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747042#action_12747042 ] 

Shai Erera commented on LUCENE-1849:
------------------------------------

But if we encourage people to extend AbstractCollector, they'll incur some overhead we've tried to eliminate. Why have a setScorer impl which does something, even if just member assignment, if I don't need the score?

Keeping Scorer private is not good either, because that'll add a method call for me (score() vs. scorer.score()). This is also something we've tried to avoid in all these "specialized" impls in TSDC/TFC.

Part of the new Collector API is to help Collector developers impl the best Collector for their needs. By having those methods abstract, we force them to think a little bit and then do what's best for them. If I don't need a score, I don't even want to have a Scorer member in my Collector (which consumes memory, even if just a pointer).

I'm afraid that by introducing this AbstractCollector, we won't encourage people to impl the best Collector for their needs. They'll just impl collect() and use whatever is available if they need to. And really, IMO, this AbstractCollector saves me nothing - I don't mind writing this.scorer = scorer if I need Scorer. And I don't think the current Collector API makes it any more harder to write a Collector ... but perhaps I'm wrong.

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Tim Smith (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12746907#action_12746907 ] 

Tim Smith commented on LUCENE-1849:
-----------------------------------

They would be convenience classes for people implementing their own Collectors (as i am)

just kinda a pain (and bloats amount of required code by about 5 lines) to have to always implement this method (when it could be inherited easily from a parent class)

Just throwing this out as an idea to see if anyone else likes it (thats why i marked it as a _Wish_)

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747067#action_12747067 ] 

Michael McCandless commented on LUCENE-1849:
--------------------------------------------

I think we should simply do nothing.  This is an expert API.  There are a many ways it's going to be used.  Each app can make helper classes if it wants, but I think Lucene should simply offer Collector.

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Tim Smith (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747075#action_12747075 ] 

Tim Smith commented on LUCENE-1849:
-----------------------------------

bq. I think we should simply do nothing. This is an expert API.

i'm ok with that

just thought this idea would potentially be of general use for other developers, but it probably gets more complex adding all the variations for subclasses of Collector and maybe even more confusing that just the raw Collector API


> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747678#action_12747678 ] 

Michael McCandless commented on LUCENE-1849:
--------------------------------------------

Good catch!  It should not be doing that.  It should be passing "true" down since it will use advance.

But it doesn't result in a bug, because 1) its Weight.scoresDocsOutOfOrder returns false, which causes IndexSearcher's sort by relevance or field methods to pick an in-order collector which then causes IndexSearcher to pass "true" for scoreDocsInOrder, and 2) it's passing down "false" for topScorer, which causes BooleanQuery to not return an out-of-order scorer.

I'll commit a fix plus a test shortly.

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Tim Smith (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747587#action_12747587 ] 

Tim Smith commented on LUCENE-1849:
-----------------------------------

bq. I would prefer not to make a default here, ie, force an explicit choice, because it is an expert API. 

very reasonable

bq. BooleanQuery gets sizable gains in performance if you let it return docs out of order.

Any stats on the performance gains here available?
didn't see any on a cursory glance through javadoc

Also, are the implications of out of order docids coming back from nextDoc() well documented (javadoc?, wiki?)?
I guess out of order docids really screw up advance(int), so you should never call advance(int) if you allowed out of order collection for a Scorer?


> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Tim Smith (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747039#action_12747039 ] 

Tim Smith commented on LUCENE-1849:
-----------------------------------

bq. I think this will get pretty messy and complicated.

yeah, this is a bit messy with the chain of inheritance in these classes (as each variant is "slightly" optimized depending on in order/out of order collection)

makes me go back to favoring InOrderCollector/OutOfOrderCollector abstract classes
or maybe just one "AbstractCollector" method which implements all methods except collect()

like so:
{code}
public abstract class AbstractCollector extends Collector {
  private final boolean allowDocsOutOfOrder;
  protected IndexReader reader;
  protected Scorer scorer;
  protected int docBase;  

  public AbstractCollector() {
    this(false);
  }

  public AbstractCollector(boolean allowDocsOutOfOrder) {
    this.allowDocsOutOfOrder = allowDocsOutOfOrder;
  }

  public void setNextReader(IndexReader reader, int docBase) {
    this.reader = reader;
    this.docBase = docBase;
  }

  public void setScorer(Scorer scorer) {
    this.scorer = scorer;
  }

  public final boolean acceptsDocsOutOfOrder() {
    return allowDocsOutOfOrder;
  }
}
{code}

bq. What exactly are we trying to solve here?
the Collector methodology has grown more complicated (because it does more to handle per segment searches)
the HitCollector api was nice and simple

this AbstractCollector (insert better name here) gets things back to being more simple
could even hide the "Scorer" as private and provide a "score()" method that returns the score for the current document, and otherwise simplify this even more

subclassing AbstractCollector instead of Collector makes it so most of the "required common things" are done for you
otherwise, every single Collector will do virtually the same thing is done in AbstractCollector here (as far as setup/etc)

Again, this is just a _Wish_ which i've thought of as i've been working through the new collector API (and found myself doing the exact same thing for every implementation of Collector)


> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Tim Smith (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747051#action_12747051 ] 

Tim Smith commented on LUCENE-1849:
-----------------------------------

i was just proposing AbstractCollector to consolidate the "variations" of abstract subclasses of Collector

I like ScoringCollector
i would also like a NonScoringCollector

in this case, i would recommend both should take the "allowDocsOutOfOrder" flag in their constructors (and store in a private final returned by acceptingOutOfOrderDocs())
otherwise, i would still want to see 2 variations on each of ScoringCollector and NonScoringCollector to handle the OutOfOrder vs InOrder variations




> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Tim Smith (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747059#action_12747059 ] 

Tim Smith commented on LUCENE-1849:
-----------------------------------

I guess the question is: what variations do we provide helper Collector implementations for?

seems like there's a bunch of possibilities (depending on how far you go)

thats why i initially proposed AbstractCollector (storing everything that was set (IndexReader, Scorer, docBase))
the amount of memory and time used to set 2 pointers and an int per segment almost seems irrelevant for this Collector implementation aid (and if you really care about those few bytes and cpu cycles, you can directly implement Collector)


> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747054#action_12747054 ] 

Uwe Schindler commented on LUCENE-1849:
---------------------------------------

There should also be not only the variation that simply stores docBase in a field. I normally never used that, I always store the reader in a field and then directly operate on the subreader (e.g. for retriving values from the field cache). For me global docIds are unimportant (at least in my collectors).

Because of this problem, I had created exactly such a abstract helper class handling this.

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Commented: (LUCENE-1849) Add OutOfOrderCollector and InOrderCollector subclasses of Collector

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12746971#action_12746971 ] 

Michael McCandless commented on LUCENE-1849:
--------------------------------------------

bq. I like the idea of this flag being private final and initialized via a Collector constructor

This seems OK to me.

bq. Collector.acceptDocsOutOfOrder() should then be made final though?

Or just make it package private?  This flag is only used by oal.search.* to mate the right scorer to the collector.

> Add OutOfOrderCollector and InOrderCollector subclasses of Collector
> --------------------------------------------------------------------
>
>                 Key: LUCENE-1849
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1849
>             Project: Lucene - Java
>          Issue Type: Wish
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>            Priority: Minor
>             Fix For: 2.9
>
>
> I find myself always having to implement these methods, and i always return a constant (depending on if the collector can handle out of order hits)
> would be nice for these two convenience abstract classes to exist that implemented acceptsDocsOutOfOrder() as final and returned the appropriate value

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org