You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Otis Gospodnetic (JIRA)" <ji...@apache.org> on 2007/04/18 00:05:18 UTC
[jira] Resolved: (LUCENE-730) Restore top level disjunction
performance
[ https://issues.apache.org/jira/browse/LUCENE-730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Otis Gospodnetic resolved LUCENE-730.
-------------------------------------
Resolution: Fixed
Lucene Fields: [New, Patch Available] (was: [Patch Available, New])
I've committed this (changed a few minor things in the patch)...without benchmarking BS vs. BS2 with < 32 prohibited clauses.
Hm, if I exposed that 32 as a static setter method, then one could easily benchmark and compare BS vs. BS2 with Doron's contrib/benchmark.
> Restore top level disjunction performance
> -----------------------------------------
>
> Key: LUCENE-730
> URL: https://issues.apache.org/jira/browse/LUCENE-730
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Reporter: Paul Elschot
> Priority: Minor
> Attachments: TopLevelDisjunction20061127.patch
>
>
> This patch restores the performance of top level disjunctions.
> The introduction of BooleanScorer2 had impacted this as reported
> on java-user on 21 Nov 2006 by Stanislav Jordanov.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Resolved: (LUCENE-730) Restore top level disjunction performance
Posted by Michael Busch <bu...@gmail.com>.
Chris Hostetter wrote:
> : With this committed it also makes sense to deprecate the setUseScorer14()
> : method and the corresponding get...() method. If you want a patch for that,
> : I'll gladly provide one.
>
> i haven't really been able to follow this issue as much as i would like,
> but docs now sometimes coming out of order and the need to hobble
> QueryUtils.check(Query,Searcher) because of this alarms me a bit ... i
> can't really think of all the cases this might cause problems for people,
> but i'm, sure there may be some (i remember it was kind of a big deal when
> BooleanScorer2 came out and people could start relying on docs coming in
> order ... it's why ConstantScoreRangeQuery because a wrapper arround
> ConstantScoreQuery -- the first version i wrote collected dos in order
> they were found while walking the TermEnum/TermDocs and people didn't like
> that this ment they weren't in order)
>
I agree. I also believe that a lot of people rely on docs in order (I do
too in some cases). I think before we make a 2.2 release we should
change this, so that the docs are still in order per default.
> perhaps this patch should be changed to only have this effect
> (BooleanScorer2 delegating to BooleanScorer) only if
> BooleanQuery.setUseScorer14(true) has been called -- and the existing use
> of BooleanQuery.getUseScorer14() to decide between BooleanWeight and
> BooleanWeight2 can be removed (and BooleanWeight can be deprecated)
>
> that way people who are okay with getting docs out of order can call
> BooleanQuery.setUseScorer14(true) and get the performance benefits when
> possible, but people who want to be sure they get documents in order have
> to accept that in some cases their queries arent' as fast asthey could be.
>
>
+1. I think this makes sense. We can implement it this way for now, and
maybe in the future we can deprecate setUseScorer14() and add something
like
void allowDocsOutOfOrer(boolean allow);
to Query as Doron suggested. I'm going to reopen LUCENE-730.
> ...i believe what i'm suggesting would keep the fundemental meaning of
> setUseScorer(true), even if the value is now used in a slightly differnet
> place ... correct?
>
>
> -Hoss
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Resolved: (LUCENE-730) Restore top level disjunction
performance
Posted by Chris Hostetter <ho...@fucit.org>.
: Also, it would be nice to make this a (non static) method of Query, and, I
: think, also of Weight and Scorer (Weight would "like-inherit" that property
: from its Query, and Scorer from its Weight). The default would be
as i recall, the staticness was so that people writing apps could garuntee
they would allways get the behavior they wanted (ie: docs in order)
regardless of what code path created the BooleanQuery (QueryParser,
PrefixQuery.rewrite etc..)
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Resolved: (LUCENE-730) Restore top level disjunction performance
Posted by Doron Cohen <DO...@il.ibm.com>.
Chris Hostetter <ho...@fucit.org> wrote on 19/04/2007 16:01:53:
>
> : With this committed it also makes sense to deprecate the
setUseScorer14()
> : method and the corresponding get...() method. If you want a patch for
that,
> : I'll gladly provide one.
>
> i haven't really been able to follow this issue as much as i would like,
> but docs now sometimes coming out of order and the need to hobble
> QueryUtils.check(Query,Searcher) because of this alarms me a bit ... i
> can't really think of all the cases this might cause problems for people,
> but i'm, sure there may be some (i remember it was kind of a big deal
when
> BooleanScorer2 came out and people could start relying on docs coming in
> order ... it's why ConstantScoreRangeQuery because a wrapper arround
> ConstantScoreQuery -- the first version i wrote collected dos in order
> they were found while walking the TermEnum/TermDocs and people didn't
like
> that this ment they weren't in order)
>
> perhaps this patch should be changed to only have this effect
> (BooleanScorer2 delegating to BooleanScorer) only if
> BooleanQuery.setUseScorer14(true) has been called -- and the existing use
> of BooleanQuery.getUseScorer14() to decide between BooleanWeight and
> BooleanWeight2 can be removed (and BooleanWeight can be deprecated)
>
> that way people who are okay with getting docs out of order can call
> BooleanQuery.setUseScorer14(true) and get the performance benefits when
> possible, but people who want to be sure they get documents in order have
> to accept that in some cases their queries arent' as fast asthey could
be.
>
> ...i believe what i'm suggesting would keep the fundemental meaning of
> setUseScorer(true), even if the value is now used in a slightly differnet
> place ... correct?
Amazing timing! - just as I was working at 697 trying to un-cancel the
checking skipTo...
This suggestion would help in 697 of course, but I think it also makes
sense in general.
Could I modify it a little - to something more general than "use scorer
number J" - perhaps something like:
void allowDocsOutOfOrer(boolean allow);
boolean isAllowedDocsOutOfOrder();
Also, it would be nice to make this a (non static) method of Query, and, I
think, also of Weight and Scorer (Weight would "like-inherit" that property
from its Query, and Scorer from its Weight). The default would be
"disallowed", and the documentation would say that allowing may enable
further optimization, though in some cases (like currently non boolean
scorers or ones with more than 32 prohibited sub scorers) this might have
no effect at all.
Doron
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Resolved: (LUCENE-730) Restore top level disjunction
performance
Posted by Chris Hostetter <ho...@fucit.org>.
: With this committed it also makes sense to deprecate the setUseScorer14()
: method and the corresponding get...() method. If you want a patch for that,
: I'll gladly provide one.
i haven't really been able to follow this issue as much as i would like,
but docs now sometimes coming out of order and the need to hobble
QueryUtils.check(Query,Searcher) because of this alarms me a bit ... i
can't really think of all the cases this might cause problems for people,
but i'm, sure there may be some (i remember it was kind of a big deal when
BooleanScorer2 came out and people could start relying on docs coming in
order ... it's why ConstantScoreRangeQuery because a wrapper arround
ConstantScoreQuery -- the first version i wrote collected dos in order
they were found while walking the TermEnum/TermDocs and people didn't like
that this ment they weren't in order)
perhaps this patch should be changed to only have this effect
(BooleanScorer2 delegating to BooleanScorer) only if
BooleanQuery.setUseScorer14(true) has been called -- and the existing use
of BooleanQuery.getUseScorer14() to decide between BooleanWeight and
BooleanWeight2 can be removed (and BooleanWeight can be deprecated)
that way people who are okay with getting docs out of order can call
BooleanQuery.setUseScorer14(true) and get the performance benefits when
possible, but people who want to be sure they get documents in order have
to accept that in some cases their queries arent' as fast asthey could be.
...i believe what i'm suggesting would keep the fundemental meaning of
setUseScorer(true), even if the value is now used in a slightly differnet
place ... correct?
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Resolved: (LUCENE-730) Restore top level disjunction performance
Posted by Paul Elschot <pa...@xs4all.nl>.
On Wednesday 18 April 2007 00:05, Otis Gospodnetic (JIRA) wrote:
>
>
[ https://issues.apache.org/jira/browse/LUCENE-730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Otis Gospodnetic resolved LUCENE-730.
> -------------------------------------
>
> Resolution: Fixed
> Lucene Fields: [New, Patch Available] (was: [Patch Available, New])
>
> I've committed this (changed a few minor things in the patch)...without
benchmarking BS vs. BS2 with < 32 prohibited clauses.
With this committed it also makes sense to deprecate the setUseScorer14()
method and the corresponding get...() method. If you want a patch for that,
I'll gladly provide one.
Actually I prefer to have these methods removed altogether now, but that is
probably not compatible with the release policy.
Regards,
Paul Elschot
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org