You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Robert Muir (JIRA)" <ji...@apache.org> on 2010/11/25 21:29:13 UTC
[jira] Created: (LUCENE-2780) optimize spantermquery
optimize spantermquery
----------------------
Key: LUCENE-2780
URL: https://issues.apache.org/jira/browse/LUCENE-2780
Project: Lucene - Java
Issue Type: Improvement
Components: Search
Reporter: Robert Muir
Fix For: 3.1, 4.0
Looking at http://www.lucidimagination.com/search/document/c2c6f660ddde4f7f/dismaxqparserplugin_and_tokenization ,
I saw a user building DisjunctionMaxQuery / BooleanQuery with SpanTermQuerys.
I wonder if users know that doing this is much slower than just using TermQuery?
I agree it makes little sense to use SpanTermQuery if you arent going to use it inside a SpanNear etc,
but on the other hand, I think its a little non-intuitive that it wouldnt be just as fast in a case like this.
I could see this complicating queryparsing etc for users that want to sometimes use positions etc.
SpanTermQuery is the same as TermQuery, except tf is computed as (#of spans * sloppyFreq(spanLength)
For this case, #ofspans = tf and spanLength for a single term is always 1.
Maybe we should optimize SpanTermQuery to return TermScorer, with just this special tf computation.
This would avoid reading positions for anyone that does this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2780) optimize spantermquery
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2780:
--------------------------------
Fix Version/s: (was: 3.1)
> optimize spantermquery
> ----------------------
>
> Key: LUCENE-2780
> URL: https://issues.apache.org/jira/browse/LUCENE-2780
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-2780.patch
>
>
> Looking at http://www.lucidimagination.com/search/document/c2c6f660ddde4f7f/dismaxqparserplugin_and_tokenization ,
> I saw a user building DisjunctionMaxQuery / BooleanQuery with SpanTermQuerys.
> I wonder if users know that doing this is much slower than just using TermQuery?
> I agree it makes little sense to use SpanTermQuery if you arent going to use it inside a SpanNear etc,
> but on the other hand, I think its a little non-intuitive that it wouldnt be just as fast in a case like this.
> I could see this complicating queryparsing etc for users that want to sometimes use positions etc.
> SpanTermQuery is the same as TermQuery, except tf is computed as (#of spans * sloppyFreq(spanLength)
> For this case, #ofspans = tf and spanLength for a single term is always 1.
> Maybe we should optimize SpanTermQuery to return TermScorer, with just this special tf computation.
> This would avoid reading positions for anyone that does this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2780) optimize spantermquery
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12935972#action_12935972 ]
Michael McCandless commented on LUCENE-2780:
--------------------------------------------
Looks good Robert! It's a sneaky trap. Maybe add a comment to createWeight explaining that this is only used when a "normal" (non-span) Query embeds a SpanTermQuery?
Someday we need to merge Span* back into the "normal" queries.
> optimize spantermquery
> ----------------------
>
> Key: LUCENE-2780
> URL: https://issues.apache.org/jira/browse/LUCENE-2780
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Reporter: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2780.patch
>
>
> Looking at http://www.lucidimagination.com/search/document/c2c6f660ddde4f7f/dismaxqparserplugin_and_tokenization ,
> I saw a user building DisjunctionMaxQuery / BooleanQuery with SpanTermQuerys.
> I wonder if users know that doing this is much slower than just using TermQuery?
> I agree it makes little sense to use SpanTermQuery if you arent going to use it inside a SpanNear etc,
> but on the other hand, I think its a little non-intuitive that it wouldnt be just as fast in a case like this.
> I could see this complicating queryparsing etc for users that want to sometimes use positions etc.
> SpanTermQuery is the same as TermQuery, except tf is computed as (#of spans * sloppyFreq(spanLength)
> For this case, #ofspans = tf and spanLength for a single term is always 1.
> Maybe we should optimize SpanTermQuery to return TermScorer, with just this special tf computation.
> This would avoid reading positions for anyone that does this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2780) optimize spantermquery
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2780:
--------------------------------
Attachment: LUCENE-2780.patch
patch
> optimize spantermquery
> ----------------------
>
> Key: LUCENE-2780
> URL: https://issues.apache.org/jira/browse/LUCENE-2780
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Reporter: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2780.patch
>
>
> Looking at http://www.lucidimagination.com/search/document/c2c6f660ddde4f7f/dismaxqparserplugin_and_tokenization ,
> I saw a user building DisjunctionMaxQuery / BooleanQuery with SpanTermQuerys.
> I wonder if users know that doing this is much slower than just using TermQuery?
> I agree it makes little sense to use SpanTermQuery if you arent going to use it inside a SpanNear etc,
> but on the other hand, I think its a little non-intuitive that it wouldnt be just as fast in a case like this.
> I could see this complicating queryparsing etc for users that want to sometimes use positions etc.
> SpanTermQuery is the same as TermQuery, except tf is computed as (#of spans * sloppyFreq(spanLength)
> For this case, #ofspans = tf and spanLength for a single term is always 1.
> Maybe we should optimize SpanTermQuery to return TermScorer, with just this special tf computation.
> This would avoid reading positions for anyone that does this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org