You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (JIRA)" <ji...@apache.org> on 2009/04/14 18:36:15 UTC
[jira] Created: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
------------------------------------------------------------------------
Key: LUCENE-1603
URL: https://issues.apache.org/jira/browse/LUCENE-1603
Project: Lucene - Java
Issue Type: Improvement
Affects Versions: 2.4, 2.9
Reporter: Uwe Schindler
This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
- Make the private members protected, to have access to them from the very special TrieRangeTermEnum
- Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
- Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
- Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
{code}
[junit] Average number of terms during random search on 'field8':
[junit] Trie query: 244.2
[junit] Classical query: 3136.94
[junit] Average number of terms during random search on 'field4':
[junit] Trie query: 38.3
[junit] Classical query: 3018.68
[junit] Average number of terms during random search on 'field2':
[junit] Trie query: 18.04
[junit] Classical query: 3539.42
{code}
All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699660#action_12699660 ]
Michael McCandless commented on LUCENE-1603:
--------------------------------------------
I think the name is good, so it's clear you have to provide a MultiTermQuery yourself (via subclass) to use it.
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch, LUCENE-1603.patch, LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12698852#action_12698852 ]
Uwe Schindler commented on LUCENE-1603:
---------------------------------------
Maybe declare the term number as transient, it should not be serialized...
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Reopened: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler reopened LUCENE-1603:
-----------------------------------
To also expose getTotalTermCount to the Filter counterparts, I provide a new supercalss for those filters MultiTermQueryWrapperFilter (name to be discussed).
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch, LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1603:
----------------------------------
Attachment: LUCENE-1603.patch
Patch that provides that filter.
Mike: Can you please test drop-in-backwards-compatibility with this patch applied, with Windows, the checkout through Ant does not work correctly? I also set the native line ending svn property of the new file in the patch.
The update of the TrieRange follows after this is committed, code not affected, compiles still fine.
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch, LUCENE-1603.patch, LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1603:
----------------------------------
Attachment: LUCENE-1603.patch
The patch is here. Maybe the javadocs of getLastNumberOfTerms could be improved...
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Attachments: LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699647#action_12699647 ]
Michael McCandless commented on LUCENE-1603:
--------------------------------------------
Patch looks good -- I'll commit shortly. Thanks Uwe!
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch, LUCENE-1603.patch, LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699087#action_12699087 ]
Uwe Schindler commented on LUCENE-1603:
---------------------------------------
Just for completeness: Should we add the number of terms methods also to the *Filter counterparts of the MultiTermQueries (as the new methods are only automatically appear in subclasses, but not in related pass-to-query-only classes)?
In trie-range I have these pass-to-query methods.
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch, LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-1603.
----------------------------------------
Resolution: Fixed
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch, LUCENE-1603.patch, LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12698887#action_12698887 ]
Michael McCandless commented on LUCENE-1603:
--------------------------------------------
Looks good Uwe, thanks. I plan to commit shortly. I changed one sentence in the javadoc to:
{code}
* If you re-use the same query instance for another
* search, be sure to first reset the term counter
* with {@link #clearTotalNumberOfTerms}.
{code}
(It was missing the "reuse" part, making it sound like you always must call clear first).
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch, LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12698831#action_12698831 ]
Michael McCandless commented on LUCENE-1603:
--------------------------------------------
Patch looks good. Should we allow lastNumberOfTerms to be the sum of all invocations? (Instead of clearing it per segment)? And maybe add a resetLastNumberOfTerms, in case one wants to re-use a MultiTermQuery and recheck that count.
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-1603.
----------------------------------------
Resolution: Fixed
Thanks Uwe!
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch, LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12698851#action_12698851 ]
Michael McCandless commented on LUCENE-1603:
--------------------------------------------
I was thinking that this count is a good way to measure how much net work was done, hence the switch to sum. EG you could compare that count vs the count you get after having optimized the index to get a sense of how much you gained by optimizing.
Whereas now, with the count only showing the # terms from the last segment searched, is not really useful at all.
bq. Are queries also rewritten per segment with the new Searchers? If not, one could use the BooleanQuery variant, if he wants to have real term numbers on unoptimized index.
They are rewritten at the MultiReader level, so you're right one could use that to get "number of unique terms" vs "amount of work (seeks) done".
If we do change it, ow about "get/clearTotalNumberOfTerms()"?
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1603:
----------------------------------
Attachment: LUCENE-1603.patch
Here the patch with the suggested changes.
TrieRange test changes and so on come later.
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch, LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699088#action_12699088 ]
Uwe Schindler commented on LUCENE-1603:
---------------------------------------
An idea how to achive this, would be to make the *Filter counterparts subclasses of a new superclass MutiTermFilter, that just passes all methods to the corresponding query. This would make the filter classes simplier and one must only add the new methods to this superclass not to every filter class manually.
Should I add a patch?
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch, LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12698846#action_12698846 ]
Uwe Schindler commented on LUCENE-1603:
---------------------------------------
This is also another idea. The sum is interesting, too, but not so helpful, if the segments share the same terms. But for the number of seeks in TermDocs its interesting. Even if it sums up, the numbers in the Trie tests are the same, because index is optimized. So the note could be: For real numbers to analyze your query, use a optimized index, for real seek counts on a complex index/query structure, do not optimize.
By the way: Are queries also rewritten per segment with the new Searchers? If not, one could use the BooleanQuery variant, if he wants to have real term numbers on unoptimized index.
I am +/- 0 about the change. In case of a change, the method should be called getCurrentNumberOfTerms() or something like that -- naming is the hardest one.
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless reassigned LUCENE-1603:
------------------------------------------
Assignee: Michael McCandless
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699654#action_12699654 ]
Uwe Schindler commented on LUCENE-1603:
---------------------------------------
Do you think the name is good? MultiTermQueryWrapperFilter or simplier MultiTermFilter? Its not really one of both, its a mix between wrapper and the real filter: It wraps the query, but does the getDocIdSet and TermEnums himself.
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch, LUCENE-1603.patch, LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699169#action_12699169 ]
Michael McCandless commented on LUCENE-1603:
--------------------------------------------
bq. An idea how to achive this, would be to make the *Filter counterparts subclasses of a new superclass MutiTermFilter, that just passes all methods to the corresponding query.
That'd be great -- can you reopen this & attach patch?
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch, LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1603) Changes for TrieRange in
FilteredTermEnum and MultiTermQuery improvement
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-1603:
---------------------------------------
Fix Version/s: 2.9
> Changes for TrieRange in FilteredTermEnum and MultiTermQuery improvement
> ------------------------------------------------------------------------
>
> Key: LUCENE-1603
> URL: https://issues.apache.org/jira/browse/LUCENE-1603
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.4, 2.9
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1603.patch
>
>
> This is a patch, that is needed for the MultiTermQuery-rewrite of TrieRange (LUCENE-1602):
> - Make the private members protected, to have access to them from the very special TrieRangeTermEnum
> - Fix a small inconsistency (docFreq() now only returns a value, if a valid term is existing)
> - Improvement of MultiTermFilter.getDocIdSet to return DocIdSet.EMPTY_DOCIDSET, if the TermEnum is empty (less memory usage) and faster.
> - Add the getLastNumberOfTerms() to MultiTermQuery for statistics on different multi term queries and how may terms they affect, using this new functionality, the improvement of TrieRange can be shown (extract from test case there, 10000 docs index, long values):
> {code}
> [junit] Average number of terms during random search on 'field8':
> [junit] Trie query: 244.2
> [junit] Classical query: 3136.94
> [junit] Average number of terms during random search on 'field4':
> [junit] Trie query: 38.3
> [junit] Classical query: 3018.68
> [junit] Average number of terms during random search on 'field2':
> [junit] Trie query: 18.04
> [junit] Classical query: 3539.42
> {code}
> All core tests pass.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org