You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Robert Muir (JIRA)" <ji...@apache.org> on 2014/09/11 14:07:34 UTC

[jira] [Commented] (LUCENE-5938) New DocIdSet implementation with random write access

    [ https://issues.apache.org/jira/browse/LUCENE-5938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14129937#comment-14129937 ] 

Robert Muir commented on LUCENE-5938:
-------------------------------------

This also sounds like a very good candidate for MultiTermQueryWrapperFilter (used by "filter" rewrite)? 

I think last i looked, the main downside to filter rewrite is for sparse queries. Because of this, its faster for those queries to do a boolean rewrite, and there is complex logic ("auto rewrite") that first rewrites as booleanquery, then if it hits too many terms / too much density it "starts over" as filter rewrite. Perhaps we can just replace all of that with this bitset?

> New DocIdSet implementation with random write access
> ----------------------------------------------------
>
>                 Key: LUCENE-5938
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5938
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Adrien Grand
>            Assignee: Adrien Grand
>         Attachments: LUCENE-5938.patch
>
>
> We have a great cost API that is supposed to help make decisions about how to best execute queries. However, due to the fact that several of our filter implementations (eg. TermsFilter and BooleanFilter) return FixedBitSets, either we use the cost API and make bad decisions, or need to fall back to heuristics which are not as good such as RandomAccessFilterStrategy.useRandomAccess which decides that random access should be used if the first doc in the set is less than 100.
> On the other hand, we also have some nice compressed and cacheable DocIdSet implementation but we cannot make use of them because TermsFilter requires a DocIdSet that has random write access, and FixedBitSet is the only DocIdSet that we have that supports random access.
> I think it would be nice to replace FixedBitSet in those filters with another DocIdSet that would also support random write access but would have a better cost?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org