You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (JIRA)" <ji...@apache.org> on 2011/06/18 13:27:47 UTC
[jira] [Created] (LUCENE-3212) Supply FilterIndexReader based on
any o.a.l.search.Filter
Supply FilterIndexReader based on any o.a.l.search.Filter
---------------------------------------------------------
Key: LUCENE-3212
URL: https://issues.apache.org/jira/browse/LUCENE-3212
Project: Lucene - Java
Issue Type: Improvement
Components: core/index, core/search
Affects Versions: 4.0
Reporter: Uwe Schindler
Assignee: Uwe Schindler
Fix For: 4.0
When coding LUCENE-2919 (PKIndexSplitter), Mike and me had the idea, how to effectively apply filters on the lowest level (before query execution). This is very useful for e.g. security Filters that simply hide some documents. Currently when you apply the filter after searching, lots of useless work was done like scoring filtered documents, iterating term positions (for Phrases),...
This patch will provide a FilterIndexReader subclass (4.0 only, 3.x is too complicated to implement), that hides filtered documents by returning them in getDeletedDocs(). In contrast to LUCENE-2919, the filtering will work on per-segment (without SlowMultiReaderWrapper), so per segment search keeps available and reopening can be done very efficient, as the filter is only calculated on openeing new or changed segments.
This filter should improve use-cases where the filter can be applied one time before all queries (like security filters) on (re-)opening the IndexReader.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3212) Supply FilterIndexReader based on
any o.a.l.search.Filter
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13055061#comment-13055061 ]
Michael McCandless commented on LUCENE-3212:
--------------------------------------------
I think this is idea is similar to the CachedFilterIndexReader on LUCENE-1536? See https://issues.apache.org/jira/browse/LUCENE-1536?focusedCommentId=12908914&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12908914
> Supply FilterIndexReader based on any o.a.l.search.Filter
> ---------------------------------------------------------
>
> Key: LUCENE-3212
> URL: https://issues.apache.org/jira/browse/LUCENE-3212
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index, core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 4.0
>
>
> When coding LUCENE-2919 (PKIndexSplitter), Mike and me had the idea, how to effectively apply filters on the lowest level (before query execution). This is very useful for e.g. security Filters that simply hide some documents. Currently when you apply the filter after searching, lots of useless work was done like scoring filtered documents, iterating term positions (for Phrases),...
> This patch will provide a FilterIndexReader subclass (4.0 only, 3.x is too complicated to implement), that hides filtered documents by returning them in getDeletedDocs(). In contrast to LUCENE-2919, the filtering will work on per-segment (without SlowMultiReaderWrapper), so per segment search keeps available and reopening can be done very efficient, as the filter is only calculated on openeing new or changed segments.
> This filter should improve use-cases where the filter can be applied one time before all queries (like security filters) on (re-)opening the IndexReader.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3212) Supply FilterIndexReader based on
any o.a.l.search.Filter
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13055146#comment-13055146 ]
Michael McCandless commented on LUCENE-3212:
--------------------------------------------
That's a good point -- I'm not sure why I didn't just override getDeletedDocs! It seems like that should work fine.
> Supply FilterIndexReader based on any o.a.l.search.Filter
> ---------------------------------------------------------
>
> Key: LUCENE-3212
> URL: https://issues.apache.org/jira/browse/LUCENE-3212
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index, core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 4.0
>
>
> When coding LUCENE-2919 (PKIndexSplitter), Mike and me had the idea, how to effectively apply filters on the lowest level (before query execution). This is very useful for e.g. security Filters that simply hide some documents. Currently when you apply the filter after searching, lots of useless work was done like scoring filtered documents, iterating term positions (for Phrases),...
> This patch will provide a FilterIndexReader subclass (4.0 only, 3.x is too complicated to implement), that hides filtered documents by returning them in getDeletedDocs(). In contrast to LUCENE-2919, the filtering will work on per-segment (without SlowMultiReaderWrapper), so per segment search keeps available and reopening can be done very efficient, as the filter is only calculated on openeing new or changed segments.
> This filter should improve use-cases where the filter can be applied one time before all queries (like security filters) on (re-)opening the IndexReader.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3212) Supply FilterIndexReader based on
any o.a.l.search.Filter
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13055185#comment-13055185 ]
Uwe Schindler commented on LUCENE-3212:
---------------------------------------
I don't think, this issue is obsolete with LUCENE-1536:
If you have one filter thats e.g. applied for one user every time, maybe for all his queries, it can live as long as the SegmentReader lives. So simply wrapping the IndexReader with a Filter has much more flexibility, as its done one time on creating the IndexReader - so I think, this filter could additionally live in contrib. If we have RandomAccessFilters, this one and also PKIndexSplitter (which will only use this FIR and drop its own impl) can directly use the Bits supplied by the Filter's DocIdSet.
> Supply FilterIndexReader based on any o.a.l.search.Filter
> ---------------------------------------------------------
>
> Key: LUCENE-3212
> URL: https://issues.apache.org/jira/browse/LUCENE-3212
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index, core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 4.0
>
>
> When coding LUCENE-2919 (PKIndexSplitter), Mike and me had the idea, how to effectively apply filters on the lowest level (before query execution). This is very useful for e.g. security Filters that simply hide some documents. Currently when you apply the filter after searching, lots of useless work was done like scoring filtered documents, iterating term positions (for Phrases),...
> This patch will provide a FilterIndexReader subclass (4.0 only, 3.x is too complicated to implement), that hides filtered documents by returning them in getDeletedDocs(). In contrast to LUCENE-2919, the filtering will work on per-segment (without SlowMultiReaderWrapper), so per segment search keeps available and reopening can be done very efficient, as the filter is only calculated on openeing new or changed segments.
> This filter should improve use-cases where the filter can be applied one time before all queries (like security filters) on (re-)opening the IndexReader.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3212) Supply FilterIndexReader based on
any o.a.l.search.Filter
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13055116#comment-13055116 ]
Uwe Schindler commented on LUCENE-3212:
---------------------------------------
It's similar, but I dont understand the impl there. I would simply override getDeletedDocs to return the deleted docs ored with the filtered. Then you dont need to override terms() and fields().
> Supply FilterIndexReader based on any o.a.l.search.Filter
> ---------------------------------------------------------
>
> Key: LUCENE-3212
> URL: https://issues.apache.org/jira/browse/LUCENE-3212
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index, core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 4.0
>
>
> When coding LUCENE-2919 (PKIndexSplitter), Mike and me had the idea, how to effectively apply filters on the lowest level (before query execution). This is very useful for e.g. security Filters that simply hide some documents. Currently when you apply the filter after searching, lots of useless work was done like scoring filtered documents, iterating term positions (for Phrases),...
> This patch will provide a FilterIndexReader subclass (4.0 only, 3.x is too complicated to implement), that hides filtered documents by returning them in getDeletedDocs(). In contrast to LUCENE-2919, the filtering will work on per-segment (without SlowMultiReaderWrapper), so per segment search keeps available and reopening can be done very efficient, as the filter is only calculated on openeing new or changed segments.
> This filter should improve use-cases where the filter can be applied one time before all queries (like security filters) on (re-)opening the IndexReader.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-3212) Supply FilterIndexReader based on
any o.a.l.search.Filter
Posted by "Uwe Schindler (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler resolved LUCENE-3212.
-----------------------------------
Resolution: Won't Fix
LUCENE-1536 superceedes this.
> Supply FilterIndexReader based on any o.a.l.search.Filter
> ---------------------------------------------------------
>
> Key: LUCENE-3212
> URL: https://issues.apache.org/jira/browse/LUCENE-3212
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index, core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 4.0
>
>
> When coding LUCENE-2919 (PKIndexSplitter), Mike and me had the idea, how to effectively apply filters on the lowest level (before query execution). This is very useful for e.g. security Filters that simply hide some documents. Currently when you apply the filter after searching, lots of useless work was done like scoring filtered documents, iterating term positions (for Phrases),...
> This patch will provide a FilterIndexReader subclass (4.0 only, 3.x is too complicated to implement), that hides filtered documents by returning them in getDeletedDocs(). In contrast to LUCENE-2919, the filtering will work on per-segment (without SlowMultiReaderWrapper), so per segment search keeps available and reopening can be done very efficient, as the filter is only calculated on openeing new or changed segments.
> This filter should improve use-cases where the filter can be applied one time before all queries (like security filters) on (re-)opening the IndexReader.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org