You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (Created) (JIRA)" <ji...@apache.org> on 2011/10/25 14:18:33 UTC
[jira] [Created] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
------------------------------------------------------------------------------------------
Key: LUCENE-3531
URL: https://issues.apache.org/jira/browse/LUCENE-3531
Project: Lucene - Java
Issue Type: Improvement
Components: core/search
Affects Versions: 4.0
Reporter: Uwe Schindler
Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler resolved LUCENE-3531.
-----------------------------------
Resolution: Fixed
Reverted backport of listeners in 3.x revision: 1214347
Reverted listeners in trunk revision: 1214349
If I have a better idea, I will open an issue.
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-3531:
----------------------------------
Attachment: LUCENE-3531-identityFix.patch
New patch, now with WeakIdentityHashMap impl from Apache CXF (excluding its inefficient Collection views and supporting null keys).
I also changed the CachingWrapperFilter to hold soft references to the DocIdSet, as otherwise e.g. the Bits==null key will never be purged (as the null key cannot be reclaimed by GC).
We should maybe also change FieldCache to hold soft references to its arrays, as this makes it more easy to purge them for the GC in low-memory conditions, even if the reader key is still alive (means field cache can be purged although reader is still alive).
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-3531:
----------------------------------
Attachment: LUCENE-3531-identityFix.patch
Here an improved patch factoring out the wrapper object to o.a.l.util, adding tests.
Once this is committed, I will review other code and maybe move to this wrapper class (e.g. MMapIndexInput's cloned inputs)
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169766#comment-13169766 ]
Uwe Schindler commented on LUCENE-3531:
---------------------------------------
Not yet, once we have split composite readers and atomic reader, yes. But at this point i cannot handle that, as SegmentReader can be replaced by SlowMultiReaderWrapper (and they have no common base class): LUCENE-2858
So I would like to commit this first and later move this with all this stuff to atomic readers only.
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-v2.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169747#comment-13169747 ]
Robert Muir commented on LUCENE-3531:
-------------------------------------
can't this just be on segmentreader only?
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-v2.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Michael McCandless (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169493#comment-13169493 ]
Michael McCandless commented on LUCENE-3531:
--------------------------------------------
Phew thanks Uwe! What a doozie that turned out to be :)
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168812#comment-13168812 ]
Uwe Schindler commented on LUCENE-3531:
---------------------------------------
I did some investigation. If you want a complete WeakIdentityHashMap witha all iterators and so on its heavy to do and you must in all cases also wrap all keys with a WeakReference even for lookup (unless you implement your completely own HashMap impl). The easy fix here is to use a wrapper "Reference" object as cache key, that simply has a final field and equals/hashCode that does the system hashcode. By comparing the wrapper object as key, two wrapper objects are only equal, if the wrapped objects are identical:
{code:java}
static final class IdentityKeyWrapper<T> {
public final T key;
private final int hashCode;
public IdentityKeyWrapper(T key) {
this.key = key;
this.hashCode = System.identityHashCode(key);
}
public int hashCode() { return hashCode; }
public boolean equals(Object o) {
if (o == this) return true;
if (o instanceof IdentityKeyWrapper) {
return ((IdentityKeyWrapper<T>)o).key == this.key;
}
return false;
}
}
{code}
The backside is that you have to wrap the Bits interface even on a lookup, but thats cheap just like boxing/unboxing (eden space).
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-3531:
----------------------------------
Attachment: LUCENE-3531-identityFix.patch
Here the fix. Mike, I think this is fine?
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168647#comment-13168647 ]
Uwe Schindler commented on LUCENE-3531:
---------------------------------------
I will take care and also use the map then for MMapDirectory.MMapIndexInput where I explicitely overrided the equals/hashcode of the keys to be 100% identity.
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169236#comment-13169236 ]
Uwe Schindler commented on LUCENE-3531:
---------------------------------------
This is why most custom implementations of WeakIdentityHashMaps found on the net internally extend WeakReference to add equals/hashCode to it. It must be one object, otherwise GC may purge objects.
There is no way around implementing an own WeakIdentityHashMap (not fully implementing the Map interface, as not needed for caches).
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167107#comment-13167107 ]
Uwe Schindler commented on LUCENE-3531:
---------------------------------------
Looks fine!
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Attachments: LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-3531:
----------------------------------
Attachment: LUCENE-3531-identityFix.patch
More cleanup and made more fields private/final.
I will commit this now.
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-3531:
----------------------------------
Attachment: LUCENE-3531-v2.patch
Patch.
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-v2.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Reopened] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Michael McCandless (Reopened) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless reopened LUCENE-3531:
----------------------------------------
Uwe found a sneaky problem here: using acceptDocs as the cache key is bad, eg FixedBitSet's equals/hashCode are horribly costly... we need a WeakIdentityHashMap.
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Michael McCandless (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-3531:
---------------------------------------
Attachment: LUCENE-3531.patch
Patch, adding optional boolean "recacheDeletes", defaulting to false. We only cache if the incoming acceptDocs == reader's liveDocs.
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler resolved LUCENE-3531.
-----------------------------------
Resolution: Fixed
Committed trunk revision: 1214551
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-v2.patch, LUCENE-3531-v2.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Michael McCandless (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-3531.
----------------------------------------
Resolution: Fixed
Fix Version/s: 4.0
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler resolved LUCENE-3531.
-----------------------------------
Resolution: Fixed
Committed trunk revision: 1214274
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-3531:
----------------------------------
Attachment: LUCENE-3531-identityFix.patch
Updated patch, now also supporting purging CachingWrapperFilter's cache when the reader(-core) is closed.
This fix should also be backported to 3.x.
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Assigned] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Michael McCandless (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless reassigned LUCENE-3531:
------------------------------------------
Assignee: Michael McCandless
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Attachments: LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169424#comment-13169424 ]
Uwe Schindler commented on LUCENE-3531:
---------------------------------------
Backported adding close listeners to 3.x.
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Reopened] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Reopened) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler reopened LUCENE-3531:
-----------------------------------
After some investigation, we should remove the purgeing on IndexReader/Core close again, as this holds strong references from the IndexReader to the CachingWrapperFilter. This would lead to problems in the following case: A app has an IndexReader that never changes and is never closed, but creates lots of CachingWrapperFilters for short-time caches. This will fill up the event listener queue.
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-3531:
----------------------------------
Attachment: LUCENE-3531-v2.patch
Updated patch with a test for the core key and combined key. I will commit this now and then work on splitting atomic and composite readers.
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-v2.patch, LUCENE-3531-v2.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169232#comment-13169232 ]
Uwe Schindler commented on LUCENE-3531:
---------------------------------------
There is one problem with this wrapper object and WeakHashMap. The wrapped object does not have a reference to the wrapper itsself, so the wrapper may get garbage collected (as nothing refers to it) and the item may get removed from the map.
So this does not work! ...digging!
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Assigned] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler reassigned LUCENE-3531:
-------------------------------------
Assignee: Uwe Schindler (was: Michael McCandless)
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Reopened] (LUCENE-3531) Improve CachingWrapperFilter to
optionally also cache acceptDocs, if identical to liveDocs
Posted by "Uwe Schindler (Reopened) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler reopened LUCENE-3531:
-----------------------------------
Mike and I discussed on IRC about easier ways to solve the caching problem. We came to the conclusion, that using the acceptDocs (may be null) as a sub-key are suboptimal, because the null key cannot be evicted from the WeakIdentityHashMap. Also the two caches inside each other are hard to manage.
The idea was to have a "combined" key on segmentreader, thats hard referenced by the instance, changing only when deletions change. This key should also only have identity equals/hashCode.
IndexReader like in 3.x now supplies two keys, one based on the core without deletions, and a second one thats equal for all segmentreaders with same deletions and same core. The trick is to use a Object field in SegmentReader thats initialized with new Object() at the beginning and transferred over to a new instance when cloned. But once deletions change, we reassign a new "new Object()", so the key changes.
CachingWrapperFilter uses either the conventional coreCacheKey (if if does not respect deletions at all and caches without acceptDocs), or it uses the combined key (IndexReader.getCombinedCoreAndDeletesKey()) for the cache.
The WeakIdentityHashMap is therefore obsolete again, I will nuke it.
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Issue Comment Edited] (LUCENE-3531) Improve
CachingWrapperFilter to optionally also cache acceptDocs, if identical to
liveDocs
Posted by "Uwe Schindler (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168840#comment-13168840 ]
Uwe Schindler edited comment on LUCENE-3531 at 12/13/11 11:16 PM:
------------------------------------------------------------------
Here the fix. Mike, I think this is fine?
The fix also removes the useless genericfication of FilterCache, as we removed the SpanCachingWrapperFilter.
was (Author: thetaphi):
Here the fix. Mike, I think this is fine?
> Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-3531
> URL: https://issues.apache.org/jira/browse/LUCENE-3531
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3531-identityFix.patch, LUCENE-3531.patch
>
>
> Spinoff from LUCENE-1536: This issue removed the different cache modes completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), the cache only contains raw DocIdSet without any deletions/acceptDocs. For IndexReaders that are seldom reopened, this might not be as performant as it could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached with liveDocs applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org