You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (JIRA)" <ji...@apache.org> on 2010/11/19 09:27:13 UTC
[jira] Created: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
---------------------------------------------------------------------------------
Key: LUCENE-2769
URL: https://issues.apache.org/jira/browse/LUCENE-2769
Project: Lucene - Java
Issue Type: Bug
Affects Versions: 4.0
Reporter: Uwe Schindler
Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933722#action_12933722 ]
Uwe Schindler commented on LUCENE-2769:
---------------------------------------
In my opinion, LUCENE-2459 should be reverted and all places that can not correctly filter per segment should use SlowMultiReaderWrapper as superclass (like e.g. MultiPassIndexSplitter)
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933729#action_12933729 ]
Uwe Schindler commented on LUCENE-2769:
---------------------------------------
Additionally to this:
The MultiPassIndexSplitter in 3.x and 3.0 should also return null in getSeqSubReaders. It works currently because we know how SegmentMerger works, but its still incorrect. To Filter the terms correctly it should imitate a atomic reader.
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-2769:
----------------------------------
Attachment: LUCENE-2769_SegmentMerger.patch
Last patch containes unused variable (a relic from very earlier times). "int starts[]" in merge terms is not needed.
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch, LUCENE-2769_SegmentMerger.patch, LUCENE-2769_SegmentMerger.patch, LUCENE-2769_SegmentMerger.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933736#action_12933736 ]
Uwe Schindler commented on LUCENE-2769:
---------------------------------------
We can use the SlowMultiWrapper also to remove norms merging in DirectoryReader.norms(field) and MultiReader.norms(field) should also throw UOE like fields(). If you need top-level norms use the SlowMultiReaderWrapper.
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933910#action_12933910 ]
Uwe Schindler commented on LUCENE-2769:
---------------------------------------
I created new issue for the norms thing: LUCENE-2771
I will commit the basic patch without "norms" soon and update the norms patch to latest trunk.
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2769:
--------------------------------
Attachment: LUCENE-2769.patch
ok try 2: we shouldn't encourage subclassing of SlowMultiReaderWrapper :)
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933907#action_12933907 ]
Robert Muir commented on LUCENE-2769:
-------------------------------------
I thought about this more, i think for this issue, we should not make SlowMultiReaderWrapper complicated and force ParallelReader to take atomic readers.
instead, we should remove norms from Multi/DirReader like my patch suggests and make ParallelReader manage its own cached norms.
For ParallelReader, this is no worse than today, but no better either. We should seriously either figure out how to
fix this ParallelReader or move it to contrib.
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-2769:
----------------------------------
Attachment: (was: LUCENE-2769_SegmentMerger.patch)
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2769:
--------------------------------
Attachment: LUCENE-2769.patch
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler resolved LUCENE-2769.
-----------------------------------
Resolution: Fixed
Lucene Fields: [New, Patch Available] (was: [New])
Committed basic patch revision: 1036977
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-2769:
----------------------------------
Attachment: LUCENE-2769_SegmentMerger.patch
Here a better patch for the segment merger. We should even apply this if we not remove top-level norms!
It saves lots of memory during merging by using ReaderUtil to go down to segment level! Don't wonder about BytesRef, but we need a reference here because of the anonymous inner class.
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch, LUCENE-2769_SegmentMerger.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2769:
--------------------------------
Attachment: LUCENE-2769_norms.patch
here is a hack patch for Uwe's idea about the norms.
we need to change SegmentMerger to not call norms on the top-level IR but populate its normBuffer from the subs.
in my opinion it seems crazy we are currently creating these big arrays this way (yeah there is the hairy code for re-open that re-uses the big merged cache for the NRT case, but still).
Maybe i am missing something.
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch, LUCENE-2769_norms.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933848#action_12933848 ]
Uwe Schindler commented on LUCENE-2769:
---------------------------------------
I removed the SegmentMerger patches from here and moved over to LUCENE-2770. The problem noted before is now solved. The SegmentMerger missed to clone TVReaders. This was a bug not recognized before.
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-2769:
----------------------------------
Attachment: LUCENE-2769_SegmentMerger.patch
Here a new SegmentMerger now working only on atomic readers. All "normal" tests pass, but it seems that addIndexesWithThreads fails during merging term vectors. This is not clear, all other tests pass and the created indexes are fine.
Mike: Do you understand that? The problem seems to be some thread safety issue in addIndexes(IndexReader...). Not sure who changes who's internal structures? Maybe suddenly subreaders change?
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch, LUCENE-2769_SegmentMerger.patch, LUCENE-2769_SegmentMerger.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2769:
--------------------------------
Attachment: LUCENE-2769_norms.patch
here's another hacky update: but still a few tests explicitly check these norms and need to be fixed.
maybe we could add an uncached "MultiNorms" or something at least in src/test for convenience,
just to fill the byte arrays so these tests can assertEquals
otherwise we are going to have to put a lot of SlowMultiReaderWrappers in these tests.
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-2769) FilterIndexReader in
trunk does not implement getSequentialSubReaders() correctly
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933722#action_12933722 ]
Uwe Schindler edited comment on LUCENE-2769 at 11/19/10 4:09 AM:
-----------------------------------------------------------------
In my opinion, LUCENE-2459 should be reverted and all places that can not correctly filter per segment should use SlowMultiReaderWrapper as superclass (like e.g. MultiPassIndexSplitter) or better simply wrap with SlowMultiReaderWrapper in the ctor:
{code}
ctor(IndexReader in) {
super(SlowMultiReaderWrapper.wrap(in));
}
{code}
was (Author: thetaphi):
In my opinion, LUCENE-2459 should be reverted and all places that can not correctly filter per segment should use SlowMultiReaderWrapper as superclass (like e.g. MultiPassIndexSplitter) or better simply wrap with SlowMultiReaderWrapper in the ctor:
{code}
ctor() {
super(SlowMultiReaderWrapper.wrap(in));
}
{code}
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-2769:
----------------------------------
Attachment: (was: LUCENE-2769_SegmentMerger.patch)
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2769) FilterIndexReader in trunk does not
implement getSequentialSubReaders() correctly
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2769:
--------------------------------
Attachment: LUCENE-2769_norms.patch
here is an updated patch, with core/contrib/solr tests passing.
For ParallelReader i forced it to require non-composite readers only (e.g. SlowMultiReaderWrap them if thats not the case).
TODO:
* ParallelReader shouldnt need multifields etc anymore
* there are 5 @Ignore'd ParallelReader-related tests, because of things like reopen/isOptimized/isCurrent
* merge in Uwe's improved SegmentsMerger
* clean up code.
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch, LUCENE-2769_norms.patch, LUCENE-2769_SegmentMerger.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-2769) FilterIndexReader in
trunk does not implement getSequentialSubReaders() correctly
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933722#action_12933722 ]
Uwe Schindler edited comment on LUCENE-2769 at 11/19/10 3:51 AM:
-----------------------------------------------------------------
In my opinion, LUCENE-2459 should be reverted and all places that can not correctly filter per segment should use SlowMultiReaderWrapper as superclass (like e.g. MultiPassIndexSplitter) or better simply wrap with SlowMultiReaderWrapper in the ctor:
{code}
ctor() {
super(SlowMultiReaderWrapper.wrap(in));
}
{code}
was (Author: thetaphi):
In my opinion, LUCENE-2459 should be reverted and all places that can not correctly filter per segment should use SlowMultiReaderWrapper as superclass (like e.g. MultiPassIndexSplitter)
> FilterIndexReader in trunk does not implement getSequentialSubReaders() correctly
> ---------------------------------------------------------------------------------
>
> Key: LUCENE-2769
> URL: https://issues.apache.org/jira/browse/LUCENE-2769
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 4.0
> Reporter: Uwe Schindler
> Attachments: LUCENE-2769.patch, LUCENE-2769.patch
>
>
> Since LUCENE-2459, getSequentialSubReaders() in FilterIndexReader returns null, so it returns an atomic reader. But If you call then any of the enum methods, it throws Exception because the underlying reader is not atomic.
> We should move the null-returning method to SlowMultiReaderWrapper and fix FilterIndexReader's default to return in.getSequentialSubReaders(). Ideally an implementation must of course also wrap the sub-readers.
> If we change this we have to look into other Impls like the MultiPassIndexSplitter if we need to add atomicity.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org