You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (Issue Comment Edited) (JIRA)" <ji...@apache.org> on 2012/02/09 09:42:59 UTC
[jira] [Issue Comment Edited] (LUCENE-3736) ParallelReader is now
atomic, rename to ParallelAtomicReader and also add a
ParallelCompositeReader (that requires LogDocMergePolicy to have identical
subreader structure)
[ https://issues.apache.org/jira/browse/LUCENE-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204361#comment-13204361 ]
Uwe Schindler edited comment on LUCENE-3736 at 2/9/12 8:41 AM:
---------------------------------------------------------------
Hi Mike,
I attached a new patch with some obsolete code removed: At the times of the builder, to detect errors early, the subreader checks were also not only inspecting the composite subs but also the leaves. As we now have no separation anymore between Builder.add() and the ctor that does the actual work, we can remove the leaves checks, as the recursive ctor will do the same checks, so its impossible to have different leaf structure.
One check could be added instead: currently only maxDocs of subs are compared, maybe also numDocs.... But also here the check is done by the invoked ctors for the wrapped subreaders already.
Otherwise I am not happy with the telescopic ctor (sorry, builder looked better for the expert case - this is now unreadable).
The good thing is that we could add freedom so storedFieldsReaders can be completely separate. It would be easy to implement: closeSubReaders/parallelReaders arrays would need to be an union of both sets (currently only readers ctor param).
I did not know that Collections.newSetFromMap() is already in 1.6. We should remove the util class MapBackedSet in trunk and replace all occurences by the same code like you did. I opened LUCENE-3764 for that.
One small thing for "safety": MultiReader currently clones the reader arrays to prevent external modification. Both ParallelReaders should do the same. The builder enforced that before as you had no access to the subs and the array was cloned on building (copy from ArrayList->array).
was (Author: thetaphi):
Hi Mike,
I attached a new patch with some obsolete code removed: At the times of the builder, to detect errors early, the subreader checks were also not only inspecting the composite subs but also the leaves. As we now have no separation anymore between Builder.add() and the ctor that does the actual work, we can remove the leaves checks, as the recursive ctor will do the same checks, so its impossible to have different leaf structure.
One check could be added instead: currently only maxDocs of subs are compared, maybe also numDocs.... But also here the check is done by the invoked ctors for the wrapped subreaders already.
Otherwise I am not happy with the telescopic ctor (sorry, builder looked better for the expert case - this is now unreadable).
The good thing is that we could add freedom so storedFieldsReaders can be completely separate. It would be easy to implement: closeSubReaders/parallelReaders arrays would need to be an union of both sets (currently only readers ctor param).
I did not know that Collections.setOfMap() is already in 1.6. We should remove the util class SetOfMaps in trunk and replace all occurences by the same code like you di. I will open an issue.
One small thing for "safety": MultiReader currently clones the reader arrays to prevent external modification. Both ParallelReaders should do the same. The builder enforced that before as you had no access to the subs and the array was cloned on building (copy from ArrayList->array).
> ParallelReader is now atomic, rename to ParallelAtomicReader and also add a ParallelCompositeReader (that requires LogDocMergePolicy to have identical subreader structure)
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-3736
> URL: https://issues.apache.org/jira/browse/LUCENE-3736
> Project: Lucene - Java
> Issue Type: Sub-task
> Components: core/index
> Reporter: Uwe Schindler
> Assignee: Michael McCandless
> Fix For: 4.0
>
> Attachments: LUCENE-3733.patch, LUCENE-3736.patch, LUCENE-3736.patch, LUCENE-3736.patch, LUCENE-3736.patch, LUCENE-3736.patch, LUCENE-3736.patch
>
>
> The plan is:
> - Move all subreaders to ctor (builder-like API. First build reader-set, then call build)
> - Rename ParallelReader to ParallelAtomicReader
> - Add a ParallelCompositeReader with same builder API, but taking any CompositeReader-set and checks them that they are aligned (docStarts identical). The subreaders are ParallelAtomicReaders.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org