You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Earwin Burrfoot (JIRA)" <ji...@apache.org> on 2009/05/21 17:46:45 UTC
[jira] Created: (LUCENE-1651) Make IndexReader.open() always return
MSR to simplify (re-)opens.
Make IndexReader.open() always return MSR to simplify (re-)opens.
-----------------------------------------------------------------
Key: LUCENE-1651
URL: https://issues.apache.org/jira/browse/LUCENE-1651
Project: Lucene - Java
Issue Type: Task
Affects Versions: 2.9
Reporter: Earwin Burrfoot
As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
The patch somewhat de-hairies (re-)open logic for MSR/SR.
SR no longer needs an ability to pose as toplevel directory-owning IR.
All related logic is moved from DIR to MSR.
DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1651) Make IndexReader.open() always
return MSR to simplify (re-)opens.
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715888#action_12715888 ]
Michael McCandless commented on LUCENE-1651:
--------------------------------------------
Hmm -- let me figure out what happened. Sorry for the false alarm!
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651-tag.patch, LUCENE-1651.patch, LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1651) Make IndexReader.open() always return
MSR to simplify (re-)opens.
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Earwin Burrfoot updated LUCENE-1651:
------------------------------------
Attachment: LUCENE-1651.patch
Okay, here's the first patch, against latest trunk.
test-core, test-contrib pass. test-tag fails due to no-longer-correct test assumptions. Should I checkout the branch and make a separate patch, or how is it done?
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Attachments: LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1651) Make IndexReader.open() always
return MSR to simplify (re-)opens.
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715908#action_12715908 ]
Earwin Burrfoot commented on LUCENE-1651:
-----------------------------------------
bq. Patch looks good Earwin, thanks!
I believe the readers can be cleaned up further, but I'm short on time and don't want to delay it for another week or two, and then rebase it against updated trunk once again. Might as well do that under a separate issue.
bq. I think we should now rename MultiSegmentReader to DirectoryIndexReader?
Maybe DirectoryReader instead of DirectoryIndexReader? But all three are in fact okay with me, I really don't have any preference here.
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651-tag.patch, LUCENE-1651.patch, LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1651) Make IndexReader.open() always
return MSR to simplify (re-)opens.
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716004#action_12716004 ]
Michael McCandless commented on LUCENE-1651:
--------------------------------------------
OK patch looks good. I plan to commit in a day or two. Thanks Earwin!
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651-tag.patch, LUCENE-1651-tag.patch, LUCENE-1651.patch, LUCENE-1651.patch, LUCENE-1651.patch, LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1651) Make IndexReader.open() always
return MSR to simplify (re-)opens.
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715672#action_12715672 ]
Earwin Burrfoot commented on LUCENE-1651:
-----------------------------------------
Hm.. okay, I've got back to work on this patch. To fix tests relying on getting SR from IR.open() on trunk I introduced a package-private utility method that extracts SR from MSR if it is the only one there. The tests in tags/XXX don't see this method, should I backport it somewhere there?
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1651) Make IndexReader.open() always
return MSR to simplify (re-)opens.
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715900#action_12715900 ]
Michael McCandless commented on LUCENE-1651:
--------------------------------------------
OK, I had one hunk fail in SegmentReader.java patch; once I fixed that, t'sall good.
Patch looks good Earwin, thanks!
I think we should now rename MultiSegmentReader to DirectoryIndexReader?
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651-tag.patch, LUCENE-1651.patch, LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1651) Make IndexReader.open() always return
MSR to simplify (re-)opens.
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Earwin Burrfoot updated LUCENE-1651:
------------------------------------
Attachment: LUCENE-1651-tag.patch
LUCENE-1651.patch
Here are the patches for current lucene trunk and back compat tag.
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651-tag.patch, LUCENE-1651.patch, LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1651) Make IndexReader.open() always
return MSR to simplify (re-)opens.
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715676#action_12715676 ]
Michael McCandless commented on LUCENE-1651:
--------------------------------------------
You can add a stub method (matches signature but does nothing) to src/java in the back-compat branch, just so the tests compile. The back-compat tests run against the trunk's jar.
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1651) Make IndexReader.open() always return
MSR to simplify (re-)opens.
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Earwin Burrfoot updated LUCENE-1651:
------------------------------------
Attachment: LUCENE-1651-tag.patch
LUCENE-1651.patch
Argh! The rename broke test-tag again :) in new and innovative ways.
New patches attached.
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651-tag.patch, LUCENE-1651-tag.patch, LUCENE-1651.patch, LUCENE-1651.patch, LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1651) Make IndexReader.open() always
return MSR to simplify (re-)opens.
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715836#action_12715836 ]
Earwin Burrfoot commented on LUCENE-1651:
-----------------------------------------
Seems the patch didn't apply completely. Your line numbers are off, also directory/readOnly are now members of SegmentReader, no way they can't be seen:
{code}
class SegmentReader extends IndexReader implements Cloneable {
protected Directory directory;
protected boolean readOnly;
private String segment;
private SegmentInfo si;
private int readBufferSize;
{code}
Here's corresponding part of the patch, I bet $Id$ is the reason.
{code}
-/**
- * @version $Id$
- */
-class SegmentReader extends DirectoryIndexReader {
+/** @version $Id$ */
+class SegmentReader extends IndexReader implements Cloneable {
+ protected Directory directory;
+ protected boolean readOnly;
+
{code}
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651-tag.patch, LUCENE-1651.patch, LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1651) Make IndexReader.open() always return
MSR to simplify (re-)opens.
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Earwin Burrfoot updated LUCENE-1651:
------------------------------------
Attachment: LUCENE-1651.patch
One more version, applies against current trunk without fuzzy hunk matching.
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651-tag.patch, LUCENE-1651-tag.patch, LUCENE-1651.patch, LUCENE-1651.patch, LUCENE-1651.patch, LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-1651) Make IndexReader.open() always
return MSR to simplify (re-)opens.
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless reassigned LUCENE-1651:
------------------------------------------
Assignee: Michael McCandless
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Attachments: LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1651) Make IndexReader.open() always return
MSR to simplify (re-)opens.
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-1651:
---------------------------------------
Fix Version/s: 2.9
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1651) Make IndexReader.open() always
return MSR to simplify (re-)opens.
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711668#action_12711668 ]
Michael McCandless commented on LUCENE-1651:
--------------------------------------------
Excellent! Thanks Earwin.
bq. test-tag fails due to no-longer-correct test assumptions. Should I checkout the branch and make a separate patch, or how is it done?
Yes, please make mods to the tags/XXX checkout, and include those mods in the patch.
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Attachments: LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1651) Make IndexReader.open() always
return MSR to simplify (re-)opens.
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713170#action_12713170 ]
Michael McCandless commented on LUCENE-1651:
--------------------------------------------
Earwin are you working on the patch for back-compat tests, here?
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1651) Make IndexReader.open() always
return MSR to simplify (re-)opens.
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713195#action_12713195 ]
Earwin Burrfoot commented on LUCENE-1651:
-----------------------------------------
Yup, will post one soon, didn't have enough time on hands.
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1651) Make IndexReader.open() always
return MSR to simplify (re-)opens.
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-1651.
----------------------------------------
Resolution: Fixed
Thanks Earwin!
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651-tag.patch, LUCENE-1651-tag.patch, LUCENE-1651.patch, LUCENE-1651.patch, LUCENE-1651.patch, LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1651) Make IndexReader.open() always
return MSR to simplify (re-)opens.
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715777#action_12715777 ]
Michael McCandless commented on LUCENE-1651:
--------------------------------------------
I'm hitting compilation errors:
{code}
common.compile-core:
[mkdir] Created dir: /lucene/src/msr.1651/build/classes/java
[javac] Compiling 374 source files to /lucene/src/msr.1651/build/classes/java
[javac] /lucene/src/msr.1651/src/java/org/apache/lucene/index/SegmentReader.java:419: directory has private access in org.apache.lucene.index.IndexReader
[javac] instance.directory = dir;
[javac] ^
[javac] /lucene/src/msr.1651/src/java/org/apache/lucene/index/SegmentReader.java:420: cannot find symbol
[javac] symbol : variable readOnly
[javac] location: class org.apache.lucene.index.SegmentReader
[javac] instance.readOnly = readOnly;
[javac] ^
[javac] /lucene/src/msr.1651/src/java/org/apache/lucene/index/SegmentReader.java:562: cannot find symbol
[javac] symbol : variable readOnly
[javac] location: class org.apache.lucene.index.SegmentReader
[javac] return clone(readOnly); // Preserve current readOnly
[javac] ^
[javac] /lucene/src/msr.1651/src/java/org/apache/lucene/index/SegmentReader.java:588: cannot find symbol
[javac] symbol : variable readOnly
[javac] location: class org.apache.lucene.index.SegmentReader
[javac] if (normsUpToDate && deletionsUpToDate && !doClone && openReadOnly && readOnly) {
[javac] ^
[javac] /lucene/src/msr.1651/src/java/org/apache/lucene/index/SegmentReader.java:611: cannot find symbol
[javac] symbol : variable readOnly
[javac] location: class org.apache.lucene.index.SegmentReader
[javac] clone.readOnly = openReadOnly;
[javac] ^
[javac] /lucene/src/msr.1651/src/java/org/apache/lucene/index/SegmentReader.java:612: directory has private access in org.apache.lucene.index.IndexReader
[javac] clone.directory = directory;
[javac] ^
[javac] /lucene/src/msr.1651/src/java/org/apache/lucene/index/SegmentReader.java:612: directory has private access in org.apache.lucene.index.IndexReader
[javac] clone.directory = directory;
[javac] ^
[javac] /lucene/src/msr.1651/src/java/org/apache/lucene/index/SegmentReader.java:1214: directory has private access in org.apache.lucene.index.IndexReader
[javac] return directory;
[javac] ^
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] 8 errors
{code}
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651-tag.patch, LUCENE-1651.patch, LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1651) Make IndexReader.open() always
return MSR to simplify (re-)opens.
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715935#action_12715935 ]
Michael McCandless commented on LUCENE-1651:
--------------------------------------------
Let's just go w/ DirectoryReader? Can you post a new patch w/ that? Thanks.
bq. I believe the readers can be cleaned up further, but I'm short on time and don't want to delay it for another week or two, and then rebase it against updated trunk once again. Might as well do that under a separate issue.
Agreed! Progress not perfection.
> Make IndexReader.open() always return MSR to simplify (re-)opens.
> -----------------------------------------------------------------
>
> Key: LUCENE-1651
> URL: https://issues.apache.org/jira/browse/LUCENE-1651
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1651-tag.patch, LUCENE-1651.patch, LUCENE-1651.patch
>
>
> As per discussion in mailing list, I'm making DirectoryIndexReader.open() always return MSR, even for single-segment indexes.
> While theoretically valid in the past (if you make sure to keep your index constantly optimized) this feature is made practically obsolete by per-segment collection.
> The patch somewhat de-hairies (re-)open logic for MSR/SR.
> SR no longer needs an ability to pose as toplevel directory-owning IR.
> All related logic is moved from DIR to MSR.
> DIR becomes almost empty, and copying two or three remaining fields over to MSR/SR, I remove it.
> Lots of tests fail, as they rely on SR returned from IR.open(), I fix by introducing SR.getOnlySegmentReader static package-private method.
> Some previous bugs are uncovered, one is fixed in LUCENE-1645, another (partially fixed in LUCENE-1648) is fixed in this patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org