You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Earwin Burrfoot (JIRA)" <ji...@apache.org> on 2009/05/19 21:16:45 UTC
[jira] Created: (LUCENE-1645) Deleted documents are visible across
reopened MSRs
Deleted documents are visible across reopened MSRs
--------------------------------------------------
Key: LUCENE-1645
URL: https://issues.apache.org/jira/browse/LUCENE-1645
Project: Lucene - Java
Issue Type: Bug
Affects Versions: 2.9
Reporter: Earwin Burrfoot
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1645) Deleted documents are visible across
reopened MSRs
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Earwin Burrfoot updated LUCENE-1645:
------------------------------------
Attachment: LUCENE-1645.patch
Here's the fix. Plus slightly modified test that fails without the fix.
> Deleted documents are visible across reopened MSRs
> --------------------------------------------------
>
> Key: LUCENE-1645
> URL: https://issues.apache.org/jira/browse/LUCENE-1645
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Fix For: 2.9
>
> Attachments: LUCENE-1645.patch, LUCENE-1645.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1645) Deleted documents are visible across
reopened MSRs
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-1645.
----------------------------------------
Resolution: Fixed
> Deleted documents are visible across reopened MSRs
> --------------------------------------------------
>
> Key: LUCENE-1645
> URL: https://issues.apache.org/jira/browse/LUCENE-1645
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Fix For: 2.9
>
> Attachments: LUCENE-1645.patch, LUCENE-1645.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1645) Deleted documents are visible
across reopened MSRs
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710872#action_12710872 ]
Michael McCandless commented on LUCENE-1645:
--------------------------------------------
Good catch!
So we are missing a clone() on the old subreaders, when referencing them from the new MultiSegmentReader?
> Deleted documents are visible across reopened MSRs
> --------------------------------------------------
>
> Key: LUCENE-1645
> URL: https://issues.apache.org/jira/browse/LUCENE-1645
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Fix For: 2.9
>
> Attachments: LUCENE-1645.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1645) Deleted documents are visible
across reopened MSRs
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710894#action_12710894 ]
Earwin Burrfoot commented on LUCENE-1645:
-----------------------------------------
Either that. Or having boolean readerShared[] as a property of MSR, and check it on each modification attempt -> lazy clone().
*rant on*
Or make readers immutable :) I already have a patch that removes norm updates, lazy loading and synchronization. Then we only need to do deletes in IndexWriter without resorting to its reader pool.
By the way, do we need public IR cloneability? Not for Lucene internals, but for actual users? That really looks like one more feature that can be safely removed.
> Deleted documents are visible across reopened MSRs
> --------------------------------------------------
>
> Key: LUCENE-1645
> URL: https://issues.apache.org/jira/browse/LUCENE-1645
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Fix For: 2.9
>
> Attachments: LUCENE-1645.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1645) Deleted documents are visible across
reopened MSRs
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Earwin Burrfoot updated LUCENE-1645:
------------------------------------
Attachment: LUCENE-1645.patch
If you reopen() MSR with unchanged segments, the resulting MSR will share SR instances for these segments.
Consequently, when you delete a doc in one of said segments through new MSR, deletion will be visible in original MSR, as no checks for shared SRs are done in delete().
The patch illustrates the bug by altering existing test to reopen MSR->MSR instead of SR->MSR.
> Deleted documents are visible across reopened MSRs
> --------------------------------------------------
>
> Key: LUCENE-1645
> URL: https://issues.apache.org/jira/browse/LUCENE-1645
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Attachments: LUCENE-1645.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1645) Deleted documents are visible
across reopened MSRs
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711305#action_12711305 ]
Michael McCandless commented on LUCENE-1645:
--------------------------------------------
Looks good... I'll commit shortly. Thanks Earwin!
> Deleted documents are visible across reopened MSRs
> --------------------------------------------------
>
> Key: LUCENE-1645
> URL: https://issues.apache.org/jira/browse/LUCENE-1645
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Fix For: 2.9
>
> Attachments: LUCENE-1645.patch, LUCENE-1645.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1645) Deleted documents are visible
across reopened MSRs
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711006#action_12711006 ]
Earwin Burrfoot commented on LUCENE-1645:
-----------------------------------------
Lazy clone() is a bad idea, since it has to be synced.
> Deleted documents are visible across reopened MSRs
> --------------------------------------------------
>
> Key: LUCENE-1645
> URL: https://issues.apache.org/jira/browse/LUCENE-1645
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Fix For: 2.9
>
> Attachments: LUCENE-1645.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1645) Deleted documents are visible
across reopened MSRs
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711139#action_12711139 ]
Earwin Burrfoot commented on LUCENE-1645:
-----------------------------------------
bq. Right; I think we should simply clone shared subreaders on reopen.
IFF the subreader being cloned is not readonly!
bq. I think it's too soon to do this.
Hey, let's reverse readonly/writeable reader inheritance? WriteableSegmentReader extends SegmentReader.
SegmentReader is readonly by behaviour and in code - there are no syncs and mutator methods throw UOE. WriteableSegmentReader adds a writeability layer over SegmentReader and all the gory syncs.
bq. I think we do? Eg it enables you to clone a readOnly reader to a read/write one in order to do deletes/setNorm.
Hey, open a new IR if you want to write. Cloning/reopening an IR while simultaneously switching readonly flag is a feature that gives user zero new functionality, absolutely nothing he can't accomplish using a new IR, but makes reopening/cloning code more complex.
> Deleted documents are visible across reopened MSRs
> --------------------------------------------------
>
> Key: LUCENE-1645
> URL: https://issues.apache.org/jira/browse/LUCENE-1645
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Fix For: 2.9
>
> Attachments: LUCENE-1645.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1645) Deleted documents are visible
across reopened MSRs
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711185#action_12711185 ]
Michael McCandless commented on LUCENE-1645:
--------------------------------------------
bq. IFF the subreader being cloned is not readonly!
OK.
{quote}
Hey, let's reverse readonly/writeable reader inheritance? WriteableSegmentReader extends SegmentReader.
SegmentReader is readonly by behaviour and in code - there are no syncs and mutator methods throw UOE. WriteableSegmentReader adds a writeability layer over SegmentReader and all the gory syncs.
{quote}
That sounds right, but let's do this as a separate issue? And leave this issue on fixing this particular bug.
{quote}
Hey, open a new IR if you want to write. Cloning/reopening an IR while simultaneously switching readonly flag is a feature that gives user zero new functionality, absolutely nothing he can't accomplish using a new IR, but makes reopening/cloning code more complex.
{quote}
Opening a new IR is quite a bit more costly than cloning/reopening.
Plus the reopen/cloning code will still need the complexity because internal to Lucene we still need to do this. We are only discussing whether the functionality should remain public, which I think it should.
> Deleted documents are visible across reopened MSRs
> --------------------------------------------------
>
> Key: LUCENE-1645
> URL: https://issues.apache.org/jira/browse/LUCENE-1645
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Fix For: 2.9
>
> Attachments: LUCENE-1645.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1645) Deleted documents are visible across
reopened MSRs
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-1645:
---------------------------------------
Fix Version/s: 2.9
> Deleted documents are visible across reopened MSRs
> --------------------------------------------------
>
> Key: LUCENE-1645
> URL: https://issues.apache.org/jira/browse/LUCENE-1645
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Fix For: 2.9
>
> Attachments: LUCENE-1645.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1645) Deleted documents are visible
across reopened MSRs
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711086#action_12711086 ]
Michael McCandless commented on LUCENE-1645:
--------------------------------------------
bq. Lazy clone() is a bad idea, since it has to be synced.
Right; I think we should simply clone shared subreaders on reopen.
bq. Or make readers immutable
I think it's too soon to do this. In theory, NRT should enable us to deprecate deletions from IW, but 1) we need NRT released and some real experience before doing so, and 2) we still need to get setNorm working from IW.
bq. By the way, do we need public IR cloneability?
I think we do? Eg it enables you to clone a readOnly reader to a read/write one in order to do deletes/setNorm.
> Deleted documents are visible across reopened MSRs
> --------------------------------------------------
>
> Key: LUCENE-1645
> URL: https://issues.apache.org/jira/browse/LUCENE-1645
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.9
> Reporter: Earwin Burrfoot
> Fix For: 2.9
>
> Attachments: LUCENE-1645.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org