You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Shai Erera (JIRA)" <ji...@apache.org> on 2009/06/21 09:44:07 UTC
[jira] Created: (LUCENE-1708) Improve the use of isDeleted in the
indexing code
Improve the use of isDeleted in the indexing code
-------------------------------------------------
Key: LUCENE-1708
URL: https://issues.apache.org/jira/browse/LUCENE-1708
Project: Lucene - Java
Issue Type: Improvement
Components: Index
Reporter: Shai Erera
Fix For: 2.9
A spin off from here: http://www.nabble.com/Some-thoughts-around-the-use-of-reader.isDeleted-and-hasDeletions-td23931216.html.
Two changes:
# Optimize SegmentMerger work when a reader has no deletions.
# IndexReader.document() will no longer check if the document is deleted.
Will post a patch shortly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1708) Improve the use of isDeleted in the
indexing code
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12722783#action_12722783 ]
Shai Erera commented on LUCENE-1708:
------------------------------------
bq. I see lots of cleanups snuck in here
Not all cleanups are bad, i.e. changing to iteration-based loops, rather than calling list.size() in each loop and list.get(i), is better, and cleaner :). I admit though that some of the cleanups just helped me understand the code better.
> Improve the use of isDeleted in the indexing code
> -------------------------------------------------
>
> Key: LUCENE-1708
> URL: https://issues.apache.org/jira/browse/LUCENE-1708
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Shai Erera
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1708.patch, LUCENE-1708.patch
>
>
> A spin off from here: http://www.nabble.com/Some-thoughts-around-the-use-of-reader.isDeleted-and-hasDeletions-td23931216.html.
> Two changes:
> # Optimize SegmentMerger work when a reader has no deletions.
> # IndexReader.document() will no longer check if the document is deleted.
> Will post a patch shortly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1708) Improve the use of isDeleted in the
indexing code
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12722789#action_12722789 ]
Michael McCandless commented on LUCENE-1708:
--------------------------------------------
bq. Not all cleanups are bad,
Oh, they are all good cleanups! I just wish they were in a dedicated "cleanup" issue, somehow, instead... because then it's harder for me to focus on the "real" changes (though I realize "clean as you go" is mighty convenient).
> Improve the use of isDeleted in the indexing code
> -------------------------------------------------
>
> Key: LUCENE-1708
> URL: https://issues.apache.org/jira/browse/LUCENE-1708
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Shai Erera
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1708.patch, LUCENE-1708.patch
>
>
> A spin off from here: http://www.nabble.com/Some-thoughts-around-the-use-of-reader.isDeleted-and-hasDeletions-td23931216.html.
> Two changes:
> # Optimize SegmentMerger work when a reader has no deletions.
> # IndexReader.document() will no longer check if the document is deleted.
> Will post a patch shortly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-1708) Improve the use of isDeleted in the
indexing code
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless reassigned LUCENE-1708:
------------------------------------------
Assignee: Michael McCandless
> Improve the use of isDeleted in the indexing code
> -------------------------------------------------
>
> Key: LUCENE-1708
> URL: https://issues.apache.org/jira/browse/LUCENE-1708
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Shai Erera
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1708.patch
>
>
> A spin off from here: http://www.nabble.com/Some-thoughts-around-the-use-of-reader.isDeleted-and-hasDeletions-td23931216.html.
> Two changes:
> # Optimize SegmentMerger work when a reader has no deletions.
> # IndexReader.document() will no longer check if the document is deleted.
> Will post a patch shortly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1708) Improve the use of isDeleted in the
indexing code
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-1708.
----------------------------------------
Resolution: Fixed
Thanks Shai!
> Improve the use of isDeleted in the indexing code
> -------------------------------------------------
>
> Key: LUCENE-1708
> URL: https://issues.apache.org/jira/browse/LUCENE-1708
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Shai Erera
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1708.patch, LUCENE-1708.patch
>
>
> A spin off from here: http://www.nabble.com/Some-thoughts-around-the-use-of-reader.isDeleted-and-hasDeletions-td23931216.html.
> Two changes:
> # Optimize SegmentMerger work when a reader has no deletions.
> # IndexReader.document() will no longer check if the document is deleted.
> Will post a patch shortly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1708) Improve the use of isDeleted in the
indexing code
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726543#action_12726543 ]
Mark Miller commented on LUCENE-1708:
-------------------------------------
I cant find the back compat discussion on this. Did I miss?
I do see where mike mentions the isDeleted change should happen with deprecation, but this patch appears to just make the change. Our current back compat policy doesn't allow that. It doesn't even really allow the really special exceptions that we have had to make (and I think it should spell that out, as well as our 'experiemental trick'). We have not officially allowed a relaxed back compat policy right?
I'm not trying to police back compat, but I think we need to do our best to live up to it until its officially changed. Having had to make a couple exceptions doesn't mean we can just toss it for this release.
Sorry if I missed the relevant discussion on this - didn't see anything in the attached email thread.
> Improve the use of isDeleted in the indexing code
> -------------------------------------------------
>
> Key: LUCENE-1708
> URL: https://issues.apache.org/jira/browse/LUCENE-1708
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Shai Erera
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1708.patch, LUCENE-1708.patch
>
>
> A spin off from here: http://www.nabble.com/Some-thoughts-around-the-use-of-reader.isDeleted-and-hasDeletions-td23931216.html.
> Two changes:
> # Optimize SegmentMerger work when a reader has no deletions.
> # IndexReader.document() will no longer check if the document is deleted.
> Will post a patch shortly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1708) Improve the use of isDeleted in the
indexing code
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726563#action_12726563 ]
Mark Miller commented on LUCENE-1708:
-------------------------------------
I suppose, to be fair, we do mention that we might change runtime behaviour and document it - its just that we don't usually say, code around it.
I guess its simple enough here thats its not really a big deal. I was just surprised I saw no mention of back compat in the discussion other than Mike mentioning that the change should be made through deprecation early on in the attached email thread.
> Improve the use of isDeleted in the indexing code
> -------------------------------------------------
>
> Key: LUCENE-1708
> URL: https://issues.apache.org/jira/browse/LUCENE-1708
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Shai Erera
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1708.patch, LUCENE-1708.patch
>
>
> A spin off from here: http://www.nabble.com/Some-thoughts-around-the-use-of-reader.isDeleted-and-hasDeletions-td23931216.html.
> Two changes:
> # Optimize SegmentMerger work when a reader has no deletions.
> # IndexReader.document() will no longer check if the document is deleted.
> Will post a patch shortly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1708) Improve the use of isDeleted in the
indexing code
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shai Erera updated LUCENE-1708:
-------------------------------
Attachment: LUCENE-1708.patch
Patch includes changes to SegmentMerger, IndexReader, SegmentReader, CHANGES, and TestSegmentReader (on tags as well).
I think this is ready to commit (all tests pass). Before commit, we should complete the "Shai Erera via" in CHANGES.
> Improve the use of isDeleted in the indexing code
> -------------------------------------------------
>
> Key: LUCENE-1708
> URL: https://issues.apache.org/jira/browse/LUCENE-1708
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Shai Erera
> Fix For: 2.9
>
> Attachments: LUCENE-1708.patch
>
>
> A spin off from here: http://www.nabble.com/Some-thoughts-around-the-use-of-reader.isDeleted-and-hasDeletions-td23931216.html.
> Two changes:
> # Optimize SegmentMerger work when a reader has no deletions.
> # IndexReader.document() will no longer check if the document is deleted.
> Will post a patch shortly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1708) Improve the use of isDeleted in the
indexing code
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726580#action_12726580 ]
Shai Erera commented on LUCENE-1708:
------------------------------------
There is a paragraph in CHANGES under "Changes to Runtime Behavior" that explains this. I think it was on the email thread and not on this issue, that people preferred the runtime change vs. the deprecation and a new method name for document(), under the assumption that it's very unlikely that someone relies on IndexReader.document() checking for isDeleted (i.e., it passes a document which may or may not be deleted).
> Improve the use of isDeleted in the indexing code
> -------------------------------------------------
>
> Key: LUCENE-1708
> URL: https://issues.apache.org/jira/browse/LUCENE-1708
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Shai Erera
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1708.patch, LUCENE-1708.patch
>
>
> A spin off from here: http://www.nabble.com/Some-thoughts-around-the-use-of-reader.isDeleted-and-hasDeletions-td23931216.html.
> Two changes:
> # Optimize SegmentMerger work when a reader has no deletions.
> # IndexReader.document() will no longer check if the document is deleted.
> Will post a patch shortly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1708) Improve the use of isDeleted in the
indexing code
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726614#action_12726614 ]
Shai Erera commented on LUCENE-1708:
------------------------------------
bq. it was a quick back and forth at the end. Got lazy towards the end
Sometimes it's a valid way to pass a decision ;).
> Improve the use of isDeleted in the indexing code
> -------------------------------------------------
>
> Key: LUCENE-1708
> URL: https://issues.apache.org/jira/browse/LUCENE-1708
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Shai Erera
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1708.patch, LUCENE-1708.patch
>
>
> A spin off from here: http://www.nabble.com/Some-thoughts-around-the-use-of-reader.isDeleted-and-hasDeletions-td23931216.html.
> Two changes:
> # Optimize SegmentMerger work when a reader has no deletions.
> # IndexReader.document() will no longer check if the document is deleted.
> Will post a patch shortly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1708) Improve the use of isDeleted in the
indexing code
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12722644#action_12722644 ]
Michael McCandless commented on LUCENE-1708:
--------------------------------------------
I see lots of cleanups snuck in here ;)
> Improve the use of isDeleted in the indexing code
> -------------------------------------------------
>
> Key: LUCENE-1708
> URL: https://issues.apache.org/jira/browse/LUCENE-1708
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Shai Erera
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1708.patch
>
>
> A spin off from here: http://www.nabble.com/Some-thoughts-around-the-use-of-reader.isDeleted-and-hasDeletions-td23931216.html.
> Two changes:
> # Optimize SegmentMerger work when a reader has no deletions.
> # IndexReader.document() will no longer check if the document is deleted.
> Will post a patch shortly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1708) Improve the use of isDeleted in the
indexing code
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-1708:
---------------------------------------
Attachment: LUCENE-1708.patch
Attached another iteration on the patch, basically cosmetic changes (removing un-needed args, naming things more verbosely, etc.). I think it's ready to be committed. I'll wait a day or two.
> Improve the use of isDeleted in the indexing code
> -------------------------------------------------
>
> Key: LUCENE-1708
> URL: https://issues.apache.org/jira/browse/LUCENE-1708
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Shai Erera
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1708.patch, LUCENE-1708.patch
>
>
> A spin off from here: http://www.nabble.com/Some-thoughts-around-the-use-of-reader.isDeleted-and-hasDeletions-td23931216.html.
> Two changes:
> # Optimize SegmentMerger work when a reader has no deletions.
> # IndexReader.document() will no longer check if the document is deleted.
> Will post a patch shortly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1708) Improve the use of isDeleted in the
indexing code
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726593#action_12726593 ]
Mark Miller commented on LUCENE-1708:
-------------------------------------
bq. There is a paragraph in CHANGES under "Changes to Runtime Behavior" that explains this.
Right, I saw that - I just wondered about the discussion to do it.
bq. I think it was on the email thread and not on this issue, that people preferred the runtime change vs. the deprecation and a new method name for document(), under the assumption that it's very unlikely that someone relies on IndexReader.document() checking for isDeleted (i.e., it passes a document which may or may not be deleted).
Thanks - thats the discussion I wasn't able to spot.
> Improve the use of isDeleted in the indexing code
> -------------------------------------------------
>
> Key: LUCENE-1708
> URL: https://issues.apache.org/jira/browse/LUCENE-1708
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Shai Erera
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1708.patch, LUCENE-1708.patch
>
>
> A spin off from here: http://www.nabble.com/Some-thoughts-around-the-use-of-reader.isDeleted-and-hasDeletions-td23931216.html.
> Two changes:
> # Optimize SegmentMerger work when a reader has no deletions.
> # IndexReader.document() will no longer check if the document is deleted.
> Will post a patch shortly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1708) Improve the use of
isDeleted in the indexing code
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726593#action_12726593 ]
Mark Miller edited comment on LUCENE-1708 at 7/2/09 11:05 AM:
--------------------------------------------------------------
bq. There is a paragraph in CHANGES under "Changes to Runtime Behavior" that explains this.
Right, I saw that - I just wondered about the discussion to do it.
bq. I think it was on the email thread and not on this issue, that people preferred the runtime change vs. the deprecation and a new method name for document(), under the assumption that it's very unlikely that someone relies on IndexReader.document() checking for isDeleted (i.e., it passes a document which may or may not be deleted).
Thanks - thats the discussion I wasn't able to spot.
I see - it was a quick back and forth at the end. Got lazy towards the end :) Thanks for pointing out.
was (Author: markrmiller@gmail.com):
bq. There is a paragraph in CHANGES under "Changes to Runtime Behavior" that explains this.
Right, I saw that - I just wondered about the discussion to do it.
bq. I think it was on the email thread and not on this issue, that people preferred the runtime change vs. the deprecation and a new method name for document(), under the assumption that it's very unlikely that someone relies on IndexReader.document() checking for isDeleted (i.e., it passes a document which may or may not be deleted).
Thanks - thats the discussion I wasn't able to spot.
> Improve the use of isDeleted in the indexing code
> -------------------------------------------------
>
> Key: LUCENE-1708
> URL: https://issues.apache.org/jira/browse/LUCENE-1708
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Shai Erera
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1708.patch, LUCENE-1708.patch
>
>
> A spin off from here: http://www.nabble.com/Some-thoughts-around-the-use-of-reader.isDeleted-and-hasDeletions-td23931216.html.
> Two changes:
> # Optimize SegmentMerger work when a reader has no deletions.
> # IndexReader.document() will no longer check if the document is deleted.
> Will post a patch shortly
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org