You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Michael McCandless (JIRA)" <ji...@apache.org> on 2007/03/03 15:42:50 UTC
[jira] Created: (LUCENE-820) SegmentReader.setNorm can fail to
remove separate norms file, on Windows
SegmentReader.setNorm can fail to remove separate norms file, on Windows
------------------------------------------------------------------------
Key: LUCENE-820
URL: https://issues.apache.org/jira/browse/LUCENE-820
Project: Lucene - Java
Issue Type: Bug
Components: Index
Affects Versions: 2.1
Reporter: Michael McCandless
Assigned To: Michael McCandless
Priority: Minor
Fix For: 2.2
While working through LUCENE-710 I hit this bug: on Windows
only, when SegmentReader.setNorm is called, but separate norms
(_X_N.sY) had already been previously saved, then, on closing the
reader, we will write the next gen separate norm file correctly
(_X_N+1.sY) but fail to delete the current one.
It's quite minor because the next writer to touch the index will
remove the stale file.
This is because the Norm class still holds the IndexInput open when
the reader commits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-820) SegmentReader.setNorm can fail to
remove separate norms file, on Windows
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12477664 ]
Yonik Seeley commented on LUCENE-820:
-------------------------------------
> This is also nice because it frees up file handles once norms are cached.
Ouch... the file handles should not have been open in the first place :-)
LUCENE-821 fixes that.
> SegmentReader.setNorm can fail to remove separate norms file, on Windows
> ------------------------------------------------------------------------
>
> Key: LUCENE-820
> URL: https://issues.apache.org/jira/browse/LUCENE-820
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.1
> Reporter: Michael McCandless
> Assigned To: Michael McCandless
> Priority: Minor
> Fix For: 2.2
>
> Attachments: LUCENE-820.patch
>
>
> While working through LUCENE-710 I hit this bug: on Windows
> only, when SegmentReader.setNorm is called, but separate norms
> (_X_N.sY) had already been previously saved, then, on closing the
> reader, we will write the next gen separate norm file correctly
> (_X_N+1.sY) but fail to delete the current one.
> It's quite minor because the next writer to touch the index will
> remove the stale file.
> This is because the Norm class still holds the IndexInput open when
> the reader commits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-820) SegmentReader.setNorm can fail to
remove separate norms file, on Windows
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12477707 ]
Michael McCandless commented on LUCENE-820:
-------------------------------------------
> Ouch... the file handles should not have been open in the first
> place :-) LUCENE-821 fixes that.
Excellent!
How about you commit LUCENE-821, then I'll update & merge, and commit
my unit test & improvements to MockRamDirectory to mimic Windows?
> SegmentReader.setNorm can fail to remove separate norms file, on Windows
> ------------------------------------------------------------------------
>
> Key: LUCENE-820
> URL: https://issues.apache.org/jira/browse/LUCENE-820
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.1
> Reporter: Michael McCandless
> Assigned To: Michael McCandless
> Priority: Minor
> Fix For: 2.2
>
> Attachments: LUCENE-820.patch
>
>
> While working through LUCENE-710 I hit this bug: on Windows
> only, when SegmentReader.setNorm is called, but separate norms
> (_X_N.sY) had already been previously saved, then, on closing the
> reader, we will write the next gen separate norm file correctly
> (_X_N+1.sY) but fail to delete the current one.
> It's quite minor because the next writer to touch the index will
> remove the stale file.
> This is because the Norm class still holds the IndexInput open when
> the reader commits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-820) SegmentReader.setNorm can fail to
remove separate norms file, on Windows
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-820:
--------------------------------------
Attachment: LUCENE-820.patch
I've attached a patch that fixes this.
The fix is to close the IndexInput after we've cached the norms in the
in-memory byte array. Once we've cached the norms, we never use that
IndexInput again (as far as I can tell?). This is also nice because
it frees up file handles once norms are cached.
Changes:
* Fixed TestMultiSearcher: it was closing a searcher and then
continuing to use it.
* Added unit test case showing this bug before the patch.
* Improved MockRAMDirectory to "act like Windows" by refusing to
delete or overwrite an open file. I removed "final" from 2
methods (deleteFile, openInput) of RAMDirectory for this.
* Changed SegmentReader.getNorms to close the Norms after caching.
> SegmentReader.setNorm can fail to remove separate norms file, on Windows
> ------------------------------------------------------------------------
>
> Key: LUCENE-820
> URL: https://issues.apache.org/jira/browse/LUCENE-820
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.1
> Reporter: Michael McCandless
> Assigned To: Michael McCandless
> Priority: Minor
> Fix For: 2.2
>
> Attachments: LUCENE-820.patch
>
>
> While working through LUCENE-710 I hit this bug: on Windows
> only, when SegmentReader.setNorm is called, but separate norms
> (_X_N.sY) had already been previously saved, then, on closing the
> reader, we will write the next gen separate norm file correctly
> (_X_N+1.sY) but fail to delete the current one.
> It's quite minor because the next writer to touch the index will
> remove the stale file.
> This is because the Norm class still holds the IndexInput open when
> the reader commits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-820) SegmentReader.setNorm can fail to
remove separate norms file, on Windows
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-820.
---------------------------------------
Resolution: Fixed
This root cause of this bug was already fixed by LUCENE-821 (thanks Yonik!).
I committed a test case that shows this bug (and now passes), plus improvements to MockRAMDirectory to make it behave like Windows ("can't delete open files").
> SegmentReader.setNorm can fail to remove separate norms file, on Windows
> ------------------------------------------------------------------------
>
> Key: LUCENE-820
> URL: https://issues.apache.org/jira/browse/LUCENE-820
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.1
> Reporter: Michael McCandless
> Assigned To: Michael McCandless
> Priority: Minor
> Fix For: 2.2
>
> Attachments: LUCENE-820.patch
>
>
> While working through LUCENE-710 I hit this bug: on Windows
> only, when SegmentReader.setNorm is called, but separate norms
> (_X_N.sY) had already been previously saved, then, on closing the
> reader, we will write the next gen separate norm file correctly
> (_X_N+1.sY) but fail to delete the current one.
> It's quite minor because the next writer to touch the index will
> remove the stale file.
> This is because the Norm class still holds the IndexInput open when
> the reader commits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org