You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Shai Erera (JIRA)" <ji...@apache.org> on 2011/06/24 13:18:47 UTC
[jira] [Created] (LUCENE-3237) FSDirectory.fsync() may not work
properly
FSDirectory.fsync() may not work properly
-----------------------------------------
Key: LUCENE-3237
URL: https://issues.apache.org/jira/browse/LUCENE-3237
Project: Lucene - Java
Issue Type: Bug
Components: core/store
Reporter: Shai Erera
Fix For: 3.4, 4.0
Spinoff from LUCENE-3230. FSDirectory.fsync() opens a new RAF, sync() its FileDescriptor and closes RAF. It is not clear that this syncs whatever was written to the file by other FileDescriptors. It would be better if we do this operation on the actual RAF/FileOS which wrote the data. We can add sync() to IndexOutput and FSIndexOutput will do that.
Directory-wise, we should stop syncing on file names, and instead sync on the IOs that performed the write operations.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3237) FSDirectory.fsync() may not work
properly
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098958#comment-13098958 ]
Yonik Seeley commented on LUCENE-3237:
--------------------------------------
The man pages on fsync (which I assume FileDescriptor.sync() uses seem to say that it affects all in-core modified copies of buffers (as opposed to just modifications done through that specific descriptor). If you think about it, it's really the only sane behavior.
> FSDirectory.fsync() may not work properly
> -----------------------------------------
>
> Key: LUCENE-3237
> URL: https://issues.apache.org/jira/browse/LUCENE-3237
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/store
> Reporter: Shai Erera
> Fix For: 3.4, 4.0
>
>
> Spinoff from LUCENE-3230. FSDirectory.fsync() opens a new RAF, sync() its FileDescriptor and closes RAF. It is not clear that this syncs whatever was written to the file by other FileDescriptors. It would be better if we do this operation on the actual RAF/FileOS which wrote the data. We can add sync() to IndexOutput and FSIndexOutput will do that.
> Directory-wise, we should stop syncing on file names, and instead sync on the IOs that performed the write operations.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3237) FSDirectory.fsync() may not work
properly
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-3237:
---------------------------------------
Fix Version/s: (was: 3.4)
3.5
> FSDirectory.fsync() may not work properly
> -----------------------------------------
>
> Key: LUCENE-3237
> URL: https://issues.apache.org/jira/browse/LUCENE-3237
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/store
> Reporter: Shai Erera
> Fix For: 3.5, 4.0
>
>
> Spinoff from LUCENE-3230. FSDirectory.fsync() opens a new RAF, sync() its FileDescriptor and closes RAF. It is not clear that this syncs whatever was written to the file by other FileDescriptors. It would be better if we do this operation on the actual RAF/FileOS which wrote the data. We can add sync() to IndexOutput and FSIndexOutput will do that.
> Directory-wise, we should stop syncing on file names, and instead sync on the IOs that performed the write operations.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-3237) FSDirectory.fsync() may not work
properly
Posted by "Shai Erera (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shai Erera resolved LUCENE-3237.
--------------------------------
Resolution: Won't Fix
Fix Version/s: (was: 3.5)
(was: 4.0)
Closing. If we ever see that this actually is a problem, we can reopen.
> FSDirectory.fsync() may not work properly
> -----------------------------------------
>
> Key: LUCENE-3237
> URL: https://issues.apache.org/jira/browse/LUCENE-3237
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/store
> Reporter: Shai Erera
>
> Spinoff from LUCENE-3230. FSDirectory.fsync() opens a new RAF, sync() its FileDescriptor and closes RAF. It is not clear that this syncs whatever was written to the file by other FileDescriptors. It would be better if we do this operation on the actual RAF/FileOS which wrote the data. We can add sync() to IndexOutput and FSIndexOutput will do that.
> Directory-wise, we should stop syncing on file names, and instead sync on the IOs that performed the write operations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3237) FSDirectory.fsync() may not work
properly
Posted by "Simon Willnauer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149555#comment-13149555 ]
Simon Willnauer commented on LUCENE-3237:
-----------------------------------------
Shai, I think we should close this. we can still reopen if we run into issues?
> FSDirectory.fsync() may not work properly
> -----------------------------------------
>
> Key: LUCENE-3237
> URL: https://issues.apache.org/jira/browse/LUCENE-3237
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/store
> Reporter: Shai Erera
> Fix For: 3.5, 4.0
>
>
> Spinoff from LUCENE-3230. FSDirectory.fsync() opens a new RAF, sync() its FileDescriptor and closes RAF. It is not clear that this syncs whatever was written to the file by other FileDescriptors. It would be better if we do this operation on the actual RAF/FileOS which wrote the data. We can add sync() to IndexOutput and FSIndexOutput will do that.
> Directory-wise, we should stop syncing on file names, and instead sync on the IOs that performed the write operations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3237) FSDirectory.fsync() may not work
properly
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098981#comment-13098981 ]
Shai Erera commented on LUCENE-3237:
------------------------------------
It could be, and generally I think that it makes sense. Only the Java documentation is not so explicit about it - I wish it was. Because if Java was explicit, it would mean we wouldn't need to wonder what do the "man pages" say about this on Windows, AIX etc.
If others think that it makes sense that sync() affects all existing buffers, then I don't mind if we close that issue. We haven't seen any problems with this implementation so far, so it kinda reassuring ...
> FSDirectory.fsync() may not work properly
> -----------------------------------------
>
> Key: LUCENE-3237
> URL: https://issues.apache.org/jira/browse/LUCENE-3237
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/store
> Reporter: Shai Erera
> Fix For: 3.4, 4.0
>
>
> Spinoff from LUCENE-3230. FSDirectory.fsync() opens a new RAF, sync() its FileDescriptor and closes RAF. It is not clear that this syncs whatever was written to the file by other FileDescriptors. It would be better if we do this operation on the actual RAF/FileOS which wrote the data. We can add sync() to IndexOutput and FSIndexOutput will do that.
> Directory-wise, we should stop syncing on file names, and instead sync on the IOs that performed the write operations.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org