You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Mark Miller (JIRA)" <ji...@apache.org> on 2009/08/30 20:29:33 UTC
[jira] Created: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Improve IndexWriter javadoc on locking
--------------------------------------
Key: LUCENE-1877
URL: https://issues.apache.org/jira/browse/LUCENE-1877
Project: Lucene - Java
Issue Type: Improvement
Components: Javadocs
Reporter: Mark Miller
Priority: Trivial
Fix For: 2.9
A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
{code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
another <code>IndexWriter</code> on the same directory will lead to a
{@link LockObtainFailedException}. The {@link LockObtainFailedException}
is also thrown if an IndexReader on the same directory is used to delete documents
from the index.</p>{code}
Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Socket and file locks
Posted by Marvin Humphrey <ma...@rectangular.com>.
On Sun, Nov 22, 2009 at 10:36:57AM +0000, Thomas Mueller (JIRA) wrote:
> Thomas Mueller commented on LUCENE-1877:
> ----------------------------------------
>
> > take it somewhere other than this closed issue.
>
> Yes, where?
The java-dev list.
> > shouldn't active code like that live in the application layer?
>
> Why?
You can all but guarantee that polling will work at the app layer, because you
can have almost full control over process priority.
If the polling code is lower down and hidden away, then it worries me that a
lock might be swept away by another process, and by the time the original
process realizes that it doesn't hold the lock anymore, the damage could
already have been done. Unless I'm missing something, it doesn't seem like a
failsafe design.
But this is "theoretical", I suppose:
> I'm just trying to say that in theory, the thread is problematic, but in
> practice it isn't. While file locking is not a problem in theory, but in
> practice.
Heh. :)
> > What happens when the app sleeps?
>
> Good question! Standby / hibernate are not supported. I didn't think about
> that. Is there a way to detect the wakeup?
Not sure.
FYI, I'm only an indirect contributor to Java Lucene. My main projects are
Lucy and KinoSearch, loose ports to C. I know the problem domain intimately,
but my Java skills are sketchy.
> > host name and the pid
>
> Yes. It is not so easy to get the PID in Java, I found:
> http://stackoverflow.com/questions/35842/process-id-in-java
> "ManagementFactory.getRuntimeMXBean().getName()".
A web search for "java process id" turns up a bazillion hits about how to hack
up a PID.
How annoying. This seems to me like a case of the perfect being the enemy of
the good. How many machines that run Java are running operating systems that
have no support for PIDs?
Hasn't somebody open sourced a "GiveMeTheFrikkinPID" library yet?
> What do you do if the lock was generated by another machine?
Require that all machines participating in the writer pool supply a unique
host ID as part of the locking API. Store that host ID in the lockfile and
only allow machines to sweep stale files that they own.
Unfortunately, that's not failsafe either, though: misconfiguration leads to
index corruption rather than deadlock, when two machines that use identical
host IDs sweep each others lockfiles and write simultaneously.
> I tried with using a server socket, so you need the IP address, but
> unfortunately, sometimes the network is not configured correctly (but maybe
> it's possible to detect that). Maybe the two machines can't access each
> other over TCP/IP.
This is an intriguing approach. Can it be designed to be failsafe?
If the server and the client can't access each other, that's failsafe at
least, because the client will simply fail to acquire the lock.
But if a client is misconfigured, could it contact the wrong host,
successfully open a port that coincidentally happens to be open, believe it
has acquired the lock and corrupt the index? If so, could some sort of
handshake prevent that?
I'm also curious if we can use this approach for read locking. For that, you
need a reference counting scheme -- one ref for each reader accessing the
index. Is that possible under the socket model?
> > hard links
>
> Yes, but it looks like this doesn't work always.
It is theoretically possible for the link() call to return false incorrectly
when the hard link has actually been created, for instance because a network
problem prevents the "success" packet from getting back to the client from the
server.
However, this is failsafe, because the requestor will not believe that the
lock has been secured and thus won't write. That process won't be able to
sweep away the orphaned lock file itself, but once it exits, a graceful
recovery will occur.
Marvin Humphrey
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1877:
----------------------------------
Attachment: LUCENE-1877.patch
Here is a patch, that changes the default for ctor-based / open() based instantiations to use NativeFSLockFactory (in fact, if the supplied param to ctor is NULL). Also change javadocs.
It also deprecates the static FSDirectory.setDisableLocks() [which we should have done already]. One should simple use a ctor/open with NoLockFactory as param to do that.
Currently only TestIndexReader fails here on windows because of some strange lockfile-delete opeartions. Maybe the testcase must be updated. I will look into this.
If we want to go this way, we have to put this in 2.9.
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1877:
----------------------------------
Attachment: LUCENE-1877.patch
Final patch.
I implemented additional:
- all FS-based lock factories use the same prefix encoding. They are now (mostly) compatible. E.g. a lock obtained with NativeFSLockFactory would also be seen as locked with SimpleFSLockFactory.
- Added LockFactory ctors with no param. The FSDir will set the lockdir to itsself in this case.
- Added test for LUCENE-1885
Please test this extensively! I hope, I found all problems.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Marvin Humphrey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12781531#action_12781531 ]
Marvin Humphrey commented on LUCENE-1877:
-----------------------------------------
>> take it somewhere other than this closed issue.
>
> Yes, where?
The java-dev list: http://markmail.org/message/ivdgmxrivs3jzhfe
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749506#action_12749506 ]
Mark Miller commented on LUCENE-1877:
-------------------------------------
Oh, okay, cool - that makes me feel a little better.
Though new users seeing it as the default now - thats not the worst situation, but I would almost prefer the change go through a dev cycle as the default.
If others are not feeling as cautious, I wouldn't vote against.
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated LUCENE-1877:
--------------------------------
Comment: was deleted
(was: Is that a bug? I can't even find that class ...)
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Marvin Humphrey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749330#action_12749330 ]
Marvin Humphrey commented on LUCENE-1877:
-----------------------------------------
> Anyone remember why NativeFSLockFactory is not the default over
> SimpleFSLockFactory?
Wasn't it because native locking is somethings implemented with Fcntl, and
Fcntl locking blows chunks? Especially for a library rather than an
application.
>From the BSD manpage on Fcntl:
{quote}
This interface follows the completely stupid semantics of System V and IEEE
Std 1003.1-1988 (``POSIX.1'') that require that all locks associated with a
file for a given process are removed when any file descriptor for that file is
closed by that process. This semantic means that applications must be aware
of any files that a subroutine library may access. For example if an
application for updating the password file locks the password file database
while making the update, and then calls getpwname(3) to retrieve a record, the
lock will be lost because getpwname(3) opens, reads, and closes the password
database. The database close will release all locks that the process has
associated with the database, even if the library routine never requested a
lock on the database. Another minor semantic problem with this interface is
that locks are not inherited by a child process created using the fork(2)
function. The flock(2) interface has much more rational last close
semantics and allows locks to be inherited by child processes. Flock(2) is
recommended for applications that want to ensure the integrity of their locks
when using library routines or wish to pass locks to their children...
{quote}
The lockfile may be annoying, but at least it's guaranteed safe on all non-shared
volumes when the OS implements atomic file opening.
Are you folks at least able to clean up orphaned lockfiles if the PID it was created
under is no longer active?
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749301#action_12749301 ]
Uwe Schindler commented on LUCENE-1877:
---------------------------------------
For IndexWriter/IndexReader this hint is no longer needed (in Lucene 2.9), as all methods taking String/File instead of Directory are deprecated and users should create directory instances and then will automatically get to the place where the LockFactory can be supplied.
The note should be added to FSDirectory instead.
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Marvin Humphrey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780647#action_12780647 ]
Marvin Humphrey commented on LUCENE-1877:
-----------------------------------------
> http://www.h2database.com/html/advanced.html#file_locking_protocols
I'm a little concerned about the suitability of the polling approach for a
low-level library like Lucene -- shouldn't active code like that live in the
application layer? Is it possible to exceed the polling interval for a low
priority process on a system under heavy load? What happens when the app
sleeps?
For removing stale lock files, another technique is to incorporate the host
name and the pid. So long as you can determine that the lock file belongs to
your machine and that the PID is not active, you can safely zap it.
Then tricky bit is how you get that information into the lock file. If you
try to write that info to the lock file itself after an O_EXCL open, creating
a fully valid lock file is no longer an atomic operation.
The approach suggested by the creat(2) man page and endorsed in the Linux NFS
FAQ involves hard links:
{noformat}
The solution for performing atomic file locking using a lockfile
is to create a unique file on the same file system (e.g.,
incorporating hostname and pid), use link(2) to make a link to the
lockfile. If link() returns 0, the lock is successful. Otherwise,
use stat(2) on the unique file to check if its link count has
increased to 2, in which case the lock is also successful.
{noformat}
This approach should also work on Windows for NTFS file systems since Windows
2000 thanks to the CreateHardLink() function. (Samba file shares, you're out
of luck.) However, I'm not sure about the state of support for hard links in
Java.
If you're interested in continuing this discussion, we should probably take it
somewhere other than this closed issue.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler reassigned LUCENE-1877:
-------------------------------------
Assignee: Uwe Schindler
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Priority: Trivial
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750117#action_12750117 ]
Uwe Schindler commented on LUCENE-1877:
---------------------------------------
As nobody else objects, I will update the tests tomorrow and switch to NativeFSLockFactory for the new ctors and FSDir.open(). The old and deprectated API is unchanged.
Will go to bed now.
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Thomas Mueller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780540#action_12780540 ]
Thomas Mueller commented on LUCENE-1877:
----------------------------------------
FYI: other Java projects also implement exclusive locking, and automatic removal of such a file.
Apache Jackrabbit uses FileChannel.lock() by default, but there are problems with some NFS implementations (some don't release the lock after restart, some don't support locks at all). Also, some operating systems / file systems allow multiple write locks within the same process (possibly in different class loaders). Jackrabbit works around that by (mis-)using a system property. See http://wiki.apache.org/jackrabbit/RepositoryLock
For the H2 Database Engine I implemented a cooperative locking mechanism: http://www.h2database.com/html/advanced.html#file_locking_protocols - I have also ported that to Apache Jackrabbit ("Cooperative File Lock Mechanism"). It always works, but needs a background thread. H2 also supports a mechanism based on a server socket (open a server socket and write the IP address and port to the file) - but this is problematic if the network is misconfigured (localhost not bound to 127.0.0.1 and such) which does happen in practice.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750655#action_12750655 ]
Michael McCandless commented on LUCENE-1877:
--------------------------------------------
Patch looks good! Don't forget to fix back compat tests.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750646#action_12750646 ]
Michael McCandless commented on LUCENE-1877:
--------------------------------------------
bq. E.g. a lock obtained with NativeFSLockFactory would also be seen as locked with SimpleFSLockFactory.
This is neat, but I don't think we should advertise it?
Ie, it's unsupported to mix different LockFactory impls. EG, in this case, the reverse is not true, right?
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler resolved LUCENE-1877.
-----------------------------------
Resolution: Fixed
Committed revision: 811157
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750581#action_12750581 ]
Mark Miller commented on LUCENE-1877:
-------------------------------------
Interesting - yeah, its hard to follow it all :) I havn't had a chance to apply and look at your patch either.
My main issue with it is that there a bunch of places where it says you can set the lock factory that way (not in deprecated javadoc sections). We should prob remove all those.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749417#action_12749417 ]
Michael McCandless commented on LUCENE-1877:
--------------------------------------------
{quote}
Let's do it in the following way:
- deprecated FSDir.getDirectory() methods return the SimpleLockFactory, as it was before.
- The new FSDir.open() methods and also the direct ctors of SimpleFSDir, MMapFSDir, NIOFSDir default to NativeLocakFactory (these ctors/methods are all new in 2.9)
{quote}
+1
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1877:
----------------------------------
Priority: Major (was: Trivial)
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750565#action_12750565 ]
Mark Miller commented on LUCENE-1877:
-------------------------------------
Is that a bug? I can't even find that class ...
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750664#action_12750664 ]
Uwe Schindler commented on LUCENE-1877:
---------------------------------------
Yes, I already prepared the BW test change (i simply disabled this test with the lock prefix).
{quote}
bq. all FS-based lock factories use the same prefix encoding. They are now (mostly) compatible. E.g. a lock obtained with NativeFSLockFactory would also be seen as locked with SimpleFSLockFactory
Ie, it's unsupported to mix different LockFactory impls.
{quote}
One important fact, because I enabled this: The deprecated methods in IndexWriter/IndexReader taking String/File args and FSDirectory.getDirectory() still use the SimpleFSLockFactory per default (or the system property). If some use mixes these deprecated calls with e.g. FSDir.open() he has still a chance to get locking work. But it is unsupported. I also added a extra note in the CHANGES.txt now, that warns because of this trap.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749322#action_12749322 ]
Michael McCandless commented on LUCENE-1877:
--------------------------------------------
bq. Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
In my testing (long ago) over NFS, I actually found "native" locks didn't work as well as "simple" locks. I was also a bit nervous on how well supported "native" locks are across different OSs.
bq. My preference would really be to make it the default (though of course not for 2.9).
+1, I think it's the better default.
People who use Lucene over NFS already have to do special things (eg make a custom deletion policy), and far too many users hit the "leftover lock file" problem. We could state in the javadocs that this default will change in 3.0?
Maybe just add one sentence in that IndexWriter locking section, referencing the discussion in NativeFSLockFactory's javadocs about not having the "leftover lock file" problem?
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1877) Improve IndexWriter
javadoc on locking
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750117#action_12750117 ]
Uwe Schindler edited comment on LUCENE-1877 at 9/1/09 2:43 PM:
---------------------------------------------------------------
As nobody else objects, I will update the tests tomorrow and switch to NativeFSLockFactory for the new ctors and FSDir.open(). The old and deprectated API is unchanged.
I will also remove the unneeded lock prefix and use the same lock file name as SimpleFSLockFactory. This would also help users mixing both lock factories together (by using deprecated code defaulting to Simple and new code defaulting to Native). The SimpleLockFactory would also detect a lock, if the NativeFSLockFactory created it (because file has same name). The tests will then also pass (which depended on file name).
Will go to bed now.
was (Author: thetaphi):
As nobody else objects, I will update the tests tomorrow and switch to NativeFSLockFactory for the new ctors and FSDir.open(). The old and deprectated API is unchanged.
Will go to bed now.
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749475#action_12749475 ]
Uwe Schindler commented on LUCENE-1877:
---------------------------------------
With the above patch some more tests are failing, mostly because of the strange lock file names. I think we should fix the tests, at least hardcode the simplelock factory, when it should be tested.
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1877:
----------------------------------
Attachment: LUCENE-1877.patch
Formatting changes in CHANGES.txt and some minor tweaks.
Also changed the isLocked() method for LUCENE-1885 to shortcut, if no lock file is present. In this case, without a lockfile it may also be not locked. This prevent NativeFSLock for creating the lock short time without really using it.
It would be good, if somebody could also test this with strange file systems. I only tested Windows and Solaris
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1877:
----------------------------------
Summary: Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open) (was: Improve IndexWriter javadoc on locking)
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1877) Use NativeFSLockFactory
as default for new API (direct ctors & FSDir.open)
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750563#action_12750563 ]
Mark Miller edited comment on LUCENE-1877 at 9/2/09 10:58 AM:
--------------------------------------------------------------
heh - my jetlag is full effect - I wasn't looking at the lockdir, I was looking at:
{code} String lockClassName = System.getProperty("org.apache.lucene.store.FSDirectoryLockFactoryClass");
{code}
*edit
I'm too tired to be emailing - how about deprecating this one though?
was (Author: markrmiller@gmail.com):
heh - my jetlag is full effect - I wasn't looking at the lockdir, I was looking at:
{code} String lockClassName = System.getProperty("org.apache.lucene.store.FSDirectoryLockFactoryClass");
{code}
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749449#action_12749449 ]
Uwe Schindler commented on LUCENE-1877:
---------------------------------------
I will be happy when 3.0 removes all this FSDirectory deprecated stuff. Its a hell to maintain!
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749459#action_12749459 ]
Michael McCandless commented on LUCENE-1877:
--------------------------------------------
I think we should also fix NativeLockFactory so that if the write lock is in the index dir it doesn't generate the large digest in the file name. That digest is problematic when two different machines access the same physical dir via different mount names, since that results in different lock file names.
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749505#action_12749505 ]
Uwe Schindler commented on LUCENE-1877:
---------------------------------------
It's only the default for new code, clearly documented; deprecated code stays as it is.
If we will not get this into 2.9, 3.0 will remove the deprecated parts and the new code (new in 2.9) will change its defaults.
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750573#action_12750573 ]
Uwe Schindler commented on LUCENE-1877:
---------------------------------------
It is indirectly deprecated, as it is only used, when FSDir.getDirectory() is used. In all other cases NativeFSLockFactory is used or the given one. Maybe we should add a note somewhere in javadocs. The same with the default FSDir class property (its also indirectly deprecated.
I know the code is very ugly, but this is how it works :-(
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1877) Use NativeFSLockFactory
as default for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750632#action_12750632 ]
Uwe Schindler edited comment on LUCENE-1877 at 9/2/09 12:53 PM:
----------------------------------------------------------------
Final patch.
I implemented additional:
- all FS-based lock factories use the same prefix encoding. They are now (mostly) compatible. E.g. a lock obtained with NativeFSLockFactory would also be seen as locked with SimpleFSLockFactory.
- Added LockFactory ctors with no param. The FSDir will set the lockdir to itsself in this case.
- Added test for LUCENE-1885
- Added note about deprecation of all FSDir related system properties, fixed some docs
Please test this extensively! I hope, I found all problems.
was (Author: thetaphi):
Final patch.
I implemented additional:
- all FS-based lock factories use the same prefix encoding. They are now (mostly) compatible. E.g. a lock obtained with NativeFSLockFactory would also be seen as locked with SimpleFSLockFactory.
- Added LockFactory ctors with no param. The FSDir will set the lockdir to itsself in this case.
- Added test for LUCENE-1885
Please test this extensively! I hope, I found all problems.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749336#action_12749336 ]
Mark Miller commented on LUCENE-1877:
-------------------------------------
bq. People who use Lucene over NFS already have to do special things (eg make a custom deletion policy), and far too many users hit the "leftover lock file" problem. We could state in the javadocs that this default will change in 3.0?
+1 from me - if it made things work out of the box with NFS, I'd vote to keep as is, but the points you mention were in my head too.
My only worry is current users counting on this default for NFS - but if we put it in the back compat break section (a break in regards to NFS anyway), that should be sufficient warning?
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Thomas Mueller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12781111#action_12781111 ]
Thomas Mueller commented on LUCENE-1877:
----------------------------------------
> take it somewhere other than this closed issue.
Yes, where?
> shouldn't active code like that live in the application layer?
Why?
> exceed the polling interval for a low priority process on a system under heavy load?
The watchdog thread runs with high priority (see the H2 docs). It's still possible the thread isn't run at all, but highly unlikely. High priority threads are quite reliable. I wrote a MP3 player in Java (mp3transform) which I used a lot, I never heard any gaps. If the thread can be avoided, that would be great of course. I'm just trying to say that in theory, the thread is problematic, but in practice it isn't. While file locking is not a problem in theory, but in practice.
> What happens when the app sleeps?
Good question! Standby / hibernate are not supported. I didn't think about that. Is there a way to detect the wakeup?
> host name and the pid
Yes. It is not so easy to get the PID in Java, I found: http://stackoverflow.com/questions/35842/process-id-in-java "ManagementFactory.getRuntimeMXBean().getName()". What do you do if the lock was generated by another machine? I tried with using a server socket, so you need the IP address, but unfortunately, sometimes the network is not configured correctly (but maybe it's possible to detect that). Maybe the two machines can't access each other over TCP/IP.
> hard links
Yes, but it looks like this doesn't work always.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Marvin Humphrey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749363#action_12749363 ]
Marvin Humphrey commented on LUCENE-1877:
-----------------------------------------
> I can see how this is not ideal, but I'm not seeing how any of the
> mentioned issues apply to our simple lock usage ...
"Simple lock usage"?! You must have a bigger brain than me...
As a matter of fact, I think you're right. Fcntl locks have two major
drawbacks, and upon review I think NativeFSLockFactory avoids both of them.
The first is that opening and closing a file releases all locks for the entire
process. Even if you never request a lock on the second filehandle, or if you
request a lock and the request is denied, closing the filehandle releases the
lock on the first filehandle. But NativeFSLockFactory avoids that problem by
keeping the HashSet of filepaths and ensuring that the same file is never
opened twice. Furthermore, since the lockfiles are private to Lucene, you can
assume that nobody else is going to open them and inadvertently spoil the lock.
The second is that child processes spawned via fork() do not inherit locks
from the parent process. If you assume that nobody's ever going to fork a
Java process, that's not relevant. (Too bad that won't work for Lucy... we
have to support fork().)
I think you're probably safe with Fcntl locks on all non-shared volumes.
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749324#action_12749324 ]
Uwe Schindler commented on LUCENE-1877:
---------------------------------------
Let's do it in the following way:
- deprecated FSDir.getDirectory() methods return the SimpleLockFactory, as it was before.
- The new FSDir.open() methods and also the direct ctors of SimpleFSDir, MMapFSDir, NIOFSDir default to NativeLocakFactory (these ctors/methods are all new in 2.9)
Because of this we have no BW problem.
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749404#action_12749404 ]
Mark Miller commented on LUCENE-1877:
-------------------------------------
My brain has never been as large as I'd like it to be, but that's
never concerned me too greatly - it's my ego that I have the trouble
with.
- Mark
http://www.lucidimagination.com (mobile)
On Aug 30, 2009, at 11:25 PM, "Marvin Humphrey (JIRA)"
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750552#action_12750552 ]
Mark Miller commented on LUCENE-1877:
-------------------------------------
bq. Any thoughts, I would like to have that.
+1 - def a good idea.
I'd kind of like to deprecate the sys property to set the lock dir as well - we have done a good job of moving away from that stuff elsewhere.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1877:
----------------------------------
Attachment: LUCENE-1877.patch
Here is the patch:
- SimpleFSLockFactory and NativeFSLockFactory now have the same abstract superclass providing setLockDir and getLockDir. Using this method, it is possible for directory instances to detect, if the locks reside in the directory itsself and so a lock prefix is switched off.
- The isLocked() bug in NativeFSLockFactory (LUCENE-1885) is solved by implementing what was described in this issue.
I have one idea (which is a new feature): How about providing a ctor to NativeFSLockFactory and SimpleFSLockFactory without param. When this LF is added to a FSDir, it would default to set the LockDir to itsself (if lf.getLockDir()==null) lf.setLockDir(this.directory)). This would prevent users from always giving the directory twice? Any thoughts, I would like to have that.
Because of the missing lockPrefix for locks inside the directory itsself one backwards test (TestLockFactory) must be changed in backwards-branch, too.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749303#action_12749303 ]
Mark Miller commented on LUCENE-1877:
-------------------------------------
My initial thought was also that it didn't really belong in IndexWriter - but I sold myself on the fact that IndexWriter talks about locking and offers the force unlock method - so it seems fine to me to mention how to use the optimal locking factory (and generally avoid using the force unlock at all - as an aside I just saw a guy trying to use that the other day as regular code so that they could use two IndexWriters with just commit rather than close - ugg).
I'm not sold either way though - I'd go with whatever. My preference would really be to make it the default.
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750588#action_12750588 ]
Michael McCandless commented on LUCENE-1877:
--------------------------------------------
Patch looks good! I like the deprecation of FSDir.set/getDisableLocks and the new FSLockFactory approach.
bq. Maybe we should add a note somewhere in javadocs. The same with the default FSDir class property (its also indirectly deprecated
+1, I think we should deprecate these global system properties.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751190#action_12751190 ]
Uwe Schindler commented on LUCENE-1877:
---------------------------------------
I will commit soon!
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750650#action_12750650 ]
Uwe Schindler commented on LUCENE-1877:
---------------------------------------
bq. This is neat, but I don't think we should advertise it?
Definitely not.
b.q. Ie, it's unsupported to mix different LockFactory impls. EG, in this case, the reverse is not true, right?
Exactly.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750425#action_12750425 ]
Uwe Schindler commented on LUCENE-1877:
---------------------------------------
bq. I think we should also fix NativeLockFactory so that if the write lock is in the index dir it doesn't generate the large digest in the file name. That digest is problematic when two different machines access the same physical dir via different mount names, since that results in different lock file names.
The digest problem is not easy to solve: It happens for all LockFactories if they are not automatically created (when LockFactory==null). As soon as you cann FSDir.open(..., new SimpleLockFactory(...)) you also get these prefix. It does not appear, when the FSDir is created by FSDir.getDirectory(), as the init() method cleans the lockPrefix directly after setting the lockfactory (the lock factory setter sets the prefix).
The prefix is only important, if the lock is not placed inside the index directory. The best would be that FSDir would simply return null in getLockId, when the LockFactory uses the same path as the Directory. For that to work, the LockFactory should have a getter for the fs path.
I will try some possibilities and post a patch.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750560#action_12750560 ]
Uwe Schindler commented on LUCENE-1877:
---------------------------------------
bq. I'd kind of like to deprecate the sys property to set the lock dir as well - we have done a good job of moving away from that stuff elsewhere.
It is already deprecated and even not used anymore:
{code}
/**
* Directory specified by <code>org.apache.lucene.lockDir</code>
* or <code>java.io.tmpdir</code> system property.
* @deprecated As of 2.1, <code>LOCK_DIR</code> is unused
* because the write.lock is now stored by default in the
* index directory. If you really want to store locks
* elsewhere you can create your own {@link
* SimpleFSLockFactory} (or {@link NativeFSLockFactory},
* etc.) passing in your preferred lock directory. Then,
* pass this <code>LockFactory</code> instance to one of
* the <code>getDirectory</code> methods that take a
* <code>lockFactory</code> (for example, {@link #getDirectory(String, LockFactory)}).
*/
public static final String LOCK_DIR = System.getProperty("org.apache.lucene.lockDir",
System.getProperty("java.io.tmpdir"));
{code}
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750563#action_12750563 ]
Mark Miller commented on LUCENE-1877:
-------------------------------------
heh - my jetlag is full effect - I wasn't looking at the lockdir, I was looking at:
{code} String lockClassName = System.getProperty("org.apache.lucene.store.FSDirectoryLockFactoryClass");
{code}
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749502#action_12749502 ]
Mark Miller commented on LUCENE-1877:
-------------------------------------
bq. If we want to go this way, we have to put this in 2.9.
I'd personally be a little (to a lot) afraid to change the default to native during freeze -
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Use NativeFSLockFactory as default
for new API (direct ctors & FSDir.open)
Posted by "Thomas Mueller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12781113#action_12781113 ]
Thomas Mueller commented on LUCENE-1877:
----------------------------------------
> detect the wakeup / polling interval exceeded
An obvious solution is to use System.currentTimeMillis() and compare with the expected delay. And then stop writing and throw a exception.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1877) Improve IndexWriter javadoc on
locking
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749334#action_12749334 ]
Mark Miller commented on LUCENE-1877:
-------------------------------------
{quote}This interface follows the completely stupid semantics of System V and IEEE
Std 1003.1-1988 (``POSIX.1'') that require that all locks associated with a
file for a given process are removed when any file descriptor for that file is
closed by that process. This semantic means that applications must be aware
of any files that a subroutine library may access. For example if an
application for updating the password file locks the password file database
while making the update, and then calls getpwname(3) to retrieve a record, the
lock will be lost because getpwname(3) opens, reads, and closes the password
database. The database close will release all locks that the process has
associated with the database, even if the library routine never requested a
lock on the database. Another minor semantic problem with this interface is
that locks are not inherited by a child process created using the fork(2)
function. The flock(2) interface has much more rational last close
semantics and allows locks to be inherited by child processes. Flock(2) is
recommended for applications that want to ensure the integrity of their locks
when using library routines or wish to pass locks to their children... {quote}
I can see how this is not ideal, but I'm not seeing how any of the mentioned issues apply to our simple lock usage ...
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1877) Use NativeFSLockFactory
as default for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750425#action_12750425 ]
Uwe Schindler edited comment on LUCENE-1877 at 9/2/09 6:12 AM:
---------------------------------------------------------------
bq. I think we should also fix NativeLockFactory so that if the write lock is in the index dir it doesn't generate the large digest in the file name. That digest is problematic when two different machines access the same physical dir via different mount names, since that results in different lock file names.
The digest problem is not easy to solve: It happens for all LockFactories if they are not automatically created (when LockFactory==null). As soon as you call FSDir.open(..., new SimpleLockFactory(...)) you also get this prefix. It does not appear, when the FSDir is created by FSDir.getDirectory(), as the init() method cleans the lockPrefix directly after setting the lockfactory (the lock factory setter sets the prefix).
The prefix is only important, if the lock is not placed inside the index directory. The best would be that FSDir would simply return null in getLockId(), when the LockFactory uses the same path as the Directory. For that to work, the LockFactory should have a getter for the fs path.
I will try some possibilities and post a patch.
was (Author: thetaphi):
bq. I think we should also fix NativeLockFactory so that if the write lock is in the index dir it doesn't generate the large digest in the file name. That digest is problematic when two different machines access the same physical dir via different mount names, since that results in different lock file names.
The digest problem is not easy to solve: It happens for all LockFactories if they are not automatically created (when LockFactory==null). As soon as you cann FSDir.open(..., new SimpleLockFactory(...)) you also get these prefix. It does not appear, when the FSDir is created by FSDir.getDirectory(), as the init() method cleans the lockPrefix directly after setting the lockfactory (the lock factory setter sets the prefix).
The prefix is only important, if the lock is not placed inside the index directory. The best would be that FSDir would simply return null in getLockId, when the LockFactory uses the same path as the Directory. For that to work, the LockFactory should have a getter for the fs path.
I will try some possibilities and post a patch.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1877) Improve IndexWriter
javadoc on locking
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749303#action_12749303 ]
Mark Miller edited comment on LUCENE-1877 at 8/30/09 11:53 AM:
---------------------------------------------------------------
My initial thought was also that it didn't really belong in IndexWriter - but I sold myself on the fact that IndexWriter talks about locking and offers the force unlock method - so it seems fine to me to mention how to use the optimal locking factory (and generally avoid using the force unlock at all - as an aside I just saw a guy trying to use that the other day as regular code so that they could use two IndexWriters with just commit rather than close - ugg).
I'm not sold either way though - I'd go with whatever. My preference would really be to make it the default (though of course not for 2.9).
was (Author: markrmiller@gmail.com):
My initial thought was also that it didn't really belong in IndexWriter - but I sold myself on the fact that IndexWriter talks about locking and offers the force unlock method - so it seems fine to me to mention how to use the optimal locking factory (and generally avoid using the force unlock at all - as an aside I just saw a guy trying to use that the other day as regular code so that they could use two IndexWriters with just commit rather than close - ugg).
I'm not sold either way though - I'd go with whatever. My preference would really be to make it the default.
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1877) Use NativeFSLockFactory
as default for new API (direct ctors & FSDir.open)
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750549#action_12750549 ]
Uwe Schindler edited comment on LUCENE-1877 at 9/2/09 10:35 AM:
----------------------------------------------------------------
Here is the patch:
- SimpleFSLockFactory and NativeFSLockFactory now have the same abstract superclass providing setLockDir and getLockDir. Using this method, it is possible for directory instances to detect, if the locks reside in the directory itsself and so a lock prefix is switched off.
- The isLocked() bug in NativeFSLockFactory (LUCENE-1885) is solved by implementing what was described in this issue.
- aquireTestLock in NativeFSLockFactory was removed from ctor and only called for the first makeLock() call. This prevents the LockFactory from creating the directory when not needed (e.g. opening non-existent index).
I have one idea (which is a new feature): How about providing a ctor to NativeFSLockFactory and SimpleFSLockFactory without param. When this LF is added to a FSDir, it would default to set the LockDir to itsself (if lf.getLockDir()==null) lf.setLockDir(this.directory)). This would prevent users from always giving the directory twice? Any thoughts, I would like to have that.
Because of the missing lockPrefix for locks inside the directory itsself one backwards test (TestLockFactory) must be changed in backwards-branch, too.
was (Author: thetaphi):
Here is the patch:
- SimpleFSLockFactory and NativeFSLockFactory now have the same abstract superclass providing setLockDir and getLockDir. Using this method, it is possible for directory instances to detect, if the locks reside in the directory itsself and so a lock prefix is switched off.
- The isLocked() bug in NativeFSLockFactory (LUCENE-1885) is solved by implementing what was described in this issue.
I have one idea (which is a new feature): How about providing a ctor to NativeFSLockFactory and SimpleFSLockFactory without param. When this LF is added to a FSDir, it would default to set the LockDir to itsself (if lf.getLockDir()==null) lf.setLockDir(this.directory)). This would prevent users from always giving the directory twice? Any thoughts, I would like to have that.
Because of the missing lockPrefix for locks inside the directory itsself one backwards test (TestLockFactory) must be changed in backwards-branch, too.
> Use NativeFSLockFactory as default for new API (direct ctors & FSDir.open)
> --------------------------------------------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch, LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1877) Improve IndexWriter
javadoc on locking
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749475#action_12749475 ]
Uwe Schindler edited comment on LUCENE-1877 at 8/31/09 5:17 AM:
----------------------------------------------------------------
With the above patch some more tests are failing, mostly because of the strange lock file names. I think we should fix the tests, at least hardcode the simplelock factory, when it should be tested.
The backwards-tests seem to pass, as they only use FSDir.getDirectory() which defaults to old standard. That's good.
was (Author: thetaphi):
With the above patch some more tests are failing, mostly because of the strange lock file names. I think we should fix the tests, at least hardcode the simplelock factory, when it should be tested.
> Improve IndexWriter javadoc on locking
> --------------------------------------
>
> Key: LUCENE-1877
> URL: https://issues.apache.org/jira/browse/LUCENE-1877
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Javadocs
> Reporter: Mark Miller
> Priority: Trivial
> Fix For: 2.9
>
> Attachments: LUCENE-1877.patch
>
>
> A user requested we add a note in IndexWriter alerting the availability of NativeFSLockFactory (allowing you to avoid retaining locks on abnormal jvm exit). Seems reasonable to me - we want users to be able to easily stumble upon this class. The below code looks like a good spot to add a note - could also improve whats there a bit - opening an IndexWriter does not necessarily create a lock file - that would depend on the LockFactory used.
> {code} <p>Opening an <code>IndexWriter</code> creates a lock file for the directory in use. Trying to open
> another <code>IndexWriter</code> on the same directory will lead to a
> {@link LockObtainFailedException}. The {@link LockObtainFailedException}
> is also thrown if an IndexReader on the same directory is used to delete documents
> from the index.</p>{code}
> Anyone remember why NativeFSLockFactory is not the default over SimpleFSLockFactory?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org