You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Philippe Marschall (JIRA)" <ji...@apache.org> on 2018/12/20 16:15:00 UTC
[jira] [Comment Edited] (LUCENE-8617) FSDirectory tries to create
MMapDirectory on non default file system
[ https://issues.apache.org/jira/browse/LUCENE-8617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16725986#comment-16725986 ]
Philippe Marschall edited comment on LUCENE-8617 at 12/20/18 4:14 PM:
----------------------------------------------------------------------
{quote}We provide mmap with non-default filesystems for testing purposes and we override the mmap() method:
{quote}
Yes you implement and override the the mmap() method but in the end you delegate to a `FileChannel` instance created on the default file system. All the file system you provide are actually "just" decorations around the default file system. You have to because you can not create a subclass or instance of `MappedByteBuffer` because all the constructors are package-private. The only ways to get your hands on a `MappedByteBuffer` is by either calling `FileChannel#map` or calling `ByteBuffer#allocateDirect()`.
Implementing `FileChannel#map` by calling `FileChannel#map` means it will only work on the default file system as only it can create instance of `MappedByteBuffer`.
Implementing `FileChannel#map` by calling `ByteBuffer#allocateDirect()` relies on implementation quirks and would be quite tricky.
{quote}That change would have a real impact on test coverage since we use non-default filesystems to check for things like file descriptor leaks{quote}
That's a valid point, I was not aware of this. My use case case is very niche, an in-memory file system for testing, and a work around exists. I don't see an easy way to detect wether a file system supports mmap() other than by calling it.
was (Author: marschall):
> We provide mmap with non-default filesystems for testing purposes and we override the mmap() method:
Yes you implement and override the the mmap() method but in the end you delegate to a `FileChannel` instance created on the default file system. All the file system you provide are actually "just" decorations around the default file system. You have to because you can not create a subclass or instance of `MappedByteBuffer` because all the constructors are package-private. The only ways to get your hands on a `MappedByteBuffer` is by either calling `FileChannel#map` or calling `ByteBuffer#allocateDirect()`.
Implementing `FileChannel#map` by calling `FileChannel#map` means it will only work on the default file system as only it can create instance of `MappedByteBuffer`.
Implementing `FileChannel#map` by calling `ByteBuffer#allocateDirect()` relies on implementation quirks and would be quite tricky.
> That change would have a real impact on test coverage since we use non-default filesystems to check for things like file descriptor leaks
That's a valid point, I was not aware of this. My use case case is very niche, an in-memory file system for testing, and a work around exists. I don't see an easy way to detect wether a file system supports mmap() other than by calling it.
> FSDirectory tries to create MMapDirectory on non default file system
> --------------------------------------------------------------------
>
> Key: LUCENE-8617
> URL: https://issues.apache.org/jira/browse/LUCENE-8617
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/store
> Affects Versions: 5.0
> Reporter: Philippe Marschall
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> `org.apache.lucene.store.FSDirectory.open(Path, LockFactory)` and `org.apache.lucene.store.FSDirectory.open(Path)` always assume the path is created on the default file system. If the path is not on the default file system `MMapDirectory` is the wrong choice as only the default file system can provide `mmap()`.
> In case of a non default file system `SimpleFSDirectory` is a good choice.
> See this bug for an example
> https://github.com/marschall/memoryfilesystem/issues/113
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org