You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Zach Chen (Jira)" <ji...@apache.org> on 2020/10/31 03:22:00 UTC
[jira] [Comment Edited] (LUCENE-8982) Make NativeUnixDirectory pure
java now that direct IO is possible
[ https://issues.apache.org/jira/browse/LUCENE-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223986#comment-17223986 ]
Zach Chen edited comment on LUCENE-8982 at 10/31/20, 3:21 AM:
--------------------------------------------------------------
Thanks Michael! I've opened an initial PR with the proposed changes, but also have a few follow-up questions:
# Are new unit tests (other than benchmarking tests) needed, given these changes may not have direct side-effects to assert on?
# Shall I go ahead and remove code that may no longer be used after these changes, such as the related method in *NativePosixUtil.cpp*?
The PR also needs to include some more updates to comment to reflect the new way of doing direct IO. I can work on that once the changes are very much ready.
was (Author: zacharymorn):
Thanks Michael! I've opened an initial PR with the proposed changes, but also have a few follow-up questions:
# Are new unit tests (other than benchmarking tests) needed, given these changes may not have direct side-effects to assert on?
# Shall I go ahead and remove code that may no longer be used after these changes, such as the related method in `NativePosixUtil.cpp`?
The PR also needs to include some more updates to comment to reflect the new way of doing direct IO. I can work on that once the changes are very much ready.
> Make NativeUnixDirectory pure java now that direct IO is possible
> -----------------------------------------------------------------
>
> Key: LUCENE-8982
> URL: https://issues.apache.org/jira/browse/LUCENE-8982
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/misc
> Reporter: Michael McCandless
> Priority: Major
>
> {{NativeUnixDirectory}} is a {{Directory}} implementation that uses direct IO to write newly merged segments. Direct IO bypasses the kernel's buffer cache and write cache, making merge writes "invisible" to the kernel, though the reads for merging the N segments are still going through the kernel.
> But today, {{NativeUnixDirectory}} uses a small JNI wrapper to access the {{O_DIRECT}} flag to {{open}} ... since JDK9 we can now pass that flag in pure java code, so we should now fix {{NativeUnixDirectory}} to not use JNI anymore.
> We should also run some more realistic benchmarks seeing if this option really helps nodes that are doing concurrent indexing (merging) and searching.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org