You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Zach Chen (Jira)" <ji...@apache.org> on 2020/11/17 01:51:00 UTC
[jira] [Comment Edited] (LUCENE-8982) Make NativeUnixDirectory pure
java now that direct IO is possible
[ https://issues.apache.org/jira/browse/LUCENE-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233183#comment-17233183 ]
Zach Chen edited comment on LUCENE-8982 at 11/17/20, 1:50 AM:
--------------------------------------------------------------
Sorry for the late response, just got off work and see this.
From the discussion it seems the assumption / reality here is that cpp toolchain may or may not be available in the VMs. However, since Lucene does have native code and scheduled build can discover any change that breaks the native-java integration early on (there was actually one commit before this that broke it), should the build in general assume cpp toolchain to be there in the VMs (and add them if they are missing) to execute the compilation and tests, but still have -Pbuild.native=false as default to not break builds for others and have a few VMs with cpp toolchain intentionally left out to test for compatibility?
was (Author: zacharymorn):
Sorry for the late response, just got off work and see this.
From the discussion it seems the assumption / reality here is that cpp toolchain may or may not be available in the VMs. However, since Lucene does have native code and scheduled build can discover any change that breaks the native-java integration early on (there was actually one commit before this that broke it), should the build in general assume cpp toolchain to be there in the VMs (and add them if they are missing), but still have -Pbuild.native=false as default to not break builds for others and have a few VMs with cpp toolchain intentionally left out to test for compatibility?
> Make NativeUnixDirectory pure java now that direct IO is possible
> -----------------------------------------------------------------
>
> Key: LUCENE-8982
> URL: https://issues.apache.org/jira/browse/LUCENE-8982
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/misc
> Reporter: Michael McCandless
> Assignee: Dawid Weiss
> Priority: Major
> Time Spent: 9h
> Remaining Estimate: 0h
>
> {{NativeUnixDirectory}} is a {{Directory}} implementation that uses direct IO to write newly merged segments. Direct IO bypasses the kernel's buffer cache and write cache, making merge writes "invisible" to the kernel, though the reads for merging the N segments are still going through the kernel.
> But today, {{NativeUnixDirectory}} uses a small JNI wrapper to access the {{O_DIRECT}} flag to {{open}} ... since JDK9 we can now pass that flag in pure java code, so we should now fix {{NativeUnixDirectory}} to not use JNI anymore.
> We should also run some more realistic benchmarks seeing if this option really helps nodes that are doing concurrent indexing (merging) and searching.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org