You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Michael McCandless (JIRA)" <ji...@apache.org> on 2009/11/11 20:04:39 UTC
[jira] Created: (LUCENE-2056) Should NIOFSDir use direct
ByteBuffers?
Should NIOFSDir use direct ByteBuffers?
---------------------------------------
Key: LUCENE-2056
URL: https://issues.apache.org/jira/browse/LUCENE-2056
Project: Lucene - Java
Issue Type: Improvement
Components: Store
Reporter: Michael McCandless
Priority: Minor
I'm trying to test NRT performance, and noticed when I dump the thread stacks that the darned threads often seem to be in {{java.nio.Bits.copyToByteArray(Native Method)}}... so I wondered whether we could/should use direct ByteBuffers, and whether that would gain performance in general. We currently just use our own byte[] buffer via BufferedIndexInput.
It's hard to test since it's likely platform specific, but if it does result in gains it could be an easy win.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2056) Should NIOFSDir use direct
ByteBuffers?
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776611#action_12776611 ]
Michael McCandless commented on LUCENE-2056:
--------------------------------------------
Hmm not quite a ringing endorsement ;) I don't think I'll look into this any time soon (plenty on my plate!!) so if someone else wants to try, maybe with Java 7, go for it!
> Should NIOFSDir use direct ByteBuffers?
> ---------------------------------------
>
> Key: LUCENE-2056
> URL: https://issues.apache.org/jira/browse/LUCENE-2056
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Store
> Reporter: Michael McCandless
> Priority: Minor
>
> I'm trying to test NRT performance, and noticed when I dump the thread stacks that the darned threads often seem to be in {{java.nio.Bits.copyToByteArray(Native Method)}}... so I wondered whether we could/should use direct ByteBuffers, and whether that would gain performance in general. We currently just use our own byte[] buffer via BufferedIndexInput.
> It's hard to test since it's likely platform specific, but if it does result in gains it could be an easy win.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2056) Should NIOFSDir use direct
ByteBuffers?
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776587#action_12776587 ]
Yonik Seeley commented on LUCENE-2056:
--------------------------------------
I have an uncomfortable feeling that it will be slower.
IIRC, there's no way to get a byte[] from a direct byte buffer, so all of our methods that get a byte at a time will be making method calls. If those calls were directly implemented by the JVM as intrinsics... perhaps it would be faster. In general though, I've learned to lower my expectations (compared to the hype we've sometimes heard from Sun) and sometimes I'm pleasantly surprised :-)
> Should NIOFSDir use direct ByteBuffers?
> ---------------------------------------
>
> Key: LUCENE-2056
> URL: https://issues.apache.org/jira/browse/LUCENE-2056
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Store
> Reporter: Michael McCandless
> Priority: Minor
>
> I'm trying to test NRT performance, and noticed when I dump the thread stacks that the darned threads often seem to be in {{java.nio.Bits.copyToByteArray(Native Method)}}... so I wondered whether we could/should use direct ByteBuffers, and whether that would gain performance in general. We currently just use our own byte[] buffer via BufferedIndexInput.
> It's hard to test since it's likely platform specific, but if it does result in gains it could be an easy win.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2056) Should NIOFSDir use direct
ByteBuffers?
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776588#action_12776588 ]
Mark Miller commented on LUCENE-2056:
-------------------------------------
Will be interesting to see what you come up with. I replaced the byte buffer in BufferedIndexInput with a direct buffer a year or two ago and it slowed things down. Then I read a bunch about how there were various issues with direct buffers - they really got NIO right on the first go or two ;) They are supposed to be much faster in java 7.
Who knows though - I was lazy and went with a pretty much straight port to direct buffers. Can prob get a much better kick with some pooling or something.
> Should NIOFSDir use direct ByteBuffers?
> ---------------------------------------
>
> Key: LUCENE-2056
> URL: https://issues.apache.org/jira/browse/LUCENE-2056
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Store
> Reporter: Michael McCandless
> Priority: Minor
>
> I'm trying to test NRT performance, and noticed when I dump the thread stacks that the darned threads often seem to be in {{java.nio.Bits.copyToByteArray(Native Method)}}... so I wondered whether we could/should use direct ByteBuffers, and whether that would gain performance in general. We currently just use our own byte[] buffer via BufferedIndexInput.
> It's hard to test since it's likely platform specific, but if it does result in gains it could be an easy win.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org