[jira] [Commented] (LUCENE-3178) Native MMapDir

["Michael McCandless (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923769#comment-13923769 ] 

Michael McCandless commented on LUCENE-3178:
--------------------------------------------

bq. ... and suddenly I got good results, this is idiopathic :S

Lovely :)

It is odd, because we do relatively "few" IO ops, since we read a big byte[] blob, and then do all decode from that (128 packed ints) in RAM.

It do think it'd be interesting to pair up a NativeMMapDir with a custom postings format that instead uses IndexInput.readLong (via Unsafe.getLong) to pull longs from disk; this should save some cost we now have in packed ints to reconstitute longs from byte[] in Java.  But, we'd need to fix the byte order in the index to match the CPU used at search time.  Or, maybe we could use a DirectByteBuffer and set the byte order (but this may mean byte swapping for every access, which maybe is not so bad).

> Native MMapDir
> --------------
>
>                 Key: LUCENE-3178
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3178
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: core/store
>            Reporter: Michael McCandless
>              Labels: gsoc2014
>         Attachments: LUCENE-3178-Native-MMap-implementation.patch, LUCENE-3178-Native-MMap-implementation.patch, LUCENE-3178-Native-MMap-implementation.patch
>
>
> Spinoff from LUCENE-2793.
> Just like we will create native Dir impl (UnixDirectory) to pass the right OS level IO flags depending on the IOContext, we could in theory do something similar with MMapDir.
> The problem is MMap is apparently quite hairy... and to pass the flags the native code would need to invoke mmap (I think?), unlike UnixDir where the code "only" has to open the file handle.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org