You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Michael Busch <bu...@gmail.com> on 2009/03/28 14:44:16 UTC
NIO.2
NIO.2 sounds great.
Though, it will probably take a pretty long time before we can switch
Lucene to Java 1.7 :(
We could write a (contrib) module that we don't ship together with the
core that has a Directory implementation which uses NIO.2.
http://jcp.org/en/jsr/detail?id=203
http://ronsoft.net/files/WhatsNewNIO2.pdf
-Michael
Re: NIO.2
Posted by Earwin Burrfoot <ea...@gmail.com>.
On Sat, Mar 28, 2009 at 16:44, Michael Busch <bu...@gmail.com> wrote:
> NIO.2 sounds great.
> Though, it will probably take a pretty long time before we can switch Lucene
> to Java 1.7 :(
>
> We could write a (contrib) module that we don't ship together with the core
> that has a Directory implementation which uses NIO.2.
>
> http://jcp.org/en/jsr/detail?id=203
> http://ronsoft.net/files/WhatsNewNIO2.pdf
>
>
> -Michael
I was excited for a second, until I noticed they somehow lost
Big*Buffers while merging into JDK7.
http://download.java.net/jdk7/docs/api/java/nio/channels/package-summary.html
<- Ctrl+F, MappedBigByteBuffer, that's the only remnant of what was
supposed to be there.
--
Kirill Zakharenko/Кирилл Захаренко (earwin@gmail.com)
Home / Mobile: +7 (495) 683-567-4 / +7 (903) 5-888-423
ICQ: 104465785
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: NIO.2
Posted by Earwin Burrfoot <ea...@gmail.com>.
> I think having async IO will be great, though I wonder how we would
> change Lucene to take advantage of it. It ought to gain us
> concurrency (eg we can score last chunk while we have an io request
> out to retrieve next chunk, of term docs / positions / etc.).
A presentation given above references Big*Buffers, including
MappedBigByteBuffer, which differ from their not-so-Big counterparts
in using long sizes/offsets. That means (woo-hoo!) a way better
MMapDirectory.
Everything else there is totally irrelevant to lucene. Okay, maybe
atomic file moves, but that part of Directory is long time deprecated
:)
--
Kirill Zakharenko/Кирилл Захаренко (earwin@gmail.com)
Home / Mobile: +7 (495) 683-567-4 / +7 (903) 5-888-423
ICQ: 104465785
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: NIO.2
Posted by Michael McCandless <lu...@mikemccandless.com>.
I think having async IO will be great, though I wonder how we would
change Lucene to take advantage of it. It ought to gain us
concurrency (eg we can score last chunk while we have an io request
out to retrieve next chunk, of term docs / positions / etc.).
Watch service sounds neat. Maybe we could use that as a way for
readers to know when to reopen themselves.
Something Lucene would really benefit from is access to madvise, so
that we could tell the OS that the massive amounts of data we are
reading & writing for merging should not be cached. But it doesn't
look like NIO2 has exposed this...
Another thing would be control over the priority of multiple
outstanding IO tasks. EG I'd like to say that the IO in a merge
thread is lower priority than indexing thread, which is lower priority
than searching threads. This is even further out (I don't think OSs
expose this control?).
Mike
On Sat, Mar 28, 2009 at 9:44 AM, Michael Busch <bu...@gmail.com> wrote:
> NIO.2 sounds great.
> Though, it will probably take a pretty long time before we can switch Lucene
> to Java 1.7 :(
>
> We could write a (contrib) module that we don't ship together with the core
> that has a Directory implementation which uses NIO.2.
>
> http://jcp.org/en/jsr/detail?id=203
> http://ronsoft.net/files/WhatsNewNIO2.pdf
>
>
> -Michael
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org