You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Koji Sekiguchi <ko...@m4.dion.ne.jp> on 2005/10/09 14:12:34 UTC
What is MMapDirectory?
Hello,
What is MMapDirectory?
I've searched mailing list archive, but cannot find it.
I could find the following explanation at Lucene 1.9 CHANGES.txt:
8. Add MMapDirectory, which uses nio to mmap input files. This is
still somewhat slower than FSDirectory. However it uses less
memory per query term, since a new buffer is not allocated per
term, which may help applications which use, e.g., wildcard
queries. It may also someday be faster. (cutting & Paul Elschot)
And I saw bug 31508:
[PATCH] MMapDirectory for huge index files
http://issues.apache.org/bugzilla/show_bug.cgi?id=31508
But I'm curious to know about MMapDirecotry in more detail about
the following point:
1. Were there any troubles in FSDirectory with huge index files?
2. Is MMapDirectory used for only searching? (not indexing?)
3. Why is MMapDirectory somewhat slower than FSDirectory?
4. Why does MMapDirectory use less memory while using mapped file (mmap)?
Thank you very much in advance,
Koji
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: What is MMapDirectory?
Posted by Koji Sekiguchi <ko...@m4.dion.ne.jp>.
Paul,
Thank you very much for your explanation.
> However, in case you have different experience, we'd like to know.
I don't have it. I'm just curious.
Thank you again,
Koji
----- Original Message -----
From: "Paul Elschot" <pa...@xs4all.nl>
To: <ja...@lucene.apache.org>
Sent: Tuesday, October 11, 2005 4:16 PM
Subject: Re: What is MMapDirectory?
> Koji,
>
> On Sunday 09 October 2005 14:12, Koji Sekiguchi wrote:
>> Hello,
>>
>> What is MMapDirectory?
>>
>> I've searched mailing list archive, but cannot find it.
>> I could find the following explanation at Lucene 1.9 CHANGES.txt:
>>
>> 8. Add MMapDirectory, which uses nio to mmap input files. This is
>> still somewhat slower than FSDirectory. However it uses less
>> memory per query term, since a new buffer is not allocated per
>> term, which may help applications which use, e.g., wildcard
>> queries. It may also someday be faster. (cutting & Paul Elschot)
>>
>> And I saw bug 31508:
>>
>> [PATCH] MMapDirectory for huge index files
>> http://issues.apache.org/bugzilla/show_bug.cgi?id=31508
>
> Bugzilla has been replaced by JIRA recently, this is now here:
> http://issues.apache.org/jira/browse/LUCENE-283
>
>>
>> But I'm curious to know about MMapDirecotry in more detail about
>> the following point:
>>
>> 1. Were there any troubles in FSDirectory with huge index files?
>
> There were some troubles with huge index files, but not in FSDirectory.
> FSDirectory is still faster than MMapDirectory, but that might change
> in the future.
>
>> 2. Is MMapDirectory used for only searching? (not indexing?)
>
> Indeed so.
>
>> 3. Why is MMapDirectory somewhat slower than FSDirectory?
>> 4. Why does MMapDirectory use less memory while using mapped file (mmap)?
>
> Both depend on the underlying implementation of buffering in the JVM
> and in the operating system. In my (few) attempts so far, memory
> mapped I/O does not currently help to improve performance of Lucene.
> However, in case you have different experience, we'd like to know.
>
> Regards,
> Paul Elschot
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: What is MMapDirectory?
Posted by Paul Elschot <pa...@xs4all.nl>.
Koji,
On Sunday 09 October 2005 14:12, Koji Sekiguchi wrote:
> Hello,
>
> What is MMapDirectory?
>
> I've searched mailing list archive, but cannot find it.
> I could find the following explanation at Lucene 1.9 CHANGES.txt:
>
> 8. Add MMapDirectory, which uses nio to mmap input files. This is
> still somewhat slower than FSDirectory. However it uses less
> memory per query term, since a new buffer is not allocated per
> term, which may help applications which use, e.g., wildcard
> queries. It may also someday be faster. (cutting & Paul Elschot)
>
> And I saw bug 31508:
>
> [PATCH] MMapDirectory for huge index files
> http://issues.apache.org/bugzilla/show_bug.cgi?id=31508
Bugzilla has been replaced by JIRA recently, this is now here:
http://issues.apache.org/jira/browse/LUCENE-283
>
> But I'm curious to know about MMapDirecotry in more detail about
> the following point:
>
> 1. Were there any troubles in FSDirectory with huge index files?
There were some troubles with huge index files, but not in FSDirectory.
FSDirectory is still faster than MMapDirectory, but that might change
in the future.
> 2. Is MMapDirectory used for only searching? (not indexing?)
Indeed so.
> 3. Why is MMapDirectory somewhat slower than FSDirectory?
> 4. Why does MMapDirectory use less memory while using mapped file (mmap)?
Both depend on the underlying implementation of buffering in the JVM
and in the operating system. In my (few) attempts so far, memory
mapped I/O does not currently help to improve performance of Lucene.
However, in case you have different experience, we'd like to know.
Regards,
Paul Elschot
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org