You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Doug Cutting <cu...@apache.org> on 2004/10/01 22:16:59 UTC
Re: Using MMapDirectory fails TestCompoundFile; MMapDirectory for
huge indexes
Paul Elschot wrote:
> I'm working on a memory mapped directory that uses multiple buffers
> for large files.
Great!
There will be a small performance hit, as each call to readByte() will
need to first check whether it's overflowed the current buffer, right?
> While trying some test runs I found that the current version fails a test:
>
> [junit] Testsuite: org.apache.lucene.index.TestCompoundFile
Thanks for testing this!
> I'm testing the version with multiple buffers using a smaller maximum
> buffer size (1024 * 128), and it does this test in the same way.
You mean it fails too?
> I have not yet looked into TestCompoundFile. When it is a good test
> case for this, I'll submit the multibuffer version as an enhancement.
Thanks, that would be great.
Doug
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: Using MMapDirectory fails TestCompoundFile; MMapDirectory for huge indexes
Posted by Paul Elschot <pa...@xs4all.nl>.
On Friday 01 October 2004 22:16, Doug Cutting wrote:
> Paul Elschot wrote:
> > I'm working on a memory mapped directory that uses multiple buffers
> > for large files.
>
> Great!
>
> There will be a small performance hit, as each call to readByte() will
> need to first check whether it's overflowed the current buffer, right?
Yes. I just simplified that test to a counter equals zero.
That counter also is decremented by readByte().
> > While trying some test runs I found that the current version fails a
> > test:
> >
> > [junit] Testsuite: org.apache.lucene.index.TestCompoundFile
>
> Thanks for testing this!
Errr, TestCompoundFile turns out to be quite extensive, so I'll share that
with Dmitry.
>
> > I'm testing the version with multiple buffers using a smaller maximum
> > buffer size (1024 * 128), and it does this test in the same way.
>
> You mean it fails too?
Yes, it behaves in the same way. It also passes the remaining tests
from TestCompoundFile for a smaller maximum buffer size.
> > I have not yet looked into TestCompoundFile. When it is a good test
> > case for this, I'll submit the multibuffer version as an enhancement.
>
> Thanks, that would be great.
A new version of MMapDirectory is on it's way with
a few comments on possible performance improvements.
Regards,
Paul Elschot
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org