You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Neil Ireson <n....@dcs.shef.ac.uk> on 2012/12/12 11:32:13 UTC
Explicit setting of NIOFSDirectory not respected
Hi all,
I run my code on a cluster where I have to preset resource limits and therefore the processes have limited virtual memory causing OOME when using MMapDirectory on large indexes.
This means I explicitly use NIOFSDirectory (i.e. Directory indexDirectory = NIOFSDirectory.open(indexFile);) however when I run the code on Java HotSpot(TM) 64-Bit Server VM Lucene still uses MMapDirectory to open the index, using a 32 bit JVM opens the index with NIOFSDirectory but I can only assign -Xmx2G.
Surely Lucene should not override my selection of NIOFSDirectory.
N
PS I'm using Lucene 4.1-SNAPSHOT
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Explicit setting of NIOFSDirectory not respected
Posted by Uwe Schindler <uw...@thetaphi.de>.
If you want a specific directory implementation, instantiate it directly (new NIOFSDirectory)? If you use FSDirectory.open() it will use the autodetected one depending on OS and bitness.
About configuring your server from MMAP in a correct way, read http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html; most important is the last section if you get OOM (which is not a OOM regarding heap space, it is something like "mmap failed" wrapped by an IOException).
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Neil Ireson [mailto:n.ireson@dcs.shef.ac.uk]
> Sent: Wednesday, December 12, 2012 11:32 AM
> To: java-user@lucene.apache.org
> Subject: Explicit setting of NIOFSDirectory not respected
>
> Hi all,
>
> I run my code on a cluster where I have to preset resource limits and
> therefore the processes have limited virtual memory causing OOME when
> using MMapDirectory on large indexes.
>
> This means I explicitly use NIOFSDirectory (i.e. Directory indexDirectory =
> NIOFSDirectory.open(indexFile);) however when I run the code on Java
> HotSpot(TM) 64-Bit Server VM Lucene still uses MMapDirectory to open the
> index, using a 32 bit JVM opens the index with NIOFSDirectory but I can only
> assign -Xmx2G.
>
> Surely Lucene should not override my selection of NIOFSDirectory.
>
> N
>
> PS I'm using Lucene 4.1-SNAPSHOT
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org