Too many open files (windows 2035 files limit)

[Victor Hadianto <vi...@nuix.com.au>]

Hi All,

Recently we came across a problem where Lucene throws too many open files 
problem. We have multiple indexes and each indexes contains over 100 fields. 

After searching the mail list archive I didn't find any usefull information, 
seems like there is a problem with Windows + java regarding on maximum open 
files.

Then I stumbled across this bug (Java bug 4189011)

http://developer.java.sun.com/developer/bugParade/bugs/4189011.html

Cut and pasted from the bug record:
" To clarify the issue, on win32 system there are three
ways to open a file:
1: Using Win32 API
2: Using MFC class framework lib.
3: using C-Library API (open() and fopen())

Other than the third option, i.e. option 1 and 2 have practically no
limitation in opening number of files. The third method is restricted
(for the reason not known to me) to open only approx. 2035 files. That
is why MS JVM is able to open unlimited (practically) files, but SUN JVM
fails after 2035 files (my guess is it is using 3rd method to open
file).

 The case is true only only on Win32 OS. The reason this is serious
problem ans we have to revert to MS JVM is because we werr writing high
scalable server in Java. I am sure you understand that Java as an API
must compatible on all platform, some of which are Solaris, NT(Win32),
Novell etc... "

Anyway this guy seems to know what he is talking about (from the bug records 
and all). Anyone else came across the same issue when working with Lucene + 
Windows?


Regards,

-- 
Victor Hadianto

NUIX Pty Ltd
Level 8, 143 York Street, Sydney 2000
Phone: (02) 9283 9010
Fax:   (02) 9283 9020

This message is intended only for the named recipient. If you are not the
intended recipient you are notified that disclosing, copying, distributing
or taking any action in reliance on the contents of this message or
attachment is strictly prohibited.