You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Yonik Seeley <yo...@apache.org> on 2008/11/12 20:59:18 UTC

NIO not working yet

NIO support in the latest Solr development versions does not work yet
(I previously advised that some people with possible lock contention
problems try it out).  We'll let you know when it's fixed, but in the
meantime you can always set the system property
"org.apache.lucene.FSDirectory.class" to
"org.apache.lucene.store.NIOFSDirectory" to try it out.

for example:

java -Dorg.apache.lucene.FSDirectory.class=org.apache.lucene.store.NIOFSDirectory
  -jar start.jar

-Yonik

Re: NIO not working yet

Posted by Yonik Seeley <yo...@apache.org>.
On Wed, Nov 12, 2008 at 3:53 PM, Feak, Todd <To...@smss.sony.com> wrote:
> Is support for setting the FSDirectory this way built into 1.3.0
> release? Or is it necessary to grab a trunk build.

It's not in 1.3, you need a very recent trunk build.

-Yonik

RE: NIO not working yet

Posted by "Feak, Todd" <To...@smss.sony.com>.
Is support for setting the FSDirectory this way built into 1.3.0
release? Or is it necessary to grab a trunk build.

-Todd Feak

-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik
Seeley
Sent: Wednesday, November 12, 2008 11:59 AM
To: solr-user@lucene.apache.org
Subject: NIO not working yet

NIO support in the latest Solr development versions does not work yet
(I previously advised that some people with possible lock contention
problems try it out).  We'll let you know when it's fixed, but in the
meantime you can always set the system property
"org.apache.lucene.FSDirectory.class" to
"org.apache.lucene.store.NIOFSDirectory" to try it out.

for example:

java
-Dorg.apache.lucene.FSDirectory.class=org.apache.lucene.store.NIOFSDirec
tory
  -jar start.jar

-Yonik


Solr with Networkn File Server

Posted by souravm <SO...@infosys.com>.
Hi,

I have huge index files to query. On a first cut calculation it looks like I would need around 3 boxes (each box not more than 125 M records of size 12.5GB) for around 25 apps - so all together 75 boxes. 

However the number of concurrent users would be lesser - may not be more than 20 at a time or max 25.

So thinking of an option where I use around 20-25 servers each with 2 GB heap size and loading all indexes in a network file server. I know this would impact the performance (especially for the first time query) but not sure how much impact it would be. 

If anybody already tried this type of solution please let me know how was the performance impact.

Regards,
Sourav

**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely 
for the use of the addressee(s). If you are not the intended recipient, please 
notify the sender by e-mail and delete the original message. Further, you are not 
to copy, disclose, or distribute this e-mail or its contents to any other person and 
any such actions are unlawful. This e-mail may contain viruses. Infosys has taken 
every reasonable precaution to minimize this risk, but is not liable for any damage 
you may sustain as a result of any virus in this e-mail. You should carry out your 
own virus checks before opening the e-mail or attachment. Infosys reserves the 
right to monitor and review the content of all messages sent to or from this e-mail 
address. Messages sent to or from this e-mail address may be stored on the 
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***

Re: NIO not working yet

Posted by wojtekpia <wo...@hotmail.com>.
I've updated my deployment to use NIOFSDirectory. Now I'd like to confirm
some previous results with the original FSDirectory. Can I turn it off with
a parameter? I tried:

java
-Dorg.apache.lucene.FSDirectory.class=org.apache.lucene.store.FSDirectory
...

but that didn't work. 

-- 
View this message in context: http://www.nabble.com/NIO-not-working-yet-tp20468152p20845732.html
Sent from the Solr - User mailing list archive at Nabble.com.


Re: NIO not working yet

Posted by Yonik Seeley <yo...@apache.org>.
On Tue, Dec 2, 2008 at 3:41 PM, Burton-West, Tom <tb...@umich.edu> wrote:
> Thanks Yonik,
>
> ->>The next nightly build (Dec-01-2008) should have the changes.
>
> The latest nightly build seems to be 30-Nov-2008 08:20,
> http://people.apache.org/builds/lucene/solr/nightly/
> has the version with the NIO fix been built?  Are we looking in the
> wrong place?

If the tests fail (which they seem to have for the last 2 days) then a
new snapshot is not uploaded.

Hudson also does solr builds though:
http://hudson.zones.apache.org/hudson/job/Solr-trunk/lastSuccessfulBuild/artifact/trunk/dist/

-Yonik


> Tom
>
> Tom Burton-West
> Information Retrieval Progammer
> Digital Library Production Services
> University of Michigan Library
>
> -----Original Message-----
> From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik
> Seeley
> Sent: Sunday, November 30, 2008 8:43 PM
> To: solr-user@lucene.apache.org
> Subject: Re: NIO not working yet
>
> OK, the development version of Solr should now be fixed (i.e. NIO should
> be the default for non-Windows platforms).  The next nightly build
> (Dec-01-2008) should have the changes.
>
> -Yonik
>
> On Wed, Nov 12, 2008 at 2:59 PM, Yonik Seeley <yo...@apache.org> wrote:
>> NIO support in the latest Solr development versions does not work yet
>> (I previously advised that some people with possible lock contention
>> problems try it out).  We'll let you know when it's fixed, but in the
>> meantime you can always set the system property
>> "org.apache.lucene.FSDirectory.class" to
>> "org.apache.lucene.store.NIOFSDirectory" to try it out.
>>
>> for example:
>>
>> java
>> -Dorg.apache.lucene.FSDirectory.class=org.apache.lucene.store.NIOFSDir
>> ectory
>>  -jar start.jar
>>
>> -Yonik
>

RE: NIO not working yet

Posted by "Burton-West, Tom" <tb...@umich.edu>.
Thanks Yonik,

->>The next nightly build (Dec-01-2008) should have the changes.

The latest nightly build seems to be 30-Nov-2008 08:20,
http://people.apache.org/builds/lucene/solr/nightly/ 
has the version with the NIO fix been built?  Are we looking in the
wrong place?

Tom

Tom Burton-West
Information Retrieval Progammer
Digital Library Production Services
University of Michigan Library

-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik
Seeley
Sent: Sunday, November 30, 2008 8:43 PM
To: solr-user@lucene.apache.org
Subject: Re: NIO not working yet

OK, the development version of Solr should now be fixed (i.e. NIO should
be the default for non-Windows platforms).  The next nightly build
(Dec-01-2008) should have the changes.

-Yonik

On Wed, Nov 12, 2008 at 2:59 PM, Yonik Seeley <yo...@apache.org> wrote:
> NIO support in the latest Solr development versions does not work yet 
> (I previously advised that some people with possible lock contention 
> problems try it out).  We'll let you know when it's fixed, but in the 
> meantime you can always set the system property 
> "org.apache.lucene.FSDirectory.class" to 
> "org.apache.lucene.store.NIOFSDirectory" to try it out.
>
> for example:
>
> java 
> -Dorg.apache.lucene.FSDirectory.class=org.apache.lucene.store.NIOFSDir
> ectory
>  -jar start.jar
>
> -Yonik

Re: NIO not working yet

Posted by Yonik Seeley <yo...@apache.org>.
On Sun, Nov 30, 2008 at 10:43 PM, Jon Baer <jo...@gmail.com> wrote:
> Sorry missed that (and probably dumb question), does that -D flag work for
> setting as a RAMDirectory as well?

Nope... that's implemented only in FSDirectory to specify a specific
subclass implementation (which RAMDirectory is not).

> - Jon
>
> On Nov 30, 2008, at 8:42 PM, Yonik Seeley wrote:
>
>> OK, the development version of Solr should now be fixed (i.e. NIO
>> should be the default for non-Windows platforms).  The next nightly
>> build (Dec-01-2008) should have the changes.
>>
>> -Yonik
>>
>> On Wed, Nov 12, 2008 at 2:59 PM, Yonik Seeley <yo...@apache.org> wrote:
>>>
>>> NIO support in the latest Solr development versions does not work yet
>>> (I previously advised that some people with possible lock contention
>>> problems try it out).  We'll let you know when it's fixed, but in the
>>> meantime you can always set the system property
>>> "org.apache.lucene.FSDirectory.class" to
>>> "org.apache.lucene.store.NIOFSDirectory" to try it out.
>>>
>>> for example:
>>>
>>> java
>>> -Dorg.apache.lucene.FSDirectory.class=org.apache.lucene.store.NIOFSDirectory
>>> -jar start.jar
>>>
>>> -Yonik
>
>

Re: NIO not working yet

Posted by Jon Baer <jo...@gmail.com>.
Sorry missed that (and probably dumb question), does that -D flag work  
for setting as a RAMDirectory as well?

- Jon

On Nov 30, 2008, at 8:42 PM, Yonik Seeley wrote:

> OK, the development version of Solr should now be fixed (i.e. NIO
> should be the default for non-Windows platforms).  The next nightly
> build (Dec-01-2008) should have the changes.
>
> -Yonik
>
> On Wed, Nov 12, 2008 at 2:59 PM, Yonik Seeley <yo...@apache.org>  
> wrote:
>> NIO support in the latest Solr development versions does not work yet
>> (I previously advised that some people with possible lock contention
>> problems try it out).  We'll let you know when it's fixed, but in the
>> meantime you can always set the system property
>> "org.apache.lucene.FSDirectory.class" to
>> "org.apache.lucene.store.NIOFSDirectory" to try it out.
>>
>> for example:
>>
>> java - 
>> Dorg 
>> .apache 
>> .lucene.FSDirectory.class=org.apache.lucene.store.NIOFSDirectory
>> -jar start.jar
>>
>> -Yonik


Re: NIO not working yet

Posted by Yonik Seeley <yo...@apache.org>.
OK, the development version of Solr should now be fixed (i.e. NIO
should be the default for non-Windows platforms).  The next nightly
build (Dec-01-2008) should have the changes.

-Yonik

On Wed, Nov 12, 2008 at 2:59 PM, Yonik Seeley <yo...@apache.org> wrote:
> NIO support in the latest Solr development versions does not work yet
> (I previously advised that some people with possible lock contention
> problems try it out).  We'll let you know when it's fixed, but in the
> meantime you can always set the system property
> "org.apache.lucene.FSDirectory.class" to
> "org.apache.lucene.store.NIOFSDirectory" to try it out.
>
> for example:
>
> java -Dorg.apache.lucene.FSDirectory.class=org.apache.lucene.store.NIOFSDirectory
>  -jar start.jar
>
> -Yonik