question about Solr directories on mounted file systems

[Jim Adams <ja...@gmail.com>]

I have an application that is using SOLR on a mounted file system.  However,
machine or human error can sometimes unmount the file system.  This causes
Solr to write index files to a different area from the index I am using.

This also means that the index instance becomes corrupt, because some
entries are in the (correct) index, and some are in the one that was created
because the mounted file system was missing.

Is there an option in Solr that says "Do not write to the index if the
directory path does not already exist"?  Or some other way of ensuring that
I don't get multiple index instances created unintentionally, in a situation
such as this?

Thanks!

Re: question about Solr directories on mounted file systems

[Chris Hostetter <ho...@fucit.org>]

: I've seen a nasty problem like this. When the mounted filesystem goes away,
: you can create regular directories and files under the mount point. When
: it comes back, the newly created files are not accessible. Yuk. --wunder

Ahhh....  i see what you mean.  isn't the solution there to make sure the 
"data" directory (on the mounted filesystem) is writable by solr but that 
the "parent" directory (ie: the mount point) isn't writabl? ... so solr 
can't create the directory in the event that the mount point isn't there?




-Hoss

Re: question about Solr directories on mounted file systems

[Walter Underwood <wu...@netflix.com>]

I've seen a nasty problem like this. When the mounted filesystem goes away,
you can create regular directories and files under the mount point. When
it comes back, the newly created files are not accessible. Yuk. --wunder

On 11/6/08 8:19 AM, "Chris Hostetter" <ho...@fucit.org> wrote:

> : machine or human error can sometimes unmount the file system.  This causes
> : Solr to write index files to a different area from the index I am using.
> 
> can you clarify what you mean by this?

Re: question about Solr directories on mounted file systems

[Chris Hostetter <ho...@fucit.org>]

: machine or human error can sometimes unmount the file system.  This causes
: Solr to write index files to a different area from the index I am using.

can you clarify what you mean by this?

Solr will only ever write to the data dir you configure it with. if the 
parent dir (or volume) doesn't exist, it should fail and refuse to write 
anything -- there's no reason why it should just start writing index 
files to some different directory.

But for the record: i heartily agree with walter: don't use remote disk 
for your index. you can always use the backup scripts to bulk copy the 
index onto remote disk if that's how you want to distrubute or backup the 
index, but your life is going to be a lot easier if Solr is working with a 
local filesystem.


-Hoss

Re: question about Solr directories on mounted file systems

[Walter Underwood <wu...@netflix.com>]

I do not recommend using Lucene or Solr on a mounted file system.
My implementation was 100X faster after I moved it from NFS to
local disk. --wunder

On 11/5/08 10:01 AM, "Jim Adams" <ja...@gmail.com> wrote:

> I have an application that is using SOLR on a mounted file system.  However,
> machine or human error can sometimes unmount the file system.  This causes
> Solr to write index files to a different area from the index I am using.
> 
> This also means that the index instance becomes corrupt, because some
> entries are in the (correct) index, and some are in the one that was created
> because the mounted file system was missing.
> 
> Is there an option in Solr that says "Do not write to the index if the
> directory path does not already exist"?  Or some other way of ensuring that
> I don't get multiple index instances created unintentionally, in a situation
> such as this?
> 
> Thanks!