Remote file systems solrcloud

[Christopher Luciano <ch...@gmail.com>]

Hi I'd like to know if anyone has had much success with remote file systems or NAS type storage backends for SolrCloud. Right now we are using local disk  in a Mesos cluster but it would be great if we could use something more tolerant to failure. We have to perform some hacks to pin instances to machines in our Mesos cluster.

We have considered systems like Ceph. We've heard from some of our storage experts that Lucene has certain issues with these types of parallel filesystems. Has anyone had experience with this or something like glusterfs?


Sent from my iPad
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: Remote file systems solrcloud

[Scott Blum <dr...@gmail.com>]

Not exactly what you're asking, but we have a Solr cluster that runs fine
on Google Compute Engine with persistent SSD.  Under the hood the
persistent SSDs are network attached.

On Sun, Mar 20, 2016 at 5:47 PM, Christopher Luciano <
christophermluciano@gmail.com> wrote:

> Hi I'd like to know if anyone has had much success with remote file
> systems or NAS type storage backends for SolrCloud. Right now we are using
> local disk  in a Mesos cluster but it would be great if we could use
> something more tolerant to failure. We have to perform some hacks to pin
> instances to machines in our Mesos cluster.
>
> We have considered systems like Ceph. We've heard from some of our storage
> experts that Lucene has certain issues with these types of parallel
> filesystems. Has anyone had experience with this or something like
> glusterfs?
>
>
> Sent from my iPad
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

Re: Remote file systems solrcloud

[Shawn Heisey <ap...@elyograg.org>]

On 3/20/2016 3:47 PM, Christopher Luciano wrote:
> Hi I'd like to know if anyone has had much success with remote file systems or NAS type storage backends for SolrCloud. Right now we are using local disk  in a Mesos cluster but it would be great if we could use something more tolerant to failure. We have to perform some hacks to pin instances to machines in our Mesos cluster.
>
> We have considered systems like Ceph. We've heard from some of our storage experts that Lucene has certain issues with these types of parallel filesystems. Has anyone had experience with this or something like glusterfs?

This question should have gone to the solr-user list, not the dev list. 
I'll go ahead and answer it, but please take any further discussion (or
new topics about Solr) to that list.

Solr/Lucene has issues with pretty much all network filesystems.

There's a lot of data out there about NFS and Lucene.

Gluster may not provide all the locking mechanisms that Solr expects,
especially if you access it with NFS or SMB.  When my company tried to
use it for production storage (not Solr), we had a lot of stability
problems.  We have finally migrated all our back end storage off of
Gluster and are now happily using Isilon via NFS.  We are not using
Isilon for Solr.

I have only a little experience with Ceph, but even the Ceph developers
don't think CephFS is ready for production -- their effort is more
focused on the object store and the network block device capability than
the filesystem.

You may be able to get Solr working with a network filesystem by
changing its LockFactory configuration, but Solr will always perform
best with fast local storage, native OS filesystems, and a lot of memory
to cache the filesystem.

For my Solr servers, we have six local 1TB or 2TB drives in RAID10,
providing either 3TB or 6TB of local storage.  In terms of space, this
is overkill, but the RAID10 across six drives provides a lot of speed.

Thanks,
Shawn


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org