You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by GW <th...@gmail.com> on 2016/12/20 13:03:53 UTC

Do I need a replication factor if I run an Hadoop file system?

Dear Users,

I'm in the process of making an app cluster aware and getting ready for
deployment.

I've been looking at running an Hadoop file system. If I have fault
tolerance at the file system it seems that I would be creating a ton of
extra drive i/o to do replication. Am I correct in this assumption?

My data is not really critical so one of the fist things I will be doing is
seeing if I can do a full reindex in a timely manner. My data needs to
change weekly. Can I have a case where replication is not required? It
seems that I might.

So, my thoughts bounce back to Solr on plain old SSD on a journaled file
system.

Thanks,

GW

Re: Do I need a replication factor if I run an Hadoop file system?

Posted by Erick Erickson <er...@gmail.com>.
It depends (tm).

For fault tolerance, probably not. For handling an increased
QPS rate (query, not update) maybe, maybe not, you haven't
given any idea what your SLAs are for queries.

Best,
Erick


On Tue, Dec 20, 2016 at 5:03 AM, GW <th...@gmail.com> wrote:
> Dear Users,
>
> I'm in the process of making an app cluster aware and getting ready for
> deployment.
>
> I've been looking at running an Hadoop file system. If I have fault
> tolerance at the file system it seems that I would be creating a ton of
> extra drive i/o to do replication. Am I correct in this assumption?
>
> My data is not really critical so one of the fist things I will be doing is
> seeing if I can do a full reindex in a timely manner. My data needs to
> change weekly. Can I have a case where replication is not required? It
> seems that I might.
>
> So, my thoughts bounce back to Solr on plain old SSD on a journaled file
> system.
>
> Thanks,
>
> GW