You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by longsan <lo...@sina.com> on 2014/01/16 03:46:43 UTC
solr cloud + hdfs issue
Hi, i'm newer for solr cloud. i met a question here.
we use hdfs as file storage service and use Solr cloud to provide indexing
service and query service.
But solr cloud will replicate index/data files between leader and replica.
And the hdfs replicates the files too.
i think it wil waste disk space. Is there any ways to disable solr cloud
replication index/data files between leader and replcas?
In other words , how to implement that hdfs provide data reliability and
solr cloud provide service reliability?
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-cloud-hdfs-issue-tp4111593.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr cloud + hdfs issue
Posted by Greg Walters <gr...@answers.com>.
> You can configure the Solr client to use a replication factor of 1 for hdfs and then let Solr replicate for you if you want to avoid this.
What is solr's behavior if the lucene files underneath it suddenly disappear? Will a core that's running and can't access its files in the case of a HDFS failure handle this gracefully and move to a DOWN state?
On Jan 21, 2014, at 4:29 AM, longsan <lo...@sina.com> wrote:
> thanks. i think it's a good option for me.
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-cloud-hdfs-issue-tp4111593p4112422.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr cloud + hdfs issue
Posted by longsan <lo...@sina.com>.
thanks. i think it's a good option for me.
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-cloud-hdfs-issue-tp4111593p4112422.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr cloud + hdfs issue
Posted by Mark Miller <ma...@gmail.com>.
You can configure the Solr client to use a replication factor of 1 for hdfs and then let Solr replicate for you if you want to avoid this.
Other than that, we will be adding further options over time.
- Mark
On Jan 15, 2014, at 9:46 PM, longsan <lo...@sina.com> wrote:
> Hi, i'm newer for solr cloud. i met a question here.
>
> we use hdfs as file storage service and use Solr cloud to provide indexing
> service and query service.
> But solr cloud will replicate index/data files between leader and replica.
> And the hdfs replicates the files too.
>
> i think it wil waste disk space. Is there any ways to disable solr cloud
> replication index/data files between leader and replcas?
>
> In other words , how to implement that hdfs provide data reliability and
> solr cloud provide service reliability?
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-cloud-hdfs-issue-tp4111593.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr cloud + hdfs issue
Posted by Erick Erickson <er...@gmail.com>.
I know it's "on the roadmap", but it's always a resource problem...
Any help appreciated, of course....
Best,
Erick
On Wed, Jan 15, 2014 at 10:57 PM, Otis Gospodnetic
<ot...@gmail.com> wrote:
> Hi,
>
> I think this is a known issue and I don't know of anyone working on
> changing this.
>
> Otis
> --
> Performance Monitoring * Log Analytics * Search Analytics
> Solr & Elasticsearch Support * http://sematext.com/
>
>
> On Wed, Jan 15, 2014 at 9:46 PM, longsan <lo...@sina.com> wrote:
>
>> Hi, i'm newer for solr cloud. i met a question here.
>>
>> we use hdfs as file storage service and use Solr cloud to provide indexing
>> service and query service.
>> But solr cloud will replicate index/data files between leader and replica.
>> And the hdfs replicates the files too.
>>
>> i think it wil waste disk space. Is there any ways to disable solr cloud
>> replication index/data files between leader and replcas?
>>
>> In other words , how to implement that hdfs provide data reliability and
>> solr cloud provide service reliability?
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/solr-cloud-hdfs-issue-tp4111593.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
Re: solr cloud + hdfs issue
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi,
I think this is a known issue and I don't know of anyone working on
changing this.
Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
On Wed, Jan 15, 2014 at 9:46 PM, longsan <lo...@sina.com> wrote:
> Hi, i'm newer for solr cloud. i met a question here.
>
> we use hdfs as file storage service and use Solr cloud to provide indexing
> service and query service.
> But solr cloud will replicate index/data files between leader and replica.
> And the hdfs replicates the files too.
>
> i think it wil waste disk space. Is there any ways to disable solr cloud
> replication index/data files between leader and replcas?
>
> In other words , how to implement that hdfs provide data reliability and
> solr cloud provide service reliability?
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/solr-cloud-hdfs-issue-tp4111593.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>