You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Arkadi Colson <ar...@smartbit.be> on 2013/03/27 08:57:44 UTC
Disc space and replication
Hi
When replication is down for some time or an instance crashed for some
reason replication will always start over again from the beginning. This
means it will copy the whole shard over of about 150GB. So we need at
least a disc of about 300 GB.
I've read somewhere that Solr will replicate everything when 100 entries
are missing? Why is that? Is it configurable?
What about optimization? Is it still needed in SolrCloud? Will it reduce
the disc usage? Does it also need twice the shard size to run successful?
Is it correct that currently the only option for now the make more
shards to reduce the disc space?
Is the any progress in the resharding option the developers are working on?
Thx!
--
Met vriendelijke groeten
Arkadi Colson
Smartbit bvba • Hoogstraat 13 • 3670 Meeuwen
T +32 11 64 08 80 • F +32 11 64 08 81
Re: Disc space and replication
Posted by Mark Miller <ma...@gmail.com>.
On Mar 27, 2013, at 3:57 AM, Arkadi Colson <ar...@smartbit.be> wrote:
> Hi
>
>
> When replication is down for some time or an instance crashed for some reason replication will always start over again from the beginning. This means it will copy the whole shard over of about 150GB. So we need at least a disc of about 300 GB.
>
> I've read somewhere that Solr will replicate everything when 100 entries are missing? Why is that? Is it configurable?
Not configurable. Are you using 4.2? It will not recopy any segment files that already exist on the replica - 4.0 and 4.1 copied all the files regardless in SolrCloud mode.
>
> What about optimization? Is it still needed in SolrCloud? Will it reduce the disc usage? Does it also need twice the shard size to run successful?
I wouldn't optimize if you will continue to add/update documents. Use merge policy settings to control the segment count.
>
> Is it correct that currently the only option for now the make more shards to reduce the disc space?
??
>
> Is the any progress in the resharding option the developers are working on?
Yes, see the JIRA issue on shard splitting.
- Mark
>
>
> Thx!
>
> --
> Met vriendelijke groeten
>
> Arkadi Colson
>
> Smartbit bvba • Hoogstraat 13 • 3670 Meeuwen
> T +32 11 64 08 80 • F +32 11 64 08 81
>