DataImportHandler while Replication

[Robin Woods <wo...@gmail.com>]

Hi,

What would happen to DataImportHandler that is setup on the master when the
slave is in the process of replicating the index.

Is there anyway to configure DataImportHandler to not do anything if
replication is in process and/or disable replication before
DataImportHandler starts its process?

Please share your thoughts..

Best,
Robin

Re: DataImportHandler while Replication

[Robin Woods <wo...@gmail.com>]

Erick,

Thanks a bunch. Good to know the internals. 

Best,
Robin



--
View this message in context: http://lucene.472066.n3.nabble.com/DataImportHandler-while-Replication-tp4138763p4138984.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: DataImportHandler while Replication

[rulinma <ru...@gmail.com>]

good.



--
View this message in context: http://lucene.472066.n3.nabble.com/DataImportHandler-while-Replication-tp4138763p4139774.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: DataImportHandler while Replication

[Erick Erickson <er...@gmail.com>]

You don't need to do anything. Essentially, when replication starts the
current state of the master's index is "frozen" in the sense that the
_closed_ segments that make up the index at the time replication starts are
the only ones that are replicated to the slave.

All indexing happens into an _open_ segment, so they are automatically
excluded from replication. And as long as the replication is happening, the
segments that are being replicated will not be deleted from the master. If
an open segment (at the time replication starts) is closed before
replication is done, it doesn't matter since it is _not_ replicated because
it wasn't closed when replication started.

IOW, you're over-thinking the problem, it's just magic, man....

Best,
Erick




On Fri, May 30, 2014 at 10:36 AM, Robin Woods <wo...@gmail.com> wrote:

> Hello Shalin,
>
> Appreciate your reply. I've not implemented DIH in production and now we
> are
> considering. hence the question, if we configure DIH on the master from
> which the Slave is replicating the index, we will need to control this
> externally? or is there any setting in DIH that will allow us to handle
> this?
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/DataImportHandler-while-Replication-tp4138763p4138948.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: DataImportHandler while Replication

[Robin Woods <wo...@gmail.com>]

Hello Shalin,

Appreciate your reply. I've not implemented DIH in production and now we are
considering. hence the question, if we configure DIH on the master from
which the Slave is replicating the index, we will need to control this
externally? or is there any setting in DIH that will allow us to handle
this?



--
View this message in context: http://lucene.472066.n3.nabble.com/DataImportHandler-while-Replication-tp4138763p4138948.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: DataImportHandler while Replication

[Shalin Shekhar Mangar <sh...@gmail.com>]

They are completely separate components in Solr. Are you seeing performance
problems in replication due to indexing or vice versa?


On Fri, May 30, 2014 at 10:10 AM, Robin Woods <wo...@gmail.com> wrote:

> Hi,
>
> What would happen to DataImportHandler that is setup on the master when the
> slave is in the process of replicating the index.
>
> Is there anyway to configure DataImportHandler to not do anything if
> replication is in process and/or disable replication before
> DataImportHandler starts its process?
>
> Please share your thoughts..
>
> Best,
> Robin
>



-- 
Regards,
Shalin Shekhar Mangar.