You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Thomas Kellerer <sp...@gmx.net> on 2011/01/20 12:53:16 UTC
Problem with replication
Hi all,
we have implemented a Solr based search in our web application. We have one master server that maintains the index which is replicated to the slaves using the built-in Solr replication.
This has been working fine so far, but suddenly the replication does not send the modified files to the slaves.
Each time the slave polls the master, the master answers with "No files to download for indexversion: 1291900430240"
The problem is, that the index version on the master **is** higher than the one on the slave.
We have something like this:
Master: Index Version: 1291900430241, Generation: 1288
Slave: Index Version: 1291900430240, Generation: 1287
But still the files are not replicated ("No files to download...")
Why would the master think that no files need to be replicated even though the slave version and generation are lower than the one from the master?
What makes the problem even more difficult is, that this isn't reproducable. Sometimes re-starting the master puts everything back to normal.
Any ideas?
Regards
Thomas
Re: Problem with replication
Posted by raulgrande83 <ra...@hotmail.com>.
Any solution to this issue? We have the different versions problem and after
restarting the Solr cluster the message "SEVERE No files to download for
index generation: xxxxxx" appears again and again...
--
View this message in context: http://lucene.472066.n3.nabble.com/Problem-with-replication-tp2294313p4041257.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem with replication
Posted by astubbs <an...@gmail.com>.
It may have been a permissions problem, or it stared working after the master
had done another fresh scheduled full-import and jumped an index version.
Timestamp issue?
--
View this message in context: http://lucene.472066.n3.nabble.com/Problem-with-replication-tp2294313p3704559.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem with replication
Posted by astubbs <an...@gmail.com>.
Actually, I get:
No files to download for index generation:
this is after deleting the data directory on the slave.
--
View this message in context: http://lucene.472066.n3.nabble.com/Problem-with-replication-tp2294313p3704457.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem with replication
Posted by astubbs <an...@gmail.com>.
I have the same problem.
--
View this message in context: http://lucene.472066.n3.nabble.com/Problem-with-replication-tp2294313p3704449.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem with replication
Posted by Thomas Kellerer <sp...@gmx.net>.
We have tried that as well, but the slave still claims to have a higher index
version, even when the index files were deleted completely
Regards
Thomas
Stevo Slavić, 20.01.2011 16:52:
> Not too elegant but valid check would be to bring slave down, delete
> it's index data directory, then to commit a change to index to master,
> then bring slave up, and wait for pollInterval to pass and replication
> to occur.
>
> Regards,
> Stevo.
>
> On Thu, Jan 20, 2011 at 4:16 PM, Thomas Kellerer<sp...@gmx.net> wrote:
>> Stevo Slavić, 20.01.2011 15:42:
>>>
>>> So if on startup index gets replicated, then commit probably isn't
>>> being called anywhere on master.
>>
>> No, the index is not replicated on startup (same behaviour: "no files to
>> download")
>>
>>> Is that index configured to autocommit on master, or do you commit
>>> from application code? If you commit from application code, check if
>>> commit actually gets issued to the slave.
>>>
>>
>> We explicitely commit() from the application when doing an update to the
>> index.
>>
>> How do I check if the commit is "sent" to the slave other than monitoring
>> the polling on the slave (which doesn't pick up the changes)
>>
>> Regards
>> Thomas
>>
>>
>
Re: Problem with replication
Posted by Stevo Slavić <ss...@gmail.com>.
Not too elegant but valid check would be to bring slave down, delete
it's index data directory, then to commit a change to index to master,
then bring slave up, and wait for pollInterval to pass and replication
to occur.
Regards,
Stevo.
On Thu, Jan 20, 2011 at 4:16 PM, Thomas Kellerer <sp...@gmx.net> wrote:
> Stevo Slavić, 20.01.2011 15:42:
>>
>> So if on startup index gets replicated, then commit probably isn't
>> being called anywhere on master.
>
> No, the index is not replicated on startup (same behaviour: "no files to
> download")
>
>> Is that index configured to autocommit on master, or do you commit
>> from application code? If you commit from application code, check if
>> commit actually gets issued to the slave.
>>
>
> We explicitely commit() from the application when doing an update to the
> index.
>
> How do I check if the commit is "sent" to the slave other than monitoring
> the polling on the slave (which doesn't pick up the changes)
>
> Regards
> Thomas
>
>
Re: Problem with replication
Posted by Thomas Kellerer <sp...@gmx.net>.
Stevo Slavić, 20.01.2011 15:42:
> So if on startup index gets replicated, then commit probably isn't
> being called anywhere on master.
No, the index is not replicated on startup (same behaviour: "no files to download")
> Is that index configured to autocommit on master, or do you commit
> from application code? If you commit from application code, check if
> commit actually gets issued to the slave.
>
We explicitely commit() from the application when doing an update to the index.
How do I check if the commit is "sent" to the slave other than monitoring the polling on the slave (which doesn't pick up the changes)
Regards
Thomas
Re: Problem with replication
Posted by Stevo Slavić <ss...@gmail.com>.
So if on startup index gets replicated, then commit probably isn't
being called anywhere on master.
Is that index configured to autocommit on master, or do you commit
from application code? If you commit from application code, check if
commit actually gets issued to the slave.
Regards,
Stevo.
On Thu, Jan 20, 2011 at 2:41 PM, Thomas Kellerer <sp...@gmx.net> wrote:
> Thomas Kellerer, 20.01.2011 12:53:
>>
>> Hi all,
>>
>> we have implemented a Solr based search in our web application. We
>> have one master server that maintains the index which is replicated
>> to the slaves using the built-in Solr replication.
>>
>> This has been working fine so far, but suddenly the replication does
>> not send the modified files to the slaves.
>>
>> Each time the slave polls the master, the master answers with "No
>> files to download for indexversion: 1291900430240"
>>
>> The problem is, that the index version on the master **is** higher
>> than the one on the slave.
>>
>
> One other thing I noticed:
>
> When I do
>
> http://host:port/solr/replication?command=indexversion
>
> on the slave, it returns 0 (zero)
>
> <response>
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">0</int>
> </lst>
> <long name="indexversion">0</long>
> <long name="generation">0</long>
> </response>
>
> But when I http://host:port/solr/core0/admin/replication/index.jsp on the
> slave, it displays a "valid" indexversion.
>
> Don't know that that means though.
>
> Regards
> Thomas
>
>
Re: Problem with replication
Posted by Thomas Kellerer <sp...@gmx.net>.
Thomas Kellerer, 20.01.2011 12:53:
> Hi all,
>
> we have implemented a Solr based search in our web application. We
> have one master server that maintains the index which is replicated
> to the slaves using the built-in Solr replication.
>
> This has been working fine so far, but suddenly the replication does
> not send the modified files to the slaves.
>
> Each time the slave polls the master, the master answers with "No
> files to download for indexversion: 1291900430240"
>
> The problem is, that the index version on the master **is** higher
> than the one on the slave.
>
One other thing I noticed:
When I do
http://host:port/solr/replication?command=indexversion
on the slave, it returns 0 (zero)
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">0</int>
</lst>
<long name="indexversion">0</long>
<long name="generation">0</long>
</response>
But when I http://host:port/solr/core0/admin/replication/index.jsp on the slave, it displays a "valid" indexversion.
Don't know that that means though.
Regards
Thomas
Re: Problem with replication
Posted by Thomas Kellerer <sp...@gmx.net>.
Here is our configuration:
<lst name="master">
<str name="enable">true</str>
<str name="replicateAfter">commit</str>
<str name="replicateAfter">startup</str>
<str name="confFiles">stopwords.txt,stopwords_de.txt,stopwords_en.txt,synonyms.txt</str>
</lst>
Stevo Slavić, 20.01.2011 13:26:
> On which events did you configure master to perform replication? replicateAfter
>
> Regards,
> Stevo.
>
> On Thu, Jan 20, 2011 at 12:53 PM, Thomas Kellerer<sp...@gmx.net> wrote:
>> Hi all,
>>
>> we have implemented a Solr based search in our web application. We have one
>> master server that maintains the index which is replicated to the slaves
>> using the built-in Solr replication.
>>
>> This has been working fine so far, but suddenly the replication does not
>> send the modified files to the slaves.
>>
>> Each time the slave polls the master, the master answers with "No files to
>> download for indexversion: 1291900430240"
>>
>> The problem is, that the index version on the master **is** higher than the
>> one on the slave.
>>
>> We have something like this:
>>
>> Master: Index Version: 1291900430241, Generation: 1288
>> Slave: Index Version: 1291900430240, Generation: 1287
>>
>> But still the files are not replicated ("No files to download...")
>>
>> Why would the master think that no files need to be replicated even though
>> the slave version and generation are lower than the one from the master?
>>
>> What makes the problem even more difficult is, that this isn't reproducable.
>> Sometimes re-starting the master puts everything back to normal.
>>
>> Any ideas?
>>
>>
>> Regards
>> Thomas
>>
>>
>
Re: Problem with replication
Posted by Stevo Slavić <ss...@gmail.com>.
On which events did you configure master to perform replication? replicateAfter
Regards,
Stevo.
On Thu, Jan 20, 2011 at 12:53 PM, Thomas Kellerer <sp...@gmx.net> wrote:
> Hi all,
>
> we have implemented a Solr based search in our web application. We have one
> master server that maintains the index which is replicated to the slaves
> using the built-in Solr replication.
>
> This has been working fine so far, but suddenly the replication does not
> send the modified files to the slaves.
>
> Each time the slave polls the master, the master answers with "No files to
> download for indexversion: 1291900430240"
>
> The problem is, that the index version on the master **is** higher than the
> one on the slave.
>
> We have something like this:
>
> Master: Index Version: 1291900430241, Generation: 1288
> Slave: Index Version: 1291900430240, Generation: 1287
>
> But still the files are not replicated ("No files to download...")
>
> Why would the master think that no files need to be replicated even though
> the slave version and generation are lower than the one from the master?
>
> What makes the problem even more difficult is, that this isn't reproducable.
> Sometimes re-starting the master puts everything back to normal.
>
> Any ideas?
>
>
> Regards
> Thomas
>
>