You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Maduranga Kannangara <mk...@infomedia.com.au> on 2009/12/22 04:54:16 UTC
Solr replication 1.3 issue
Hi All,
We're trying to replicate indexes on Solr 1.3 across from Dev->QA->Staging->Prod etc.
So at each stage other than Dev and Prod, each would live as a master and a slave at a given time.
We hit a bottle neck (may be?) when we try to start rsyncd-start on the master from the slave machine.
Commands used:
ssh -o StrictHostKeyChecking=no admin@192.168.22.1 /solr/SolrHome/bin/rsyncd-enable
ssh -o StrictHostKeyChecking=no admin@192.168.22.1 /solr / SolrHome /bin/rsyncd-start -p 18003
On slave following error is displayed:
@RSYNCD: 29
@ERROR: protocol startup error
On master logs following were found:
2009/12/21 22:46:05 enabled by admin
2009/12/21 22:46:05 command: / solr/SolrHome /bin/rsyncd-enable
2009/12/21 22:46:05 ended (elapsed time: 0 sec)
2009/12/21 22:46:09 started by admin
2009/12/21 22:46:09 command: /solr/SolrHome/bin/rsyncd-start -p 18993
2009/12/21 22:46:09 [16964] forward name lookup for devserver002 failed: ai_family not supported
2009/12/21 22:46:09 [16964] connect from UNKNOWN (localhost)
2009/12/21 22:46:29 [16964] rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
2009/12/21 22:46:29 [16964] rsync error: error in rsync protocol data stream (code 12) at io.c(463) [receiver=2.6.8]
2009/12/21 22:46:44 rsyncd not accepting connections, exiting
2009/12/21 22:46:57 enabled by admin
2009/12/21 22:46:57 command: /solr/SolrHome/bin/rsyncd-enable
2009/12/21 22:46:57 rsyncd already currently enabled
2009/12/21 22:46:57 exited (elapsed time: 0 sec)
2009/12/21 22:47:00 started by admin
2009/12/21 22:47:00 command: /solr/SolrHome/bin/rsyncd-start -p 18993
2009/12/21 22:47:00 [17115] forward name lookup for devserver002 failed: ai_family not supported
2009/12/21 22:47:00 [17115] connect from UNKNOWN (localhost)
2009/12/21 22:49:18 rsyncd not accepting connections, exiting
Is it not possible to start the rsync daemon on master from the slave?
The user that we use is on the sudoers list as well.
Thanks
Madu
RE: Solr replication 1.3 issue
Posted by Maduranga Kannangara <mk...@infomedia.com.au>.
Yes, that makes sense Lance. Thanks.
For the moment we broke our script to do the master's part (starting rsyncd) on the master server itself. Our problem is that we have so many instances running in different environment and we'd really love to minimize the number of them. :-)
Thanks again.
Madu
-----Original Message-----
From: Lance Norskog [mailto:goksron@gmail.com]
Sent: Wednesday, 23 December 2009 9:33 AM
To: solr-user@lucene.apache.org
Subject: Re: Solr replication 1.3 issue
This is a Unix security question. "rsyncd' is a system daemon and
should be managed in the OS scripts. The rsyncd-* scripts include a
security setting for the 'solr' account that limits the account to the
solr data directory (and this code does not support multicore).
For security reasons, I personally would not make a sudoer out of a
user with an automated remote login; but every site is different.
On Mon, Dec 21, 2009 at 7:54 PM, Maduranga Kannangara
<mk...@infomedia.com.au> wrote:
> Hi All,
>
> We're trying to replicate indexes on Solr 1.3 across from Dev->QA->Staging->Prod etc.
> So at each stage other than Dev and Prod, each would live as a master and a slave at a given time.
>
> We hit a bottle neck (may be?) when we try to start rsyncd-start on the master from the slave machine.
>
> Commands used:
>
> ssh -o StrictHostKeyChecking=no admin@192.168.22.1 /solr/SolrHome/bin/rsyncd-enable
> ssh -o StrictHostKeyChecking=no admin@192.168.22.1 /solr / SolrHome /bin/rsyncd-start -p 18003
>
> On slave following error is displayed:
>
> @RSYNCD: 29
> @ERROR: protocol startup error
>
> On master logs following were found:
>
> 2009/12/21 22:46:05 enabled by admin
> 2009/12/21 22:46:05 command: / solr/SolrHome /bin/rsyncd-enable
> 2009/12/21 22:46:05 ended (elapsed time: 0 sec)
> 2009/12/21 22:46:09 started by admin
> 2009/12/21 22:46:09 command: /solr/SolrHome/bin/rsyncd-start -p 18993
> 2009/12/21 22:46:09 [16964] forward name lookup for devserver002 failed: ai_family not supported
> 2009/12/21 22:46:09 [16964] connect from UNKNOWN (localhost)
> 2009/12/21 22:46:29 [16964] rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
> 2009/12/21 22:46:29 [16964] rsync error: error in rsync protocol data stream (code 12) at io.c(463) [receiver=2.6.8]
> 2009/12/21 22:46:44 rsyncd not accepting connections, exiting
> 2009/12/21 22:46:57 enabled by admin
> 2009/12/21 22:46:57 command: /solr/SolrHome/bin/rsyncd-enable
> 2009/12/21 22:46:57 rsyncd already currently enabled
> 2009/12/21 22:46:57 exited (elapsed time: 0 sec)
> 2009/12/21 22:47:00 started by admin
> 2009/12/21 22:47:00 command: /solr/SolrHome/bin/rsyncd-start -p 18993
> 2009/12/21 22:47:00 [17115] forward name lookup for devserver002 failed: ai_family not supported
> 2009/12/21 22:47:00 [17115] connect from UNKNOWN (localhost)
> 2009/12/21 22:49:18 rsyncd not accepting connections, exiting
>
>
> Is it not possible to start the rsync daemon on master from the slave?
> The user that we use is on the sudoers list as well.
>
> Thanks
> Madu
>
>
>
>
--
Lance Norskog
goksron@gmail.com
Re: Solr replication 1.3 issue
Posted by Lance Norskog <go...@gmail.com>.
This is a Unix security question. "rsyncd' is a system daemon and
should be managed in the OS scripts. The rsyncd-* scripts include a
security setting for the 'solr' account that limits the account to the
solr data directory (and this code does not support multicore).
For security reasons, I personally would not make a sudoer out of a
user with an automated remote login; but every site is different.
On Mon, Dec 21, 2009 at 7:54 PM, Maduranga Kannangara
<mk...@infomedia.com.au> wrote:
> Hi All,
>
> We're trying to replicate indexes on Solr 1.3 across from Dev->QA->Staging->Prod etc.
> So at each stage other than Dev and Prod, each would live as a master and a slave at a given time.
>
> We hit a bottle neck (may be?) when we try to start rsyncd-start on the master from the slave machine.
>
> Commands used:
>
> ssh -o StrictHostKeyChecking=no admin@192.168.22.1 /solr/SolrHome/bin/rsyncd-enable
> ssh -o StrictHostKeyChecking=no admin@192.168.22.1 /solr / SolrHome /bin/rsyncd-start -p 18003
>
> On slave following error is displayed:
>
> @RSYNCD: 29
> @ERROR: protocol startup error
>
> On master logs following were found:
>
> 2009/12/21 22:46:05 enabled by admin
> 2009/12/21 22:46:05 command: / solr/SolrHome /bin/rsyncd-enable
> 2009/12/21 22:46:05 ended (elapsed time: 0 sec)
> 2009/12/21 22:46:09 started by admin
> 2009/12/21 22:46:09 command: /solr/SolrHome/bin/rsyncd-start -p 18993
> 2009/12/21 22:46:09 [16964] forward name lookup for devserver002 failed: ai_family not supported
> 2009/12/21 22:46:09 [16964] connect from UNKNOWN (localhost)
> 2009/12/21 22:46:29 [16964] rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
> 2009/12/21 22:46:29 [16964] rsync error: error in rsync protocol data stream (code 12) at io.c(463) [receiver=2.6.8]
> 2009/12/21 22:46:44 rsyncd not accepting connections, exiting
> 2009/12/21 22:46:57 enabled by admin
> 2009/12/21 22:46:57 command: /solr/SolrHome/bin/rsyncd-enable
> 2009/12/21 22:46:57 rsyncd already currently enabled
> 2009/12/21 22:46:57 exited (elapsed time: 0 sec)
> 2009/12/21 22:47:00 started by admin
> 2009/12/21 22:47:00 command: /solr/SolrHome/bin/rsyncd-start -p 18993
> 2009/12/21 22:47:00 [17115] forward name lookup for devserver002 failed: ai_family not supported
> 2009/12/21 22:47:00 [17115] connect from UNKNOWN (localhost)
> 2009/12/21 22:49:18 rsyncd not accepting connections, exiting
>
>
> Is it not possible to start the rsync daemon on master from the slave?
> The user that we use is on the sudoers list as well.
>
> Thanks
> Madu
>
>
>
>
--
Lance Norskog
goksron@gmail.com