Distribution without SSH?

[Justin Knoll <jk...@bittorrent.com>]

Hello,
I recently set up Solr with distribution on a couple of servers. I  
just learned that our network policies do not permit us to use SSH  
with passphraseless keys, and the snappuller script uses SSH to  
examine the master Solr instance's state before it pulls the newest  
index via rsync.

We plan to attempt to rewrite the snappuller (and possibly other  
distribution scripts, as required) to eliminate this dependency on  
SSH. I thought I ask the list in case anyone has experience with this  
same situation or any insights into the reasoning behind requiring  
SSH access to the master instance.

Thanks,
Justin Knoll

Re: Distribution without SSH?

[Chris Hostetter <ho...@fucit.org>]

: I recently set up Solr with distribution on a couple of servers. I just
: learned that our network policies do not permit us to use SSH with
: passphraseless keys, and the snappuller script uses SSH to examine the master
: Solr instance's state before it pulls the newest index via rsync.

you may want to question/clarify this policy ... while it's generally a 
good idea to have a policy like this for *users* there's very little 
reason for it when you're dealing with "role users" ... accounts that 
exists solely to execute specific applications nad have limitited 
permissions.  if you have a "solruser" with a passphraseless key, which 
only works on the specific machines running solr, and solruser can only 
read/write the specific files it needs to for replication, there's very 
little downside.

: scripts, as required) to eliminate this dependency on SSH. I thought I ask the
: list in case anyone has experience with this same situation or any insights
: into the reasoning behind requiring SSH access to the master instance.

i haven't looked at those scripts in a while, but i believe it's two fold:
  1) get the name of hte most current snapshoot
  2) notify the master which snapshot is being used (for the status page)



-Hoss

Re: Distribution without SSH?

[Marcus Stratmann <st...@gmx.de>]

Justin Knoll wrote:
> We plan to attempt to rewrite the snappuller (and possibly other 
> distribution scripts, as required) to eliminate this dependency on SSH. 
> I thought I ask the list in case anyone has experience with this same 
> situation or any insights into the reasoning behind requiring SSH access 
> to the master instance.
We use our database to store the master's state. Both master and 
slave(s) have access to the database and can exchange "messages" using a 
field in a table where we store miscellaneous information about our 
system. After an update of the master's index a flag in that field 
signals that a new index is available. The slaves regularly read this 
field an pull the new index on demand.

Marcus

Re: Distribution without SSH?

[Matt Kangas <ka...@gmail.com>]

Your company's network policies seem to be a good thing. I've worked  
at places with this same policy, for good reason. But it does tend to  
complicate operations sometimes. Some options you might pursue:

* Set up ssh-agent on the clients and use passphrase-protected keys.  
Downside to this, someone on your ops team will be inevitably awoken  
at 4am to type in the password.
* Try to get an exception to the policy by running Solr under a new  
user account inside a jail. Use a restricted login shell to make sure  
it can do only what you intend. So when the key is compromised,  
damage is contained.

Or, write a custom server/client running on a different port. In this  
case you lose over-the-wire encryption, and if your server is buggy,  
you get pwn3d anyway.

--Matt

On Nov 29, 2007, at 7:48 PM, Justin Knoll wrote:

> Hello,
> I recently set up Solr with distribution on a couple of servers. I  
> just learned that our network policies do not permit us to use SSH  
> with passphraseless keys, and the snappuller script uses SSH to  
> examine the master Solr instance's state before it pulls the newest  
> index via rsync.
>
> We plan to attempt to rewrite the snappuller (and possibly other  
> distribution scripts, as required) to eliminate this dependency on  
> SSH. I thought I ask the list in case anyone has experience with  
> this same situation or any insights into the reasoning behind  
> requiring SSH access to the master instance.
>
> Thanks,
> Justin Knoll

--
Matt Kangas / kangas@gmail.com