You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Garrett McGrath <gm...@Princeton.EDU> on 2006/12/11 16:30:13 UTC

svn and cron

 
Has anybody had any success running cron based svn scripts?  Mine can all be
executed from command line but for some reason running then from cron they
start discovering these non exsistent temporary files and manage to lock my
svn archive for no aparent reason.  I understand that svn isn't supposed to
be automated but this small side project would be a fairly huge boon in the
documentation of our centers server configurations.  However running the
scripts manually effectively eliminates the boon gained by the automation.
If anybody has been able to successfully execute svn cleanup, svn commit or
svn update from a cron job please let me know.
-Garrett McGrath

RE: RE: svn and cron

Posted by Ch...@qimonda.com.
You can always redefine your PATH variable in your
script. For example, in the cron expect script I run, I
have 

set env(PATH) /mypath

>-----Original Message-----
>From: Garrett McGrath [mailto:gmcgrath@Princeton.EDU] 
>Sent: Tuesday, December 12, 2006 2:30 PM
>To: 'John Rouillard'; users@subversion.tigris.org
>Subject: RE: svn and cron
>
>I believe I have found the problem in my particular instance.  
>On centos (and quiet probobly more generally) cron uses a 
>restricted path for the commands it issues.  My version of svn 
>wasn't found in that path and a core distribution version was 
>being used instead.  Defining the full path of the svn binary 
>and the svn working copy 'appears' to have fixed the problem 
>it'll take another day or so to completely test it however.
>-Garrett 
>
>-----Original Message-----
>From: John Rouillard [mailto:rouilj@renesys.com]
>Sent: Tuesday, December 12, 2006 12:02 PM
>To: Mikko Ruohola
>Cc: users@subversion.tigris.org
>Subject: Re: svn and cron
>
>On Tue, Dec 12, 2006 at 02:01:53PM +0200, Mikko Ruohola wrote:
>> Vincent Lefevre kirjoitti:
>> >On 2006-12-11 20:50:49 -0600, Ryan Schmidt wrote:
>> >  
>> >>Curious. What do you need expect for in this scenario? svn 
>is not an 
>> >>interactive program. The only thing I think it can prompt 
>for is the 
>> >>username and password, and you can provide that with the 
>--username 
>> >>and --password options without resorting to something like expect.
>> >>    
>> >
>> >The --password option may be a security hole, e.g. if there are 
>> >several users on the machine.
>> --password `cat /var/etc/thefilewiththepassword` And file 
>can have all 
>> the fancy permissions to prevent people from seeing it. or does it 
>> show on process list apps like top or ps?
>
>You bet it does.
>
>> atleast ncftp hides some command line parameters from process list.
>
>That is usually done by rewriting argv, so it introduces a 
>race condition where the password is visible in plaintext 
>format.  Also not all systems allow you to rewrite argv and 
>have the kernel pick up the changed version.
>
>-- 
>				-- rouilj
>
>John Rouillard
>System Administrator
>Renesys Corporation
>603-643-9300 x 111
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
>For additional commands, e-mail: users-help@subversion.tigris.org
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
>For additional commands, e-mail: users-help@subversion.tigris.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org


RE: svn and cron

Posted by Garrett McGrath <gm...@Princeton.EDU>.
I believe I have found the problem in my particular instance.  On centos
(and quiet probobly more generally) cron uses a restricted path for the
commands it issues.  My version of svn wasn't found in that path and a core
distribution version was being used instead.  Defining the full path of the
svn binary and the svn working copy 'appears' to have fixed the problem
it'll take another day or so to completely test it however.
-Garrett 

-----Original Message-----
From: John Rouillard [mailto:rouilj@renesys.com] 
Sent: Tuesday, December 12, 2006 12:02 PM
To: Mikko Ruohola
Cc: users@subversion.tigris.org
Subject: Re: svn and cron

On Tue, Dec 12, 2006 at 02:01:53PM +0200, Mikko Ruohola wrote:
> Vincent Lefevre kirjoitti:
> >On 2006-12-11 20:50:49 -0600, Ryan Schmidt wrote:
> >  
> >>Curious. What do you need expect for in this scenario? svn is not an 
> >>interactive program. The only thing I think it can prompt for is the 
> >>username and password, and you can provide that with the --username 
> >>and --password options without resorting to something like expect.
> >>    
> >
> >The --password option may be a security hole, e.g. if there are 
> >several users on the machine.
> --password `cat /var/etc/thefilewiththepassword` And file can have all 
> the fancy permissions to prevent people from seeing it. or does it 
> show on process list apps like top or ps?

You bet it does.

> atleast ncftp hides some command line parameters from process list.

That is usually done by rewriting argv, so it introduces a race condition
where the password is visible in plaintext format.  Also not all systems
allow you to rewrite argv and have the kernel pick up the changed version.

-- 
				-- rouilj

John Rouillard
System Administrator
Renesys Corporation
603-643-9300 x 111

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: svn and cron

Posted by John Rouillard <ro...@renesys.com>.
On Tue, Dec 12, 2006 at 02:01:53PM +0200, Mikko Ruohola wrote:
> Vincent Lefevre kirjoitti:
> >On 2006-12-11 20:50:49 -0600, Ryan Schmidt wrote:
> >  
> >>Curious. What do you need expect for in this scenario? svn is not an  
> >>interactive program. The only thing I think it can prompt for is the  
> >>username and password, and you can provide that with the --username  
> >>and --password options without resorting to something like expect.
> >>    
> >
> >The --password option may be a security hole, e.g. if there are
> >several users on the machine.
> --password `cat /var/etc/thefilewiththepassword`
> And file can have all the fancy permissions to prevent people from 
> seeing it. or does it show on process list apps like top or ps?

You bet it does.

> atleast ncftp hides some command line parameters from process list.

That is usually done by rewriting argv, so it introduces a race
condition where the password is visible in plaintext format.  Also not
all systems allow you to rewrite argv and have the kernel pick up the
changed version.

-- 
				-- rouilj

John Rouillard
System Administrator
Renesys Corporation
603-643-9300 x 111

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: svn and cron

Posted by Mikko Ruohola <po...@taajama.fi>.
Vincent Lefevre kirjoitti:
> On 2006-12-11 20:50:49 -0600, Ryan Schmidt wrote:
>   
>> Curious. What do you need expect for in this scenario? svn is not an  
>> interactive program. The only thing I think it can prompt for is the  
>> username and password, and you can provide that with the --username  
>> and --password options without resorting to something like expect.
>>     
>
> The --password option may be a security hole, e.g. if there are
> several users on the machine.
>
>   
--password `cat /var/etc/thefilewiththepassword`
And file can have all the fancy permissions to prevent people from 
seeing it. or does it show on process list apps like top or ps? atleast 
ncftp hides some command line parameters from process list.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: svn and cron

Posted by Erik Huelsmann <eh...@gmail.com>.
On 12/12/06, Vincent Lefevre <vi...@vinc17.org> wrote:
> On 2006-12-11 20:50:49 -0600, Ryan Schmidt wrote:
> > Curious. What do you need expect for in this scenario? svn is not an
> > interactive program. The only thing I think it can prompt for is the
> > username and password, and you can provide that with the --username
> > and --password options without resorting to something like expect.
>
> The --password option may be a security hole, e.g. if there are
> several users on the machine.

Yup. If you want authentication, it'll be hard to get right from a
cron job, especially with several users on one machine (ie you can't
whitelist an IP).

Also note that if you want to prevent user interaction in your script,
you should pass the --non-interactive command line option.

HTH,

Erik.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: svn and cron

Posted by Thomas Harold <tg...@tgharold.com>.
Les Mikesell wrote:
> 
> As someone who deals with about 10 gigs (compressed) of web logs a
> month,
> I have to wonder why anyone would want to put timestamped lines that
> never
> change into a revision control repository that won't allow you to
> ever delete anything.  'rsync -essh' works if you just need a transport.
> 

Our logs only grow about 1GB per month (uncompressed).  If/when the 
repository gets too big and I decide that I don't want to keep log files 
from 2000 any longer, we'll simply do a dump/restore to a new repository 
every few years.  Or the same repository.

I think one advantage over rsync might be that the web server can't 
delete older log files.  Due to the locked down nature of the SSH key 
along with a heavily locked down SSHD configuration, the web server 
machine shouldn't be able to do damage.  The worst it can probably do is 
an "svn del" against the repository, which can easily be undone.  I 
might also implement an IP address filter on the SSH key.

It's useful for us because we already have an SSH key structure in 
place, SVN is installed, and if a developer / admin wants to look at log 
files, they're as easy to get to as any other repository.  So there's 
little training or hand-holding needed.

But mostly it was done as a test exercise to see how well it worked. 
Necessary?  Maybe not.  But it sharpened my SVN admin skills.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: svn and cron

Posted by Les Mikesell <le...@gmail.com>.
On Wed, 2006-12-13 at 18:37 -0500, Thomas Harold wrote:

> You might also look into using a no-password SSH keyfile that has 
> limited rights to the SVN repository (i.e. it can only read).  Plus you 
> can restrict the SSH key to only executing the "svnserve -t" command on 
> the SVN server.
> 
> Which should be fairly secure and low-risk.
> 
> (We us something similar to push log files from our web servers to a 
> central SVN server.  Works well, only transmits the changes and we have 
> it set to transmit hourly.)

As someone who deals with about 10 gigs (compressed) of web logs a
month,
I have to wonder why anyone would want to put timestamped lines that
never
change into a revision control repository that won't allow you to
ever delete anything.  'rsync -essh' works if you just need a transport.

-- 
  Les Mikesell
   lesmikesell@gmail.com


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

RE: svn and cron

Posted by Garrett McGrath <gm...@Princeton.EDU>.
 It actually boiled down to permissions and the restricted execution
enviroment for cron.  Specifying the complete path for the svn binary and
the working directory fixed the problem.  The SSH issues and other net
related issues were moot points due to the repository being accessed via the
file:// option.  The working directory is what we write to since it's
mounted on all but 1 of the servers, then the workhorse server just does the
commit for the entire working area.
-Garrett

-----Original Message-----
From: Thomas Harold [mailto:tgh@tgharold.com] 
Sent: Wednesday, December 13, 2006 6:38 PM
To: users@subversion.tigris.org
Subject: Re: svn and cron

Vincent Lefevre wrote:
> On 2006-12-11 20:50:49 -0600, Ryan Schmidt wrote:
>> Curious. What do you need expect for in this scenario? svn is not an 
>> interactive program. The only thing I think it can prompt for is the 
>> username and password, and you can provide that with the --username 
>> and --password options without resorting to something like expect.
> 
> The --password option may be a security hole, e.g. if there are 
> several users on the machine.
> 

You might also look into using a no-password SSH keyfile that has limited
rights to the SVN repository (i.e. it can only read).  Plus you can restrict
the SSH key to only executing the "svnserve -t" command on the SVN server.

Which should be fairly secure and low-risk.

(We us something similar to push log files from our web servers to a central
SVN server.  Works well, only transmits the changes and we have it set to
transmit hourly.)

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: svn and cron

Posted by Thomas Harold <tg...@tgharold.com>.
Vincent Lefevre wrote:
> On 2006-12-11 20:50:49 -0600, Ryan Schmidt wrote:
>> Curious. What do you need expect for in this scenario? svn is not an  
>> interactive program. The only thing I think it can prompt for is the  
>> username and password, and you can provide that with the --username  
>> and --password options without resorting to something like expect.
> 
> The --password option may be a security hole, e.g. if there are
> several users on the machine.
> 

You might also look into using a no-password SSH keyfile that has 
limited rights to the SVN repository (i.e. it can only read).  Plus you 
can restrict the SSH key to only executing the "svnserve -t" command on 
the SVN server.

Which should be fairly secure and low-risk.

(We us something similar to push log files from our web servers to a 
central SVN server.  Works well, only transmits the changes and we have 
it set to transmit hourly.)

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: svn and cron

Posted by Vincent Lefevre <vi...@vinc17.org>.
On 2006-12-11 20:50:49 -0600, Ryan Schmidt wrote:
> Curious. What do you need expect for in this scenario? svn is not an  
> interactive program. The only thing I think it can prompt for is the  
> username and password, and you can provide that with the --username  
> and --password options without resorting to something like expect.

The --password option may be a security hole, e.g. if there are
several users on the machine.

-- 
Vincent Lefèvre <vi...@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: svn and cron

Posted by Ryan Schmidt <su...@ryandesign.com>.
On Dec 11, 2006, at 11:37, Chris Fouts wrote:

>> Has anybody had any success running cron based svn scripts?  Mine  
>> can all be executed from command line but for some reason running  
>> then from cron they start discovering these non exsistent  
>> temporary files and manage to lock my svn archive for no aparent  
>> reason.  I understand that svn isn't supposed to be automated but  
>> this small side project would be a fairly huge boon in the  
>> documentation of our centers server configurations.  However  
>> running the scripts manually effectively eliminates the boon  
>> gained by the automation.  If anybody has been able to  
>> successfully execute svn cleanup, svn commit or svn update from a  
>> cron job please let me know.
>
> I run expect scripts that do an "svn co" for my nightly regression
> tests on the trunk.

Curious. What do you need expect for in this scenario? svn is not an  
interactive program. The only thing I think it can prompt for is the  
username and password, and you can provide that with the --username  
and --password options without resorting to something like expect.


-- 
To reply to the mailing list, please use your mailer's Reply To All  
function


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

RE: svn and cron

Posted by Ch...@qimonda.com.
I run expect scripts that do an "svn co" for my nightly regression
tests on the trunk.


________________________________

	From: Garrett McGrath [mailto:gmcgrath@Princeton.EDU] 
	Sent: Monday, December 11, 2006 11:30 AM
	To: users@subversion.tigris.org
	Subject: svn and cron
	
	
	 
	Has anybody had any success running cron based svn scripts?
Mine can all be executed from command line but for some reason running
then from cron they start discovering these non exsistent temporary
files and manage to lock my svn archive for no aparent reason.  I
understand that svn isn't supposed to be automated but this small side
project would be a fairly huge boon in the documentation of our centers
server configurations.  However running the scripts manually effectively
eliminates the boon gained by the automation.  If anybody has been able
to successfully execute svn cleanup, svn commit or svn update from a
cron job please let me know.
	-Garrett McGrath


RE: svn and cron

Posted by Thomas Hemmer <th...@go-engineering.de>.
Garrett,

did you take into account that cron scripts will be running within a
rather restricted environment?
Especially your $PATH does not contain everything what you would expect
from an interactive shell.


HTH,

Thomas


________________________________

	From: Garrett McGrath [mailto:gmcgrath@Princeton.EDU]
	Sent: Monday, December 11, 2006 5:30 PM
	To: users@subversion.tigris.org
	Subject: svn and cron



	Has anybody had any success running cron based svn scripts?
Mine can all be executed from command line but for some reason running
then from cron they start discovering these non exsistent temporary
files and manage to lock my svn archive for no aparent reason.  I
understand that svn isn't supposed to be automated but this small side
project would be a fairly huge boon in the documentation of our centers
server configurations.  However running the scripts manually effectively
eliminates the boon gained by the automation.  If anybody has been able
to successfully execute svn cleanup, svn commit or svn update from a
cron job please let me know.
	-Garrett McGrath




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org