You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Ryan Hunt <rh...@hp.com> on 2003/12/09 23:14:43 UTC
Re: WC permissions
On Dec 9, 2003, at 12:15 PM, y2w2de9j001@sneakemail.com wrote:
>
> What are the fundamental problems with allowing more than two people
> from working within the same WC, other than that svn is chmod'ing
> files that it might not need to, then failing with EPERM?
>
> Thanks,
>
> Scott
I would like to know this too as it is becoming more and more essential
that all users of my repository use the same WC. My full WC is
currently 200GB and scheduled to be 5TB within 6-9 months. This will
be impossible to maintain more than a couple of WCs.
Is this going to be allowed in future versions, or does anyone have a
work around?
-Ryan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by Sigfred Håversen <su...@mumak.com>.
On Wednesday 10 December 2003 00:14, Ryan Hunt wrote:
[snip]
> My full WC is
> currently 200GB and scheduled to be 5TB within 6-9 months. This will
> be impossible to maintain more than a couple of WCs.
I'm curious, what kind of data do you have since you need such large WC? How
do you actually manage such a huge amount of data in Subversion?
/Sigfred
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by Branko Čibej <br...@xbc.nu>.
Philip Martin wrote:
>Branko ÄŒibej <br...@xbc.nu> writes:
>
>
>
>>Philip Martin wrote:
>>
>>
>>
>>>Branko ÄŒibej <br...@xbc.nu> writes:
>>>
>>>
>>>>The only way I know to do this on Unix (if your filesystem doesn't have
>>>>inheritable ACLs) is to put all your users into the same group (doesn't
>>>>have to be the primary group), use umask 002 and set the sticky-group
>>>>bit on the directories in the subversion WC.
>>>>
>>>>
>>>On my system chmod(2) says
>>>
>>>"The effective UID of the process must be zero or must match the
>>> owner of the file."
>>>
>>>so the sticky bit does not appear to solve the shared working copy
>>>problem. It does solve the shared repository problem.
>>>
>>>
>>>
>>Hm? What does chmod have to do with sticky group ownership?
>>
>>
>
>Perhaps I misunderstood, I thought you were suggesting that group
>access (hence the sticky bit) would be sufficient for shared working
>copies.
>
>
No, you didn't misunderstand, but I forgot about us making files
read-only, and of course chmod requires ownership. How boring.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by Branko Čibej <br...@xbc.nu>.
Philip Martin wrote:
>Branko ÄŒibej <br...@xbc.nu> writes:
>
>
>
>>Philip Martin wrote:
>>
>>
>>
>>>Branko ÄŒibej <br...@xbc.nu> writes:
>>>
>>>
>>>>The only way I know to do this on Unix (if your filesystem doesn't have
>>>>inheritable ACLs) is to put all your users into the same group (doesn't
>>>>have to be the primary group), use umask 002 and set the sticky-group
>>>>bit on the directories in the subversion WC.
>>>>
>>>>
>>>On my system chmod(2) says
>>>
>>>"The effective UID of the process must be zero or must match the
>>> owner of the file."
>>>
>>>so the sticky bit does not appear to solve the shared working copy
>>>problem. It does solve the shared repository problem.
>>>
>>>
>>>
>>Hm? What does chmod have to do with sticky group ownership?
>>
>>
>
>Perhaps I misunderstood, I thought you were suggesting that group
>access (hence the sticky bit) would be sufficient for shared working
>copies.
>
>
No, you didn't misunderstand, but I forgot about us making files
read-only, and of course chmod requires ownership. How boring.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by Philip Martin <ph...@codematters.co.uk>.
Branko Äibej <br...@xbc.nu> writes:
> Philip Martin wrote:
>
>>Branko ÃÅibej <br...@xbc.nu> writes:
>>>>
>>>The only way I know to do this on Unix (if your filesystem doesn't have
>>>inheritable ACLs) is to put all your users into the same group (doesn't
>>>have to be the primary group), use umask 002 and set the sticky-group
>>>bit on the directories in the subversion WC.
>>
>>On my system chmod(2) says
>>
>> "The effective UID of the process must be zero or must match the
>> owner of the file."
>>
>>so the sticky bit does not appear to solve the shared working copy
>>problem. It does solve the shared repository problem.
>>
> Hm? What does chmod have to do with sticky group ownership?
Perhaps I misunderstood, I thought you were suggesting that group
access (hence the sticky bit) would be sufficient for shared working
copies.
> What I described works on every Unix-ish system i've laid my hands
> on. Not many, I'll admit -- only AIX, HP-UX, Solaris, FreeBSD and
> Linux.
On my Linux box only the owner of a file can change the permissions on
that file. So it doesn't matter how I set up groups, umask or sticky
bits, only the wc owner can run update.
$ umask
0002
$ groups
pm
$ svnadmin create repo
$ svn co file://`pwd`/repo wc
$ chmod -R +t wc
$ svn mkdir -m "" file://`pwd`/repo/foo
$ su corris
Password:
$ newgrp pm
Password:
$ groups
pm corris
$ umask
0002
$ svn up wc
../svn/subversion/libsvn_subr/io.c:749: (apr_err=1)
svn: Can't set file 'wc/.svn/entries' read-write: Operation not permitted
What's really nasty is that this leaves wc/.svn/tmp/entries owned by
the "other" user and it gets reused when the wc owner runs update.
Thus when wc/.svn/tmp/entries gets renamed to wc/.svn/entries it is
still owned by the other user, and so the wc owner cannot set its
permissions and the update fails.
Shared working copies are not really possible.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by Philip Martin <ph...@codematters.co.uk>.
Branko Äibej <br...@xbc.nu> writes:
> Philip Martin wrote:
>
>>Branko ÃÅibej <br...@xbc.nu> writes:
>>>>
>>>The only way I know to do this on Unix (if your filesystem doesn't have
>>>inheritable ACLs) is to put all your users into the same group (doesn't
>>>have to be the primary group), use umask 002 and set the sticky-group
>>>bit on the directories in the subversion WC.
>>
>>On my system chmod(2) says
>>
>> "The effective UID of the process must be zero or must match the
>> owner of the file."
>>
>>so the sticky bit does not appear to solve the shared working copy
>>problem. It does solve the shared repository problem.
>>
> Hm? What does chmod have to do with sticky group ownership?
Perhaps I misunderstood, I thought you were suggesting that group
access (hence the sticky bit) would be sufficient for shared working
copies.
> What I described works on every Unix-ish system i've laid my hands
> on. Not many, I'll admit -- only AIX, HP-UX, Solaris, FreeBSD and
> Linux.
On my Linux box only the owner of a file can change the permissions on
that file. So it doesn't matter how I set up groups, umask or sticky
bits, only the wc owner can run update.
$ umask
0002
$ groups
pm
$ svnadmin create repo
$ svn co file://`pwd`/repo wc
$ chmod -R +t wc
$ svn mkdir -m "" file://`pwd`/repo/foo
$ su corris
Password:
$ newgrp pm
Password:
$ groups
pm corris
$ umask
0002
$ svn up wc
../svn/subversion/libsvn_subr/io.c:749: (apr_err=1)
svn: Can't set file 'wc/.svn/entries' read-write: Operation not permitted
What's really nasty is that this leaves wc/.svn/tmp/entries owned by
the "other" user and it gets reused when the wc owner runs update.
Thus when wc/.svn/tmp/entries gets renamed to wc/.svn/entries it is
still owned by the other user, and so the wc owner cannot set its
permissions and the update fails.
Shared working copies are not really possible.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by Branko Čibej <br...@xbc.nu>.
Philip Martin wrote:
>Branko ÄŒibej <br...@xbc.nu> writes:
>
>
>
>>Ryan Hunt wrote:
>>
>>
>>
>>>On Dec 9, 2003, at 12:15 PM, y2w2de9j001@sneakemail.com wrote:
>>>
>>>
>>>>What are the fundamental problems with allowing more than two people
>>>>from working within the same WC, other than that svn is chmod'ing
>>>>files that it might not need to, then failing with EPERM?
>>>>
>>>>
>>>I would like to know this too as it is becoming more and more
>>>essential that all users of my repository use the same WC.
>>>
>>>
>>The only way I know to do this on Unix (if your filesystem doesn't have
>>inheritable ACLs) is to put all your users into the same group (doesn't
>>have to be the primary group), use umask 002 and set the sticky-group
>>bit on the directories in the subversion WC.
>>
>>
>
>On my system chmod(2) says
>
> "The effective UID of the process must be zero or must match the
> owner of the file."
>
>so the sticky bit does not appear to solve the shared working copy
>problem. It does solve the shared repository problem.
>
>
Hm? What does chmod have to do with sticky group ownership? What I
described works on every Unix-ish system i've laid my hands on. Not
many, I'll admit -- only AIX, HP-UX, Solaris, FreeBSD and Linux.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by Branko Čibej <br...@xbc.nu>.
Philip Martin wrote:
>Branko ÄŒibej <br...@xbc.nu> writes:
>
>
>
>>Ryan Hunt wrote:
>>
>>
>>
>>>On Dec 9, 2003, at 12:15 PM, y2w2de9j001@sneakemail.com wrote:
>>>
>>>
>>>>What are the fundamental problems with allowing more than two people
>>>>from working within the same WC, other than that svn is chmod'ing
>>>>files that it might not need to, then failing with EPERM?
>>>>
>>>>
>>>I would like to know this too as it is becoming more and more
>>>essential that all users of my repository use the same WC.
>>>
>>>
>>The only way I know to do this on Unix (if your filesystem doesn't have
>>inheritable ACLs) is to put all your users into the same group (doesn't
>>have to be the primary group), use umask 002 and set the sticky-group
>>bit on the directories in the subversion WC.
>>
>>
>
>On my system chmod(2) says
>
> "The effective UID of the process must be zero or must match the
> owner of the file."
>
>so the sticky bit does not appear to solve the shared working copy
>problem. It does solve the shared repository problem.
>
>
Hm? What does chmod have to do with sticky group ownership? What I
described works on every Unix-ish system i've laid my hands on. Not
many, I'll admit -- only AIX, HP-UX, Solaris, FreeBSD and Linux.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by Philip Martin <ph...@codematters.co.uk>.
Branko Äibej <br...@xbc.nu> writes:
> Ryan Hunt wrote:
>
>> On Dec 9, 2003, at 12:15 PM, y2w2de9j001@sneakemail.com wrote:
>>>
>>> What are the fundamental problems with allowing more than two people
>>> from working within the same WC, other than that svn is chmod'ing
>>> files that it might not need to, then failing with EPERM?
>>
>> I would like to know this too as it is becoming more and more
>> essential that all users of my repository use the same WC.
>
> The only way I know to do this on Unix (if your filesystem doesn't have
> inheritable ACLs) is to put all your users into the same group (doesn't
> have to be the primary group), use umask 002 and set the sticky-group
> bit on the directories in the subversion WC.
On my system chmod(2) says
"The effective UID of the process must be zero or must match the
owner of the file."
so the sticky bit does not appear to solve the shared working copy
problem. It does solve the shared repository problem.
One way to solve the shared working copy problem is to have a
dedicated svn user and use a setuid client so that all the Subversion
commands are run as the same user. Now, a setuid svn client
effectively provides shell access, so you may want a setuid wrapper
that checks for appropriate conditions (having a suitable user and/or
group ID say) before invoking the normal client.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by Sigfred Håversen <su...@mumak.com>.
On Wednesday 10 December 2003 00:54, Ryan Hunt wrote:
> On Dec 9, 2003, at 4:29 PM, Branko Čibej wrote:
[snip]
> > and set the sticky-group
> > bit on the directories in the subversion WC.
>
> how do I do this?
You use chmod for this. For more information about the sticky bit, see
the man page for sticky on http://www.openbsd.org/cgi-bin/man.cgi
/Sigfred
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by Philip Martin <ph...@codematters.co.uk>.
Branko Äibej <br...@xbc.nu> writes:
> Ryan Hunt wrote:
>
>> On Dec 9, 2003, at 12:15 PM, y2w2de9j001@sneakemail.com wrote:
>>>
>>> What are the fundamental problems with allowing more than two people
>>> from working within the same WC, other than that svn is chmod'ing
>>> files that it might not need to, then failing with EPERM?
>>
>> I would like to know this too as it is becoming more and more
>> essential that all users of my repository use the same WC.
>
> The only way I know to do this on Unix (if your filesystem doesn't have
> inheritable ACLs) is to put all your users into the same group (doesn't
> have to be the primary group), use umask 002 and set the sticky-group
> bit on the directories in the subversion WC.
On my system chmod(2) says
"The effective UID of the process must be zero or must match the
owner of the file."
so the sticky bit does not appear to solve the shared working copy
problem. It does solve the shared repository problem.
One way to solve the shared working copy problem is to have a
dedicated svn user and use a setuid client so that all the Subversion
commands are run as the same user. Now, a setuid svn client
effectively provides shell access, so you may want a setuid wrapper
that checks for appropriate conditions (having a suitable user and/or
group ID say) before invoking the normal client.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by Ryan Hunt <rh...@hp.com>.
On Dec 9, 2003, at 4:29 PM, Branko Čibej wrote:
>
> The only way I know to do this on Unix (if your filesystem doesn't have
> inheritable ACLs) is to put all your users into the same group
done
> (doesn't
> have to be the primary group), use umask 002
done
> and set the sticky-group
> bit on the directories in the subversion WC.
how do I do this?
-Ryan
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by Branko Čibej <br...@xbc.nu>.
Ryan Hunt wrote:
> On Dec 9, 2003, at 12:15 PM, y2w2de9j001@sneakemail.com wrote:
>
>>
>> What are the fundamental problems with allowing more than two people
>> from working within the same WC, other than that svn is chmod'ing
>> files that it might not need to, then failing with EPERM?
>>
>> Thanks,
>>
>> Scott
>
>
> I would like to know this too as it is becoming more and more
> essential that all users of my repository use the same WC. My full WC
> is currently 200GB and scheduled to be 5TB within 6-9 months. This
> will be impossible to maintain more than a couple of WCs.
>
> Is this going to be allowed in future versions, or does anyone have a
> work around?
The only way I know to do this on Unix (if your filesystem doesn't have
inheritable ACLs) is to put all your users into the same group (doesn't
have to be the primary group), use umask 002 and set the sticky-group
bit on the directories in the subversion WC.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by Branko Čibej <br...@xbc.nu>.
Ryan Hunt wrote:
> On Dec 9, 2003, at 12:15 PM, y2w2de9j001@sneakemail.com wrote:
>
>>
>> What are the fundamental problems with allowing more than two people
>> from working within the same WC, other than that svn is chmod'ing
>> files that it might not need to, then failing with EPERM?
>>
>> Thanks,
>>
>> Scott
>
>
> I would like to know this too as it is becoming more and more
> essential that all users of my repository use the same WC. My full WC
> is currently 200GB and scheduled to be 5TB within 6-9 months. This
> will be impossible to maintain more than a couple of WCs.
>
> Is this going to be allowed in future versions, or does anyone have a
> work around?
The only way I know to do this on Unix (if your filesystem doesn't have
inheritable ACLs) is to put all your users into the same group (doesn't
have to be the primary group), use umask 002 and set the sticky-group
bit on the directories in the subversion WC.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by kf...@collab.net.
Ryan Hunt <rh...@hp.com> writes:
> > Working copies are intended to be limited to a single user, and not
> > just for the obvious umask reasons; this is true for CVS just as
> > much as it is for subversion.
>
> This may be how they are intended, I don't know. However, just
> because that is how they are intended to work, doesn't mean that they
> couldn't be used in other ways. Just need to find a work around.
I think John may not have been aware of
http://subversion.tigris.org/issues/show_bug.cgi?id=1509
(scheduled for after 1.0, though).
As for whether 5TB of data will overwhelm Subversion's client-side
code, well, I guess you're our tester :-).
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by John Peacock <jp...@rowman.com>.
kfogel@collab.net wrote:
>>both SVN and CVS are written to assume that the WC is under
>>the exclusive control of one process
>
>
> Actually, both are written to be safe if multiple processes access the
> WC. We deliberately tried to match CVS in this respect -- this is why
> we have wc locks, in fact.
I was not aware of that it worked under CVS, either (see below). Good job!
> (FWIW, I've done this different-users-sharing-a-working-copy thing
> with CVS in real life. It worked out okay, though we had to do some
> tweaking of Unix groups and umasks to make it go smoothly.)
It doesn't work (in my experience) under cvsnt, which is strange since the
concept of umask is foreign to Win32. It's been so long since I tried to make
it work, I don't remember the details, but I seem to remember it messed up the WC...
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5748
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by kf...@collab.net.
John Peacock <jp...@rowman.com> writes:
> Apart from the bandwidth required to transfer enormous files over a
> LAN link, both SVN and CVS are written to assume that the WC is under
> the exclusive control of one process. I don't know exactly what would
> be the worst case, but I could see one user in the middle of a large
> update while another begins a commit as being a problem. I don't want
> to spread FUD without cause, but I'm pretty confident that shared WC's
> are a bad idea.
Shared WCs are only a bad idea because our current implementation
doesn't handle the permissions sophisticatedly enough for it to work.
Here's the key mis-assumption:
> both SVN and CVS are written to assume that the WC is under
> the exclusive control of one process
Actually, both are written to be safe if multiple processes access the
WC. We deliberately tried to match CVS in this respect -- this is why
we have wc locks, in fact. But the real problem here is not multiple
processes per se, rather it is that the ownerships/perms on our
metadata currently make it difficult for different users to share a
working copy. That's what issue #1509 is about fixing.
(FWIW, I've done this different-users-sharing-a-working-copy thing
with CVS in real life. It worked out okay, though we had to do some
tweaking of Unix groups and umasks to make it go smoothly.)
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by kf...@collab.net.
John Peacock <jp...@rowman.com> writes:
> Apart from the bandwidth required to transfer enormous files over a
> LAN link, both SVN and CVS are written to assume that the WC is under
> the exclusive control of one process. I don't know exactly what would
> be the worst case, but I could see one user in the middle of a large
> update while another begins a commit as being a problem. I don't want
> to spread FUD without cause, but I'm pretty confident that shared WC's
> are a bad idea.
Shared WCs are only a bad idea because our current implementation
doesn't handle the permissions sophisticatedly enough for it to work.
Here's the key mis-assumption:
> both SVN and CVS are written to assume that the WC is under
> the exclusive control of one process
Actually, both are written to be safe if multiple processes access the
WC. We deliberately tried to match CVS in this respect -- this is why
we have wc locks, in fact. But the real problem here is not multiple
processes per se, rather it is that the ownerships/perms on our
metadata currently make it difficult for different users to share a
working copy. That's what issue #1509 is about fixing.
(FWIW, I've done this different-users-sharing-a-working-copy thing
with CVS in real life. It worked out okay, though we had to do some
tweaking of Unix groups and umasks to make it go smoothly.)
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by Sander Striker <st...@apache.org>.
On Wed, 2003-12-10 at 21:16, John Peacock wrote:
> Ryan Hunt wrote:
>
> > As far as I was aware the only issue with mounted file systems was with the
> > repository, not the WC. If this is not the case can you identify the issues
> > that will arise due to accessing a WC across a network file system?
>
> Apart from the bandwidth required to transfer enormous files over a LAN link,
> both SVN and CVS are written to assume that the WC is under the exclusive
> control of one process. I don't know exactly what would be the worst case, but
> I could see one user in the middle of a large update while another begins a
> commit as being a problem. I don't want to spread FUD without cause, but I'm
> pretty confident that shared WC's are a bad idea.
Be prepared to be told that this is a common practice ;)
Sander
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: WC permissions
Posted by Alan <ju...@alan2.com>.
John Peacock Wrote:
> There is also the issue that if the individual files are very large, the time
to produce > the version delta may be so large that the system would have
difficulty keeping up
> (depending on the speed of the changes). Scaling has a lot more to do with
the whole
> process, not just the storage.
I don't know the internals of how svn does its binary diffing... But from my
past experences with other binary diff tools, it is VERY memory intensive...
I have worked with bsdiff, which takes around 17 times the size of the file to
do its diffs... http://www.daemonology.net/bsdiff/
Granted this is a little extreme, but in general, unless you really are making
small changes to binary files, as opposed to not outright changing them
dramatically, this may not be the best tool for the job...
-Alan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by Sander Striker <st...@apache.org>.
On Wed, 2003-12-10 at 21:16, John Peacock wrote:
> Ryan Hunt wrote:
>
> > As far as I was aware the only issue with mounted file systems was with the
> > repository, not the WC. If this is not the case can you identify the issues
> > that will arise due to accessing a WC across a network file system?
>
> Apart from the bandwidth required to transfer enormous files over a LAN link,
> both SVN and CVS are written to assume that the WC is under the exclusive
> control of one process. I don't know exactly what would be the worst case, but
> I could see one user in the middle of a large update while another begins a
> commit as being a problem. I don't want to spread FUD without cause, but I'm
> pretty confident that shared WC's are a bad idea.
Be prepared to be told that this is a common practice ;)
Sander
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by Ryan Hunt <rh...@hp.com>.
On Dec 10, 2003, at 1:16 PM, John Peacock wrote:
> Ryan Hunt wrote:
>> As far as I was aware the only limitations in this capacity were
>> those due to
>> the berkeley bd.
>
> There is also the issue that if the individual files are very large,
> the time to produce the version delta may be so large that the system
> would have difficulty keeping up (depending on the speed of the
> changes). Scaling has a lot more to do with the whole process, not
> just the storage.
So does this mean that the diff is created for the entire ci, or is
each file diffed individually and then readied for ci?
>>> Unless the differences between versions are themselves significant
>>> (as
>>> opposed to just being able to extract any given version),
>> Is there any significant difference?
>
> Yes. If you only need to retrieve an arbitrary version of a file, you
> can just
> as easily copy the file off to another location (c.f. snapshots
> below). With
> source-code (i.e. non-binary), frequently the diff between versions is
> itself of
> some important (i.e. what code changed).
While the diff itself is not so much of importance the fact that only
the diff is stored is.
For example: I just checked in a change to a 1.5GB disk image that had
some minor modifications. Excluding log files which can be removed
after backup the repository grew by just over 100MB. From what I can
see of the snapshot feature this would have taken 1.5GB of space to
save the new version. I can therefore get 15 levels of history in the
same space with SVN if I have the patience to wait for a checkin that
take approximately 1 hour per Gig of original source file.
This is the time it takes on rather old and dilapidated equipment. I
am in the process of trying to order 2 proc 2GHz G5s with 4GB of RAM
with a series of 2.5TB XRaids attached for the repositories. The
expectation is that this new equipment will be even faster, though I
have not been able to benchmark it at all.
-Ryan
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by Ryan Hunt <rh...@hp.com>.
On Dec 10, 2003, at 1:16 PM, John Peacock wrote:
> Ryan Hunt wrote:
>> As far as I was aware the only limitations in this capacity were
>> those due to
>> the berkeley bd.
>
> There is also the issue that if the individual files are very large,
> the time to produce the version delta may be so large that the system
> would have difficulty keeping up (depending on the speed of the
> changes). Scaling has a lot more to do with the whole process, not
> just the storage.
So does this mean that the diff is created for the entire ci, or is
each file diffed individually and then readied for ci?
>>> Unless the differences between versions are themselves significant
>>> (as
>>> opposed to just being able to extract any given version),
>> Is there any significant difference?
>
> Yes. If you only need to retrieve an arbitrary version of a file, you
> can just
> as easily copy the file off to another location (c.f. snapshots
> below). With
> source-code (i.e. non-binary), frequently the diff between versions is
> itself of
> some important (i.e. what code changed).
While the diff itself is not so much of importance the fact that only
the diff is stored is.
For example: I just checked in a change to a 1.5GB disk image that had
some minor modifications. Excluding log files which can be removed
after backup the repository grew by just over 100MB. From what I can
see of the snapshot feature this would have taken 1.5GB of space to
save the new version. I can therefore get 15 levels of history in the
same space with SVN if I have the patience to wait for a checkin that
take approximately 1 hour per Gig of original source file.
This is the time it takes on rather old and dilapidated equipment. I
am in the process of trying to order 2 proc 2GHz G5s with 4GB of RAM
with a series of 2.5TB XRaids attached for the repositories. The
expectation is that this new equipment will be even faster, though I
have not been able to benchmark it at all.
-Ryan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by John Peacock <jp...@rowman.com>.
Ryan Hunt wrote:
> As far as I was aware the only issue with mounted file systems was with the
> repository, not the WC. If this is not the case can you identify the issues
> that will arise due to accessing a WC across a network file system?
Apart from the bandwidth required to transfer enormous files over a LAN link,
both SVN and CVS are written to assume that the WC is under the exclusive
control of one process. I don't know exactly what would be the worst case, but
I could see one user in the middle of a large update while another begins a
commit as being a problem. I don't want to spread FUD without cause, but I'm
pretty confident that shared WC's are a bad idea.
>
>> Secondly, I would seriously doubt that subversion will scale to 5TB WC's
>> (not to mention what this means the repository must look like).
>>
>
> As far as I was aware the only limitations in this capacity were those due to
> the berkeley bd.
There is also the issue that if the individual files are very large, the time to
produce the version delta may be so large that the system would have difficulty
keeping up (depending on the speed of the changes). Scaling has a lot more to
do with the whole process, not just the storage.
>> Unless the differences between versions are themselves significant (as
>> opposed to just being able to extract any given version),
>
>
> Is there any significant difference?
>
Yes. If you only need to retrieve an arbitrary version of a file, you can just
as easily copy the file off to another location (c.f. snapshots below). With
source-code (i.e. non-binary), frequently the diff between versions is itself of
some important (i.e. what code changed).
> I am not terribly familiar with SANs and the snapshot feature. Do you have
> any good links that describe this process?
They are just automatic backups to a protected portion of the volume. Snapshots
are available from several NAS vendors (NetworkAppliance for example has hourly
snapshots available). They require 100% additional storage per file per
snapshot (but drives are cheap these days). Here's something from NetApps website:
http://www.netapp.com/tech_library/3002.html
(not a recommendation for their product, just I know they have them).
HTH
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5748
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by Ryan Hunt <rh...@hp.com>.
On Dec 11, 2003, at 10:01 AM, Mike Mason wrote:
> Ryan Hunt wrote:
>
>>
>> If there are other limitations that svn is placing on the versioned
>> file system I would very much like to know. If there are I will have
>> to rethink my strategy.
>
>
> Have you thought about AlienBrain? Not used it myself, but apparently
> very good for creative types with big binary files.
> http://www.nxn-software.com/
>
> Mike.
AlienBrain is something I looked at a while ago. As far as I know
however, they only have a solution for Windows, and my solution has to
be Unix based.
-Ryan
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by Ryan Hunt <rh...@hp.com>.
On Dec 11, 2003, at 10:01 AM, Mike Mason wrote:
> Ryan Hunt wrote:
>
>>
>> If there are other limitations that svn is placing on the versioned
>> file system I would very much like to know. If there are I will have
>> to rethink my strategy.
>
>
> Have you thought about AlienBrain? Not used it myself, but apparently
> very good for creative types with big binary files.
> http://www.nxn-software.com/
>
> Mike.
AlienBrain is something I looked at a while ago. As far as I know
however, they only have a solution for Windows, and my solution has to
be Unix based.
-Ryan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by Mike Mason <mg...@thoughtworks.net>.
Ryan Hunt wrote:
>
> If there are other limitations that svn is placing on the versioned
> file system I would very much like to know. If there are I will have
> to rethink my strategy.
Have you thought about AlienBrain? Not used it myself, but apparently
very good for creative types with big binary files.
http://www.nxn-software.com/
Mike.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by John Peacock <jp...@rowman.com>.
Ryan Hunt wrote:
> As far as I was aware the only issue with mounted file systems was with the
> repository, not the WC. If this is not the case can you identify the issues
> that will arise due to accessing a WC across a network file system?
Apart from the bandwidth required to transfer enormous files over a LAN link,
both SVN and CVS are written to assume that the WC is under the exclusive
control of one process. I don't know exactly what would be the worst case, but
I could see one user in the middle of a large update while another begins a
commit as being a problem. I don't want to spread FUD without cause, but I'm
pretty confident that shared WC's are a bad idea.
>
>> Secondly, I would seriously doubt that subversion will scale to 5TB WC's
>> (not to mention what this means the repository must look like).
>>
>
> As far as I was aware the only limitations in this capacity were those due to
> the berkeley bd.
There is also the issue that if the individual files are very large, the time to
produce the version delta may be so large that the system would have difficulty
keeping up (depending on the speed of the changes). Scaling has a lot more to
do with the whole process, not just the storage.
>> Unless the differences between versions are themselves significant (as
>> opposed to just being able to extract any given version),
>
>
> Is there any significant difference?
>
Yes. If you only need to retrieve an arbitrary version of a file, you can just
as easily copy the file off to another location (c.f. snapshots below). With
source-code (i.e. non-binary), frequently the diff between versions is itself of
some important (i.e. what code changed).
> I am not terribly familiar with SANs and the snapshot feature. Do you have
> any good links that describe this process?
They are just automatic backups to a protected portion of the volume. Snapshots
are available from several NAS vendors (NetworkAppliance for example has hourly
snapshots available). They require 100% additional storage per file per
snapshot (but drives are cheap these days). Here's something from NetApps website:
http://www.netapp.com/tech_library/3002.html
(not a recommendation for their product, just I know they have them).
HTH
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5748
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by Mike Mason <mg...@thoughtworks.net>.
Ryan Hunt wrote:
>
> If there are other limitations that svn is placing on the versioned
> file system I would very much like to know. If there are I will have
> to rethink my strategy.
Have you thought about AlienBrain? Not used it myself, but apparently
very good for creative types with big binary files.
http://www.nxn-software.com/
Mike.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by kf...@collab.net.
Ryan Hunt <rh...@hp.com> writes:
> > Working copies are intended to be limited to a single user, and not
> > just for the obvious umask reasons; this is true for CVS just as
> > much as it is for subversion.
>
> This may be how they are intended, I don't know. However, just
> because that is how they are intended to work, doesn't mean that they
> couldn't be used in other ways. Just need to find a work around.
I think John may not have been aware of
http://subversion.tigris.org/issues/show_bug.cgi?id=1509
(scheduled for after 1.0, though).
As for whether 5TB of data will overwhelm Subversion's client-side
code, well, I guess you're our tester :-).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by Ryan Hunt <rh...@hp.com>.
On Dec 10, 2003, at 9:14 AM, John Peacock wrote:
> Ryan Hunt wrote:
>
>> I would like to know this too as it is becoming more and more
>> essential that all users of my repository use the same WC. My full
>> WC is currently 200GB and scheduled to be 5TB within 6-9 months.
>> This will be impossible to maintain more than a couple of WCs.
>
> EWRONGTOOL
>
> Working copies are intended to be limited to a single user, and not
> just for the obvious umask reasons; this is true for CVS just as much
> as it is for subversion.
>
This may be how they are intended, I don't know. However, just because
that is how they are intended to work, doesn't mean that they couldn't
be used in other ways. Just need to find a work around.
> For one thing, WC's are supposed to be on a local (not mapped) drive;
> are all of your users accessing this massive WC from a single computer
> with this WC stored locally to that machine?
>
As far as I was aware the only issue with mounted file systems was with
the repository, not the WC. If this is not the case can you identify
the issues that will arise due to accessing a WC across a network file
system?
> Secondly, I would seriously doubt that subversion will scale to 5TB
> WC's (not to mention what this means the repository must look like).
>
As far as I was aware the only limitations in this capacity were those
due to the berkeley bd. Quoted from sleepycat's online documentation:
http://www.sleepycat.com/docs/ref/am_misc/dbsizes.html
>>> "The largest database file that Berkeley DB can handle depends on
>>> the page size selected by the application. Berkeley DB stores
>>> database file page numbers as unsigned 32-bit numbers and database
>>> file page sizes as unsigned 16-bit numbers. Using the maximum
>>> database page size of 65536, this results in a maximum database file
>>> size of 248 (256 terabytes). The minimum database page size is 512
>>> bytes, which results in a minimum maximum database size of 241 (2
>>> terabytes).
>>>
>>> The largest database file Berkeley DB can support is potentially
>>> further limited if the host system does not have filesystem support
>>> for files larger than 232, including the ability to seek to absolute
>>> offsets within those files."
If there are other limitations that svn is placing on the versioned
file system I would very much like to know. If there are I will have
to rethink my strategy.
> I'm guessing you have large binary files that you are trying to
> maintain as multiple versions (correct me if I'm wrong).
You are correct.
> Unless the differences between versions are themselves significant (as
> opposed to just being able to extract any given version),
Is there any significant difference?
> there is little functionality that Subversion can provide for you
> (IMHO). I'd have to say a NAS/SAN with online snapshots is probably
> more likely to fit your actual needs.
I am not terribly familiar with SANs and the snapshot feature. Do you
have any good links that describe this process?
-Ryan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: WC permissions
Posted by Ryan Hunt <rh...@hp.com>.
On Dec 10, 2003, at 9:14 AM, John Peacock wrote:
> Ryan Hunt wrote:
>
>> I would like to know this too as it is becoming more and more
>> essential that all users of my repository use the same WC. My full
>> WC is currently 200GB and scheduled to be 5TB within 6-9 months.
>> This will be impossible to maintain more than a couple of WCs.
>
> EWRONGTOOL
>
> Working copies are intended to be limited to a single user, and not
> just for the obvious umask reasons; this is true for CVS just as much
> as it is for subversion.
>
This may be how they are intended, I don't know. However, just because
that is how they are intended to work, doesn't mean that they couldn't
be used in other ways. Just need to find a work around.
> For one thing, WC's are supposed to be on a local (not mapped) drive;
> are all of your users accessing this massive WC from a single computer
> with this WC stored locally to that machine?
>
As far as I was aware the only issue with mounted file systems was with
the repository, not the WC. If this is not the case can you identify
the issues that will arise due to accessing a WC across a network file
system?
> Secondly, I would seriously doubt that subversion will scale to 5TB
> WC's (not to mention what this means the repository must look like).
>
As far as I was aware the only limitations in this capacity were those
due to the berkeley bd. Quoted from sleepycat's online documentation:
http://www.sleepycat.com/docs/ref/am_misc/dbsizes.html
>>> "The largest database file that Berkeley DB can handle depends on
>>> the page size selected by the application. Berkeley DB stores
>>> database file page numbers as unsigned 32-bit numbers and database
>>> file page sizes as unsigned 16-bit numbers. Using the maximum
>>> database page size of 65536, this results in a maximum database file
>>> size of 248 (256 terabytes). The minimum database page size is 512
>>> bytes, which results in a minimum maximum database size of 241 (2
>>> terabytes).
>>>
>>> The largest database file Berkeley DB can support is potentially
>>> further limited if the host system does not have filesystem support
>>> for files larger than 232, including the ability to seek to absolute
>>> offsets within those files."
If there are other limitations that svn is placing on the versioned
file system I would very much like to know. If there are I will have
to rethink my strategy.
> I'm guessing you have large binary files that you are trying to
> maintain as multiple versions (correct me if I'm wrong).
You are correct.
> Unless the differences between versions are themselves significant (as
> opposed to just being able to extract any given version),
Is there any significant difference?
> there is little functionality that Subversion can provide for you
> (IMHO). I'd have to say a NAS/SAN with online snapshots is probably
> more likely to fit your actual needs.
I am not terribly familiar with SANs and the snapshot feature. Do you
have any good links that describe this process?
-Ryan
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: WC permissions
Posted by John Peacock <jp...@rowman.com>.
Ryan Hunt wrote:
> I would like to know this too as it is becoming more and more essential
> that all users of my repository use the same WC. My full WC is
> currently 200GB and scheduled to be 5TB within 6-9 months. This will be
> impossible to maintain more than a couple of WCs.
>
EWRONGTOOL
Working copies are intended to be limited to a single user, and not just for the
obvious umask reasons; this is true for CVS just as much as it is for subversion.
For one thing, WC's are supposed to be on a local (not mapped) drive; are all of
your users accessing this massive WC from a single computer with this WC stored
locally to that machine?
Secondly, I would seriously doubt that subversion will scale to 5TB WC's (not to
mention what this means the repository must look like).
I'm guessing you have large binary files that you are trying to maintain as
multiple versions (correct me if I'm wrong). Unless the differences between
versions are themselves significant (as opposed to just being able to extract
any given version), there is little functionality that Subversion can provide
for you (IMHO). I'd have to say a NAS/SAN with online snapshots is probably
more likely to fit your actual needs.
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5748
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Lost my Subversion/config file & can't recreate it.
Posted by Steve Williams <st...@kromestudios.com>.
I think I had to delete the entire directory in order to get the file back.
Sly
> I was trying to update the config file to ignore VIM
> .*.swp files, but got an error. Following the advice
> from Chapter 6: Advanced Topics:
>
> If at any time you wish to have the default settings
> restore to one or more of the configuration files, you
> can simply delete the file, and run some innocuous svn
> command, such as svn --version, and the missing file
> will be recreated in its default state.
>
> So, I deleted the Subversion/config file. However,
> "svn --version" does not recreate it. Any suggestions
> on how to get it back?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Lost my Subversion/config file & can't recreate it.
Posted by Alan Thompson <ca...@yahoo.com>.
Hi,
I was trying to update the config file to ignore VIM
.*.swp files, but got an error. Following the advice
from Chapter 6: Advanced Topics:
If at any time you wish to have the default settings
restore to one or more of the configuration files, you
can simply delete the file, and run some innocuous svn
command, such as svn --version, and the missing file
will be recreated in its default state.
So, I deleted the Subversion/config file. However,
"svn --version" does not recreate it. Any suggestions
on how to get it back?
Thanks,
Alan
__________________________________
Do you Yahoo!?
New Yahoo! Photos - easier uploading and sharing.
http://photos.yahoo.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Lost my Subversion/config file & can't recreate it.
Posted by Alan Thompson <ca...@yahoo.com>.
Hi,
I was trying to update the config file to ignore VIM
.*.swp files, but got an error. Following the advice
from Chapter 6: Advanced Topics:
If at any time you wish to have the default settings
restore to one or more of the configuration files, you
can simply delete the file, and run some innocuous svn
command, such as svn --version, and the missing file
will be recreated in its default state.
So, I deleted the Subversion/config file. However,
"svn --version" does not recreate it. Any suggestions
on how to get it back?
Thanks,
Alan
__________________________________
Do you Yahoo!?
New Yahoo! Photos - easier uploading and sharing.
http://photos.yahoo.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org