You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Mark Kneisler <ma...@tceq.texas.gov> on 2013/12/11 02:45:01 UTC
Update-Only Checkout Enhancement
I have several environments where I'd like to use a SVN checkout, but where I'd never ever want to make changes to the files or perform a commit. For these environments, I'd only want to perform an update or an update to revision.
I'm hoping that this type of checkout would require far less space for the .svn directory.
Re: Update-Only Checkout Enhancement
Posted by Thorsten Schöning <ts...@am-soft.de>.
Guten Tag Mark Kneisler,
am Mittwoch, 11. Dezember 2013 um 02:45 schrieben Sie:
> I’m hoping that this type of checkout would require far less space for the .svn directory.
No it won't, during checkouts Subversion retrieves and locally saves
all the files it needs for the checked path and revision and stores
them regardless of future use. It currently simply can't know what you
plan to do with the working copy.
What's the size of your working copy that you are concerned about it?
Maybe you should use an export rather than a working copy? Depending
on your use case you can simply sync exports using rsync or such from
a machine where working copy size doesn't matter this much.
Mit freundlichen Grüßen,
Thorsten Schöning
--
Thorsten Schöning E-Mail:Thorsten.Schoening@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/
Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04
AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Re: Update-Only Checkout Enhancement
Posted by Thomas Harold <th...@nybeta.com>.
On 12/11/2013 2:19 PM, Bob Archer wrote:
>> On 11.12.2013 17:21, Mark Kneisler wrote:
>>>
>>> I think making the pristine files optional would work for me.
>>>
>>>
>>>
>>> Here’s an idea.
>>>
>>>
>>>
>>> Instead of having pristine copies of all files, how about adding
>>> to the pristine directory only when a file is changed?
>>>
>>
>> You know, that's a great idea! I wonder why we never thought of it
>> ourselves? :)
>
> Wouldn't that mean that you need to have some daemon service (or file
> watcher or something) running to determine if a file is modified?
> Also, it would mean you would need a constant connection to the
> server to use a subversion working copy.
>
Not necessarily. Take a look at how FSVS does its magic.
http://fsvs.tigris.org/
It functions in a similar manner to the svn command-line tool, but works
without requring a .svn folder. Which is why I prefer it for doing
version control of system configuration files on a Linux server.
Re: Update-Only Checkout Enhancement
Posted by Les Mikesell <le...@gmail.com>.
On Fri, Dec 13, 2013 at 7:21 AM, Branko Čibej <br...@wandisco.com> wrote:
> On 12.12.2013 17:18, Les Mikesell wrote:
>> On Thu, Dec 12, 2013 at 4:02 AM, Branko Čibej <br...@wandisco.com> wrote:
>>>> Some things... But not the things you really need to complete any
>>>> amount of actual work - like updates and commits.
>>> You're forgetting diff. If you use Subversion daily, you've become so
>>> used to it being local that you can't appreciate how slow it would be
>>> without locally cached pristine copies.
>> But (a) it is trivial
>
> Frankly, I'm a bit tired of people who have no idea what they're talking
> about telling us what's trivial and what isn't. If it's trivial, I'll be
> happy to take time to review the patch you produce in the next couple of
> days.
I meant it is trivial for a user to make his own snapshot copy of a
file/directory at any state if he thinks he is going to have some need
to diff against that particular state later without server-side
support. That has nothing to do with coding or patching anything.
It just requires an assumption of plenty of local disk space to stash
things (which seems to have already been made).
--
Les Mikesell
lesmikesell@gmail.com
Re: Update-Only Checkout Enhancement
Posted by Branko Čibej <br...@wandisco.com>.
On 12.12.2013 17:18, Les Mikesell wrote:
> On Thu, Dec 12, 2013 at 4:02 AM, Branko Čibej <br...@wandisco.com> wrote:
>>> Some things... But not the things you really need to complete any
>>> amount of actual work - like updates and commits.
>> You're forgetting diff. If you use Subversion daily, you've become so
>> used to it being local that you can't appreciate how slow it would be
>> without locally cached pristine copies.
> But (a) it is trivial
Frankly, I'm a bit tired of people who have no idea what they're talking
about telling us what's trivial and what isn't. If it's trivial, I'll be
happy to take time to review the patch you produce in the next couple of
days.
--
Branko Čibej | Director of Subversion
WANdisco // Non-Stop Data
e. brane@wandisco.com
Re: Update-Only Checkout Enhancement
Posted by Les Mikesell <le...@gmail.com>.
On Thu, Dec 12, 2013 at 4:02 AM, Branko Čibej <br...@wandisco.com> wrote:
>
>> Some things... But not the things you really need to complete any
>> amount of actual work - like updates and commits.
>
> You're forgetting diff. If you use Subversion daily, you've become so
> used to it being local that you can't appreciate how slow it would be
> without locally cached pristine copies.
But (a) it is trivial to make your own snapshot copy of file versions
if you expect to need them (as you must anyway for any offline state
except 'as checked out' and 'completely current'. And (b), I'm as
likely to want to diff against a different branch or someone else's
subsequent commit as my own pristine version (and no, that isn't
unreasonably slow...). Also, most of this discussion is related to
scenarios where active development is not happening - like the jenkins
build server example.
> Anyway, this thread has gone way off any kind of reasonable topic. The
> short answer to the original poster is that Subversion is not intended
> to be a replacement for rsync. You can take that as a hint as to what
> the solution to your problem might be.
But it _was_ intended to be a replacement for cvs, which has nowhere
near the client-side resource requirements.
--
Les Mikesell
lesmikesell@gmail.com
Re: Update-Only Checkout Enhancement
Posted by Branko Čibej <br...@wandisco.com>.
On 12.12.2013 08:30, Les Mikesell wrote:
> On Wed, Dec 11, 2013 at 11:01 PM, Ryan Schmidt
> <su...@ryandesign.com> wrote:
>> On Dec 11, 2013, at 19:19, Les Mikesell wrote:
>>
>>>>>> Also, it would mean you would need a constant connection to the server to
>>>>> use a subversion working copy.
>>> That's hardly a problem these days,
>> You apparently don’t try to work at the kinds of coffee shops I go to, where 50 college students are all watching youtube videos and making the network unbearable and Subversion’s ability to do some things offline very valuable to me.
> Some things... But not the things you really need to complete any
> amount of actual work - like updates and commits.
You're forgetting diff. If you use Subversion daily, you've become so
used to it being local that you can't appreciate how slow it would be
without locally cached pristine copies.
Anyway, this thread has gone way off any kind of reasonable topic. The
short answer to the original poster is that Subversion is not intended
to be a replacement for rsync. You can take that as a hint as to what
the solution to your problem might be.
-- Brane
--
Branko Čibej | Director of Subversion
WANdisco // Non-Stop Data
e. brane@wandisco.com
Re: Update-Only Checkout Enhancement
Posted by Les Mikesell <le...@gmail.com>.
On Wed, Dec 11, 2013 at 11:01 PM, Ryan Schmidt
<su...@ryandesign.com> wrote:
>
> On Dec 11, 2013, at 19:19, Les Mikesell wrote:
>
>>>>> Also, it would mean you would need a constant connection to the server to
>>>> use a subversion working copy.
>>
>> That's hardly a problem these days,
>
> You apparently don’t try to work at the kinds of coffee shops I go to, where 50 college students are all watching youtube videos and making the network unbearable and Subversion’s ability to do some things offline very valuable to me.
Some things... But not the things you really need to complete any
amount of actual work - like updates and commits.
--
Les Mikesell
lesmikesell@gmail.com
Re: Update-Only Checkout Enhancement
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Dec 11, 2013, at 19:19, Les Mikesell wrote:
>>>> Also, it would mean you would need a constant connection to the server to
>>> use a subversion working copy.
>
> That's hardly a problem these days,
You apparently don’t try to work at the kinds of coffee shops I go to, where 50 college students are all watching youtube videos and making the network unbearable and Subversion’s ability to do some things offline very valuable to me.
Re: Update-Only Checkout Enhancement
Posted by Les Mikesell <le...@gmail.com>.
On Wed, Dec 11, 2013 at 3:05 PM, Bob Archer <Bo...@amsi.com> wrote:
>>> > Wouldn't that mean that you need to have some daemon service (or file
>> watcher or something) running to determine if a file is modified?
>>
>> Yes.
Why would you need that in real-time instead of only when an svn
operation is done (possibly never...).
>> > Also, it would mean you would need a constant connection to the server to
>> use a subversion working copy.
That's hardly a problem these days,
>> Not necessarily; we don't need a pristine copy to check if a file is modified, or
>> if 's out of date WRT the repository. But the former problem (requiring a
>> daemon) is already a non-starter.
>
> Right, but if a file is modified you would need to contact the repository to get the pristine because you are going to get an event "after" the file is modified. There may be some transactional file systems that allow you to get an event before the modification is committed to the file system so you can access the original copy, but I think they are few and far between.
Assuming you have metadata to know the revision of the file, what
possible scenario can there be that you could not get a copy of that
revision back from the server if you happen to need it? Isn't that
why you put it there in the first place? Or if you were headed this
route, wouldn't it be better to send the new copy to the server and
let it do the server work? Unless your pending operation is a
revert, in which case you would want that copy from the server.
--
Les Mikesell
lesmikesell@gmail.com
Re: Update-Only Checkout Enhancement
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Bob Archer wrote on Wed, Dec 11, 2013 at 21:05:46 +0000:
> There may be some transactional file systems that allow you to get an
> event before the modification is committed to the file system so you
> can access the original copy, but I think they are few and far
> between.
And also would require new code on svn's side to be written.
There are, though, deduplicating filesystems, that store a file just
once if there are several copies of it. That means a file and its
pristine won't consume twice the platter space, assuming the file
doesn't use either svn:keywords or svn:eol-style=native.
(I'm thinking of zfs. And BTW, the pristine store also deduplicates, so
if two pristines have the same sha1, they're stored just once.)
RE: Update-Only Checkout Enhancement
Posted by Bob Archer <Bo...@amsi.com>.
> On 11.12.2013 20:19, Bob Archer wrote:
> >> On 11.12.2013 17:21, Mark Kneisler wrote:
> >>> I think making the pristine files optional would work for me.
> >>>
> >>>
> >>>
> >>> Here’s an idea.
> >>>
> >>>
> >>>
> >>> Instead of having pristine copies of all files, how about adding to
> >>> the pristine directory only when a file is changed?
> >>>
> >> You know, that's a great idea! I wonder why we never thought of it
> >> ourselves? :)
> > Wouldn't that mean that you need to have some daemon service (or file
> watcher or something) running to determine if a file is modified?
>
> Yes.
>
> > Also, it would mean you would need a constant connection to the server to
> use a subversion working copy.
>
> Not necessarily; we don't need a pristine copy to check if a file is modified, or
> if 's out of date WRT the repository. But the former problem (requiring a
> daemon) is already a non-starter.
Right, but if a file is modified you would need to contact the repository to get the pristine because you are going to get an event "after" the file is modified. There may be some transactional file systems that allow you to get an event before the modification is committed to the file system so you can access the original copy, but I think they are few and far between.
(So I guess your original response of "great idea" was sarcastic and I didn't see it.)
BOb
Re: Update-Only Checkout Enhancement
Posted by Branko Čibej <br...@wandisco.com>.
On 11.12.2013 20:19, Bob Archer wrote:
>> On 11.12.2013 17:21, Mark Kneisler wrote:
>>> I think making the pristine files optional would work for me.
>>>
>>>
>>>
>>> Here’s an idea.
>>>
>>>
>>>
>>> Instead of having pristine copies of all files, how about adding to
>>> the pristine directory only when a file is changed?
>>>
>> You know, that's a great idea! I wonder why we never thought of it
>> ourselves? :)
> Wouldn't that mean that you need to have some daemon service (or file watcher or something) running to determine if a file is modified?
Yes.
> Also, it would mean you would need a constant connection to the server to use a subversion working copy.
Not necessarily; we don't need a pristine copy to check if a file is
modified, or if 's out of date WRT the repository. But the former
problem (requiring a daemon) is already a non-starter.
-- Brane
--
Branko Čibej | Director of Subversion
WANdisco // Non-Stop Data
e. brane@wandisco.com
RE: Update-Only Checkout Enhancement
Posted by Bob Archer <Bo...@amsi.com>.
> On 11.12.2013 17:21, Mark Kneisler wrote:
> >
> > I think making the pristine files optional would work for me.
> >
> >
> >
> > Here’s an idea.
> >
> >
> >
> > Instead of having pristine copies of all files, how about adding to
> > the pristine directory only when a file is changed?
> >
>
> You know, that's a great idea! I wonder why we never thought of it
> ourselves? :)
Wouldn't that mean that you need to have some daemon service (or file watcher or something) running to determine if a file is modified? Also, it would mean you would need a constant connection to the server to use a subversion working copy.
BOb
Re: Update-Only Checkout Enhancement
Posted by Branko Čibej <br...@wandisco.com>.
On 11.12.2013 17:21, Mark Kneisler wrote:
>
> I think making the pristine files optional would work for me.
>
>
>
> Here’s an idea.
>
>
>
> Instead of having pristine copies of all files, how about adding to
> the pristine directory only when a file is changed?
>
You know, that's a great idea! I wonder why we never thought of it
ourselves? :)
--
Branko Čibej | Director of Subversion
WANdisco // Non-Stop Data
e. brane@wandisco.com
RE: Update-Only Checkout Enhancement
Posted by Mark Kneisler <ma...@tceq.texas.gov>.
I think making the pristine files optional would work for me.
Here's an idea.
Instead of having pristine copies of all files, how about adding to the pristine directory only when a file is changed?
From: Mark Phippard [mailto:markphip@gmail.com]
Sent: Wednesday, December 11, 2013 9:58 AM
To: Mark Kneisler
Cc: Cooke, Mark; users@subversion.apache.org
Subject: Re: Update-Only Checkout Enhancement
On Wed, Dec 11, 2013 at 10:46 AM, Mark Kneisler <ma...@tceq.texas.gov>> wrote:
Yes, I understand the export function. I want functionality for release management into test and production environments.
For these environments I have a few requirements:
Files in these environments will NEVER be edited
For new releases I will need to perform an update to revision, which will add, update and delete needed files
I want as small of a .svn directory as possible
I'm fairly certain that this functionality is currently not present, but I think there are many installations where this would be valuable.
I'd like to create an enhancement request, but according to the website it was recommended that I post to this mailing list first.
Pristine files do not exist solely so you can edit files. It is convenient they exist so that you can diff your changes or revert your changes without needing to contact the server, or even be online, but obviously those commands could be made to use the server for that and simply require a connection when no pristine is available.
The main reason the pristine files exist is so that the client and server can exchange deltas with each other and minimize network traffic. This is just as true for update as it is for commit. One of the original mantra's for the 1.0 release was "disk is cheap, network is expensive".
There is an existing request in the system for making these files optional:
http://subversion.tigris.org/issues/show_bug.cgi?id=525
There is also an existing request for storing the pristine files in compressed format:
http://subversion.tigris.org/issues/show_bug.cgi?id=908
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Update-Only Checkout Enhancement
Posted by Mark Phippard <ma...@gmail.com>.
On Wed, Dec 11, 2013 at 10:46 AM, Mark Kneisler <
mark.kneisler@tceq.texas.gov> wrote:
> Yes, I understand the export function. I want functionality for release
> management into test and production environments.
>
> For these environments I have a few requirements:
> Files in these environments will NEVER be edited
> For new releases I will need to perform an update to revision,
> which will add, update and delete needed files
> I want as small of a .svn directory as possible
>
> I'm fairly certain that this functionality is currently not present, but I
> think there are many installations where this would be valuable.
>
> I'd like to create an enhancement request, but according to the website it
> was recommended that I post to this mailing list first.
>
>
Pristine files do not exist solely so you can edit files. It is convenient
they exist so that you can diff your changes or revert your changes without
needing to contact the server, or even be online, but obviously those
commands could be made to use the server for that and simply require a
connection when no pristine is available.
The main reason the pristine files exist is so that the client and server
can exchange deltas with each other and minimize network traffic. This is
just as true for update as it is for commit. One of the original mantra's
for the 1.0 release was "disk is cheap, network is expensive".
There is an existing request in the system for making these files optional:
http://subversion.tigris.org/issues/show_bug.cgi?id=525
There is also an existing request for storing the pristine files in
compressed format:
http://subversion.tigris.org/issues/show_bug.cgi?id=908
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Update-Only Checkout Enhancement
Posted by Philip Martin <ph...@wandisco.com>.
Mark Kneisler <ma...@tceq.texas.gov> writes:
> Yes, I understand the export function. I want functionality for release management into test and production environments.
>
> For these environments I have a few requirements:
> Files in these environments will NEVER be edited
> For new releases I will need to perform an update to revision, which will add, update and delete needed files
> I want as small of a .svn directory as possible
>
> I'm fairly certain that this functionality is currently not present, but I think there are many installations where this would be valuable.
>
> I'd like to create an enhancement request, but according to the website it was recommended that I post to this mailing list first.
You probably want the "optional text base" feature:
http://subversion.tigris.org/issues/show_bug.cgi?id=525
--
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*
Re: Update-Only Checkout Enhancement
Posted by Warren Block <wb...@wonkity.com>.
On Wed, 11 Dec 2013, Les Mikesell wrote:
> On Wed, Dec 11, 2013 at 11:16 AM, Bob Archer <Bo...@amsi.com> wrote:
>>> Yes, I understand the export function. I want functionality for release
>>> management into test and production environments.
>>>
>>> For these environments I have a few requirements:
>>> Files in these environments will NEVER be edited
>>> For new releases I will need to perform an update to revision, which
>>> will add, update and delete needed files
>>> I want as small of a .svn directory as possible
>>
>> Why? Disk is cheap. Much cheaper than the time it would take to modify svn to work the way you are requesting. Heck, I just bought a 1TB SSD for ~$500. The spinning version was about $120.
>
> Sometimes it is a good idea to distribute work out to clients and
> sometimes you really want the client to just be a lightweight client
> and make the server do the work of serving. I've always liked the
> minimal amount of data that CVS needs on the clients and sometimes
> wished that svn could match that.
When FreeBSD switched from CVS to Subversion, there were a couple of
projects attempting to replace our cvsup/csup commands for simple,
history-less, source checkouts.
One of these projects was svnup, http://jcm.dsl.visi.com/freebsd/svnup/
Not sure of the present status.
Re: Update-Only Checkout Enhancement
Posted by Ben Reser <be...@reser.org>.
On 12/11/13 7:16 PM, Les Mikesell wrote:
> I guess I don't understand why it couldn't be as simple as having the
> library get a pristine copy on demand if some operation needs it.
The original wc code had almost no abstraction around the pristines (as far as
I can recall, it's been a long time since I looked at it). WC-NG has more
abstraction around them than I recall (I just went and looked). There is a
function that retrieves a stream for a pristine, but it doesn't seem to be used
everywhere. There are an awful lot of places where it seems we're just passing
around the filename (many of which appear to me to be passing through several
different layers of the libraries). Someone needs to untangle this mess and
get every use of the pristine using a definitive abstraction that can then be
made to handle missing pristines. That's not a trivial task.
> Isn't there already a recovery procedure for a missing pristine file?
No there is not. You delete a pristine the working copy is busted.
Re: Update-Only Checkout Enhancement
Posted by Les Mikesell <le...@gmail.com>.
On Wed, Dec 11, 2013 at 8:26 PM, Ben Reser <be...@reser.org> wrote:
>
> Absolutely, the answer here isn't a one size fits all. Nobody is objecting to
> the idea of allowing this. The problem is that the code is not designed to
> allow this and it's a ton of work to change that. I can think of a good 10
> other things that would require the same level of effort that would improve
> things for a lot more people
>
> Once you no longer need a pristine there are a lot of potential scenarios that
> require different behaviors. So it's not even a simple matter of just changing
> the working copy library to support pristines not existing. It becomes
> thinking about how to deal with these scenarios (not that all of them need to
> be implemented immediately, but you probably want to not pigeon hole yourself
> into an implementation that doesn't support them).
I guess I don't understand why it couldn't be as simple as having the
library get a pristine copy on demand if some operation needs it.
Isn't there already a recovery procedure for a missing pristine file?
And then make saving it optional.
As a case in point, consider the accumulation of cruft on a typical
jenkins build server where a large set of projects are built and
rarely removed - you have to allow much more disk space to each build
slave to accommodate the pristine files that don't have a whole lot of
use.
--
Les Mikesell
lesmikesell@gmail.com
RE: Update-Only Checkout Enhancement
Posted by "Gleason, Todd" <To...@elekta.com>.
> From: Ben Reser [mailto:ben@reser.org]
> Absolutely, the answer here isn't a one size fits all. Nobody is
> objecting to
> the idea of allowing this. The problem is that the code is not
> designed to
> allow this and it's a ton of work to change that. I can think of a
> good 10
> other things that would require the same level of effort that would
> improve
> things for a lot more people
Should you get into doing this, I hope it will be highly configurable both from the server side and client side.
I can definitely see running a checkout operation where you'd like to specify that for anything under a certain URL within your checkout, you don't want pristine (just what I presume would be sufficient metadata like dates+hashes to detect any changes that may occur). Or after fetching, you might want to "prune" out the pristines for a file or directory tree. You may or may not change the files afterward, and as folks have noted, you'd have to fetch down from the server in any commit operation, and you'd have to do a double-fetch of sorts (working base plus intended revision) in order to update in cases where such a file had been locally altered.
Note that this pruning sort of behavior may require maintaining client-side metadata, perhaps similar to how changelists are client-side.
You would also want to be able to configure on the server something like svn:no-pristines to say that by default there would be no pristine in any working copy that downloaded a given file/path, and then you'd need to do some opposite logic from any client that did want the pristines. This could be good when most people don't work in a given area, but you want to accommodate the few people who do by letting them demand the usual structure.
And yes, it's icing on the cake, though I can certainly imagine using it. I have enough working copies that disk space is very much an issue for me.
> Once you no longer need a pristine there are a lot of potential
> scenarios that
> require different behaviors. So it's not even a simple matter of just
> changing
> the working copy library to support pristines not existing. It becomes
> thinking about how to deal with these scenarios (not that all of them
> need to
> be implemented immediately, but you probably want to not pigeon hole
> yourself
> into an implementation that doesn't support them).
>
> Some files don't delta very well and essentially change every bit when
> the file
> changes, so those files update and commit are better off just
> transmitting full
> text. Some files do delta fine but are just large, if you change the
> file you
> may want to run a diff and update/commit would benefit from deltas.
>
> If you have a file that deltas well should you download the full text
> from the
> server and produce a delta locally or just send the full text to the
> server?
> The answer to this depends on the network. If you're on a asymmetric
> connection where upload or download or different whichever method that
> sends
> the largest data over the fastest direction is better (e.g. a typical
> DSL or
> Cable internet connection where download is faster it may be far better
> to
> download full text and upload the delta you calculate). If you're on a
> symmetric network you just do whichever takes the least work (directly
> sending/retrieving full text). Due to laptops that came move between
> networks
> you also can't just configure the file on the server side (i.e. set a
> property).
You do what you have to do based on what's in the WC (which is determined by server and client configurations). If you're committing without a pristine, I think you must either send the whole new file, or else re-receive the working base file from the server. That is, unless you do something like running checksums/hashes on fragments of the file to pinpoint changes, which you might do if the file is really large, or (my preference) if a particular svn: attribute was set on the file/folder.
Of course for doing updates, if you can determine based on the file's metadata (checksum/hash info) that the file hasn't changed, then the usual diff should still function.
> What about compressing the file. If it compresses well then you could
> have
> your cake and eat it too... You save disk space and you still have the
> pristine.
I don't think this gets you very far. Probably worth doing, but I wouldn't see it as the ultimate answer.
> The user may want us to retain the pristine data we retrieved to
> execute a
> command or they may just want us to destroy it immediately to limit
> disk space.
Exactly, and that's where the client-side configuration I described above would come into play.
> For that matter the pristine store as of 1.7 doesn't even remove files.
> So if
> you're updating a working copy it will just continue to grow. As of
> 1.8 we use
> this to allow faster updates when you already have a copy of a version
> of a
> file in the pristine store. So we need some sort of "garbage
> collector" for
> the pristine store even for the setup that has it:
> http://subversion.tigris.org/issues/show_bug.cgi?id=4071
Yes, very much so.
> So all in all it's not a matter of throwing a simple option in and
> being done
> with it. I realize that your particular use case is probably one of
> the
> simpler ones to deal with. But it's by far the least common use case
> that we
> hear complaints about with respect to the pristine store.
>
> Please realize that considerable thought has gone into this. It's just
> not
> done yet.
As always you have to prioritize. Note though that http://subversion.tigris.org/issues/show_bug.cgi?id=525 is 12 years old. With WC-NG available now, perhaps it's time to consider this.
Todd C. Gleason | Senior Software Engineer
Elekta
100 Mathilda Place, Fifth Floor
Sunnyvale, CA 94086, United States
Office: +14086782640 |
Todd.Gleason@elekta.com | www.elekta.com
Please consider the environment before printing this e-mail.
The contents of this e-mail message (including any attachments) are confidential to and are intended to be conveyed for the use of the recipient to whom it is addressed only. If you receive this transmission in error, please notify the sender of this immediately and delete the message from your system. Any distribution, reproduction or use of this message by someone other than recipient is not authorized and may be unlawful.
Re: Update-Only Checkout Enhancement
Posted by Ben Reser <be...@reser.org>.
On 12/11/13 5:10 PM, Les Mikesell wrote:
> Not exactly. Network traffic is generally bursty. Clients rarely
> spend 100% of their time checking out files, so a very large number
> could share a local network even if they always deleted their
> workspaces and checked out fresh copies. But when storing the
> pristine copies, they can't share anything - even if you map a shared
> network volume you don't want to share the workspaces.
Key phrase there local network. When Subversion was written the vast majority
of the users were using it over a WAN. This lead to the assumption that disk
space was cheaper than improving the network. That's not necessarily
reflective of the use today, but it shaped the assumption that we would always
have pristines in the code that was written.
> Well, that's why programs have options - all situations are not the same...
Absolutely, the answer here isn't a one size fits all. Nobody is objecting to
the idea of allowing this. The problem is that the code is not designed to
allow this and it's a ton of work to change that. I can think of a good 10
other things that would require the same level of effort that would improve
things for a lot more people
Once you no longer need a pristine there are a lot of potential scenarios that
require different behaviors. So it's not even a simple matter of just changing
the working copy library to support pristines not existing. It becomes
thinking about how to deal with these scenarios (not that all of them need to
be implemented immediately, but you probably want to not pigeon hole yourself
into an implementation that doesn't support them).
Some files don't delta very well and essentially change every bit when the file
changes, so those files update and commit are better off just transmitting full
text. Some files do delta fine but are just large, if you change the file you
may want to run a diff and update/commit would benefit from deltas.
If you have a file that deltas well should you download the full text from the
server and produce a delta locally or just send the full text to the server?
The answer to this depends on the network. If you're on a asymmetric
connection where upload or download or different whichever method that sends
the largest data over the fastest direction is better (e.g. a typical DSL or
Cable internet connection where download is faster it may be far better to
download full text and upload the delta you calculate). If you're on a
symmetric network you just do whichever takes the least work (directly
sending/retrieving full text). Due to laptops that came move between networks
you also can't just configure the file on the server side (i.e. set a property).
What about compressing the file. If it compresses well then you could have
your cake and eat it too... You save disk space and you still have the pristine.
The user may want us to retain the pristine data we retrieved to execute a
command or they may just want us to destroy it immediately to limit disk space.
For that matter the pristine store as of 1.7 doesn't even remove files. So if
you're updating a working copy it will just continue to grow. As of 1.8 we use
this to allow faster updates when you already have a copy of a version of a
file in the pristine store. So we need some sort of "garbage collector" for
the pristine store even for the setup that has it:
http://subversion.tigris.org/issues/show_bug.cgi?id=4071
So all in all it's not a matter of throwing a simple option in and being done
with it. I realize that your particular use case is probably one of the
simpler ones to deal with. But it's by far the least common use case that we
hear complaints about with respect to the pristine store.
Please realize that considerable thought has gone into this. It's just not
done yet.
Re: Update-Only Checkout Enhancement
Posted by Les Mikesell <le...@gmail.com>.
On Wed, Dec 11, 2013 at 12:24 PM, Ben Reser <be...@reser.org> wrote:
> On 12/11/13 9:47 AM, Les Mikesell wrote:
>> Within reasonable limits it doesn't cost anything more to send more
>> network traffic. But the cost of client disks scales up by the
>> number of clients. Sometimes you can get by mounting a network disk
>> into all the clients, but then performance suffers, especially with
>> windows clients.
>
> Network traffic has scaling costs just like storage space.
Not exactly. Network traffic is generally bursty. Clients rarely
spend 100% of their time checking out files, so a very large number
could share a local network even if they always deleted their
workspaces and checked out fresh copies. But when storing the
pristine copies, they can't share anything - even if you map a shared
network volume you don't want to share the workspaces.
> If we'd made the
> decision to not store pristines and you had to go to the server for pristine
> copies then the discussion here would be reversed. Someone would be asking why
> we don't just store pristines and pointing out how disk space is cheap compared
> to the cost of converting their entire network to have more capacity.
Sure - if you aren't local or the server is overloaded it is nice to
have the pristine copies.
> Can't
> make everyone happy all the time.
Well, that's why programs have options - all situations are not the same...
--
Les Mikesell
lesmikesell@gmail.com
Re: Update-Only Checkout Enhancement
Posted by Ben Reser <be...@reser.org>.
On 12/11/13 9:47 AM, Les Mikesell wrote:
> Within reasonable limits it doesn't cost anything more to send more
> network traffic. But the cost of client disks scales up by the
> number of clients. Sometimes you can get by mounting a network disk
> into all the clients, but then performance suffers, especially with
> windows clients.
Network traffic has scaling costs just like storage space. If we'd made the
decision to not store pristines and you had to go to the server for pristine
copies then the discussion here would be reversed. Someone would be asking why
we don't just store pristines and pointing out how disk space is cheap compared
to the cost of converting their entire network to have more capacity. Can't
make everyone happy all the time.
Re: Update-Only Checkout Enhancement
Posted by Les Mikesell <le...@gmail.com>.
On Wed, Dec 11, 2013 at 11:16 AM, Bob Archer <Bo...@amsi.com> wrote:
>> Yes, I understand the export function. I want functionality for release
>> management into test and production environments.
>>
>> For these environments I have a few requirements:
>> Files in these environments will NEVER be edited
>> For new releases I will need to perform an update to revision, which
>> will add, update and delete needed files
>> I want as small of a .svn directory as possible
>
> Why? Disk is cheap. Much cheaper than the time it would take to modify svn to work the way you are requesting. Heck, I just bought a 1TB SSD for ~$500. The spinning version was about $120.
Sometimes it is a good idea to distribute work out to clients and
sometimes you really want the client to just be a lightweight client
and make the server do the work of serving. I've always liked the
minimal amount of data that CVS needs on the clients and sometimes
wished that svn could match that.
> I would recommend you write a script that does an export rather than using the update feature. Although, this would probably mean the task would take longer, since you don't have the benefit of the .svn pristines to know what changed files need to be requested from the server. But, it sounds like you care more about disk space than network traffic and/or time. Of course, that is your decision to make.
Within reasonable limits it doesn't cost anything more to send more
network traffic. But the cost of client disks scales up by the
number of clients. Sometimes you can get by mounting a network disk
into all the clients, but then performance suffers, especially with
windows clients.
--
Les Mikesell
lesmikesell@gmail.com
RE: Update-Only Checkout Enhancement
Posted by Bob Archer <Bo...@amsi.com>.
> Yes, I understand the export function. I want functionality for release
> management into test and production environments.
>
> For these environments I have a few requirements:
> Files in these environments will NEVER be edited
> For new releases I will need to perform an update to revision, which
> will add, update and delete needed files
> I want as small of a .svn directory as possible
Why? Disk is cheap. Much cheaper than the time it would take to modify svn to work the way you are requesting. Heck, I just bought a 1TB SSD for ~$500. The spinning version was about $120.
I would recommend you write a script that does an export rather than using the update feature. Although, this would probably mean the task would take longer, since you don't have the benefit of the .svn pristines to know what changed files need to be requested from the server. But, it sounds like you care more about disk space than network traffic and/or time. Of course, that is your decision to make.
>
> I'm fairly certain that this functionality is currently not present, but I think
> there are many installations where this would be valuable.
>
> I'd like to create an enhancement request, but according to the website it
> was recommended that I post to this mailing list first.
>
> -----Original Message-----
> From: Cooke, Mark [mailto:mark.cooke@siemens.com]
> Sent: Wednesday, December 11, 2013 1:44 AM
> To: Mark Kneisler; users@subversion.apache.org
> Subject: RE: Update-Only Checkout Enhancement
>
> > -----Original Message-----
> > From: Mark Kneisler [mailto:mark.kneisler@tceq.texas.gov]
> > Sent: 11 December 2013 01:45
> >
> > I have several environments where I'd like to use a SVN checkout, but
> > where I'd never ever want to make changes to the files or perform a
> > commit. For these environments, I'd only want to perform an update or
> > an update to revision.
> >
> > I'm hoping that this type of checkout would require far less space for
> > the .svn directory.
> >
> AFAIK there is no such option for a "checkout", however, you can get the
> source without the .svn folder using the "export" command:-
>
> http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.export.html
>
> ...you can specify the revision etc. Note, however, that it will not remove
> unwanted files like an update can.
>
> ~ mark c
>
> P.s. I think Thorsten just beat me to it...
Re: Update-Only Checkout Enhancement
Posted by Ben Reser <be...@reser.org>.
On 12/11/13 7:46 AM, Mark Kneisler wrote:
> Yes, I understand the export function. I want functionality for release management into test and production environments.
>
> For these environments I have a few requirements:
> Files in these environments will NEVER be edited
> For new releases I will need to perform an update to revision, which will add, update and delete needed files
> I want as small of a .svn directory as possible
>
> I'm fairly certain that this functionality is currently not present, but I think there are many installations where this would be valuable.
>
> I'd like to create an enhancement request, but according to the website it was recommended that I post to this mailing list first.
Just to set some expectations here. The working copy code is highly
complicated and assumes that pristine files are available throughout it.
Making this change is not simple. Which is the reason it hasn't already been done.
RE: Update-Only Checkout Enhancement
Posted by Mark Kneisler <ma...@tceq.texas.gov>.
Yes, I understand the export function. I want functionality for release management into test and production environments.
For these environments I have a few requirements:
Files in these environments will NEVER be edited
For new releases I will need to perform an update to revision, which will add, update and delete needed files
I want as small of a .svn directory as possible
I'm fairly certain that this functionality is currently not present, but I think there are many installations where this would be valuable.
I'd like to create an enhancement request, but according to the website it was recommended that I post to this mailing list first.
-----Original Message-----
From: Cooke, Mark [mailto:mark.cooke@siemens.com]
Sent: Wednesday, December 11, 2013 1:44 AM
To: Mark Kneisler; users@subversion.apache.org
Subject: RE: Update-Only Checkout Enhancement
> -----Original Message-----
> From: Mark Kneisler [mailto:mark.kneisler@tceq.texas.gov]
> Sent: 11 December 2013 01:45
>
> I have several environments where I'd like to use a SVN checkout, but
> where I'd never ever want to make changes to the files or perform a
> commit. For these environments, I'd only want to perform an update or
> an update to revision.
>
> I'm hoping that this type of checkout would require far less space for
> the .svn directory.
>
AFAIK there is no such option for a "checkout", however, you can get the source without the .svn folder using the "export" command:-
http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.export.html
...you can specify the revision etc. Note, however, that it will not remove unwanted files like an update can.
~ mark c
P.s. I think Thorsten just beat me to it...
RE: Update-Only Checkout Enhancement
Posted by "Cooke, Mark" <ma...@siemens.com>.
> -----Original Message-----
> From: Mark Kneisler [mailto:mark.kneisler@tceq.texas.gov]
> Sent: 11 December 2013 01:45
>
> I have several environments where I'd like to use a SVN
> checkout, but where I'd never ever want to make changes to
> the files or perform a commit. For these environments, I'd
> only want to perform an update or an update to revision.
>
> I'm hoping that this type of checkout would require far less
> space for the .svn directory.
>
AFAIK there is no such option for a "checkout", however, you can get the source without the .svn folder using the "export" command:-
http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.export.html
...you can specify the revision etc. Note, however, that it will not remove unwanted files like an update can.
~ mark c
P.s. I think Thorsten just beat me to it...
Re: Update-Only Checkout Enhancement
Posted by Thomas Harold <th...@nybeta.com>.
On 12/10/2013 8:45 PM, Mark Kneisler wrote:
> I have several environments where I’d like to use a SVN checkout, but
> where I’d never ever want to make changes to the files or perform a
> commit. For these environments, I’d only want to perform an update or
> an update to revision.
>
In cases where you do not want a .svn directory and you are using Linux,
take a look at FSVS:
http://fsvs.tigris.org/
This is a command line tool that works very similar to the "svn"
command-line tool and talks to an SVN repository. We make heavy use of
it to version-control our Linux servers (especially the files under
/usr/local, /boot and /etc).
The big difference over using FSVS vs SVN on the Linux box is that FSVS
does not create a .svn folder in the root.
I don't know off-hand if FSVS can be used in Cygwin under Windows.