You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Peter <bf...@telenet.be> on 2007/03/15 11:22:25 UTC
Enhancement suggestions
Hi,
A couple of years ago, I developed a source control system - called
Evolver - for a company that hired me. The company made video-games, and
wanted a source control system that was powerful enough for developers,
yet very easy to use for the other kinds of users (mainly "artists").
Evolver worked exclusive on Windows/NTFS, but I guess most NTFS features
are also found in Unix-like systems.
Although subversion goes way beyond the system I developed, I do miss a
couple of things. I must admit I'm new to subversion, so some of these
might already be implemented.
- Automatic detection of renames and moves. Evolver used NTFS
object-identifiers and the change-journal to track this.
- No "cleanup" needed. Because the above, "cleanup" did not exist. All
possible changes allowed by the file system got tracked automatically.
The only "cleanup" needed was when a new file with the same name as a
deleted file was created. Although most of the time this issue was
handled internally by NTFS file tunneling, it could happen when software
moved a file with the same name across different volumes.
- Properly handling of "file with same name exists in working copy".
- Concept of "generated" files. When a conflict occured on these files,
they got deleted by default, because they can be re-generated. Of course
a build-system like SCONS that uses caching is much more suitable to
handle this, but this is not always usable with content creation tools
that often do not support batch / command-line support.
- Semi atomic updates. Using a method of file locking and renaming, and
update of a working folder either completely succeeded or failed.
Rollback after a failure was possible because a special naming scheme.
- No ".svn" folders. Evolver stored all information in extra NTFS
streams, removing clutter.
- The concept of a "unit". Currently, it is possible to commit just one
file, but doing this is a bit like hacking. Usually one works on a
"project" which is an atomic unit. Committing only a subset of changes
is dangerous because they cannot be tested, and this quickly converges
into a "but it works on my machine" situation. In Subversion a "unit"
would be just some parent directory that is always used when committing,
unless explicitly overriden.
- Dependencies to other units. A unit usually makes use of other units.
Changes in the dependencies must usually be committed at the same time.
- NTFS junctions and hardlinks. I've seen this in discussions but have
no idea what the current status is. We used junctions to dependencies,
so that all required files were located under a single project
directory. The problem is that Windows itself does not support
junctions/hardlinks (e.g. Windows Explorer does not understand them).
- Concept of automatic locking of files. Although subversion supports
locking now, it must still be performed manually? Evolver performed an
automatic lock as soon as a file was overwritten. After all, binary
files cannot be merged, and parallel evolution of such a files should be
prohibited as soon as possible. Binary files are evil, but they are very
common in content development systems (Adobe Flash, Autodesk 3D Studio
MAX, Adobe Photoshop, etc).
- Multiple branches in the same working folder. One directory could
contain assets for multiple branches at the same time. One just selects
the current branch, and commits/updates automatically pick a parent
branch for finding the "common/base" revision. I have a feeling
subversion supports this, but I haven't figured out yet how.
- Inter-branch locking. A binary file that lives in multiple branches
could be locked in all branches at the same time.
- By default, reverting a directory deletes local files in it. After
all, reverting restores the previous revision, and since the file did
not exist in the previous revision, its previous state is void.
- An experimental feature was "refactor friendly structured storage". So
instead of storing dumb source code, a DOM was stored instead, using
unique-object-identifiers instead of names to refer to symbols. So
basically, source control did not stop on the file level, but at the
object level. Merging was done on this tree, which eased extreme
programming and refactoring. For example, renaming a class was just one
change to the DOM in which the class was declared, although it might
change dozens of source text files that used it. This was never used
because it would also require different source code editors that work on
the symbolic level.
I hope this information was useful, although I'm not expecting to see
any of this in subversion soon, as you guys probably have really good
reasons not to have implemented these "features".
Cheers,
Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Enhancement suggestions
Posted by Mark Phippard <ma...@gmail.com>.
On 3/15/07, Peter <bf...@telenet.be> wrote:
>
> >> - Proper handling of "file with same name exists in working copy".
> > AKA obstructions- a major pain in some cases. Ever try to fix an
> > obstructed directory that was scheduled for deletion but not yet
> > committed?
>
> Indeed, horrible. In the beginning I was almost tempted to develop a new
> "Evolver2" (as I don't own the rights of Evolver) because I was cleaning
> up and manually deleting files all the time to get SVN updates working,
> grr... Luckily for me, I had no spare time to do this - it would be at
> least 9 man-months of full-time development, and it's not my focus.
Subversion trunk can tolerate obstructions if you want to build from
source. Otherwise, at least it will be in next release.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Enhancement suggestions
Posted by Peter <bf...@telenet.be>.
Miller, Eric wrote:
> Wow. All of my issues with svn summed up in a single post!
>
> Just a few of my favorites:
>
>> - Automatic detection of renames and moves.
> And copies- This one is a pain and is mostly related to...
You mean that a copy is actually a branched revision of the original?
Yeah that would be cool. But that would be very difficult to do using
the NTFS change journal I guess.
>> - No ".svn" folders.
> When these get deleted, copied, moved, or looked at funny bad things can
> happen. It'd be great to get them out of the working copy at the very
> least. I know there has been considerable discussion regarding this in
> the past but I don't know if there is any implementation plan.
Evolver stored all metadata using NTFS Alternate Data Streams. It stored
it in the root directory of the "unit/project", but if you want to get
rid of the unit concept, it could just as well be stored as an ADS in
the root drive (e.g. C:/)
>> - Proper handling of "file with same name exists in working copy".
> AKA obstructions- a major pain in some cases. Ever try to fix an
> obstructed directory that was scheduled for deletion but not yet
> committed?
Indeed, horrible. In the beginning I was almost tempted to develop a new
"Evolver2" (as I don't own the rights of Evolver) because I was cleaning
up and manually deleting files all the time to get SVN updates working,
grr... Luckily for me, I had no spare time to do this - it would be at
least 9 man-months of full-time development, and it's not my focus.
>> - The concept of a "unit".
> While I think your definition of a 'unit' is sufficient for most, I'd
> like to see a configurable option that can define multiple 'unit types'
> that can auto-select your path list for versioning operations.
> OTOH it's probably just easier to implement this in a wrapper on a
> per-case basis than try to come up with a general solution.
You are right. This is just some kind of "company commit policy".
>> - Inter-branch locking.
> Hallelujah!
>
> Also brought up before - working copy lock tracking.
> Tracking down locks based on uid alone is not sufficient - hostname and
> working copy path all should be logged when a lock is acquired.
Well, in Evolver that was handled by introducing an extra "proxy" object
on the server; no matter how many working copies of an "asset" (=file or
directory) existed, they all tracked back to this proxy object, and all
locking was done on it.
>
> Great stuff!
Thank you. It seems the quest for the ultimate version control system is
not over yet ;)
> Eric
>
>> -----Original Message-----
>> From: Peter [mailto:bf3@telenet.be]
>> Sent: Thursday, March 15, 2007 5:22 AM
>> To: dev@subversion.tigris.org
>> Subject: Enhancement suggestions
>>
>> Hi,
>>
>> A couple of years ago, I developed a source control system - called
>> Evolver - for a company that hired me. The company made video-games,
> and
>> wanted a source control system that was powerful enough for
> developers,
>> yet very easy to use for the other kinds of users (mainly "artists").
>> Evolver worked exclusive on Windows/NTFS, but I guess most NTFS
> features
>> are also found in Unix-like systems.
>>
>> Although subversion goes way beyond the system I developed, I do miss
> a
>> couple of things. I must admit I'm new to subversion, so some of these
>> might already be implemented.
>>
>> - Automatic detection of renames and moves. Evolver used NTFS
>> object-identifiers and the change-journal to track this.
>>
>> - No "cleanup" needed. Because the above, "cleanup" did not exist. All
>> possible changes allowed by the file system got tracked automatically.
>> The only "cleanup" needed was when a new file with the same name as a
>> deleted file was created. Although most of the time this issue was
>> handled internally by NTFS file tunneling, it could happen when
> software
>> moved a file with the same name across different volumes.
>>
>> - Properly handling of "file with same name exists in working copy".
>>
>> - Concept of "generated" files. When a conflict occured on these
> files,
>> they got deleted by default, because they can be re-generated. Of
> course
>> a build-system like SCONS that uses caching is much more suitable to
>> handle this, but this is not always usable with content creation tools
>> that often do not support batch / command-line support.
>>
>> - Semi atomic updates. Using a method of file locking and renaming,
> and
>> update of a working folder either completely succeeded or failed.
>> Rollback after a failure was possible because a special naming scheme.
>>
>> - No ".svn" folders. Evolver stored all information in extra NTFS
>> streams, removing clutter.
>>
>> - The concept of a "unit". Currently, it is possible to commit just
> one
>> file, but doing this is a bit like hacking. Usually one works on a
>> "project" which is an atomic unit. Committing only a subset of changes
>> is dangerous because they cannot be tested, and this quickly converges
>> into a "but it works on my machine" situation. In Subversion a "unit"
>> would be just some parent directory that is always used when
> committing,
>> unless explicitly overriden.
>>
>> - Dependencies to other units. A unit usually makes use of other
> units.
>> Changes in the dependencies must usually be committed at the same
> time.
>> - NTFS junctions and hardlinks. I've seen this in discussions but have
>> no idea what the current status is. We used junctions to dependencies,
>> so that all required files were located under a single project
>> directory. The problem is that Windows itself does not support
>> junctions/hardlinks (e.g. Windows Explorer does not understand them).
>>
>> - Concept of automatic locking of files. Although subversion supports
>> locking now, it must still be performed manually? Evolver performed an
>> automatic lock as soon as a file was overwritten. After all, binary
>> files cannot be merged, and parallel evolution of such a files should
> be
>> prohibited as soon as possible. Binary files are evil, but they are
> very
>> common in content development systems (Adobe Flash, Autodesk 3D
> Studio
>> MAX, Adobe Photoshop, etc).
>>
>> - Multiple branches in the same working folder. One directory could
>> contain assets for multiple branches at the same time. One just
> selects
>> the current branch, and commits/updates automatically pick a parent
>> branch for finding the "common/base" revision. I have a feeling
>> subversion supports this, but I haven't figured out yet how.
>>
>> - Inter-branch locking. A binary file that lives in multiple branches
>> could be locked in all branches at the same time.
>>
>> - By default, reverting a directory deletes local files in it. After
>> all, reverting restores the previous revision, and since the file did
>> not exist in the previous revision, its previous state is void.
>>
>> - An experimental feature was "refactor friendly structured storage".
> So
>> instead of storing dumb source code, a DOM was stored instead, using
>> unique-object-identifiers instead of names to refer to symbols. So
>> basically, source control did not stop on the file level, but at the
>> object level. Merging was done on this tree, which eased extreme
>> programming and refactoring. For example, renaming a class was just
> one
>> change to the DOM in which the class was declared, although it might
>> change dozens of source text files that used it. This was never used
>> because it would also require different source code editors that work
> on
>> the symbolic level.
>>
>> I hope this information was useful, although I'm not expecting to see
>> any of this in subversion soon, as you guys probably have really good
>> reasons not to have implemented these "features".
>>
>> Cheers,
>> Peter
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
>> For additional commands, e-mail: dev-help@subversion.tigris.org
>>
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: Enhancement suggestions
Posted by "Miller, Eric" <Er...@amd.com>.
Wow. All of my issues with svn summed up in a single post!
Just a few of my favorites:
> - Automatic detection of renames and moves.
And copies- This one is a pain and is mostly related to...
> - No ".svn" folders.
When these get deleted, copied, moved, or looked at funny bad things can
happen. It'd be great to get them out of the working copy at the very
least. I know there has been considerable discussion regarding this in
the past but I don't know if there is any implementation plan.
> - Properly handling of "file with same name exists in working copy".
AKA obstructions- a major pain in some cases. Ever try to fix an
obstructed directory that was scheduled for deletion but not yet
committed?
> - The concept of a "unit".
While I think your definition of a 'unit' is sufficient for most, I'd
like to see a configurable option that can define multiple 'unit types'
that can auto-select your path list for versioning operations.
OTOH it's probably just easier to implement this in a wrapper on a
per-case basis than try to come up with a general solution.
> - Inter-branch locking.
Hallelujah!
Also brought up before - working copy lock tracking.
Tracking down locks based on uid alone is not sufficient - hostname and
working copy path all should be logged when a lock is acquired.
Great stuff!
Eric
> -----Original Message-----
> From: Peter [mailto:bf3@telenet.be]
> Sent: Thursday, March 15, 2007 5:22 AM
> To: dev@subversion.tigris.org
> Subject: Enhancement suggestions
>
> Hi,
>
> A couple of years ago, I developed a source control system - called
> Evolver - for a company that hired me. The company made video-games,
and
> wanted a source control system that was powerful enough for
developers,
> yet very easy to use for the other kinds of users (mainly "artists").
> Evolver worked exclusive on Windows/NTFS, but I guess most NTFS
features
> are also found in Unix-like systems.
>
> Although subversion goes way beyond the system I developed, I do miss
a
> couple of things. I must admit I'm new to subversion, so some of these
> might already be implemented.
>
> - Automatic detection of renames and moves. Evolver used NTFS
> object-identifiers and the change-journal to track this.
>
> - No "cleanup" needed. Because the above, "cleanup" did not exist. All
> possible changes allowed by the file system got tracked automatically.
> The only "cleanup" needed was when a new file with the same name as a
> deleted file was created. Although most of the time this issue was
> handled internally by NTFS file tunneling, it could happen when
software
> moved a file with the same name across different volumes.
>
> - Properly handling of "file with same name exists in working copy".
>
> - Concept of "generated" files. When a conflict occured on these
files,
> they got deleted by default, because they can be re-generated. Of
course
> a build-system like SCONS that uses caching is much more suitable to
> handle this, but this is not always usable with content creation tools
> that often do not support batch / command-line support.
>
> - Semi atomic updates. Using a method of file locking and renaming,
and
> update of a working folder either completely succeeded or failed.
> Rollback after a failure was possible because a special naming scheme.
>
> - No ".svn" folders. Evolver stored all information in extra NTFS
> streams, removing clutter.
>
> - The concept of a "unit". Currently, it is possible to commit just
one
> file, but doing this is a bit like hacking. Usually one works on a
> "project" which is an atomic unit. Committing only a subset of changes
> is dangerous because they cannot be tested, and this quickly converges
> into a "but it works on my machine" situation. In Subversion a "unit"
> would be just some parent directory that is always used when
committing,
> unless explicitly overriden.
>
> - Dependencies to other units. A unit usually makes use of other
units.
> Changes in the dependencies must usually be committed at the same
time.
>
> - NTFS junctions and hardlinks. I've seen this in discussions but have
> no idea what the current status is. We used junctions to dependencies,
> so that all required files were located under a single project
> directory. The problem is that Windows itself does not support
> junctions/hardlinks (e.g. Windows Explorer does not understand them).
>
> - Concept of automatic locking of files. Although subversion supports
> locking now, it must still be performed manually? Evolver performed an
> automatic lock as soon as a file was overwritten. After all, binary
> files cannot be merged, and parallel evolution of such a files should
be
> prohibited as soon as possible. Binary files are evil, but they are
very
> common in content development systems (Adobe Flash, Autodesk 3D
Studio
> MAX, Adobe Photoshop, etc).
>
> - Multiple branches in the same working folder. One directory could
> contain assets for multiple branches at the same time. One just
selects
> the current branch, and commits/updates automatically pick a parent
> branch for finding the "common/base" revision. I have a feeling
> subversion supports this, but I haven't figured out yet how.
>
> - Inter-branch locking. A binary file that lives in multiple branches
> could be locked in all branches at the same time.
>
> - By default, reverting a directory deletes local files in it. After
> all, reverting restores the previous revision, and since the file did
> not exist in the previous revision, its previous state is void.
>
> - An experimental feature was "refactor friendly structured storage".
So
> instead of storing dumb source code, a DOM was stored instead, using
> unique-object-identifiers instead of names to refer to symbols. So
> basically, source control did not stop on the file level, but at the
> object level. Merging was done on this tree, which eased extreme
> programming and refactoring. For example, renaming a class was just
one
> change to the DOM in which the class was declared, although it might
> change dozens of source text files that used it. This was never used
> because it would also require different source code editors that work
on
> the symbolic level.
>
> I hope this information was useful, although I'm not expecting to see
> any of this in subversion soon, as you guys probably have really good
> reasons not to have implemented these "features".
>
> Cheers,
> Peter
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org