You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by John Peacock <jp...@rowman.com> on 2003/07/30 13:49:06 UTC
Yet another Case-Insensitive File System Problem
While working on my copy of the converted Perl repository, I discovered yet
another problem with case-insensitive files systems. When Unicode support was
added to Perl, the core routines were added under lib/unicode/. Later on, a
userspace module was added, under lib/Unicode/; this was recognized as a problem
under NT and OS/X 10.x and the lib/unicode directory was renamed lib/unicore.
This is even though the actual files stored in the directories are unique.
However, it seems that under Perforce, like CVS, empty directories still persist
in the repository. Consequently, it is impossible to check out the entire Perl
repository under NT at any point where both directories exist (svn refuses to
continue the checkout), like today. I can recreate the subversion repository
and delete the offending directories at the point in time where the rename
occurred, but it still means that there exists a window of revisions where
checkout is impossible under NT and OS/X.
The current subversion behavior, i.e. just refusing to continue, is suboptimal.
At the very least, there should be a way to just force a checkout, which
should merge the directories in the wc (as long as the actual filenames don't
conflict in a case-insensitive way).
The rub, of course, is what to do with files that need to be committed, that are
located in those merged directories, since the entries and text-base cannot
really be merged. I suppose the second directory could be added to the entries
file and then the files located therein could be flagged with a property that
would clue svn to handle those files differently when committing. That smells
very bad to me, and I doubt that it will fly with the list as well... ;~)
In any case, should I create an incident to track this as a post-1.0 task?
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5748
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Yet another Case-Insensitive File System Problem
Posted by Branko Čibej <br...@xbc.nu>.
Michael Wood wrote:
>On Wed, Jul 30, 2003 at 02:24:11PM -0400, John Peacock wrote:
>[snip]
>
>
>>Giving us poor NTFS souls a way, even a limited way, to continue to
>>participate is much better, IMHO, than simply erroring out due to the
>>idiotic limitations of the local filesystem. Being able to exclude a
>>portion of the tree during checkout seems a reasonable concession in
>>that direction. Think of it as an svn:antiexternal feature.
>>
>>
>
>Well I don't know about OS/2's filesystem(s) (hpfs?) but I believe it's
>possible to switch NTFS into case-sensitive mode. Unfortunately, I
>don't know how to do this or what problems it may cause. Would this
>help? :)
>
>
It's worse than that -- you can decide, at file open/creation time,
whether or not you want the operation to be case-sensitive. This is done
by useing the fabled FILE_FLAG_POSIX_SEMANTICS. Of course, it won't help
in general because your editor, build tools etc. will get confused. And
this flag isn't available when you create a directory, heh.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Yet another Case-Insensitive File System Problem
Posted by John Peacock <jp...@rowman.com>.
Michael Wood wrote:
> Well I don't know about OS/2's filesystem(s) (hpfs?) but I believe it's
> possible to switch NTFS into case-sensitive mode. Unfortunately, I
> don't know how to do this or what problems it may cause. Would this
> help? :)
Note the last paragraph here:
http://techsupt.winbatch.com/TS/T000001036004F26.html
so it appears that it is possible to turn it on, but you cannot use it!
And even if you somehow used exclusively the POSIX interface, see this link:
http://support.microsoft.com/default.aspx?scid=http://support.microsoft.com:80/support/kb/articles/Q246/0/26.ASP&NoWebContent=1
Yes, that's right! The only way to recover from this is the backup the drive
sans the offending files, reformat and reinstall!
Apparently, this is what M$loth considers POSIX compatibility!
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4720 Boston Way
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5747
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Yet another Case-Insensitive File System Problem
Posted by Michael Wood <mw...@its.uct.ac.za>.
On Wed, Jul 30, 2003 at 02:24:11PM -0400, John Peacock wrote:
[snip]
> Giving us poor NTFS souls a way, even a limited way, to continue to
> participate is much better, IMHO, than simply erroring out due to the
> idiotic limitations of the local filesystem. Being able to exclude a
> portion of the tree during checkout seems a reasonable concession in
> that direction. Think of it as an svn:antiexternal feature.
Well I don't know about OS/2's filesystem(s) (hpfs?) but I believe it's
possible to switch NTFS into case-sensitive mode. Unfortunately, I
don't know how to do this or what problems it may cause. Would this
help? :)
--
Michael Wood <mw...@its.uct.ac.za>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Yet another Case-Insensitive File System Problem
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
John Peacock wrote:
> Giving us poor NTFS souls a way, even a limited way, to continue to
> participate is much better, IMHO, than simply erroring out due to the
> idiotic limitations of the local filesystem. Being able to exclude a
> portion of the tree during checkout seems a reasonable concession in
> that direction. Think of it as an svn:antiexternal feature.
Believe me, I hope someone can come up with a good solution, as I live
in 'not quite case sensitive' hell myself, on OS X.
Personally, I'm planning on putting together a hook script that will
catch these kind of issues at commit time and error out, so at least
this kind of thing can be stopped going forward. Won't help the old
repositories though, and I still have to find a round tuit.
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Yet another Case-Insensitive File System Problem
Posted by John Peacock <jp...@rowman.com>.
Garrett Rooney wrote:
> The problem is, this is rather core to the Subversion philosophy with
> regard to files and directories. We version tree structures, and you
> are talking about two different parts of the tree. Automatically
> merging them just because they have similar names is not a trivial thing
> to do, and I'm not convinced it's the right thing to do.
And that's a fair analysis of the merge proposal. That may be an unworkable
scheme, and I fully accept that. Branko came up with some things I had not
considered that strongly argues against even attempting that route. Based on
what he suggested, I can ameliorate the worst effects of the mistake in this
particular repository. The problem won't go away in the general case, however.
Giving us poor NTFS souls a way, even a limited way, to continue to participate
is much better, IMHO, than simply erroring out due to the idiotic limitations of
the local filesystem. Being able to exclude a portion of the tree during
checkout seems a reasonable concession in that direction. Think of it as an
svn:antiexternal feature.
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5748
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Yet another Case-Insensitive File System Problem
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
John Peacock wrote:
> What bugs me the most is that this is not a very nice way for the svn
> client to act. It means that the repository is virtually inaccessible,
> but only from a specific O/S; everywhere else it is fine. If I could
> check out the repos, with warnings, rather than having it error out,
> that would be acceptable I think.
The problem is, this is rather core to the Subversion philosophy with
regard to files and directories. We version tree structures, and you
are talking about two different parts of the tree. Automatically
merging them just because they have similar names is not a trivial thing
to do, and I'm not convinced it's the right thing to do. There are
entirely too many platform dependent restrictions on the names of files
and directories, we can't hope to catch them all, and even if we detect
it, it still seems like the 'right thing to do' is just to give a nicer
error message explaining the problem. This is what you get when you
version directories, it's not a bug, IMO, it's a fundamental feature of
how the system works.
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Yet another Case-Insensitive File System Problem
Posted by John Peacock <jp...@rowman.com>.
Branko Čibej wrote:
>
> Ah, I think I see -- you're saying there was a period of time when both
> lib/unicode and lib/Unicode existed and were live. I'm sure this caused
> problems with _any_ version control system on NT and OS/X, and the only
I don't know how the Perforce client deals with it under NT. The historical
situation was that at the time this occurred, no one accessed the repository via
p4 except the committers (pumpking et al), who were all (AFAIK) running some
Unix. Everyone else got the source via a snapshot tar file; at least under NT,
tar would merge the directories silently. It appears that under OSX, tar would
automatically rename the conflicting directory (which oddly enough caused
different problems ;). Only filename conflicts were a problem.
> "solution" I can think of is to actually fiddle with the SVN repo dump
> file to rename lib/unicode to lib/unicore at the point where lib/Unicode
> was first created. This is a bit drastic, as it would mean that while
> you can check out revisions from that period on NT, you can't build the
> source anywhere any more.
Ewwwww. Not going there, thanks!
> Heh, I'd turn this around and say it was not a very nice way for the
> Perl repository admin to act when the directories with conflicting names
> were created in the first place. It's not as if systems with
> case-insensitive filenames were an obscure niche market. :-)
I fully and freely admit that this was a self-inflicted problem on the part of
the Perl pumpking! But as I said, since the majority of the developer base was
using tar-based extracts, it never caused a major problem at the time. And only
a completely bass-ackwards OS would preserve case and yet not be able to be
case-sensitive! :~o
But, the facts on the ground are that the repository is broken for a specific
sequence of revisions on (AFAIK) only these two OS's, but works fine everywhere
else. It would be a very bad precedent to retroactively change the repository
to account for a specific client OS problem.
> The only marginally sensible way of handling this -- apart from imposing
> a project-wide naming convention that prevents such conflicts -- would
> be to add directory renaming capabilities to the SVN client.
OR, as I already suggested, give the client the ability to proceed past the
conflict (with appropriately dire warnings). It is the inability to proceed at
all, on these two platforms, without performing a completely manual checkout,
that bugs me the most. Since the subversion client can browse the repository,
regardless of case, the poor NT developer should be able to intelligently decide
what directory to exclude/include. It is only at the point of actually
instantiating the wc that the local OS has a problem.
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4720 Boston Way
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5747
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Yet another Case-Insensitive File System Problem
Posted by Branko Čibej <br...@xbc.nu>.
John Peacock wrote:
> Branko Čibej wrote:
>
>> Nah. Especially as we happen to have one, nay, two workarounds in place:
>>
>> * svn rm URL. Simply remove the offending empty directory from the
>> repository, end of problem. This works from all platforms, since
>> URLs are always case-sensitive.
>
>
> This would leave a 2 year hole in the repository from the standpoint
> of NT and OSX.
I'm not quite sure what you mean by a "hole". It certainly wouldn't help
with checkouts of revisions from before the deletion, yes.
> * svn dump | svndumpfilter | svn load. Remove the offending
>
>> directory from _all_ revisions in the repository, or with a bit
>> more work, just from the point in time when it was renamed. While
>> this is more work than the previous solution, it does mean that
>> you
>> get consistent state in pre-HEAD revisions.
>
>
> This, at least, only leaves a ~4 month hole in the repository.
Ah, I think I see -- you're saying there was a period of time when both
lib/unicode and lib/Unicode existed and were live. I'm sure this caused
problems with _any_ version control system on NT and OS/X, and the only
"solution" I can think of is to actually fiddle with the SVN repo dump
file to rename lib/unicode to lib/unicore at the point where lib/Unicode
was first created. This is a bit drastic, as it would mean that while
you can check out revisions from that period on NT, you can't build the
source anywhere any more.
> It's probably what I will recommend for the perl.org repository, or
> rather to manufacture a commit at the correct moment which will delete
> the directories.
>
> What bugs me the most is that this is not a very nice way for the svn
> client to act.
Heh, I'd turn this around and say it was not a very nice way for the
Perl repository admin to act when the directories with conflicting names
were created in the first place. It's not as if systems with
case-insensitive filenames were an obscure niche market. :-)
> It means that the repository is virtually inaccessible, but only
> from a specific O/S; everywhere else it is fine. If I could check out
> the repos, with warnings, rather than having it error out, that would
> be acceptable I think.
The only marginally sensible way of handling this -- apart from imposing
a project-wide naming convention that prevents such conflicts -- would
be to add directory renaming capabilities to the SVN client. This would
be a tricky thing to design, as there are many edge cases and
interactions that have to be taken into account. But it's definitely
easier to get correct than the merging functionality you proposed.
However, I'd hesitate to propose this even as a post-1.0 feature until a
survey showed a lot more such cases in important open source
repositories than just a 4 month period in Perl's.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Yet another Case-Insensitive File System Problem
Posted by John Peacock <jp...@rowman.com>.
Branko Čibej wrote:
> Huh? This would "solve" your particular problem, but would cause any
> number of new ones. What if the directories are totally unrelated?
If there was a way to say, 'svn co --ignore=this/Stupid/dIrectory' then I could
get around the problem directory. Yes, I know I could build a series of
non-recursive checkouts that would have the appropriate effect. It would be
very painful.
Or, an option to have svn only warn (and then skip) when there are conflicting
directory names, so at least the wc is consistent (no merged directories). Or
even prompt and say "I can give you either path/to/dir or path/to/Dir, which do
you want?" and then only copy the requested directory/contents.
> Nah. Especially as we happen to have one, nay, two workarounds in place:
>
> * svn rm URL. Simply remove the offending empty directory from the
> repository, end of problem. This works from all platforms, since
> URLs are always case-sensitive.
This would leave a 2 year hole in the repository from the standpoint of NT and OSX.
> * svn dump | svndumpfilter | svn load. Remove the offending
> directory from _all_ revisions in the repository, or with a bit
> more work, just from the point in time when it was renamed. While
> this is more work than the previous solution, it does mean that you
> get consistent state in pre-HEAD revisions.
This, at least, only leaves a ~4 month hole in the repository. It's probably
what I will recommend for the perl.org repository, or rather to manufacture a
commit at the correct moment which will delete the directories.
What bugs me the most is that this is not a very nice way for the svn client to
act. It means that the repository is virtually inaccessible, but only from a
specific O/S; everywhere else it is fine. If I could check out the repos, with
warnings, rather than having it error out, that would be acceptable I think.
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5748
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Yet another Case-Insensitive File System Problem
Posted by Branko Čibej <br...@xbc.nu>.
John Peacock wrote:
> The current subversion behavior, i.e. just refusing to continue, is
> suboptimal. At the very least, there should be a way to just force a
> checkout, which should merge the directories in the wc (as long as the
> actual filenames don't conflict in a case-insensitive way).
Huh? This would "solve" your particular problem, but would cause any
number of new ones. What if the directories are totally unrelated?
> The rub, of course, is what to do with files that need to be
> committed, that are located in those merged directories, since the
> entries and text-base cannot really be merged.
That's the least of your prohlems, since you could easily add
information to the entries database about where a file came from. What
will you do with added files? What about merge behaviour?
> I suppose the second directory could be added to the entries file
> and then the files located therein could be flagged with a property
> that would clue svn to handle those files differently when
> committing. That smells very bad to me, and I doubt that it will fly
> with the list as well... ;~)
The whole idea is out of whack, if you ask me. :-)
> In any case, should I create an incident to track this as a post-1.0
> task?
Nah. Especially as we happen to have one, nay, two workarounds in place:
* svn rm URL. Simply remove the offending empty directory from the
repository, end of problem. This works from all platforms, since
URLs are always case-sensitive.
* svn dump | svndumpfilter | svn load. Remove the offending
directory from _all_ revisions in the repository, or with a bit
more work, just from the point in time when it was renamed. While
this is more work than the previous solution, it does man that you
get consistent state in pre-HEAD revisions.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org