You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by von Löwis Martin <Ma...@hpi.uni-potsdam.de> on 2006/01/20 18:53:56 UTC
Files with trailing dots on Windows
I would like subversion to support files with trailing dots on Windows.
I mentioned the issue on #svn, and darix suggested I report it here.
One use case for that is
http://svn.python.org/projects/python/trunk/Mac/IDE%20scripts/
where "Insert folder name..." has trailing dots. On checking out the
tree, subversion creates a file "Insert folder name" instead, and
subsequently reports that the tree was modified (with the original file
missing). svn revert will try to create the missing file, but fail
just as the original checkout failed.
I know that it is not subversion which drops the trailing dots, but
CreateFile.
However, there are solutions/work-arounds available, which I think
subversion should use.
For this specific problem, escaping the path name with \\?\ would
help. To see how this works, try running
notepad.exe \\?\c:\foo.txt...
This will happily create a file with trailing dots in ist name;
Microsoft even documents it in the Win32 API (somewhere).
May I file a bug report for that issue?
Regards,
Martin
P.S. If you were interested in supporting case-sensitive file names
also (i.e. having two files in the same directory that only differ
in case): that would be possible through FILE_FLAGS_POSIX_SEMANTICS
for CreateFile - unfortunately, you would have to use NtDeleteFile
for removal...
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Files with trailing dots on Windows
Posted by Giovanni Bajo <ra...@develer.com>.
Max Bowsher <ma...@ukf.net> wrote:
>> P.S. If you were interested in supporting case-sensitive file names
>> also (i.e. having two files in the same directory that only differ
>> in case): that would be possible through FILE_FLAGS_POSIX_SEMANTICS
>> for CreateFile - unfortunately, you would have to use NtDeleteFile
>> for removal...
>
> Sure, you can play all sorts of devious tricks with low level APIs,
> but that leaves you with files on your disk that cause 99% of existing
> software tools to then break in mysterious ways when you attempt to
> work with them.
>
> In my personal opinion, this 'solution' is totally nonviable.
I don't understand. It's not that Subversion is *creating* files with weird
filenames out of the blue. If *I* added it to the repository, it means that *I*
have tools to handle them. *I* will also make sure other people accessing the
repository know what to do with them. It could also be that those files can't /
needn't be accessed at all under Windows and people will access them under
different OS: if this is fine by me (in my repository), I'm not sure why you,
as a Subversion developer, should care. It's up to the users to decide what to
do. If I know how to create files with trailing dots, well, let me shoot myself
in the foot, don't tell me that I should be playing nice.
I believe Subversion should do everything in its power to behave correctly,
even with respect to files with wrong filenames. Surely being unable to check
out the correct files, and then complain because the repository is changed
while it really isn't, doesn't look like a correct behaviour.
Giovanni Bajo
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: Files with trailing dots on Windows
Posted by "Peter N. Lundblad" <pe...@famlundblad.se>.
On Fri, 20 Jan 2006, Arlie Davis wrote:
> So, while it IS possible (and easy) to create files on Windows systems that
> do not conform to the Win32 rules (which exist only for compatibility with
> applications), it may not be terribly useful, because most applications do
> NOT use the \\?\ syntax, and therefore will not be able to manipulate the
> files that you create.
>
> I agree with Max. It's a bad idea, mostly because of the potential for
> causing problems that most users will be completely unable to fix (filenames
> that cannot be manipulated with ordinary tools and programs).
>
While this is true, using absolute paths (which makes APR prepend \\?\ if
needed) will allow longer paths than 255 characters. This is a real
problem that has been reported by Java people.
See for example:
http://svn.haxx.se/users/archive-2005-02/1088.shtml
Regards,
//Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Files with trailing dots on Windows
Posted by "Martin v. Löwis" <Ma...@hpi.uni-potsdam.de>.
Arlie Davis wrote:
> So, while it IS possible (and easy) to create files on Windows systems that
> do not conform to the Win32 rules (which exist only for compatibility with
> applications), it may not be terribly useful, because most applications do
> NOT use the \\?\ syntax, and therefore will not be able to manipulate the
> files that you create.
While it may not be easy to process these files with local tools, I
wouldn't be asking for it if it wouldn't be useful to me, plus
I actually gave the reason why it is useful:
Currently, subversion will incorrectly report that there are local
modifications, and I have no way to tell it that it is wrong.
This is confusing both through TortoiseSVN,
So in this specific case, I don't *want* to edit the files. I just
want them to be consistent with the repository.
Plus, I *can* manipulate them just fine in the POSIX (Interix)
subsystem. If I need to, I can also maniuplate them in Win32
applications. For example, notepad happily accepts \\?\ file
names.
Regards,
Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: Files with trailing dots on Windows
Posted by Arlie Davis <ad...@stonestreetone.com>.
Actually, that's quite easy. Use the \\?\... syntax to override the Win32
filename processing. This also gets around limitations like not being able
to create a file named "com1" or "null".
For more information on this, check the docs on CreateFile, which describe
the \\?\ prefix. Basically, if CreateFile sees that its filename argument
begins with \\?\, then it will scan over (ignore) the prefix, and pass the
rest of the path directly to NtCreateFile (the internal / native API), and
will use the RootDirectory handle (in ObjectAttributes) to identify the \??
kernel directory namespace. This lets you do a lot of fun stuff.
HOWEVER, a lot of other parts of the Win32 subsystem will NOT properly
support this. For example:
[D:\temp\example] dir
Volume in drive D is Data
Volume Serial Number is XXXX-XXXX
Directory of D:\temp\example
01/20/2006 04:19 PM <DIR> .
01/20/2006 04:19 PM <DIR> ..
0 File(s) 0 bytes
2 Dir(s) 186,969,300,992 bytes free
[D:\temp\example] echo foo >\\?\d:\temp\example\foo...
[D:\temp\example] dir
Volume in drive D is Data
Volume Serial Number is XXXX-XXXX
Directory of D:\temp\example
01/20/2006 04:19 PM <DIR> .
01/20/2006 04:19 PM <DIR> ..
01/20/2006 04:19 PM 6 foo...
1 File(s) 6 bytes
2 Dir(s) 186,969,300,992 bytes free
[D:\temp\example] type foo...
The system cannot find the file specified.
[D:\temp\example] del foo...
Could Not Find D:\temp\example\foo...
[D:\temp\example] type \\?\d:\temp\example\foo...
foo
[D:\temp\example] del \\?\d:\temp\example\foo...
[D:\temp\example]
So, while it IS possible (and easy) to create files on Windows systems that
do not conform to the Win32 rules (which exist only for compatibility with
applications), it may not be terribly useful, because most applications do
NOT use the \\?\ syntax, and therefore will not be able to manipulate the
files that you create.
I agree with Max. It's a bad idea, mostly because of the potential for
causing problems that most users will be completely unable to fix (filenames
that cannot be manipulated with ordinary tools and programs).
-- arlie
-----Original Message-----
From: Max Bowsher [mailto:maxb1@ukf.net]
Sent: Friday, January 20, 2006 3:58 PM
To: von Löwis Martin
Cc: dev@subversion.tigris.org
Subject: Re: Files with trailing dots on Windows
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
von Löwis Martin wrote:
> I would like subversion to support files with trailing dots on Windows.
Well, you'd better go and convince Microsoft to make Windows to support
filenames with trailing dots, then.
> One use case for that is
>
> http://svn.python.org/projects/python/trunk/Mac/IDE%20scripts/
>
> where "Insert folder name..." has trailing dots. On checking out the
> tree, subversion creates a file "Insert folder name" instead, and
> subsequently reports that the tree was modified (with the original
> file missing). svn revert will try to create the missing file, but
> fail just as the original checkout failed.
Some kind of mapping logic to avoid the bizarre restrictions on Windows file
naming isn't totally out of the question, but that's a very different topic.
> However, there are solutions/work-arounds available, which I think
> subversion should use.
>
> For this specific problem, escaping the path name with \\?\ would
> help. To see how this works, try running
>
> notepad.exe \\?\c:\foo.txt...
>
> This will happily create a file with trailing dots in ist name;
> Microsoft even documents it in the Win32 API (somewhere).
>
> May I file a bug report for that issue?
>
> Regards,
> Martin
>
> P.S. If you were interested in supporting case-sensitive file names
> also (i.e. having two files in the same directory that only differ in
> case): that would be possible through FILE_FLAGS_POSIX_SEMANTICS for
> CreateFile - unfortunately, you would have to use NtDeleteFile for
> removal...
Sure, you can play all sorts of devious tricks with low level APIs, but that
leaves you with files on your disk that cause 99% of existing software tools
to then break in mysterious ways when you attempt to work with them.
In my personal opinion, this 'solution' is totally nonviable.
Max.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Cygwin)
iD8DBQFD0U7bfFNSmcDyxYARAkH+AKCjNJXlBKMeA6Q8cpnN0UNH4k1C8QCfZtKr
dfcvWhJE7QzS8+JS1akJLIk=
=L5mE
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Files with trailing dots on Windows
Posted by Max Bowsher <ma...@ukf.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
von Löwis Martin wrote:
> I would like subversion to support files with trailing dots on Windows.
Well, you'd better go and convince Microsoft to make Windows to support
filenames with trailing dots, then.
> One use case for that is
>
> http://svn.python.org/projects/python/trunk/Mac/IDE%20scripts/
>
> where "Insert folder name..." has trailing dots. On checking out the
> tree, subversion creates a file "Insert folder name" instead, and
> subsequently reports that the tree was modified (with the original file
> missing). svn revert will try to create the missing file, but fail
> just as the original checkout failed.
Some kind of mapping logic to avoid the bizarre restrictions on Windows
file naming isn't totally out of the question, but that's a very
different topic.
> However, there are solutions/work-arounds available, which I think
> subversion should use.
>
> For this specific problem, escaping the path name with \\?\ would
> help. To see how this works, try running
>
> notepad.exe \\?\c:\foo.txt...
>
> This will happily create a file with trailing dots in ist name;
> Microsoft even documents it in the Win32 API (somewhere).
>
> May I file a bug report for that issue?
>
> Regards,
> Martin
>
> P.S. If you were interested in supporting case-sensitive file names
> also (i.e. having two files in the same directory that only differ
> in case): that would be possible through FILE_FLAGS_POSIX_SEMANTICS
> for CreateFile - unfortunately, you would have to use NtDeleteFile
> for removal...
Sure, you can play all sorts of devious tricks with low level APIs, but
that leaves you with files on your disk that cause 99% of existing
software tools to then break in mysterious ways when you attempt to work
with them.
In my personal opinion, this 'solution' is totally nonviable.
Max.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Cygwin)
iD8DBQFD0U7bfFNSmcDyxYARAkH+AKCjNJXlBKMeA6Q8cpnN0UNH4k1C8QCfZtKr
dfcvWhJE7QzS8+JS1akJLIk=
=L5mE
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Files with trailing dots on Windows
Posted by "Martin v. Löwis" <Ma...@hpi.uni-potsdam.de>.
Branko Čibej wrote:
> a) Only work on Win NT and later (not Win9x) -- not that I care about
> those platforms, and
True.
> b) APR already does that with file names on Windows, so it doesn't
> always seem to help.
Maybe I'm missing something. However, looking at utf8_to_unicode_path,
I see that this only happens if (srcremains > MAX_PATH); with the
comment
/* This is correct, we don't twist the filename if it is will
* definately be shorter than MAX_PATH. It merits some
* performance testing to see if this has any effect, but there
* seem to be applications that get confused by the resulting
* Unicode \\?\ style file names, especially if they use argv[0]
* or call the Win32 API functions such as GetModuleName, etc.
* Not every application is prepared to handle such names.
*
* Note that a utf-8 name can never result in more wide chars
* than the original number of utf-8 narrow chars.
*/
So APR doesn't do this always; I my case, the path much shorter
than MAX_PATH.
> Anyway, this isn't the only kind of file name that won't work on
> Windows; file names with colons or backslashes in them (which can be
> created on Unix) will also break the Windows client.
Right. I wonder how subversion behaves when confronted with such
a file. Will it create an addition data stream?
Interix uses some mapping algorithm to support such file names,
mapping them into the Private Use Area of Unicode.
> Our position in similar situations in the past is that it's not
> Subversion's job to support file names that the native OS can't handle.
Ok, may I then file a bug that Subversion should stop claiming that
there were local modifications, when there weren't any?
> We know that, but we also know that it would be a pretty pointless thing
> to do, because other apps couldn't use such files.
That's not true. Interix apps can use them with no problems, and
Win32 apps typically can use them when passed the fully-escaped file
name.
Regards,
Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Files with trailing dots on Windows
Posted by "Peter N. Lundblad" <pe...@famlundblad.se>.
On Fri, 20 Jan 2006, [UTF-8] Branko �^Libej wrote:
> von Löwis Martin wrote:
> > I would like subversion to support files with trailing dots on Windows.
> > I mentioned the issue on #svn, and darix suggested I report it here.
> >
> b) APR already does that with file names on Windows, so it doesn't
> always seem to help.
>
AFAICT, it only does this for pathnames longer than 255 characters, which
might explain why it "doesn't always help", and it only does this with
absolute path names.
Thanks,
//Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Files with trailing dots on Windows
Posted by Branko Čibej <br...@xbc.nu>.
von Löwis Martin wrote:
> I would like subversion to support files with trailing dots on Windows.
> I mentioned the issue on #svn, and darix suggested I report it here.
>
> One use case for that is
>
> http://svn.python.org/projects/python/trunk/Mac/IDE%20scripts/
>
> where "Insert folder name..." has trailing dots. On checking out the
> tree, subversion creates a file "Insert folder name" instead, and
> subsequently reports that the tree was modified (with the original file
> missing). svn revert will try to create the missing file, but fail
> just as the original checkout failed.
>
> I know that it is not subversion which drops the trailing dots, but
> CreateFile.
>
> However, there are solutions/work-arounds available, which I think
> subversion should use.
>
> For this specific problem, escaping the path name with \\?\ would
> help. To see how this works, try running
>
> notepad.exe \\?\c:\foo.txt...
>
> This will happily create a file with trailing dots in ist name;
> Microsoft even documents it in the Win32 API (somewhere).
>
This will
a) Only work on Win NT and later (not Win9x) -- not that I care about
those platforms, and
b) APR already does that with file names on Windows, so it doesn't
always seem to help.
Anyway, this isn't the only kind of file name that won't work on
Windows; file names with colons or backslashes in them (which can be
created on Unix) will also break the Windows client.
> May I file a bug report for that issue?
>
Our position in similar situations in the past is that it's not
Subversion's job to support file names that the native OS can't handle.
> Regards,
> Martin
>
> P.S. If you were interested in supporting case-sensitive file names
> also (i.e. having two files in the same directory that only differ
> in case): that would be possible through FILE_FLAGS_POSIX_SEMANTICS
> for CreateFile - unfortunately, you would have to use NtDeleteFile
> for removal...
>
We know that, but we also know that it would be a pretty pointless thing
to do, because other apps couldn't use such files.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Files with trailing dots on Windows
Posted by "Martin v. Löwis" <Ma...@hpi.uni-potsdam.de>.
James Courtier-Dutton wrote:
> Before considering this a bug, check which file system you are using on
> Windows. FAT32 will fail, NTFS will work.
I checked, it's NTFS, it it doesn't work.
Regards,
Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Files with trailing dots on Windows
Posted by James Courtier-Dutton <Ja...@superbug.co.uk>.
von Löwis Martin wrote:
> I would like subversion to support files with trailing dots on Windows.
> I mentioned the issue on #svn, and darix suggested I report it here.
>
> One use case for that is
>
> http://svn.python.org/projects/python/trunk/Mac/IDE%20scripts/
>
> where "Insert folder name..." has trailing dots. On checking out the
> tree, subversion creates a file "Insert folder name" instead, and
> subsequently reports that the tree was modified (with the original file
> missing). svn revert will try to create the missing file, but fail
> just as the original checkout failed.
>
> I know that it is not subversion which drops the trailing dots, but
> CreateFile.
>
> However, there are solutions/work-arounds available, which I think
> subversion should use.
>
> For this specific problem, escaping the path name with \\?\ would
> help. To see how this works, try running
>
> notepad.exe \\?\c:\foo.txt...
>
> This will happily create a file with trailing dots in ist name;
> Microsoft even documents it in the Win32 API (somewhere).
>
> May I file a bug report for that issue?
>
> Regards,
> Martin
>
Before considering this a bug, check which file system you are using on
Windows. FAT32 will fail, NTFS will work.
If your's is FAT32, there is a simple windows tool to switch to NTFS.
James
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org