You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Michael Sinz <Mi...@sinz.org> on 2007/02/03 16:45:49 UTC
Bug in dump/restore...
I just ran into this problem and have not looked further than there is a problem.
In the development of Insurrection, I put together a repository with some test
cases that have (and in some cases still do) cause problems for Insurrection.
This includes file names with various characters in them that are not "normal"
(or at least not normal in the sense that they are rarely used in the
traditional sense.)
This repository can be seen at http://svn.sinz.com/svn/example/
The problem is that I was going to use that in testing some code on my local
machine so I was going to do a dump-restore of the repository rather than just
pulling the whole directory down. However, this plan ran into a bit of a snag
when svnadmin load could not process the output of svnadmin dump.
There is what I did:
In a fresh/cleanly built Subversion 1.4.3 setup, I used svnadmin dump to
generate the dump file and then just load that into a new repository as such:
$ svnadmin dump repositories/example >example.dump
$ svnadmin load repositories/new-ex <example.dump
The dump part claimed to be successful, doing all 36 revisions.
The load failed on revision #9 - where some of the special case file names
were added to the repository.
Output (trimmed) is:
* adding path : trunk/tests/TestCase-~-/test.txt ... done.
svnadmin: Invalid control character '0x7f' in path 'trunk/tests/TestCase--'
* adding path : trunk/tests/TestCase-- ...
You can see this at: http://svn.sinz.com/svn/example/trunk/tests/TestCase-%7F-/
It seems that this character sequence (0x7F) is not escaped correctly in the
dump and is placed there "in the raw" but then is not correctly handled in
the load process. (I don't blame the load too much since this is a "DEL" code,
but it works find in the bas
Note that the repository does correctly check out (svn co) for any revision on
my Linux box. (I do know it has problems on Windows due to some file names -
see http://svn.sinz.com/svn/example/trunk/tests/Win32-Tests/ for some details
on the problems within Windows...)
I have the example.dump file available at http://svn.sinz.com/example.dump.gz
Trying to load that into a repository reproduces the problem nicely...
Good thing I use FSFS and do the backups via filesystem snapshots.
--
Michael Sinz Technology and Engineering Director/Consultant
"Starting Startups" mailto:michael.sinz@sinz.org
My place on the web http://www.sinz.org/Michael.Sinz
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Bug in dump/restore...
Posted by Michael Sinz <Mi...@sinz.org>.
On 2/6/07, Michael Sinz <Mi...@sinz.org> wrote:
>
> On 2/6/07, Michael Sinz <Mi...@sinz.org> wrote:
> >
> > On 2/6/07, Ben Collins-Sussman < sussman@red-bean.com> wrote:
> >
> > > On 2/5/07, Michael Sinz <Mi...@sinz.org> wrote:
> >
> >
> [...]
>
> You must have done this a long, long, time ago. We stopped allowing
> > > control-characters to enter the repository (either in filenames or log
> > > messages) in some pre-1.0 release.
> > >
> >
> > Looking at the date in the repository, it looks like this happened in
> > the 1.2 timeframe (that server never ran less than SVN 1.1 since it is
> > all FSFS and the April 25, 2005 date shows that it may have been running the
> > 1.2 RC1 - don't know for sure)
> >
> > However, testing this now with SVN 1.4 trying to create a new repository
> > and adding that file, it seems to not to add this to the repository.
> >
> > So, now I need to filter this out of the test repository as it is no
> > longer a valid test case (and can not be added to the repository)
> >
> > I will see if I have time today to build a 1.1 SVN client and try it
> > again.
>
>
> I did build the client that my logs show I was using during this testing.
> It was the 1.1.4 release of Subversion. It did allow me to add the file
> with the "del" character (0x7f) but when I try to commit that to the server
> running 1.4.3, it does not allow it.
>
> Committing it to a local FSFS repository using Subversion 1.1.4 did work.
> (I have not tried BDB as my build environment is not set up for doing that
> test right now)
>
> So, this "bug" is not a bug in current ( 1.4.3) release and the server
> does prevent the entry of this data. However it does look like Subversion
> 1.1.4 (and most likely all 1.1.x releases) do/did allow this.
>
And a quick check of the 1.2.0 release shows that it correctly rejects the
"del" character. That means the change happened as part of the 1.2 update.
Note, I don't need this to work, I just was trying to figure out my test
cases and why I could not load a dump from the test repository. Now I
know. (And it is interesting that I did not notice this until this past
weekend... I guess I had not had to do much with my test cases in quite
some time)
--
Michael Sinz Technology and Engineering Director/Consultant
"Starting Startups" mailto:Michael.Sinz@sinz.org
My place on the web http://www.sinz.org/Michael.Sinz
Re: Bug in dump/restore...
Posted by Michael Sinz <Mi...@sinz.org>.
On 2/6/07, Michael Sinz <Mi...@sinz.org> wrote:
>
> On 2/6/07, Ben Collins-Sussman <su...@red-bean.com> wrote:
>
> > On 2/5/07, Michael Sinz <Mi...@sinz.org> wrote:
>
>
[...]
You must have done this a long, long, time ago. We stopped allowing
> > control-characters to enter the repository (either in filenames or log
> > messages) in some pre-1.0 release.
> >
>
> Looking at the date in the repository, it looks like this happened in the
> 1.2 timeframe (that server never ran less than SVN 1.1 since it is all
> FSFS and the April 25, 2005 date shows that it may have been running the
> 1.2 RC1 - don't know for sure)
>
> However, testing this now with SVN 1.4 trying to create a new repository
> and adding that file, it seems to not to add this to the repository.
>
> So, now I need to filter this out of the test repository as it is no
> longer a valid test case (and can not be added to the repository)
>
> I will see if I have time today to build a 1.1 SVN client and try it
> again.
I did build the client that my logs show I was using during this testing.
It was the 1.1.4 release of Subversion. It did allow me to add the file
with the "del" character (0x7f) but when I try to commit that to the server
running 1.4.3, it does not allow it.
Committing it to a local FSFS repository using Subversion 1.1.4 did work.
(I have not tried BDB as my build environment is not set up for doing that
test right now)
So, this "bug" is not a bug in current (1.4.3) release and the server does
prevent the entry of this data. However it does look like Subversion
1.1.4(and most likely all
1.1.x releases) do/did allow this.
--
Michael Sinz Technology and Engineering Director/Consultant
"Starting Startups" mailto:Michael.Sinz@sinz.org
My place on the web http://www.sinz.org/Michael.Sinz
Re: Bug in dump/restore...
Posted by Michael Sinz <Mi...@sinz.org>.
On 2/6/07, Ben Collins-Sussman <su...@red-bean.com> wrote:
>
> On 2/5/07, Michael Sinz <Mi...@sinz.org> wrote:
>
> > The question should now turn to why the file was allowed to be created
> in the
> > first place. I did nothing other than to create the file and then "svn
> add" it
> > to the repository.
>
> You must have done this a long, long, time ago. We stopped allowing
> control-characters to enter the repository (either in filenames or log
> messages) in some pre-1.0 release.
>
Looking at the date in the repository, it looks like this happened in the
1.2 timeframe (that server never ran less than SVN 1.1 since it is all FSFS
and the April 25, 2005 date shows that it may have been running the 1.2 RC1
- don't know for sure)
However, testing this now with SVN 1.4 trying to create a new repository and
adding that file, it seems to not to add this to the repository.
So, now I need to filter this out of the test repository as it is no longer
a valid test case (and can not be added to the repository)
I will see if I have time today to build a 1.1 SVN client and try it again.
--
Michael Sinz Technology and Engineering Director/Consultant
"Starting Startups" mailto:Michael.Sinz@sinz.org
My place on the web http://www.sinz.org/Michael.Sinz
Re: Bug in dump/restore...
Posted by Ben Collins-Sussman <su...@red-bean.com>.
On 2/5/07, Michael Sinz <Mi...@sinz.org> wrote:
> The question should now turn to why the file was allowed to be created in the
> first place. I did nothing other than to create the file and then "svn add" it
> to the repository.
You must have done this a long, long, time ago. We stopped allowing
control-characters to enter the repository (either in filenames or log
messages) in some pre-1.0 release.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Bug in dump/restore...
Posted by Michael Sinz <Mi...@sinz.org>.
Michael Sinz wrote:
> I just ran into this problem and have not looked further than there is a
> problem.
[...]
> In a fresh/cleanly built Subversion 1.4.3 setup, I used svnadmin dump to
> generate the dump file and then just load that into a new repository as
> such:
>
> $ svnadmin dump repositories/example >example.dump
> $ svnadmin load repositories/new-ex <example.dump
>
> The dump part claimed to be successful, doing all 36 revisions.
>
> The load failed on revision #9 - where some of the special case file names
> were added to the repository.
>
> Output (trimmed) is:
>
> * adding path : trunk/tests/TestCase-~-/test.txt ... done.
> svnadmin: Invalid control character '0x7f' in path 'trunk/tests/TestCase--'
> * adding path : trunk/tests/TestCase-- ...
>
> You can see this at:
> http://svn.sinz.com/svn/example/trunk/tests/TestCase-%7F-/
[...]
So, looking at the load code, it uses the svn_ctype table to determine that the
0x7F character is a control character and does not allow it as part of the node
name during the svnadmin load process.
The question now shifts to what the correct behavior should be. Given that the
repository does contain a file with this character in it, how would one correct
the situation?
Clearly, a dump/load does not work very well. At least not as long as the 0x7F
character is marked as a control character (not that I am saying that it should
not be - it clearly is special) yet the system allowed me to create this.
It is also clear that the node names are output directly from the node.
The question should now turn to why the file was allowed to be created in the
first place. I did nothing other than to create the file and then "svn add" it
to the repository. As such, given that the file name was correctly added and
that the file correctly extracts during an "svn co" or "svn up" then the
question remains "why does svnadmin load not accept it?"
Getting rid of the SVN_CTYPE_CNTRL from 0x7F is a "quick hack" that allows
the reload of the repository. However, I am unsure that this is a safe thing
to do:
--- subversion-1.4.3-release/subversion/libsvn_subr/ctype.c Thu Dec 2
01:53:20 2004
+++ subversion-1.4.3/subversion/libsvn_subr/ctype.c Mon Feb 5 22:44:23 2007
@@ -150,9 +150,9 @@
/* { */ SVN_CTYPE_ASCII | SVN_CTYPE_PUNCT,
/* | */ SVN_CTYPE_ASCII | SVN_CTYPE_PUNCT,
/* } */ SVN_CTYPE_ASCII | SVN_CTYPE_PUNCT,
/* ~ */ SVN_CTYPE_ASCII | SVN_CTYPE_PUNCT,
- /* del */ SVN_CTYPE_ASCII | SVN_CTYPE_CNTRL,
+ /* del */ SVN_CTYPE_ASCII,
/* x80 */ SVN_CTYPE_UTF8CONT,
/* x81 */ SVN_CTYPE_UTF8CONT,
/* x82 */ SVN_CTYPE_UTF8CONT,
/* x83 */ SVN_CTYPE_UTF8CONT,
Note, also, that the output from "svn log" also will end up using the raw "0x7F"
(aka 'del') character in what it generates. This may also be invalid for XML.
Now, I don't really know how this file made it into the system as it seems like
the "svn and" command has a flaw in it as it seems to allow the creation and
storage for the very same file that "svnadmin load" fails to support.
--
Michael Sinz Technology and Engineering Director/Consultant
"Starting Startups" mailto:michael.sinz@sinz.org
My place on the web http://www.sinz.org/Michael.Sinz
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org