You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Julian Foad <ju...@btopenworld.com> on 2006/01/18 23:33:53 UTC
'svnadmin load' doesn't detect premature end of dump stream
While experimenting for issue #2441, "'svnadmin load' malformed dumpfile errors
are intolerably vague", I discovered the following more serious problem.
When the dump stream being read by 'svnadmin load' ends prematurely, in several
cases no error is detected. The following script demonstrates this:
# Make a non-empty repository and dump it as "dump1"
rm -rf repos1 wc1
svnadmin create repos1
mkdir wc1
echo "Blah" > wc1/file1
mkdir wc1/dir1
svn import -m "" wc1 file://`pwd`/repos1
svnadmin dump repos1 > dump1
# Try to load a various fractions of "dump1" into a new repository
for SIZE in $(seq 100 50 $(filesize dump1)); do
rm -rf repos2
svnadmin create repos2
echo
echo "Trying to load the first $SIZE bytes..."
head -c $SIZE dump1 |
svnadmin load repos2 2>&1 >/dev/null &&
svnadmin verify repos2
done
This outputs the following:
> Trying to load the first 100 bytes...
> * Verified revision 0.
>
> Trying to load the first 150 bytes...
> svnadmin: Premature end of content data in dumpstream
>
> Trying to load the first 200 bytes...
> * Verified revision 0.
>
> Trying to load the first 250 bytes...
> svnadmin: Incomplete or unterminated property block
>
> Trying to load the first 300 bytes...
> svnadmin: Premature end of content data in dumpstream
>
> Trying to load the first 350 bytes...
> svnadmin: Premature end of content data in dumpstream
>
> Trying to load the first 400 bytes...
> * Verified revision 0.
> * Verified revision 1.
>
> Trying to load the first 450 bytes...
> svnadmin: Incomplete or unterminated property block
>
> Trying to load the first 500 bytes...
> * Verified revision 0.
> * Verified revision 1.
>
> Trying to load the first 550 bytes...
> svnadmin: Incomplete or unterminated property block
>
> Trying to load the first 600 bytes...
> svnadmin: Incomplete or unterminated property block
>
> Trying to load the first 650 bytes...
> svnadmin: Premature end of content data in dumpstream
I think this is a cause for concern.
- Julian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH] 'svnadmin load' doesn't detect premature end of dump
stream
Posted by Julian Foad <ju...@btopenworld.com>.
Garrett Rooney wrote:
> On 2/6/06, Julian Foad <ju...@btopenworld.com> wrote:
>
>>The attached patch adds missing checks for premature EOF. Does it look sane?
>
> The only comment I have is that wrappers like the stream_ran_dry
> function are evil because they make the file/line info for an error
> useless, and if you can remove their used where you see them and
> replace with macros instead it would make the world a better place.
Yup, I agree, and maybe I'll address that some time.
> Other than that looks fine.
Thanks. Committed in r18445.
- Julian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH] 'svnadmin load' doesn't detect premature end of dump stream
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 2/6/06, Julian Foad <ju...@btopenworld.com> wrote:
> The attached patch adds missing checks for premature EOF. Does it look sane?
The only comment I have is that wrappers like the stream_ran_dry
function are evil because they make the file/line info for an error
useless, and if you can remove their used where you see them and
replace with macros instead it would make the world a better place.
Other than that looks fine.
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
[PATCH] 'svnadmin load' doesn't detect premature end of dump stream
Posted by Julian Foad <ju...@btopenworld.com>.
Julian Foad wrote:
>
> When the dump stream being read by 'svnadmin load' ends prematurely, in
> several cases no error is detected.
[...]
> head -c $SIZE dump1 |
> svnadmin load repos2 2>&1 >/dev/null &&
> svnadmin verify repos2
[...]
>> Trying to load the first 200 bytes...
>> * Verified revision 0.
The dump stream format is not designed for being able to detect all
modifications, so if it happens to be cut off accidentally at a clean break,
such as between one node or revision and the next, this will never be detected
as an error. Here's a snippet:
> Revision-number: 5
(EOF here is not detected; I expect it could be.)
> Prop-content-length: 116
> Content-length: 116
>
> K 7
> svn:log
> V 10
> Check in 2
> K 10
> svn:author
> V 10
> julianfoad
> K 8
> svn:date
> V 27
> 2003-07-18T21:56:02.803550Z
> PROPS-END
>
(EOF here is not detected, and can't be, I think.)
> Node-path: one
> Node-kind: file
> Node-action: add
(EOF here not detected; difficult, as following are not always required.)
> Prop-content-length: 10
> Text-content-length: 4
> Text-content-md5: 5bbf5a52328e7439ae6e719dfe712200
> Content-length: 14
>
> PROPS-END
> one
>
The attached patch adds missing checks for premature EOF. Does it look sane?
EOF part-way through a line is now detected, and it thereby catches and errors
out on a much higher proportion of arbitrarily cut-off dump streams (like maybe
98% compared to 80%).
My concern was that accidental truncation would be missed. That case is fairly
well addressed by this patch. There's not much more to gain by trying to catch
some of the remaining EOF cases.
*Special attention* At the end of the patch I make sure that missing
node-action and copyfrom-rev headers don't assume a valid value. Is that safe?
- Julian