You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by olli hauer <oh...@gmx.de> on 2013/06/24 19:30:11 UTC
svnsync 1.8.0 fails if there is a non-printable characters in the
commit message
svnsync 1.8.0 fails if there is a non-printable characters in the commit message
Error message:
svnsync: E000022: Safe data 'MFC r251249, r251251, r251252, r' was followed by non-ASCII byte 24: unable to convert to/from UTF-8
No issue with svnsync 1.7.10 (neon based)
Parts of the file (synced with svnsync 1.7.10)
$> cat $repo/db/revprops/251/251614
...
MFC r251249, r251251, r251252, r2512, r251254 and r251515:
$> more $repo/db/revprops/251/251614
...
MFC r251249, r251251, r251252, r^X2512, r251254 and r251515:
$> hexdump -C $repo/db/revprops/251/251614
...
00000070 31 2c 20 72 32 35 31 32 35 32 2c 20 72 18 32 35 |1, r251252, r.25|
Culprit is the hex(18) sign.
--
olli
Re: svnsync 1.8.0 fails if there is a non-printable characters in
the commit message
Posted by olli hauer <oh...@gmx.de>.
On 2013-06-26 01:05, Daniel Shahaf wrote:
> olli hauer wrote on Tue, Jun 25, 2013 at 17:06:35 +0200:
>> On 2013-06-25 16:02, Daniel Shahaf wrote:
>>> Daniel Shahaf wrote on Tue, Jun 25, 2013 at 16:55:24 +0300:
>>>> Daniel Shahaf wrote on Tue, Jun 25, 2013 at 13:50:58 +0000:
>>>>> On Mon, Jun 24, 2013 at 09:47:17PM +0200, Ben Smith-Mannschott wrote:
>>>>>> 0x18 is ^X, the ASCII control code for CANCEL. Seems to be working as
>>>>>> designed. ;-)
>>>>>>
>>>>>> No more seriously though, it sure looks like a bug. 0x18 is a perfectly
>>>>>> legal UTF-8 encoding of the unicode character U+0018. Every US-ASCII
>>>>>> character is encoded as itself in UTF-8 and the first 128 Unicode code
>>>>>> points are exactly US-ASCII.
>>>>>>
>>>>>
>>>>> Works for me:
>>>>>
>>>>> % svnadmin create r
>>>>> % mv =( printf 'K 1\nx\nV 1\n\030\nEND\n' > 0 ) r/db/revprops/0/0
>>>>> mv: try to overwrite `r/db/revprops/0/0', overriding mode 0444 (r--r--r--)? y
>>>>
>>>> That last command should have been (equivalently, but without
>>>> munging internals):
>>>>
>>>> % ln -s =true r/hooks/pre-revprop-change
>>>> % svn ps -F =(printf '\030') --revprop -r0 x file://$PWD/r
>>>> property 'x' set on repository revision 0
>>>>
>>>
>>> It still works when I use "svn:log" rather than "x" as the property
>>> name.
>>>
>>> Daniel
>>
>>
>> Thanks for your first diagnostic,
>>
>> I don't have all the details, the issue is part of an FreeBSD PR where this happened to a user.
>> Since I have the same repository in sync (but with subversion 1.7.10) I've taken a look into the file which breaks the sync for the user with subversion 1.8.
>>
>> It is the following FreeBSD PR:
>> http://www.freebsd.org/cgi/query-pr.cgi?pr=179760
>>
>
> So it indeed is FreeBSD - I thought the revnums looked familiar.
>
> I'll note the path of least resistance to getting svnsync to work is to
> ask someone with access to edit the log message on svn.freebsd.org.
>
> Daniel
>
>> Additional I've ask the OP to join the users@ list so you can get more details from first hand.
>
Hi Daniel,
it turns out the OP was not using subversion from ports and the original apache subversion 1.7.x/1.8 has not this issue.
Sorry for the noise and thanks for your help!
--
Regards,
olli
Re: svnsync 1.8.0 fails if there is a non-printable characters in
the commit message
Posted by Daniel Shahaf <da...@elego.de>.
olli hauer wrote on Tue, Jun 25, 2013 at 17:06:35 +0200:
> On 2013-06-25 16:02, Daniel Shahaf wrote:
> > Daniel Shahaf wrote on Tue, Jun 25, 2013 at 16:55:24 +0300:
> >> Daniel Shahaf wrote on Tue, Jun 25, 2013 at 13:50:58 +0000:
> >>> On Mon, Jun 24, 2013 at 09:47:17PM +0200, Ben Smith-Mannschott wrote:
> >>>> 0x18 is ^X, the ASCII control code for CANCEL. Seems to be working as
> >>>> designed. ;-)
> >>>>
> >>>> No more seriously though, it sure looks like a bug. 0x18 is a perfectly
> >>>> legal UTF-8 encoding of the unicode character U+0018. Every US-ASCII
> >>>> character is encoded as itself in UTF-8 and the first 128 Unicode code
> >>>> points are exactly US-ASCII.
> >>>>
> >>>
> >>> Works for me:
> >>>
> >>> % svnadmin create r
> >>> % mv =( printf 'K 1\nx\nV 1\n\030\nEND\n' > 0 ) r/db/revprops/0/0
> >>> mv: try to overwrite `r/db/revprops/0/0', overriding mode 0444 (r--r--r--)? y
> >>
> >> That last command should have been (equivalently, but without
> >> munging internals):
> >>
> >> % ln -s =true r/hooks/pre-revprop-change
> >> % svn ps -F =(printf '\030') --revprop -r0 x file://$PWD/r
> >> property 'x' set on repository revision 0
> >>
> >
> > It still works when I use "svn:log" rather than "x" as the property
> > name.
> >
> > Daniel
>
>
> Thanks for your first diagnostic,
>
> I don't have all the details, the issue is part of an FreeBSD PR where this happened to a user.
> Since I have the same repository in sync (but with subversion 1.7.10) I've taken a look into the file which breaks the sync for the user with subversion 1.8.
>
> It is the following FreeBSD PR:
> http://www.freebsd.org/cgi/query-pr.cgi?pr=179760
>
So it indeed is FreeBSD - I thought the revnums looked familiar.
I'll note the path of least resistance to getting svnsync to work is to
ask someone with access to edit the log message on svn.freebsd.org.
Daniel
> Additional I've ask the OP to join the users@ list so you can get more details from first hand.
Re: svnsync 1.8.0 fails if there is a non-printable characters in
the commit message
Posted by olli hauer <oh...@gmx.de>.
On 2013-06-25 16:02, Daniel Shahaf wrote:
> Daniel Shahaf wrote on Tue, Jun 25, 2013 at 16:55:24 +0300:
>> Daniel Shahaf wrote on Tue, Jun 25, 2013 at 13:50:58 +0000:
>>> On Mon, Jun 24, 2013 at 09:47:17PM +0200, Ben Smith-Mannschott wrote:
>>>> 0x18 is ^X, the ASCII control code for CANCEL. Seems to be working as
>>>> designed. ;-)
>>>>
>>>> No more seriously though, it sure looks like a bug. 0x18 is a perfectly
>>>> legal UTF-8 encoding of the unicode character U+0018. Every US-ASCII
>>>> character is encoded as itself in UTF-8 and the first 128 Unicode code
>>>> points are exactly US-ASCII.
>>>>
>>>
>>> Works for me:
>>>
>>> % svnadmin create r
>>> % mv =( printf 'K 1\nx\nV 1\n\030\nEND\n' > 0 ) r/db/revprops/0/0
>>> mv: try to overwrite `r/db/revprops/0/0', overriding mode 0444 (r--r--r--)? y
>>
>> That last command should have been (equivalently, but without
>> munging internals):
>>
>> % ln -s =true r/hooks/pre-revprop-change
>> % svn ps -F =(printf '\030') --revprop -r0 x file://$PWD/r
>> property 'x' set on repository revision 0
>>
>
> It still works when I use "svn:log" rather than "x" as the property
> name.
>
> Daniel
Thanks for your first diagnostic,
I don't have all the details, the issue is part of an FreeBSD PR where this happened to a user.
Since I have the same repository in sync (but with subversion 1.7.10) I've taken a look into the file which breaks the sync for the user with subversion 1.8.
It is the following FreeBSD PR:
http://www.freebsd.org/cgi/query-pr.cgi?pr=179760
Additional I've ask the OP to join the users@ list so you can get more details from first hand.
--
Thanks,
olli
Re: svnsync 1.8.0 fails if there is a non-printable characters in
the commit message
Posted by Daniel Shahaf <da...@apache.org>.
Daniel Shahaf wrote on Tue, Jun 25, 2013 at 16:55:24 +0300:
> Daniel Shahaf wrote on Tue, Jun 25, 2013 at 13:50:58 +0000:
> > On Mon, Jun 24, 2013 at 09:47:17PM +0200, Ben Smith-Mannschott wrote:
> > > 0x18 is ^X, the ASCII control code for CANCEL. Seems to be working as
> > > designed. ;-)
> > >
> > > No more seriously though, it sure looks like a bug. 0x18 is a perfectly
> > > legal UTF-8 encoding of the unicode character U+0018. Every US-ASCII
> > > character is encoded as itself in UTF-8 and the first 128 Unicode code
> > > points are exactly US-ASCII.
> > >
> >
> > Works for me:
> >
> > % svnadmin create r
> > % mv =( printf 'K 1\nx\nV 1\n\030\nEND\n' > 0 ) r/db/revprops/0/0
> > mv: try to overwrite `r/db/revprops/0/0', overriding mode 0444 (r--r--r--)? y
>
> That last command should have been (equivalently, but without
> munging internals):
>
> % ln -s =true r/hooks/pre-revprop-change
> % svn ps -F =(printf '\030') --revprop -r0 x file://$PWD/r
> property 'x' set on repository revision 0
>
It still works when I use "svn:log" rather than "x" as the property
name.
Daniel
> > % svnadmin create r2
> > % ln -s =true r2/hooks/pre-revprop-change
> > % svnsync init file://$PWD/r2 file://$PWD/r
> > Copied properties for revision 0.
> > % svnsync copy-revprops file://$PWD/r2 file://$PWD/r
> > Copied properties for revision 0.
> > % tail -5 r2/db/revprops/0/0 | xxd
> > 0000000: 4b20 310a 780a 5620 310a 180a 454e 440a K 1.x.V 1...END.
> >
> > Did you pass --source-prop-encoding ?
> >
> > Daniel
> >
> > > // Ben
> > >
> > >
> > > On Mon, Jun 24, 2013 at 7:30 PM, olli hauer <oh...@gmx.de> wrote:
> > >
> > > > svnsync 1.8.0 fails if there is a non-printable characters in the commit
> > > > message
> > > >
> > > > Error message:
> > > > svnsync: E000022: Safe data 'MFC r251249, r251251, r251252, r' was
> > > > followed by non-ASCII byte 24: unable to convert to/from UTF-8
> > > >
> > > > No issue with svnsync 1.7.10 (neon based)
> > > >
> > > > Parts of the file (synced with svnsync 1.7.10)
> > > >
> > > > $> cat $repo/db/revprops/251/251614
> > > > ...
> > > > MFC r251249, r251251, r251252, r2512, r251254 and r251515:
> > > >
> > > >
> > > > $> more $repo/db/revprops/251/251614
> > > > ...
> > > > MFC r251249, r251251, r251252, r^X2512, r251254 and r251515:
> > > >
> > > >
> > > > $> hexdump -C $repo/db/revprops/251/251614
> > > > ...
> > > > 00000070 31 2c 20 72 32 35 31 32 35 32 2c 20 72 18 32 35 |1, r251252,
> > > > r.25|
> > > >
> > > > Culprit is the hex(18) sign.
> > > >
> > > >
> > > > --
> > > > olli
> > > >
Re: svnsync 1.8.0 fails if there is a non-printable characters in
the commit message
Posted by Daniel Shahaf <da...@apache.org>.
Daniel Shahaf wrote on Tue, Jun 25, 2013 at 13:50:58 +0000:
> On Mon, Jun 24, 2013 at 09:47:17PM +0200, Ben Smith-Mannschott wrote:
> > 0x18 is ^X, the ASCII control code for CANCEL. Seems to be working as
> > designed. ;-)
> >
> > No more seriously though, it sure looks like a bug. 0x18 is a perfectly
> > legal UTF-8 encoding of the unicode character U+0018. Every US-ASCII
> > character is encoded as itself in UTF-8 and the first 128 Unicode code
> > points are exactly US-ASCII.
> >
>
> Works for me:
>
> % svnadmin create r
> % mv =( printf 'K 1\nx\nV 1\n\030\nEND\n' > 0 ) r/db/revprops/0/0
> mv: try to overwrite `r/db/revprops/0/0', overriding mode 0444 (r--r--r--)? y
That last command should have been (equivalently, but without
munging internals):
% ln -s =true r/hooks/pre-revprop-change
% svn ps -F =(printf '\030') --revprop -r0 x file://$PWD/r
property 'x' set on repository revision 0
> % svnadmin create r2
> % ln -s =true r2/hooks/pre-revprop-change
> % svnsync init file://$PWD/r2 file://$PWD/r
> Copied properties for revision 0.
> % svnsync copy-revprops file://$PWD/r2 file://$PWD/r
> Copied properties for revision 0.
> % tail -5 r2/db/revprops/0/0 | xxd
> 0000000: 4b20 310a 780a 5620 310a 180a 454e 440a K 1.x.V 1...END.
>
> Did you pass --source-prop-encoding ?
>
> Daniel
>
> > // Ben
> >
> >
> > On Mon, Jun 24, 2013 at 7:30 PM, olli hauer <oh...@gmx.de> wrote:
> >
> > > svnsync 1.8.0 fails if there is a non-printable characters in the commit
> > > message
> > >
> > > Error message:
> > > svnsync: E000022: Safe data 'MFC r251249, r251251, r251252, r' was
> > > followed by non-ASCII byte 24: unable to convert to/from UTF-8
> > >
> > > No issue with svnsync 1.7.10 (neon based)
> > >
> > > Parts of the file (synced with svnsync 1.7.10)
> > >
> > > $> cat $repo/db/revprops/251/251614
> > > ...
> > > MFC r251249, r251251, r251252, r2512, r251254 and r251515:
> > >
> > >
> > > $> more $repo/db/revprops/251/251614
> > > ...
> > > MFC r251249, r251251, r251252, r^X2512, r251254 and r251515:
> > >
> > >
> > > $> hexdump -C $repo/db/revprops/251/251614
> > > ...
> > > 00000070 31 2c 20 72 32 35 31 32 35 32 2c 20 72 18 32 35 |1, r251252,
> > > r.25|
> > >
> > > Culprit is the hex(18) sign.
> > >
> > >
> > > --
> > > olli
> > >
Re: svnsync 1.8.0 fails if there is a non-printable characters in
the commit message
Posted by Daniel Shahaf <da...@apache.org>.
On Mon, Jun 24, 2013 at 09:47:17PM +0200, Ben Smith-Mannschott wrote:
> 0x18 is ^X, the ASCII control code for CANCEL. Seems to be working as
> designed. ;-)
>
> No more seriously though, it sure looks like a bug. 0x18 is a perfectly
> legal UTF-8 encoding of the unicode character U+0018. Every US-ASCII
> character is encoded as itself in UTF-8 and the first 128 Unicode code
> points are exactly US-ASCII.
>
Works for me:
% svnadmin create r
% mv =( printf 'K 1\nx\nV 1\n\030\nEND\n' > 0 ) r/db/revprops/0/0
mv: try to overwrite `r/db/revprops/0/0', overriding mode 0444 (r--r--r--)? y
% svnadmin create r2
% ln -s =true r2/hooks/pre-revprop-change
% svnsync init file://$PWD/r2 file://$PWD/r
Copied properties for revision 0.
% svnsync copy-revprops file://$PWD/r2 file://$PWD/r
Copied properties for revision 0.
% tail -5 r2/db/revprops/0/0 | xxd
0000000: 4b20 310a 780a 5620 310a 180a 454e 440a K 1.x.V 1...END.
Did you pass --source-prop-encoding ?
Daniel
> // Ben
>
>
> On Mon, Jun 24, 2013 at 7:30 PM, olli hauer <oh...@gmx.de> wrote:
>
> > svnsync 1.8.0 fails if there is a non-printable characters in the commit
> > message
> >
> > Error message:
> > svnsync: E000022: Safe data 'MFC r251249, r251251, r251252, r' was
> > followed by non-ASCII byte 24: unable to convert to/from UTF-8
> >
> > No issue with svnsync 1.7.10 (neon based)
> >
> > Parts of the file (synced with svnsync 1.7.10)
> >
> > $> cat $repo/db/revprops/251/251614
> > ...
> > MFC r251249, r251251, r251252, r2512, r251254 and r251515:
> >
> >
> > $> more $repo/db/revprops/251/251614
> > ...
> > MFC r251249, r251251, r251252, r^X2512, r251254 and r251515:
> >
> >
> > $> hexdump -C $repo/db/revprops/251/251614
> > ...
> > 00000070 31 2c 20 72 32 35 31 32 35 32 2c 20 72 18 32 35 |1, r251252,
> > r.25|
> >
> > Culprit is the hex(18) sign.
> >
> >
> > --
> > olli
> >
Re: svnsync 1.8.0 fails if there is a non-printable characters in the
commit message
Posted by Ben Smith-Mannschott <bs...@gmail.com>.
0x18 is ^X, the ASCII control code for CANCEL. Seems to be working as
designed. ;-)
No more seriously though, it sure looks like a bug. 0x18 is a perfectly
legal UTF-8 encoding of the unicode character U+0018. Every US-ASCII
character is encoded as itself in UTF-8 and the first 128 Unicode code
points are exactly US-ASCII.
// Ben
On Mon, Jun 24, 2013 at 7:30 PM, olli hauer <oh...@gmx.de> wrote:
> svnsync 1.8.0 fails if there is a non-printable characters in the commit
> message
>
> Error message:
> svnsync: E000022: Safe data 'MFC r251249, r251251, r251252, r' was
> followed by non-ASCII byte 24: unable to convert to/from UTF-8
>
> No issue with svnsync 1.7.10 (neon based)
>
> Parts of the file (synced with svnsync 1.7.10)
>
> $> cat $repo/db/revprops/251/251614
> ...
> MFC r251249, r251251, r251252, r2512, r251254 and r251515:
>
>
> $> more $repo/db/revprops/251/251614
> ...
> MFC r251249, r251251, r251252, r^X2512, r251254 and r251515:
>
>
> $> hexdump -C $repo/db/revprops/251/251614
> ...
> 00000070 31 2c 20 72 32 35 31 32 35 32 2c 20 72 18 32 35 |1, r251252,
> r.25|
>
> Culprit is the hex(18) sign.
>
>
> --
> olli
>