You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by LiuYan 刘研 <lo...@21cn.com> on 2010/11/17 15:06:19 UTC
Prevent mod_dav_svn REPORT log failure if files under $REPOS/db/revprops/ are not in UTF-8 encoding
Stefan Sperling <stsp <at> elego.de> writes:
>
> On Wed, Nov 17, 2010 at 11:48:07AM +0000, LiuYan 刘研 wrote:
> > I converted CVSNT respositoy to SVN repository some days ago, and config
> > mod_dav_svn in Apache http.conf.
> >
> >
> > When I try 'svn log http://192.168.105.4:8000/svn/cmcc', it shown some logs
but
> > failed/terminated at some revisions with a message like:
> > svn: REPORT of '/svn/cmcc/!svn/bc/1': 200 OK (http://192.168.105.4:8000)
> >
> >
> > I try to the svn:// protocol to get log of the failed revisions, it can get
the
> > log, but the log is not human-readable like the following:
> > svn log svn://192.168.105.4 -r 1
> > ------------------------------------------------------------------------
> > r1 | (no author) | 2005-11-10 17:28:50 +0800 (星期四, 10 十一月 2005) | 2
lines
> >
> > Standard project directories initialized by cvs2svn.?\211?\201 cvs2svn
> > ?\179?\245?\202?\188?\187?\175?\181?\196?\177?\234?\215?\188?\207?\238?\196?
> > \191?\206?\196?\188?\254?\188?\208
> > ------------------------------------------------------------------------
> >
> >
> > It seems a charset encoding problem of 'revprops' files because I
> > set 'ctx.initial_project_commit_message' cvs2svn options config file to a
> > string contains Chinese characters, and cvs2svn options config file itself
is
> > encode in GBK, and it seems cvs2svn does not converted the charset encoding
of
> > initial_project_commit_message to UTF-8.
> >
> >
> > Does this encoding issue can caused mod_dav_svn REPORT failed?
>
> The revprops should be encoded in UTF-8.
>
> You could write each revprop to a temporary file (get it with 'svn
> propget --revprop'), and convert that file to UTF-8 encoding.
> Then set the revprop again e.g. by using the svnadmin setrevprop command.
>
> Stefan
>
Thank you Stefan! And sorry for my poor English.
I've already converted these revprops files by operating them directly (I know
it's a dangerous way) via a little script, now 'svn log
http://svnserver/svnrepos/svnrepo' works fine.
My orginal idea is:
If this "svn: REPORT of " error is returned by mod_dav_svn, can it act friendly
as the same as svnserve ?
I mean, if the revprops files are not in UTF-8 encoding, don't return REPORT
Failed to client side, but still return the malformed log instead, so that the
end user can notice IT'S WRONG, and can easily figure out WHY IT'S WRONG.
Liu Yan
Re: Prevent mod_dav_svn REPORT log failure if files
under?$REPOS/db/revprops/ are not in UTF-8 encoding
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
DON'T DO IT THIS WAY. Follow Stefan's advice upthread.
Re: Prevent mod_dav_svn REPORT log failure if files under $REPOS/db/revprops/ are not in UTF-8 encoding
Posted by LiuYan 刘研 <lo...@21cn.com>.
Daniel Shahaf <d.s <at> daniel.shahaf.name> writes:
>
> LiuYan 刘研 wrote on Thu, Nov 18, 2010 at 02:53:37 +0000:
> > Daniel Shahaf <d.s <at> daniel.shahaf.name> writes:
> >
> > >
> > > Stefan Sperling wrote on Wed, Nov 17, 2010 at 18:13:44 +0100:
> > > > On Wed, Nov 17, 2010 at 03:06:19PM +0000, LiuYan 刘研 wrote:
> > > > > I mean, if the revprops files are not in UTF-8 encoding, don't return
> > REPORT
> > >
> > > Small correction: it's meaningless to talk about the encoding of the
> > > revprop files; it's only meaningful to talk about the encoding of the
> > > value of a given property.
> > >
> > > (At the revprop files level, the values are binary, and the rest of the
> > > data in those files is always ASCII.)
> > >
> > >
> >
> > You're right Daniel, but in such situation, these revprop files can be
treated
> > as readable text files:
>
> This is simply not true: if you apply 'iconv -f latin1 -t utf-8' to
> a revprop file, you will CORRUPT that revprop file.
>
>
You're right Daniel, simply apply an 'iconv' operation to a revprop file will
CORRUPT it, there's data length value should be changed too.
So I wrote a small script to do the conversion as I mentioned at
http://article.gmane.org/gmane.comp.version-control.subversion.user/101383
The script do the following operations:
1. find out the affected revprop files
2. change the svn:log value length from "V 85" to "V 98"
3. change/convert the svn:log value to a UTF-8 encoded string
Here's the small script, and be aware of this script file is in GBK encoding.
Administrator@CMTEL-SVR-HR-DB /cygdrive/d/SVNRepositories/repos/cmcc/db
$ cat fix-cvs2svn.sh
IFS=$'\n'
grep -i -r -n "Standard project directories initialized by cvs2svn" revprops/*
| cut -d ":" -f 1 > affected_files.txt
#grep result sample
#0/1:8:Standard project directories initialized by cvs2svn.由 cvs2svn
#0/133:8:Standard project directories initialized by cvs2svn.由 cvs2svn
for file in `cat affected_files.txt`
do
echo $file
#${file:9}: strip out 'revprops/'
dest_file="fix/${file:9}"
cp --force --preserve --verbose $file "fix-backup/${file:9}"
gawk 'FNR==7 {print "V 98"} FNR==8{print "Standard project directories
initialized by cvs2svn."} FNR==9{print "由 cvs2svn 初始化的标准项目文件夹"} FNR<7
|| FNR==10 {print $0}' $file | iconv --from GBK --to UTF-8 > "$dest_file"
done
Re: Prevent mod_dav_svn REPORT log failure if files under
$REPOS/db/revprops/ are not in UTF-8 encoding
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
LiuYan 刘研 wrote on Thu, Nov 18, 2010 at 02:53:37 +0000:
> Daniel Shahaf <d.s <at> daniel.shahaf.name> writes:
>
> >
> > Stefan Sperling wrote on Wed, Nov 17, 2010 at 18:13:44 +0100:
> > > On Wed, Nov 17, 2010 at 03:06:19PM +0000, LiuYan 刘研 wrote:
> > > > I mean, if the revprops files are not in UTF-8 encoding, don't return
> REPORT
> >
> > Small correction: it's meaningless to talk about the encoding of the
> > revprop files; it's only meaningful to talk about the encoding of the
> > value of a given property.
> >
> > (At the revprop files level, the values are binary, and the rest of the
> > data in those files is always ASCII.)
> >
> >
>
> You're right Daniel, but in such situation, these revprop files can be treated
> as readable text files:
This is simply not true: if you apply 'iconv -f latin1 -t utf-8' to
a revprop file, you will CORRUPT that revprop file.
Re: Prevent mod_dav_svn REPORT log failure if files under $REPOS/db/revprops/ are not in UTF-8 encoding
Posted by LiuYan 刘研 <lo...@21cn.com>.
Daniel Shahaf <d.s <at> daniel.shahaf.name> writes:
>
> Stefan Sperling wrote on Wed, Nov 17, 2010 at 18:13:44 +0100:
> > On Wed, Nov 17, 2010 at 03:06:19PM +0000, LiuYan 刘研 wrote:
> > > I mean, if the revprops files are not in UTF-8 encoding, don't return
REPORT
>
> Small correction: it's meaningless to talk about the encoding of the
> revprop files; it's only meaningful to talk about the encoding of the
> value of a given property.
>
> (At the revprop files level, the values are binary, and the rest of the
> data in those files is always ASCII.)
>
>
You're right Daniel, but in such situation, these revprop files can be treated
as readable text files:
Administrator@CMTEL-SVR-HR-DB /cygdrive/d/SVNRepositories/repos/cmcc/db
$ file revprops/0/1 fix-backup/0/1
revprops/0/1: UTF-8 Unicode text
fix-backup/0/1: ISO-8859 text
Thanks for the correction Daniel! (^_^)
And, I've post this to buglist here:
http://subversion.tigris.org/issues/show_bug.cgi?id=3753
Wish it can be accepted and fixed.
Liu Yan
Re: Prevent mod_dav_svn REPORT log failure if files under
$REPOS/db/revprops/ are not in UTF-8 encoding
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Stefan Sperling wrote on Wed, Nov 17, 2010 at 18:13:44 +0100:
> On Wed, Nov 17, 2010 at 03:06:19PM +0000, LiuYan 刘研 wrote:
> > I mean, if the revprops files are not in UTF-8 encoding, don't return REPORT
Small correction: it's meaningless to talk about the encoding of the
revprop files; it's only meaningful to talk about the encoding of the
value of a given property.
(At the revprop files level, the values are binary, and the rest of the
data in those files is always ASCII.)
Re: Prevent mod_dav_svn REPORT log failure if files under
$REPOS/db/revprops/ are not in UTF-8 encoding
Posted by Stefan Sperling <st...@elego.de>.
On Wed, Nov 17, 2010 at 03:06:19PM +0000, LiuYan 刘研 wrote:
> My orginal idea is:
> If this "svn: REPORT of " error is returned by mod_dav_svn, can it act friendly
> as the same as svnserve ?
> I mean, if the revprops files are not in UTF-8 encoding, don't return REPORT
> Failed to client side, but still return the malformed log instead, so that the
> end user can notice IT'S WRONG, and can easily figure out WHY IT'S WRONG.
>
I agree that mod_dav_svn's behaviour should match the behaviour of
svnserve in this case. I'm not sure why that isn't the case.
It's probably due to some slight difference in implementation.
If you want to file an issue about this in our issue tracker, feel free
to do so: http://subversion.tigris.org/issue-tracker.html
Thanks,
Stefan