You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Thomas STEININGER <ts...@racon-linz.at> on 2011/02/18 13:53:43 UTC
problem with mutated vowel in log-message-contents
i need a tip to solve my problem with log-messages in our
subversion-repository. (see the mail-conversation with the tortoisesvn
team below).
there are message that contain mutated vowel and i need to find all them
and correct them.
f.e. i want to replace 'übergabe' by 'uebergabe' or if this is to
difficult only by '_bergabe'.
can anybode give me tips / commands to do that or probably there is an
tool (because the problem was already solved for another user)?
thanks for help
-thomas
Mails with TortoiseSvn-Team:
>>>>
>>>> Hello i have an really *big problem *using tortoise-svn.
>>>>
>>>> The same problem has - by the way - eclipse-subversive with
>>>> javahl-connector, but not with svnkit-connector.
>>>>
>>>> if i try to show the logs of ein entry/file/.. in tortoisesvn where
the
>>>> log-messages contain a mutated vowel (german: deutscher umlaut)
>>>> then it brings an failure msg:
>>>>
>>>> and then it shows
>>>>
>>>> instead of the messages.
>>>>
>>>> can anybody help?
>>>> Thomas
>>>
>>>log messages must be in utf8 encoding. That's a requirement of the svn
>>>library. If they're not, then anything can happen. For example if you
>>>access the repository with http(s), then those messages have to be sent
>>>in an xml response, and non-utf8 strings result in invalid xml.
>>>
>>>When you commit with TSVN, TSVN ensures the encoding. But it might be
>>>that other svn clients have a bug and don't properly encode the log
>>>messages. In that case you have to fix the repository on the server
>>>directly using the command line tools (svnadmin).
>>>
>>>Stefan
>>we already want to correct the log-messages, but we did not find an way
to
>>get all the log messages with failures and then to correct them.
>>is there a known way - a tip to help us?
>>
>>thanks
>>thomas
>>
>You have to ask for help with this on the Subversion users list. This is
>an issue with the server/repository, not with TSVN and you'll get much
>more help about this over there.
>
>Stefan
>
Der Austausch von Nachrichten mit o.a. Absender via e-mail dient ausschließlich Informationszwecken. Rechtsgeschäftliche Erklärungen dürfen über dieses Medium nicht ausgetauscht werden.
Correspondence with a.m. sender via e-mail is only for information purposes. This medium is not to be used for the exchange of legally-binding communications.
Re: problem with mutated vowel in log-message-contents
Posted by Stephen Connolly <st...@gmail.com>.
bash will handle the multiline in my case (try and see) but your example
will handle if log contains "
- Stephen
---
Sent from my Android phone, so random spelling mistakes, random nonsense
words and other nonsense are a direct result of using swype to type on the
screen
On 21 Feb 2011 07:18, "Lorenz" <lo...@yahoo.com> wrote:
> Stephen Connolly wrote:
>>unix shell scripting could solved it for you
>>
>>bash
>>for rev in $(svn log ... | sed -n -e "..."); do svn ps --revprop svn:log
>>"$(svn pg svn:log -r $rev | sed -e "s/oldstring/newstring/g;")" ... ; done
>
> svn:log may be multi-line ...
>
>
> ... but
>
>> svn pg svn:log --revprop -r $rev $url | sed ... | svn ps svn:log
--revprop -r $rev --username ... --password ... --file - $url
>
> should do it (... == fill in the blanks 8-)
> --
>
> Lorenz
>
Re: problem with mutated vowel in log-message-contents
Posted by Lorenz <lo...@yahoo.com>.
Stephen Connolly wrote:
>unix shell scripting could solved it for you
>
>bash
>for rev in $(svn log ... | sed -n -e "..."); do svn ps --revprop svn:log
>"$(svn pg svn:log -r $rev | sed -e "s/oldstring/newstring/g;")" ... ; done
svn:log may be multi-line ...
... but
> svn pg svn:log --revprop -r $rev $url | sed ... | svn ps svn:log --revprop -r $rev --username ... --password ... --file - $url
should do it (... == fill in the blanks 8-)
--
Lorenz
Re: Re: Antwort: Re: problem with mutated vowel in
log-message-contents
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Tony Sweeney wrote on Tue, Feb 22, 2011 at 11:52:09 -0000:
> Which is why you should probably use iconv(1) or any of the APIs listed here:
>
> http://www.unicodetools.com/
>
+1 to iconv.
In reality, the editor-cmd.sh script will check whether the log message
(which, again, will be argv[1]) is in UTF-8 already or not, and run it
through 'iconv -f latin1 -t utf-8' if not.
It won't be long, but it won't be a one-liner.
> instead of dicking around with perl or sed and hard coded hand crafted single character mappings. There's potentially a lot more than just u-umlaut to worry about.
Thanks,
Daniel
RE: Re: Antwort: Re: problem with mutated vowel in log-message-contents
Posted by Tony Sweeney <ts...@omnifone.com>.
> -----Original Message-----
> From: Daniel Shahaf [mailto:d.s@daniel.shahaf.name]
> Sent: 22 February 2011 09:34
> To: Johan Corveleyn
> Cc: Thomas STEININGER; Stephen Connolly; users@subversion.apache.org
> Subject: Re: Re: Antwort: Re: problem with mutated vowel in
> log-message-contents
>
> Daniel Shahaf wrote on Tue, Feb 22, 2011 at 11:26:25 +0200:
> > Johan Corveleyn wrote on Tue, Feb 22, 2011 at 09:43:25 +0100:
> > > So, all that being said, what Daniel means is that you
> could apply
> > > something like:
> > >
> > > svn propedit --revprop -r $REV --editor-cmd 'perl -pi -e
> > > "s/\\xfc/\\xc3\\xbc/g"'
> > >
> > > to all revisions (REV) that need to be corrected (either
> a list that
> > > you make up manually, or something automated with "svn propget
> > > --revprop" combined with "sed", or something similar ...).
> >
> > By the way, please don't consider this a generic solution. It's a
> > *shortcut*, which is probably okay for ü, but WILL corrupt your log
> > messages if you adapt it for §.
>
> ... because the latin1 byte sequence for § is part of some
> UTF-8 byte sequences.
Which is why you should probably use iconv(1) or any of the APIs listed here:
http://www.unicodetools.com/
instead of dicking around with perl or sed and hard coded hand crafted single character mappings. There's potentially a lot more than just u-umlaut to worry about.
Tony.
>
> (I'm assuming that at least some log messages are already in UTF-8.)
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit
> http://www.messagelabs.com/email
> ______________________________________________________________________
>
Re: Re: Antwort: Re: problem with mutated vowel in
log-message-contents
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Daniel Shahaf wrote on Tue, Feb 22, 2011 at 11:26:25 +0200:
> Johan Corveleyn wrote on Tue, Feb 22, 2011 at 09:43:25 +0100:
> > So, all that being said, what Daniel means is that you could apply
> > something like:
> >
> > svn propedit --revprop -r $REV --editor-cmd 'perl -pi -e
> > "s/\\xfc/\\xc3\\xbc/g"'
> >
> > to all revisions (REV) that need to be corrected (either a list that
> > you make up manually, or something automated with "svn propget
> > --revprop" combined with "sed", or something similar ...).
>
> By the way, please don't consider this a generic solution. It's
> a *shortcut*, which is probably okay for ü, but WILL corrupt your log
> messages if you adapt it for §.
... because the latin1 byte sequence for § is part of some UTF-8
byte sequences.
(I'm assuming that at least some log messages are already in UTF-8.)
Re: Re: Antwort: Re: problem with mutated vowel in
log-message-contents
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Johan Corveleyn wrote on Tue, Feb 22, 2011 at 09:43:25 +0100:
> So, all that being said, what Daniel means is that you could apply
> something like:
>
> svn propedit --revprop -r $REV --editor-cmd 'perl -pi -e
> "s/\\xfc/\\xc3\\xbc/g"'
>
> to all revisions (REV) that need to be corrected (either a list that
> you make up manually, or something automated with "svn propget
> --revprop" combined with "sed", or something similar ...).
By the way, please don't consider this a generic solution. It's
a *shortcut*, which is probably okay for ü, but WILL corrupt your log
messages if you adapt it for §.
Re: Re: Antwort: Re: problem with mutated vowel in log-message-contents
Posted by Johan Corveleyn <jc...@gmail.com>.
[ Please do not top-post on this list, i.e. put your reply inline or
below the thing you're replying to. Also, if possible, use plain-text
email. More below ... ]
On Tue, Feb 22, 2011 at 8:27 AM, Thomas STEININGER
<ts...@racon-linz.at> wrote:
> Daniel Shahaf <d....@daniel.shahaf.name>
> > Thomas STEININGER wrote on Tue, Feb 22, 2011 at 07:39:49 +0100:
> > > do i really understand, that i have to execute this:
> > > propedit --editor-cmd 'sed --flags'
> > > on a file? on all urls that are in my svn-repository?
> > > or how you mean?
> > >
> >
> > On all revisions.
> >
> > What you'll want is to write a script that takes a filename in argv[1]
> > and converts that file (in-place) from latin1 to utf8. Then you can use
> > that file as --editor-cmd.
> >
> > Something like
> > % svn propedit --revprop -r $REV --editor-cmd 'perl -pi -e "s/\\xfc/\\xc3\\xbc/g"'
> > might just do the trick. (untested)
> >
> Ok - you mean that i start a script that iterates over all files and within over all revisions of the repository
> and execute on it your command.
> But you say 'and converts that file (in-place) from latin1 to utf8' and i have no problem with the file-contents itself.
> The files have correct contents - including mutated vowel.
>
> My Problem is in the logs.
> The log-messages (commit-comments) have mutated vowel and if i try to show the log-comments of some file with Tortoisesvn or Eclipse-subversive with JavaHL-Connector
> i have a error and see nothing.
>
> So i must correct the checkin-comments.
>
> How can i get this done?
>
> Or does i have misunderstood your trick?
Yes, you've misunderstood :-).
The important thing to realize is that log message (aka
checkin-comments) are *revision properties*, which means they are
associated with *revisions* directly, not with the files touched by
that revision. As opposed to *versioned properties*, which are
meta-data associated with files (like svn:eol-style, svn:keywords,
svn:mime-type etc...), and those properties are themselves versioned
(i.e., they can change from one revision to the other). You can find
some more background about this in the svn book:
http://svnbook.red-bean.com/nightly/en/svn.advanced.props.html
This means that you can change a revision property, like "svn:log"
(i.e. the checkin-comment), without specifying a file. Just a
revision, together with the --revprop option, should do the trick.
That is, if the change is allowed by the server, which means that the
"pre-revprop-change" hook should allow the change (by default it
doesn't). See this chapter for some more info about that:
http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.html#svn.reposadmin.maint.setlog
So, all that being said, what Daniel means is that you could apply
something like:
svn propedit --revprop -r $REV --editor-cmd 'perl -pi -e
"s/\\xfc/\\xc3\\xbc/g"'
to all revisions (REV) that need to be corrected (either a list that
you make up manually, or something automated with "svn propget
--revprop" combined with "sed", or something similar ...).
HTH,
--
Johan
Re: Antwort: Re: Antwort: Re: problem with mutated vowel in log-message-contents
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Feb 22, 2011, at 01:27, Thomas STEININGER wrote:
> Ok - you mean that i start a script that iterates over all files and within over all revisions of the repository
> and execute on it your command.
No, just iterate over all revisions. As you've said before and again below, you're not going to be modifying any of the files in your repository, just the log messages of each revision.
> But you say 'and converts that file (in-place) from latin1 to utf8' and i have no problem with the file-contents itself.
> The files have correct contents - including mutated vowel.
The file he's talking about here is a file that will (temporarily) contain the log message of each revision as the script runs.
> My Problem is in the logs.
> The log-messages (commit-comments) have mutated vowel and if i try to show the log-comments of some file with Tortoisesvn or Eclipse-subversive with JavaHL-Connector
> i have a error and see nothing.
>
> So i must correct the checkin-comments.
>
> How can i get this done?
>
> Or does i have misunderstood your trick?
Antwort: Re: Antwort: Re: problem with mutated vowel in log-message-contents
Posted by Thomas STEININGER <ts...@racon-linz.at>.
Ok - you mean that i start a script that iterates over all files and
within over all revisions of the repository
and execute on it your command.
But you say 'and converts that file (in-place) from latin1 to utf8' and i
have no problem with the file-contents itself.
The files have correct contents - including mutated vowel.
My Problem is in the logs.
The log-messages (commit-comments) have mutated vowel and if i try to
show the log-comments of some file with Tortoisesvn or Eclipse-subversive
with JavaHL-Connector
i have a error and see nothing.
So i must correct the checkin-comments.
How can i get this done?
Or does i have misunderstood your trick?
-Thomas
Daniel Shahaf <d....@daniel.shahaf.name>
22.02.2011 07:58
An
Thomas STEININGER <ts...@racon-linz.at>
Kopie
Stephen Connolly <st...@gmail.com>,
users@subversion.apache.org
Thema
Re: Antwort: Re: problem with mutated vowel in log-message-contents
Thomas STEININGER wrote on Tue, Feb 22, 2011 at 07:39:49 +0100:
> do i really understand, that i have to execute this:
> propedit --editor-cmd 'sed --flags'
> on a file? on all urls that are in my svn-repository?
> or how you mean?
>
On all revisions.
What you'll want is to write a script that takes a filename in argv[1]
and converts that file (in-place) from latin1 to utf8. Then you can use
that file as --editor-cmd.
Something like
% svn propedit --revprop -r $REV --editor-cmd 'perl -pi -e
"s/\\xfc/\\xc3\\xbc/g"'
might just do the trick. (untested)
> -Thomas
>
>
>
>
> Daniel Shahaf <d....@daniel.shahaf.name>
> 21.02.2011 19:34
>
> An
> Stephen Connolly <st...@gmail.com>
> Kopie
> Thomas STEININGER <ts...@racon-linz.at>,
users@subversion.apache.org
> Thema
> Re: problem with mutated vowel in log-message-contents
>
>
>
>
>
>
> Stephen Connolly wrote on Fri, Feb 18, 2011 at 14:01:52 +0000:
> > unix shell scripting could solved it for you
> >
> > bash
> > for rev in $(svn log ... | sed -n -e "..."); do svn ps --revprop
svn:log
> > "$(svn pg svn:log -r $rev | sed -e "s/oldstring/newstring/g;")" ... ;
> done
> >
> > I leave the ...'s as an exercise to tgeur reader
>
> Simpler:
> propedit --editor-cmd 'sed --flags'
>
>
>
>
>
> Der Austausch von Nachrichten mit o.a. Absender via e-mail dient
ausschließlich Informationszwecken. Rechtsgeschäftliche Erklärungen dürfen
über dieses Medium nicht ausgetauscht werden.
>
> Correspondence with a.m. sender via e-mail is only for information
purposes. This medium is not to be used for the exchange of
legally-binding communications.
Der Austausch von Nachrichten mit o.a. Absender via e-mail dient ausschließlich Informationszwecken. Rechtsgeschäftliche Erklärungen dürfen über dieses Medium nicht ausgetauscht werden.
Correspondence with a.m. sender via e-mail is only for information purposes. This medium is not to be used for the exchange of legally-binding communications.
Re: Antwort: Re: problem with mutated vowel in log-message-contents
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Thomas STEININGER wrote on Tue, Feb 22, 2011 at 07:39:49 +0100:
> do i really understand, that i have to execute this:
> propedit --editor-cmd 'sed --flags'
> on a file? on all urls that are in my svn-repository?
> or how you mean?
>
On all revisions.
What you'll want is to write a script that takes a filename in argv[1]
and converts that file (in-place) from latin1 to utf8. Then you can use
that file as --editor-cmd.
Something like
% svn propedit --revprop -r $REV --editor-cmd 'perl -pi -e "s/\\xfc/\\xc3\\xbc/g"'
might just do the trick. (untested)
> -Thomas
>
>
>
>
> Daniel Shahaf <d....@daniel.shahaf.name>
> 21.02.2011 19:34
>
> An
> Stephen Connolly <st...@gmail.com>
> Kopie
> Thomas STEININGER <ts...@racon-linz.at>, users@subversion.apache.org
> Thema
> Re: problem with mutated vowel in log-message-contents
>
>
>
>
>
>
> Stephen Connolly wrote on Fri, Feb 18, 2011 at 14:01:52 +0000:
> > unix shell scripting could solved it for you
> >
> > bash
> > for rev in $(svn log ... | sed -n -e "..."); do svn ps --revprop svn:log
> > "$(svn pg svn:log -r $rev | sed -e "s/oldstring/newstring/g;")" ... ;
> done
> >
> > I leave the ...'s as an exercise to tgeur reader
>
> Simpler:
> propedit --editor-cmd 'sed --flags'
>
>
>
>
>
> Der Austausch von Nachrichten mit o.a. Absender via e-mail dient ausschließlich Informationszwecken. Rechtsgeschäftliche Erklärungen dürfen über dieses Medium nicht ausgetauscht werden.
>
> Correspondence with a.m. sender via e-mail is only for information purposes. This medium is not to be used for the exchange of legally-binding communications.
Antwort: Re: problem with mutated vowel in log-message-contents
Posted by Thomas STEININGER <ts...@racon-linz.at>.
do i really understand, that i have to execute this:
propedit --editor-cmd 'sed --flags'
on a file? on all urls that are in my svn-repository?
or how you mean?
-Thomas
Daniel Shahaf <d....@daniel.shahaf.name>
21.02.2011 19:34
An
Stephen Connolly <st...@gmail.com>
Kopie
Thomas STEININGER <ts...@racon-linz.at>, users@subversion.apache.org
Thema
Re: problem with mutated vowel in log-message-contents
Stephen Connolly wrote on Fri, Feb 18, 2011 at 14:01:52 +0000:
> unix shell scripting could solved it for you
>
> bash
> for rev in $(svn log ... | sed -n -e "..."); do svn ps --revprop svn:log
> "$(svn pg svn:log -r $rev | sed -e "s/oldstring/newstring/g;")" ... ;
done
>
> I leave the ...'s as an exercise to tgeur reader
Simpler:
propedit --editor-cmd 'sed --flags'
Der Austausch von Nachrichten mit o.a. Absender via e-mail dient ausschließlich Informationszwecken. Rechtsgeschäftliche Erklärungen dürfen über dieses Medium nicht ausgetauscht werden.
Correspondence with a.m. sender via e-mail is only for information purposes. This medium is not to be used for the exchange of legally-binding communications.
Re: problem with mutated vowel in log-message-contents
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Stephen Connolly wrote on Fri, Feb 18, 2011 at 14:01:52 +0000:
> unix shell scripting could solved it for you
>
> bash
> for rev in $(svn log ... | sed -n -e "..."); do svn ps --revprop svn:log
> "$(svn pg svn:log -r $rev | sed -e "s/oldstring/newstring/g;")" ... ; done
>
> I leave the ...'s as an exercise to tgeur reader
Simpler:
propedit --editor-cmd 'sed --flags'
Re: problem with mutated vowel in log-message-contents
Posted by Stephen Connolly <st...@gmail.com>.
unix shell scripting could solved it for you
bash
for rev in $(svn log ... | sed -n -e "..."); do svn ps --revprop svn:log
"$(svn pg svn:log -r $rev | sed -e "s/oldstring/newstring/g;")" ... ; done
I leave the ...'s as an exercise to tgeur reader
- Stephen
---
Sent from my Android phone, so random spelling mistakes, random nonsense
words and other nonsense are a direct result of using swype to type on the
screen
On 18 Feb 2011 13:16, "Thomas STEININGER" <ts...@racon-linz.at> wrote:
> i need a tip to solve my problem with log-messages in our
> subversion-repository. (see the mail-conversation with the tortoisesvn
> team below).
> there are message that contain mutated vowel and i need to find all them
> and correct them.
> f.e. i want to replace 'übergabe' by 'uebergabe' or if this is to
> difficult only by '_bergabe'.
>
> can anybode give me tips / commands to do that or probably there is an
> tool (because the problem was already solved for another user)?
>
> thanks for help
> -thomas
>
> Mails with TortoiseSvn-Team:
>>>>>
>>>>> Hello i have an really *big problem *using tortoise-svn.
>>>>>
>>>>> The same problem has - by the way - eclipse-subversive with
>>>>> javahl-connector, but not with svnkit-connector.
>>>>>
>>>>> if i try to show the logs of ein entry/file/.. in tortoisesvn where
> the
>>>>> log-messages contain a mutated vowel (german: deutscher umlaut)
>>>>> then it brings an failure msg:
>>>>>
>>>>> and then it shows
>>>>>
>>>>> instead of the messages.
>>>>>
>>>>> can anybody help?
>>>>> Thomas
>>>>
>>>>log messages must be in utf8 encoding. That's a requirement of the svn
>>>>library. If they're not, then anything can happen. For example if you
>>>>access the repository with http(s), then those messages have to be sent
>
>>>>in an xml response, and non-utf8 strings result in invalid xml.
>>>>
>>>>When you commit with TSVN, TSVN ensures the encoding. But it might be
>>>>that other svn clients have a bug and don't properly encode the log
>>>>messages. In that case you have to fix the repository on the server
>>>>directly using the command line tools (svnadmin).
>>>>
>>>>Stefan
>>>we already want to correct the log-messages, but we did not find an way
> to
>>>get all the log messages with failures and then to correct them.
>>>is there a known way - a tip to help us?
>>>
>>>thanks
>>>thomas
>>>
>>You have to ask for help with this on the Subversion users list. This is
>>an issue with the server/repository, not with TSVN and you'll get much
>>more help about this over there.
>>
>>Stefan
>>
>
>
>
> Der Austausch von Nachrichten mit o.a. Absender via e-mail dient
ausschließlich Informationszwecken. Rechtsgeschäftliche Erklärungen dürfen
über dieses Medium nicht ausgetauscht werden.
>
> Correspondence with a.m. sender via e-mail is only for information
purposes. This medium is not to be used for the exchange of legally-binding
communications.
Re: problem with mutated vowel in log-message-contents
Posted by Thorsten Schöning <ts...@am-soft.de>.
Guten Tag Thomas STEININGER,
am Freitag, 18. Februar 2011 um 13:53 schrieben Sie:
> i need a tip to solve my problem with log-messages in our
> subversion-repository. (see the mail-conversation with the tortoisesvn
> team below).
> there are message that contain mutated vowel and i need to find all them
> and correct them.
> f.e. i want to replace 'übergabe' by 'uebergabe' or if this is to
> difficult only by '_bergabe'.
> can anybode give me tips / commands to do that or probably there is an
> tool (because the problem was already solved for another user)?
First of all you should examine how this could happen and what
exactly is present in the log messages in your repositories, you
should make sure that the log messages contain the wrong bytes for ü
etc. German umlauts are not a problem, normally. Before repairing your
repository I would make sure that it's really damaged.
You should attach the errors you got from TortoiseSVN as well.
Mit freundlichen Grüßen,
Thorsten Schöning
--
Thorsten Schöning
AM-SoFT IT-Systeme - Hameln | Potsdam | Leipzig
Telefon: Potsdam: 0331-743881-0
E-Mail: tschoening@am-soft.de
Web: http://www.am-soft.de
AM-SoFT GmbH IT-Systeme, Konsumhof 1-5, 14482 Potsdam
Amtsgericht Potsdam HRB 21278 P, Geschäftsführer: Andreas Muchow
Re: problem with mutated vowel in log-message-contents
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
I think what you really want is to replace a latin1 ü by a UTF-8 ü
(byte sequence C3 BC).
Thomas STEININGER wrote on Fri, Feb 18, 2011 at 13:53:43 +0100:
> i need a tip to solve my problem with log-messages in our
> subversion-repository. (see the mail-conversation with the tortoisesvn
> team below).
> there are message that contain mutated vowel and i need to find all them
> and correct them.
> f.e. i want to replace 'übergabe' by 'uebergabe' or if this is to
> difficult only by '_bergabe'.
>
> can anybode give me tips / commands to do that or probably there is an
> tool (because the problem was already solved for another user)?
>
> thanks for help
> -thomas