You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Arfrever Frehtes Taifersar Arahesis <Ar...@GMail.Com> on 2009/03/18 14:54:58 UTC
Re: Translation stuff to do before a release (Was: Re: svn commit:
r36625 - in branches/1.6.x: . subversion/po subversion/svn)
2009-03-18 10:55 Jens Seidel <je...@users.sf.net> napisał(a):
> On Tue, Mar 17, 2009 at 11:13:19PM -0500, Hyrum K. Wright wrote:
>> We are asking you to trust the rules of the community, and if there are
>> inefficiencies, to help us find and correct them. For instance, would it
>> be possible to script the merging of translations to the release branch?
>
> This code exists already: It's mentioned in TRANSLATING:
> "Maintenance on branches". This is not optimal as it mentions e.g. the
> po-merge.py script but doesn't use the full paths in the example command.
> Normally it is something I would just change but I always hesitated because
> I don't have permission (OK, it's an obvious fix, but once I start updating
> I'm not willing to stop until everything is perfect) ...
>
> Merging translations is indeed not trivial but there exist tools such
> as msgmerge (from the gettext package). I don't know why po-merge.py exists
> and suspect it is just a msgmerge replacement for Windows people.
>
> This po-merge.py gave me the last day the error message:
>
> 1.6.x/$ svn cat http://svn.collab.net/repos/svn/trunk/subversion/po/de.po |
> tools/dev/po-merge.py subversion/po/de.po
> Traceback (most recent call last):
> File "tools/dev/po-merge.py", line 171, in <module>
> main(sys.argv)
> File "tools/dev/po-merge.py", line 160, in main
> for m in msgstr:
> TypeError: 'NoneType' object is not iterable
It was fixed some days ago in trunk.
> This needs to be analysed (and the script should be moved into tools/po/).
>
> I used the following workaround (the non-trivial task, but every translator
> should know at least the basic gettext commands) (in 1.6.x/):
>
> $ svn revert subversion/po/de.po
> $ tools/po/po-update.sh de
> $ cp subversion/po/de.po subversion/po/de.po.updated
> $ cp ../trunk/subversion/po/de.po subversion/po/
> $ msgmerge -U subversion/po/de.po subversion/po/de.po.updated
>
> Using the -C (compendia) option may be useful as well, to reuse strings
> available in the release branch but absent in trunk.
>
> All this should be done by po-merge.py and it worked in the past.
>
> The document (TRANSLATING) misses also information about the new environment
> variable SVN_LOCALE_DIR (I tested it with
> export SVN_LOCALE_DIR=/home/jens/local/subversion1.6.old/share/locale/
> and it worked well finding the translations after I moved the installed path
> to ".old"). I also noticed that current i18n tools/scripts
> assumes very old gettext versions. Example:
> tools/po/po-update.sh:
> # In a display of truly bizarre behaviour, msgmerge (at least, the
> # GNU gettext-tools 0.14.6 implementation) inverts the order of obsolete
> # messages every time it is run. Therefore, run it twice, to invert and
> # then re-invert, to minimize spurious diffs.
> $MSGMERGE --sort-by-file --no-wrap --update $i subversion.pot
> $MSGMERGE --sort-by-file --no-wrap --update $i subversion.pot
>
> A single call to $MSGMERGE should be sufficient and I would add the
> --previous option (new since 2(?) years), as this adds information about
> what changed to make a string fuzzy (so that the translation is outdated and
> not used).
I prefer `svn di XX.po | less` in another terminal.
> This --previous option adds special comments which need to
> be supported by some python scripts (because it's a reimplementation of some
> gettext tools).
>
> OK, this needs to be properly explained/tested/... and I should provide a
> patch, I know.
>
> But at least you can add the following to the release preparations:
>
> Bring PO files in the release branch in sync with source (to match all
> strings from source and not only the strings as present many years ago).
pl.po before r36624 intentionally contained some strings present in the future.
> This can be done using
> $ tools/po/po-update.sh
>
> This is useful so that the final tar-ball ships current PO files. This are
> made available to the world of translators by the Subversion repository
> (or just a web frontend) but also by pages such as
> http://www.debian.org/intl/l10n/po/de
>
> And you don't want such pages to contain completely outdated files, right?
I don't see benefit in .po files with bigger number of fuzzy messages.
> This is what I did in
>
> r36624 | jensseidel | 2009-03-17 14:59:34 +0100 (Di, 17. Mär 2009) | 19 Zeilen
>
> Update source references in PO files (no changes to context). This is useful
> as some distributions (e.g. Debian) publish these released PO files and most
> people don't try to update these themself and sometimes wrk on outdated files.
> msgfmt gives now also proper statistics.
>
> * subversion/po/pt_BR.po,
> subversion/po/es.po,
> subversion/po/fr.po,
> subversion/po/de.po,
> subversion/po/sv.po,
> subversion/po/ko.po,
> subversion/po/ja.po,
> subversion/po/pl.po,
Please don't change pl.po in the future.
> subversion/po/zh_TW.po,
> subversion/po/it.po,
> subversion/po/nb.po,
> subversion/po/zh_CN.po:
> Call tools/po/po-update.sh to sync PO files with the source.
--
Arfrever Frehtes Taifersar Arahesis
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=1348891
Re: Translation stuff to do before a release
Posted by Jens Seidel <je...@users.sf.net>.
On Wed, Mar 18, 2009 at 05:09:25PM +0100, Arfrever Frehtes Taifersar Arahesis wrote:
> 2009-03-18 16:25 Jens Seidel <je...@users.sf.net> napisał(a):
> > On Wed, Mar 18, 2009 at 03:54:58PM +0100, Arfrever Frehtes Taifersar Arahesis wrote:
> >> 2009-03-18 10:55 Jens Seidel <je...@users.sf.net> napisał(a):
> >> > Bring PO files in the release branch in sync with source (to match all
> >> > strings from source and not only the strings as present many years ago).
> >>
> >> pl.po before r36624 intentionally contained some strings present in the future.
> >
> > And these strings are not lost. They can still be found as special comments at
> > the bottom of the file and will be automatically reused once they occur
> > after syncing with the source via po-update. Right?
To be honest I wasn't thinking about the case I assumed here (that a
translator added manually some msgid/msgstr pairs for future use).
Nevertheless once the PO files are updated before the release this case
is also covered.
> I meant messages which were marked as fuzzy in r36624.
Hmh, so you worry that commit r36624 demonstrated that you forgot to update
some strings? The current status of the Polish translation is
$ LANG=C msgfmt -cv subversion/po/pl.po
1970 translated messages, 14 fuzzy translations, 1 untranslated message.
This is completely unrelated to my commit! Without r36624 it would maybe
tell that pl.po is fully translated but this would be wrong as only the old
strings are considered. 15 messages are untranslated independent of any
action from my side!
But you're right, it is indeed possible that syncing with the source code
could harm: If a string for which a translation exists is changed in the
source a following sync would mark this string fuzzy. If this change to
the source is reverted the message in the PO file would still be marked
fuzzy, even after the next sync as gettext doesn't know what English string
was translated. This *could* be fixed using the --previous option of
msgmerge as a PO file would consist of 3 parts for fuzzy messages:
* the translated text
* the corresponding English text which was translated (in a comment)
* the current (changed) English text
Once the old (commented) English text matchs again the current text
the fuzzy flag could be automatically removed. Never tested this, though.
If msgmerge doesn't deal with this case a wishlist error should be reported.
Is this what you worry about?
Again let me summarize: PO is a simple, special file format which is
self-explanatory. It doesn't depend on further knowledge about the source,
special project rules, ... but just separates translations from code
development. As such the PO file syntax should be respected (and extensions
in the PO format such as --previous should not be rejected as it happens
with the self written Python scripts) and supported. Subversion doesn't need
any special rules compared to other projects.
The ordinary workflow for translating should also work with Subversion and
this works best if the PO file are up-to-date and in sync with the code.
I worked in the past on hundreds of projects and gettext simplifies
translations a lot. So let's assume that all normal PO file operations
are valid and OK for Subversion as well. If not, please write bugreports
against gettext (and yes, gettext isn't perfect).
Jens
Re: Translation stuff to do before a release
Posted by Jens Seidel <je...@users.sf.net>.
On Thu, Mar 19, 2009 at 08:57:29AM +0800, Dongsheng Song wrote:
> 2009/3/18 Jens Seidel <je...@users.sf.net>:
> > This works only if you are able to translate all new fuzzy strings at once
> > after updating the PO file. What if you want to update the file (or parts of
> > it) later? You have to dig into the history, use svn annotate, ...
I did not referred to changed source references such as the first line in
#: ../include/svn_error_codes.h:171
msgid "Bogus mime-type"
msgstr "Ungültiger Mime-Typ"
but to multiple changes in the msgid. If since the initial translation
5 changes happened to a string (e.g. a large help message) one would
need to perform 5 svn operations (remember that a stanza could also be
moved in the PO file, lines could be differently wrapped, ...).
Nevertheless I agree that often a single svn diff -cREV could be sufficient and
that many update-po calls require additional digging for affected revision
numbers. Having the old translated string as comment via --previous does
nevertheless not harm and avoids digging in the history at all, right?
And I suggest not to call update-po every day or week (some projects, such
as the Debian installer do this) but once (or maybe twice) before a release.
(Once before the "please update your translation" mail is sent out and
immediately before the final tarball is created.) This does *not* harm.
Do you use a very outdated version of gettext? --previous is supported
since version 0.16, from October 2006. From the gettext changelog:
* msgmerge has a new option --previous that has the effect of saving the
previous msgid of message when making them fuzzy. These previous msgids are
stored in the resulting PO file, using a pseudo-comment syntax like this:
#, fuzzy
#| msgid "too many arguments"
msgid "too few arguments"
msgstr "trop d'arguments"
The translator then only needs to compare the previous and the current
msgid ("too many arguments" and "too few arguments"), and infer which
parts of the translation she needs to change.
msgattrib has a new option --clear-previous that removes these #| lines.
> As I said, I don't think source references in PO files is necessary.
It is necessary for Subversion as many, many translation comments are
missing. I very often have to dig into the source to see how a string is
used. I have a few minor patches laying around ...
PO file format is indeed not suitable for ordinary patch based merging.
But there exists msgmerge ...
> On the contrary, in my opinion, source references should removed for
> minimize diffs.
I partly agree but as the repository size is probably no problem and using
source references is standard and the default I would prefer keeping it.
But I understand your arguments ...
> Otherwise, the output of 'svn annotate' is hard to read.
I normally ignore lines containing #:. Just use
svn annotate ko.po | grep -v " #: "
Are there real arguments against using --previous as default and calling
update-po once before a release? Please mention these.
Jens
Re: Translation stuff to do before a release (Was: Re: svn commit:
r36625 - in branches/1.6.x: . subversion/po subversion/svn)
Posted by Arfrever Frehtes Taifersar Arahesis <Ar...@GMail.Com>.
2009-03-18 16:25 Jens Seidel <je...@users.sf.net> napisał(a):
> On Wed, Mar 18, 2009 at 03:54:58PM +0100, Arfrever Frehtes Taifersar Arahesis wrote:
>> 2009-03-18 10:55 Jens Seidel <je...@users.sf.net> napisał(a):
>> > Bring PO files in the release branch in sync with source (to match all
>> > strings from source and not only the strings as present many years ago).
>>
>> pl.po before r36624 intentionally contained some strings present in the future.
>
> And these strings are not lost. They can still be found as special comments at
> the bottom of the file and will be automatically reused once they occur
> after syncing with the source via po-update. Right?
I meant messages which were marked as fuzzy in r36624.
--
Arfrever Frehtes Taifersar Arahesis
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=1349247
Re: Translation stuff to do before a release (Was: Re: svn commit:
r36625 - in branches/1.6.x: . subversion/po subversion/svn)
Posted by Dongsheng Song <do...@gmail.com>.
2009/3/18 Jens Seidel <je...@users.sf.net>:
> This works only if you are able to translate all new fuzzy strings at once
> after updating the PO file. What if you want to update the file (or parts of
> it) later? You have to dig into the history, use svn annotate, ...
>
As I said, I don't think source references in PO files is necessary.
On the contrary, in my opinion, source references should removed for
minimize diffs.
Otherwise, the output of 'svn annotate' is hard to read.
---
Dongsheng Song
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=1352000
Re: Translation stuff to do before a release (Was: Re: svn commit:
r36625 - in branches/1.6.x: . subversion/po subversion/svn)
Posted by Jens Seidel <je...@users.sf.net>.
On Wed, Mar 18, 2009 at 03:54:58PM +0100, Arfrever Frehtes Taifersar Arahesis wrote:
> 2009-03-18 10:55 Jens Seidel <je...@users.sf.net> napisał(a):
> > On Tue, Mar 17, 2009 at 11:13:19PM -0500, Hyrum K. Wright wrote:
> >
> > 1.6.x/$ svn cat http://svn.collab.net/repos/svn/trunk/subversion/po/de.po |
> > tools/dev/po-merge.py subversion/po/de.po
> > Traceback (most recent call last):
> > File "tools/dev/po-merge.py", line 171, in <module>
> > main(sys.argv)
> > File "tools/dev/po-merge.py", line 160, in main
> > for m in msgstr:
> > TypeError: 'NoneType' object is not iterable
>
> It was fixed some days ago in trunk.
Ah, good to know.
> > $MSGMERGE --sort-by-file --no-wrap --update $i subversion.pot
> >
> > A single call to $MSGMERGE should be sufficient and I would add the
> > --previous option (new since 2(?) years), as this adds information about
> > what changed to make a string fuzzy (so that the translation is outdated and
> > not used).
>
> I prefer `svn di XX.po | less` in another terminal.
This works only if you are able to translate all new fuzzy strings at once
after updating the PO file. What if you want to update the file (or parts of
it) later? You have to dig into the history, use svn annotate, ...
--previous is the official and standard way to solve this kind of trouble.
It is supported by multiple graphical (and text based) PO editors, it
doesn't require access to the repository and it works offline. Sounds bad,
indeed.
Do you really want to force all people to follow your current work flow?
You can continue using "svn diff" even if some comments were added, right?
> > Bring PO files in the release branch in sync with source (to match all
> > strings from source and not only the strings as present many years ago).
>
> pl.po before r36624 intentionally contained some strings present in the future.
And these strings are not lost. They can still be found as special comments at
the bottom of the file and will be automatically reused once they occur
after syncing with the source via po-update. Right?
> > This can be done using
> > $ tools/po/po-update.sh
> >
> > This is useful so that the final tar-ball ships current PO files. This are
> > made available to the world of translators by the Subversion repository
> > (or just a web frontend) but also by pages such as
> > http://www.debian.org/intl/l10n/po/de
> >
> > And you don't want such pages to contain completely outdated files, right?
>
> I don't see benefit in .po files with bigger number of fuzzy messages.
It clearly shows that there is work to do without comparing with the source
code and calling update-po again (which many people are not able to).
I don't see benefit in shipping outdated files, do you? (OK: one benefit:
the repository history is cleaner and backups of the repository are smaller,
but how often happens a release?)
> Please don't change pl.po in the future.
So you don't care having untranslated strings just because a typo in the
source was fixed and this wasn't unfuzzied in pl.po? Such as
s/effecient/efficient/ in r36585?
They priority should not be your ego but the users!
Jens
Re: Translation stuff to do before a release
Posted by Karl Fogel <kf...@red-bean.com>.
Greg Stein <gs...@gmail.com> writes:
> On Wed, Mar 18, 2009 at 15:54, Arfrever Frehtes Taifersar Arahesis
> <Ar...@gmail.com> wrote:
>> Please don't change pl.po in the future.
>
> Anybody in this project is allowed to change pl.po. That is not
> "yours" to solely maintain.
>
> If Jens made a mistake on pl.po, then you can provide a review of his
> change. Or you can fix it. But you're never allowed to tell somebody
> they cannot work on something in Subversion (assuming it is within
> their commit area, according to COMMITTERS).
+1
It's okay to suggest to a committer that they need more experience
before they make changes to something within their commit area. That's
usually enough to solve the problem; if it's not enough, and someone
continue to make bad commits, then the solution is to have a discussion
here (or on the private committers list, if necessary). But no one
person can unilaterally tell another to stop committing somewhere in
contradiction to COMMITTERS.
Remember: it's under version control; mistakes can always be reverted.
-K, who hasn't hardly committed in ages, and expects to exercise careful
judgement when he does
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=1349173
Re: Translation stuff to do before a release (Was: Re: svn commit:
r36625 - in branches/1.6.x: . subversion/po subversion/svn)
Posted by Greg Stein <gs...@gmail.com>.
On Wed, Mar 18, 2009 at 15:54, Arfrever Frehtes Taifersar Arahesis
<Ar...@gmail.com> wrote:
>...
>> r36624 | jensseidel | 2009-03-17 14:59:34 +0100 (Di, 17. Mär 2009) | 19 Zeilen
>>
>> Update source references in PO files (no changes to context). This is useful
>> as some distributions (e.g. Debian) publish these released PO files and most
>> people don't try to update these themself and sometimes wrk on outdated files.
>> msgfmt gives now also proper statistics.
>>
>> * subversion/po/pt_BR.po,
>> subversion/po/es.po,
>> subversion/po/fr.po,
>> subversion/po/de.po,
>> subversion/po/sv.po,
>> subversion/po/ko.po,
>> subversion/po/ja.po,
>> subversion/po/pl.po,
>
> Please don't change pl.po in the future.
Anybody in this project is allowed to change pl.po. That is not
"yours" to solely maintain.
If Jens made a mistake on pl.po, then you can provide a review of his
change. Or you can fix it. But you're never allowed to tell somebody
they cannot work on something in Subversion (assuming it is within
their commit area, according to COMMITTERS).
Thanks,
-g
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=1348993