[PATCH] generate subversion.pot in dist.sh

[Mattias Engdegård <ma...@bredband.net>]

Translators will need the subversion.pot file; it should be present in  
the source distribution. Presumably, the right way to ensure that is  
to have dist.sh generate it, similar to the way the configure script  
is created.

[[[
Have dist.sh generate subversion.pot for the benefit of translators  
who need
it in the source distribution.

* tools/dist/dist.sh: Generate subversion.pot by running po-update.sh.
]]]

Re: [PATCH] generate subversion.pot in dist.sh

[Julian Foad <ju...@btopenworld.com>]

Mattias Engdegård wrote:
> Sorry about the reminder, but are there any objections to the patch or could it 
> be applied?

It looks good to me, although I haven't tested it. Ben?

- Julian


> 6 maj 2014 kl. 23.52 skrev Mattias Engdegård:
>> Translators will need the subversion.pot file; it should be present in the 
>> source distribution. Presumably, the right way to ensure that is to have dist.sh 
>> generate it, similar to the way the configure script is created.
>> 
>>  [[[
>>  Have dist.sh generate subversion.pot for the benefit of translators who need
>>  it in the source distribution.
>> 
>>  * tools/dist/dist.sh: Generate subversion.pot by running po-update.sh.
>>  ]]]

Index: tools/dist/dist.sh
===================================================================
--- tools/dist/dist.sh    (revision 1592160)
+++ tools/dist/dist.sh    (arbetskopia)
@@ -292,6 +292,10 @@
   (cd "$DISTPATH" && ./autogen.sh --release) || exit 1
 fi

+# Generate the .pot file, for use by translators.
+echo "Running po-update.sh in sandbox, to create subversion.pot..."
+(cd "$DISTPATH" && tools/po/po-update.sh pot) || exit 1
+
 # Pre-translate the various sql-derived header files
 echo "Generating SQL-derived headers..."
 for f in `find "$DISTPATH/subversion" -name '*.sql'`; do

Re: [PATCH] generate subversion.pot in dist.sh

[Mattias Engdegård <ma...@bredband.net>]

Sorry about the reminder, but are there any objections to the patch or  
could it be applied?
I have received no comments on it, but perhaps the Apache mail server  
gremlins swallowed them.

6 maj 2014 kl. 23.52 skrev Mattias Engdegård:

> Translators will need the subversion.pot file; it should be present  
> in the source distribution. Presumably, the right way to ensure that  
> is to have dist.sh generate it, similar to the way the configure  
> script is created.
>
> [[[
> Have dist.sh generate subversion.pot for the benefit of translators  
> who need
> it in the source distribution.
>
> * tools/dist/dist.sh: Generate subversion.pot by running po-update.sh.
> ]]]
> <dist-po-update.patch.txt>

RE: [PATCH] generate subversion.pot in dist.sh

[Bert Huijben <be...@qqmail.nl>]

> -----Original Message-----
> From: Ben Reser [mailto:ben@reser.org]
> Sent: maandag 9 juni 2014 22:03
> To: Mattias Engdegård
> Cc: dev@subversion.apache.org Development
> Subject: Re: [PATCH] generate subversion.pot in dist.sh
> 
> On 6/8/14, 1:59 PM, Mattias Engdegård wrote:
> > Forgive me for bringing this up again, but we didn't really get
anywhere. No
> > doubt the thread got lost in the mail traffic somehow.
> >
> > Are you satisfied with my explanation, or is there still something that
> makes
> > you object to the suggested 2-line patch? Having the .pot file in the
source
> > distribution, like many other projects do (in particular those using TP,
> where
> > it is standard procedure), shouldn't be much to argue about, should it?
> 
> I don't really understand the desire to build a supposedly better
translation
> workflow around a system that requires a tarball to start working on
> translations.  The fact that lots of other projects have this poor
workflow is
> not particularly convincing to me.
> 
> With respect to Subversion for minor releases you can obviously use the
> release
> candidates.  For patch releases you don't get release candidate tarballs.
> Once
> we've produced a tarball there isn't any opportunity to change anything.
> String changes are not unheard of in patch releases.  Meaning translators
will
> only ever be able to be ahead of our release process with 1.x.0 releases
> given
> this process.
> 
> A process where we produced something for translators to work off on a
> nightly
> basis for trunk and release branches seems far better for me.  Obviously,
the
> release branches are better for translators to work off since trunk may
have
> a
> lot of useless churn.  If they work off a nightly tarball that includes a
.pot
> from a release branch they'll be able to start working on strings as soon
as a
> release branch is cut rather than waiting for a release candidate.  I
don't
> think there's that much string churn once the release branch is made.  But
> when
> they decide to start working is up to the translator.

+1 on all this.

	Bert

Re: [PATCH] generate subversion.pot in dist.sh

[Mattias Engdegård <ma...@bredband.net>]

9 jun 2014 kl. 22.02 skrev Ben Reser:

> With respect to Subversion for minor releases you can obviously use  
> the release
> candidates.  For patch releases you don't get release candidate  
> tarballs.  Once
> we've produced a tarball there isn't any opportunity to change  
> anything.
> String changes are not unheard of in patch releases.  Meaning  
> translators will
> only ever be able to be ahead of our release process with 1.x.0  
> releases given
> this process.

We must have been talking past each other then, or more likely I have  
not explained what I meant well enough. For this I apologise. It is  
not important what the source code archive files sent to translators  
are called; what matters is that they are sent at all, that sufficient  
time is given for the translators to work, and that strings are not  
expected to change too much prior to the actual release.

It is not always possible to avoid some changes to the message  
catalogue after the last translation phase, although this is usually  
not a severe problem. If the strings have changed a lot since the  
previous release, such as when a new major release is being prepared,  
then it makes sense to send out candidates for translation at more  
than one point during development.

In no way do I claim to possess answers to all or even most problems  
related to the translation process. Perhaps it won't work at all, but  
then at least we tried and would be no worse off than we are now.

There are more things that can be done to make the job easier. For  
example, I'm considering splitting up the very large strings that form  
the command help texts into one string per paragraph. That way, a tiny  
change to the documentation of "svn merge" won't result in a single  
massive fuzzy string that drives a translator to despair.

Thank you for applying the change, and I'm sorry it caused trouble on  
Windows; I really didn't anticipate that, although I should have.

Re: [PATCH] generate subversion.pot in dist.sh

[Ben Reser <be...@reser.org>]

On 6/8/14, 1:59 PM, Mattias Engdegård wrote:
> Forgive me for bringing this up again, but we didn't really get anywhere. No
> doubt the thread got lost in the mail traffic somehow.
> 
> Are you satisfied with my explanation, or is there still something that makes
> you object to the suggested 2-line patch? Having the .pot file in the source
> distribution, like many other projects do (in particular those using TP, where
> it is standard procedure), shouldn't be much to argue about, should it?

I don't really understand the desire to build a supposedly better translation
workflow around a system that requires a tarball to start working on
translations.  The fact that lots of other projects have this poor workflow is
not particularly convincing to me.

With respect to Subversion for minor releases you can obviously use the release
candidates.  For patch releases you don't get release candidate tarballs.  Once
we've produced a tarball there isn't any opportunity to change anything.
String changes are not unheard of in patch releases.  Meaning translators will
only ever be able to be ahead of our release process with 1.x.0 releases given
this process.

A process where we produced something for translators to work off on a nightly
basis for trunk and release branches seems far better for me.  Obviously, the
release branches are better for translators to work off since trunk may have a
lot of useless churn.  If they work off a nightly tarball that includes a .pot
from a release branch they'll be able to start working on strings as soon as a
release branch is cut rather than waiting for a release candidate.  I don't
think there's that much string churn once the release branch is made.  But when
they decide to start working is up to the translator.

I've gone ahead and committed your change in r1601496.  But I still think we
should do something better for translators than what you're thinking of.  If we
do I think we can remove this from needing to be in every tarball.  Since
again, I see no reason for including a generated file in the tarball for a very
small subset of our user base.  Especially, since we ought to be able to do
something better for that user base.

Re: [PATCH] generate subversion.pot in dist.sh

[Mattias Engdegård <ma...@bredband.net>]

Forgive me for bringing this up again, but we didn't really get  
anywhere. No doubt the thread got lost in the mail traffic somehow.

Are you satisfied with my explanation, or is there still something  
that makes you object to the suggested 2-line patch? Having the .pot  
file in the source distribution, like many other projects do (in  
particular those using TP, where it is standard procedure), shouldn't  
be much to argue about, should it?

21 maj 2014 kl. 22.36 skrev Mattias Engdegård:

> 20 maj 2014 kl. 23.11 skrev Ben Reser:
>
>> That seems like a really strange workflow to me.  If translators  
>> are dependent
>> on source distributions how are we supposed to have translations  
>> done for
>> upcoming unreleased changes?  I think we can come up with a better  
>> way of doing
>> this than including this file in all our source distribution files.
>
> I'm not sure why this would be controversial; it's a fairly standard  
> mode of operation. Look in any GCC source distribution, say, and you  
> will a .pot file.
>
> There may at first sight appear to be a conflict between the  
> translators' need of a finished release to work on and the  
> developers' desire to have the translations before putting that  
> release together, but in practice it's not much of a problem. A pre- 
> release, release candidate or just a selected sufficiently stable  
> nightly build can be used as the base for translation, as long as  
> the translators are given a reasonable amount of time to do their job.
>
> Even so, the .pot file hardly costs anything to have in the final  
> release. It also permits easy translation of pre-built binary  
> distributions.
>
> An alternative model is to distribute the translations (.po files)  
> separately from the source to which they belong. That way, they can  
> be updated independently after the source release. There are obvious  
> drawbacks and advantages; I would suggest going with pre-releases.
>
>>> That kind of data is highly compressible, and even more so given  
>>> the fact that
>>> it will be compressed together with the rest of the source which  
>>> already
>>> includes all the strings in the file.
>>
>> Right, I realize the data size isn't really all that big of a  
>> deal.  But I see
>> no reason to expand the size of our download package for little  
>> value.
>
> Adding the .pot to the 1.8.9 source distribution makes the .bz2 file  
> grow by less than 31 KiB, or less than 0.5 %. I suppose that  
> technically counts as "expansion".
>
>> Then translators work off that nightly package and they're able to  
>> work off
>> reasonably up to date data?  We can produce similar files for the  
>> branches if
>> someone wants to improve the strings that might be different in a  
>> branch from
>> trunk.
>
> Translation of code in motion is generally wasted time. There is  
> rarely any point in doing any work until things have settled down.  
> Unless I have misunderstood Subversion's development model,  
> translations of "trunk" is probably fairly pointless under normal  
> circumstances.

Re: [PATCH] generate subversion.pot in dist.sh

[Mattias Engdegård <ma...@bredband.net>]

20 maj 2014 kl. 23.11 skrev Ben Reser:

> That seems like a really strange workflow to me.  If translators are  
> dependent
> on source distributions how are we supposed to have translations  
> done for
> upcoming unreleased changes?  I think we can come up with a better  
> way of doing
> this than including this file in all our source distribution files.

I'm not sure why this would be controversial; it's a fairly standard  
mode of operation. Look in any GCC source distribution, say, and you  
will a .pot file.

There may at first sight appear to be a conflict between the  
translators' need of a finished release to work on and the developers'  
desire to have the translations before putting that release together,  
but in practice it's not much of a problem. A pre-release, release  
candidate or just a selected sufficiently stable nightly build can be  
used as the base for translation, as long as the translators are given  
a reasonable amount of time to do their job.

Even so, the .pot file hardly costs anything to have in the final  
release. It also permits easy translation of pre-built binary  
distributions.

An alternative model is to distribute the translations (.po files)  
separately from the source to which they belong. That way, they can be  
updated independently after the source release. There are obvious  
drawbacks and advantages; I would suggest going with pre-releases.

>> That kind of data is highly compressible, and even more so given  
>> the fact that
>> it will be compressed together with the rest of the source which  
>> already
>> includes all the strings in the file.
>
> Right, I realize the data size isn't really all that big of a deal.   
> But I see
> no reason to expand the size of our download package for little value.

Adding the .pot to the 1.8.9 source distribution makes the .bz2 file  
grow by less than 31 KiB, or less than 0.5 %. I suppose that  
technically counts as "expansion".

> Then translators work off that nightly package and they're able to  
> work off
> reasonably up to date data?  We can produce similar files for the  
> branches if
> someone wants to improve the strings that might be different in a  
> branch from
> trunk.

Translation of code in motion is generally wasted time. There is  
rarely any point in doing any work until things have settled down.  
Unless I have misunderstood Subversion's development model,  
translations of "trunk" is probably fairly pointless under normal  
circumstances.

Re: [PATCH] generate subversion.pot in dist.sh

[Ben Reser <be...@reser.org>]

On 5/20/14, 12:21 PM, Mattias Engdegård wrote:
> The goal is to permit us to use a different translation model, where
> translators aren't working directly in the SVN tree but on .pot files or source
> distributions that we give them.
> 
> The Translation Project, which I'd like to try first, works by having the
> developers sending either the .pot file, or an URL to a source archive
> containing the .pot, to the translators:
> 
> http://translationproject.org/html/maintainers.html
> 
> Since we already have a number of existing more or less complete translations,
> and (from my own experience) having access to the source code is very valuable
> for several of the strings in the catalogue, we should include the .pot in the
> source distribution.

That seems like a really strange workflow to me.  If translators are dependent
on source distributions how are we supposed to have translations done for
upcoming unreleased changes?  I think we can come up with a better way of doing
this than including this file in all our source distribution files.

> That kind of data is highly compressible, and even more so given the fact that
> it will be compressed together with the rest of the source which already
> includes all the strings in the file.

Right, I realize the data size isn't really all that big of a deal.  But I see
no reason to expand the size of our download package for little value.

> If translators need to run build scripts in order to generate the .pot file
> they need, we may never get anything from them.

Sure, but having them translate something that's already stale seems like a
waste of their time.

Would the following behavior make more sense?

* Add the code to build the .pot file but make it conditional on an option.
* Enable that option in a nightly packaging.

Then translators work off that nightly package and they're able to work off
reasonably up to date data?  We can produce similar files for the branches if
someone wants to improve the strings that might be different in a branch from
trunk.

This avoids expanding the source packaging for end users and seems to me like
it'd better support translators?

What do you think?

Re: [PATCH] generate subversion.pot in dist.sh

[Mattias Engdegård <ma...@bredband.net>]

20 maj 2014 kl. 17.57 skrev Ben Reser:

> Why should the pot file be present in the distribution package?   
> Shouldn't
> translators be working against a SVN checkout and not a distribution  
> package?

The goal is to permit us to use a different translation model, where  
translators aren't working directly in the SVN tree but on .pot files  
or source distributions that we give them.

The Translation Project, which I'd like to try first, works by having  
the developers sending either the .pot file, or an URL to a source  
archive containing the .pot, to the translators:

http://translationproject.org/html/maintainers.html

Since we already have a number of existing more or less complete  
translations, and (from my own experience) having access to the source  
code is very valuable for several of the strings in the catalogue, we  
should include the .pot in the source distribution.

> Looking at the output on 1.8.x it adds 348 KB of uncompressed size  
> to our
> source.  That's obviously not a huge problem, but I just don't  
> understand what
> value this really creates.

That kind of data is highly compressible, and even more so given the  
fact that it will be compressed together with the rest of the source  
which already includes all the strings in the file.

>  For most of users this is useless (they aren't
> translators).  For those that need it, it's trivially created.

If translators need to run build scripts in order to generate the .pot  
file they need, we may never get anything from them.

Re: [PATCH] generate subversion.pot in dist.sh

[Ben Reser <be...@reser.org>]

On 5/6/14, 2:52 PM, Mattias Engdegård wrote:
> Translators will need the subversion.pot file; it should be present in the
> source distribution. Presumably, the right way to ensure that is to have
> dist.sh generate it, similar to the way the configure script is created.
> 
> [[[
> Have dist.sh generate subversion.pot for the benefit of translators who need
> it in the source distribution.
> 
> * tools/dist/dist.sh: Generate subversion.pot by running po-update.sh.
> ]]]

Why should the pot file be present in the distribution package?  Shouldn't
translators be working against a SVN checkout and not a distribution package?

Looking at the output on 1.8.x it adds 348 KB of uncompressed size to our
source.  That's obviously not a huge problem, but I just don't understand what
value this really creates.  For most of users this is useless (they aren't
translators).  For those that need it, it's trivially created.

Configure, which you're using as an example is used by everyone that builds on
anything other than Windows.

I see no problem with the patch itself, but I don't understand the purpose.