Backport mixup last night

[Johan Corveleyn <jc...@gmail.com>]

Just a quick report about what happened and what I've done (things are
okay now).

There was an svn-role (automatic backport) mixup last night: some
revisions were still correct, but the last 6 automated backport
commits were wrong (mixed STATUS changes with wrong actual changes
etc). Cause: the cronjob of backport.pl was running simultaneously on
svn-qavm2 (old machine) and svn-qavm3 (new machine we're migrating to,
see [1]), creating a race.

For now, I've just renamed the backport.pl script on qavm2, so it
can't be run from cron anymore. I've reverted the incorrect backport
commits, and manually ran the backport.pl script on qavm3 to redo the
correct backports. That seems to have worked out OK, so all is fine
now ... carry on (or: keep calm and resolve conflicts :-).

[1] https://issues.apache.org/jira/browse/INFRA-12289

-- 
Johan

Re: Backport mixup last night

[Johan Corveleyn <jc...@gmail.com>]

On Thu, Oct 13, 2016 at 4:17 PM, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> Johan Corveleyn wrote on Thu, Oct 13, 2016 at 11:37:25 +0200:
>> [...] the cronjob of backport.pl was running simultaneously on
>> svn-qavm2 (old machine) and svn-qavm3 (new machine we're migrating to,
>> see [1]), creating a race.
>>
>> For now, I've just renamed the backport.pl script on qavm2, so it
>> can't be run from cron anymore.
>
> Thanks for diagnosing and fixing the problem.  For belt and braces we
> might do «mkdir ~svnsvn/src/svn/trunk/tools/dist/backport.pl» on the old
> machine to ensure that an errant 'svn up' wouldn't reintroduce the
> script run from cron.
>
> (I'd do this myself but I'm separated from my ssh keys right now)

Ah, good idea. Done!

-- 
Johan

Re: Backport mixup last night

[Daniel Shahaf <d....@daniel.shahaf.name>]

Johan Corveleyn wrote on Thu, Oct 13, 2016 at 11:37:25 +0200:
> [...] the cronjob of backport.pl was running simultaneously on
> svn-qavm2 (old machine) and svn-qavm3 (new machine we're migrating to,
> see [1]), creating a race.
> 
> For now, I've just renamed the backport.pl script on qavm2, so it
> can't be run from cron anymore.

Thanks for diagnosing and fixing the problem.  For belt and braces we
might do �mkdir ~svnsvn/src/svn/trunk/tools/dist/backport.pl� on the old
machine to ensure that an errant 'svn up' wouldn't reintroduce the
script run from cron.

(I'd do this myself but I'm separated from my ssh keys right now)