svn merge --record-only and svn log -g

[Philippe Combes <Ph...@enseeiht.fr>]

Dear Subversion users,

The root cause of my question is the need to automatically process the
commit logs to generate a proper ChangeLog. In order to really list all
modifications done, I have got to use svn log --use-merge-history.
Since this command is based on the property svn:mergeinfo, it lists all
merged revisions, even those which were added by svn merge
--record-only. In other words, it lists some modifications which were
not actually done.
Looking for a solution to this issue, I discovered the existence of the
tool svnmerge.py. If I understand well how it used to work, two lists of
changesets were maintained in subversion properties, so that it kept
track of the changesets blocked. I have noticed that this tool has been
removed from subversion 1.8, yet svn merge appears not to be a
one-to-one replacement for it, since it looses track of the "recorded
only" changesets.
Am I missing something ? Is there any way to segregate the recorded only
changesets from the other with the svn commands ?

I know some of you may answer that using --record-only is bad practice,
but I am working on a project where it is used intensively, and it would
be very hard to rollback on these habits. By the way, my guess is that
as long as the option is provided, the other commands should be
consistent with its behaviour.
Has anyone already met this kind of issue ?

Thanks in advance for any piece of help or advice,
Philippe

Re: svn merge --record-only and svn log -g

[Stefan Sperling <st...@stsp.name>]

On Wed, Sep 21, 2016 at 12:49:46PM +0200, Philippe Combes wrote:
> 
> > Perhaps your changelog generator could be taught to ignore revisions
> > which changed only svn:mergeinfo properties? Would that work around
> > the problem?
> 
> Stefan,
> 
> Thanks again for your time.
> As you point it out, it would only be a workaround, but it sounds a good
> lead. I plan to use svn2cl as the base of my changelog generator.
> Unfortunately, it relies on a language which I have no skill for: xslt.
> So I need to keep things quite simple, if I intend to customize the
> tool. Have you got any clue on a simple (based on subversion commands)
> and reliable way of identifying revisions which changed only
> svn:mergeinfo ?

svn log -v will show the list of changed paths and with --xml the output
can be parsed as XML. svn proplist lists the properties for a path, and
also has an --xml switch.

With this you should have all the information you need to find out
what changed in any revision. You might need to iterate over many
paths in large commits but I'm not sure how to optimize for that.
Perhaps somebody else has an idea.

If XML doesn't suite you, consider writing a script which uses one of
the language bindings (Python, Perl, Ruby, Java). Unfortunately the
documentation for bindings is relatively sparse, so it might take some
time to figure out how things work.

Re: svn merge --record-only and svn log -g

[Philippe Combes <Ph...@enseeiht.fr>]

> Perhaps your changelog generator could be taught to ignore revisions
> which changed only svn:mergeinfo properties? Would that work around
> the problem?

Stefan,

Thanks again for your time.
As you point it out, it would only be a workaround, but it sounds a good
lead. I plan to use svn2cl as the base of my changelog generator.
Unfortunately, it relies on a language which I have no skill for: xslt.
So I need to keep things quite simple, if I intend to customize the
tool. Have you got any clue on a simple (based on subversion commands)
and reliable way of identifying revisions which changed only
svn:mergeinfo ?

Regards,
Philippe

Re: svn merge --record-only and svn log -g

[Stefan Sperling <st...@elego.de>]

On Wed, Sep 14, 2016 at 10:51:54AM +0200, Philippe Combes wrote:
> The only solution here is to add another option to svn merge, --block,
> with another reserved property, just as svnmerge.py used to do. I have
> no idea of how deep the impacts on the svn merge code would be (from my
> external point of view, it is just merging two lists of changesets into
> one at the beginning of the merge process), but svn log -g would then
> work as expected with no additional developments.
> I hope we will have some feedback from the Subversion developers on this
> point.

There is no 1:1 correspondance between the svnmerge.py script dating
back to 2004, and the built-in merge-tracking feature as it was designed
during the release cycle of Subversion 1.5 in 2007/2008.
This goes both ways: merge-tracking has features which svnmerge.py
did not have. Obviously all this was decided many years ago and is
now water under the bridge.

It's possible that nobody at the time thought of your use case for
svn log --use-merge-history and how it would interact with the ability
to run --record-only merges. I think you've spotted a design flaw.

But I don't think making the current merge system even more complicated by
adding yet another special property is something we would want to do now.  
The implications for any change in merge-tracking are huge. Such a change
needs very careful planning and a lot of testing. There are many edge cases
which could be affected by such a change. I doubt anyone would manage to
fully design and implement your idea in a releasable state without investing
several weeks, if not months, of serious full-time work.

> With regards to your advice of using tags in the logs, it does not apply
> easily to my research projects, where the decision of merging a
> changeset or not is made at merge time only. This would mean changing
> all logs at merge time, which can be quite a long task because of
> administration rights issues. Moreover we already use tags in logs for
> other kinds of segregation and it may overload the logs.

Perhaps your changelog generator could be taught to ignore revisions
which changed only svn:mergeinfo properties? Would that work around
the problem?

Re: svn merge --record-only and svn log -g

[Philippe Combes <Ph...@enseeiht.fr>]

Stefan, 
Thanks a lot for your time and your answer. I am sorry I could not read
it sooner...

Now I understand well the issue of manual merges, but I don't understand
the choices made regarding it. I have the feeling that --record-only has
been first introduced for such manual merges. Then it has been used as a
workaround to the missing functionality of purely blocking some merges,
which is clearly quite different. This missing functionality was
supported by svnmerge.py but it has been removed.

The only solution here is to add another option to svn merge, --block,
with another reserved property, just as svnmerge.py used to do. I have
no idea of how deep the impacts on the svn merge code would be (from my
external point of view, it is just merging two lists of changesets into
one at the beginning of the merge process), but svn log -g would then
work as expected with no additional developments.
I hope we will have some feedback from the Subversion developers on this
point.


With regards to your advice of using tags in the logs, it does not apply
easily to my research projects, where the decision of merging a
changeset or not is made at merge time only. This would mean changing
all logs at merge time, which can be quite a long task because of
administration rights issues. Moreover we already use tags in logs for
other kinds of segregation and it may overload the logs.


Thanks again,
Philippe Combes

Re: svn merge --record-only and svn log -g

[Stefan Hett <st...@egosoft.com>]

On 7/27/2016 6:18 PM, Philippe Combes wrote:
> Dear Subversion users,
>
> The root cause of my question is the need to automatically process the
> commit logs to generate a proper ChangeLog. In order to really list all
> modifications done, I have got to use svn log --use-merge-history.
> Since this command is based on the property svn:mergeinfo, it lists all
> merged revisions, even those which were added by svn merge
> --record-only. In other words, it lists some modifications which were
> not actually done.
> Looking for a solution to this issue, I discovered the existence of the
> tool svnmerge.py. If I understand well how it used to work, two lists of
> changesets were maintained in subversion properties, so that it kept
> track of the changesets blocked. I have noticed that this tool has been
> removed from subversion 1.8, yet svn merge appears not to be a
> one-to-one replacement for it, since it looses track of the "recorded
> only" changesets.
> Am I missing something ? Is there any way to segregate the recorded only
> changesets from the other with the svn commands ?
>
> I know some of you may answer that using --record-only is bad practice,
> but I am working on a project where it is used intensively, and it would
> be very hard to rollback on these habits. By the way, my guess is that
> as long as the option is provided, the other commands should be
> consistent with its behaviour.
> Has anyone already met this kind of issue ?
>
> Thanks in advance for any piece of help or advice,
> Philippe

Can't provide much help for your case, but the limitation of indicating 
whether some revision was merged or blocked is not new.

Unfortunately, one can't simply say that everything which uses the 
--record-only merge flag would correspond to indicating the revision was 
blocked. It can also mean that the corresponding changes were already 
manually merged (or no longer apply) to the target branch but still 
should be seen/considered as merged. We use this concept quite regularly 
and indicating in general that a --record-only merge would mean a 
revision was not integrated in the target branch would be wrong for 
these cases.

Hence the whole problem is a bit more complicated than it seems to be.

We don't run into this issue, because we do not use the mergeinfo to 
indicate a revision was blocked. In this case we simply don't merge it 
and skip it. For some automated tools we use a specific flag in the 
commit log of revisions which are not meant to be merged into some other 
branch(es). The tools check for these flags in the log and simply skip 
these revisions. Maybe this approach might be usable for you as well?

-- 
Regards,
Stefan Hett