Git smudge / Clean / Filter alike in Subversion ?

[Laurent Alebarde <l....@free.fr>]

Hi all,

I am looking for some Git smudge / clean capabilities in subversion to 
be able to "clean" code before pushing it into the repository, as long 
as filters do be able to diff binary files with for example some 
embedded tags.

Here is FYI the Git documentation page that explain it exactly : 
http://git-scm.com/book/en/Customizing-Git-Git-Attributes

Any help please ?

Laurent.

Re: Git smudge / Clean / Filter alike in Subversion ?

[Laurent Alebarde <l....@free.fr>]

Le 12/09/2012 13:20, Stefan Sperling a écrit :
> On Wed, Sep 12, 2012 at 11:41:05AM +0200, Laurent Alebarde wrote:
>> I could find examples of pre-commit hooks, for example to convert
>> indentation from spaces to tabs or the opposite.
> This is very wrong. Pre-commit hooks are not supposed to modify
> the content of commit transactions. Doing so causes the working
> copy and the repository to get out of sync, having conflicting
> ideas about what was committed. Don't do this!
THANKS for the warning ! So I will use a WS dummy copy to perform my 
strange things with external tools, and use SVN on this copy.
>> Then I could find
>> the lists of available hooks in the documentation. If I am not
>> mistaken, there is no hook available in the opposite direction, say
>> in the check-out process ?
> No, there are no hooks during checkout.
>
> TortoiseSVN has client-side hooks, however. You might be able
> to leverage those. But I don't know details of how they work.
> And this would force all developers to use TortoiseSVN.
I agree, that would be bad to require it.

Re: Git smudge / Clean / Filter alike in Subversion ?

[Stefan Sperling <st...@elego.de>]

On Wed, Sep 12, 2012 at 11:41:05AM +0200, Laurent Alebarde wrote:
> I could find examples of pre-commit hooks, for example to convert
> indentation from spaces to tabs or the opposite.

This is very wrong. Pre-commit hooks are not supposed to modify
the content of commit transactions. Doing so causes the working
copy and the repository to get out of sync, having conflicting
ideas about what was committed. Don't do this!

> Then I could find
> the lists of available hooks in the documentation. If I am not
> mistaken, there is no hook available in the opposite direction, say
> in the check-out process ?

No, there are no hooks during checkout.

TortoiseSVN has client-side hooks, however. You might be able
to leverage those. But I don't know details of how they work.
And this would force all developers to use TortoiseSVN.

Re: Git smudge / Clean / Filter alike in Subversion ?

[Laurent Alebarde <l....@free.fr>]

Le 11/09/2012 17:47, Ryan Schmidt a écrit :
> On Sep 11, 2012, at 09:48, Laurent Alebarde wrote:
>
>> Le 11/09/2012 15:49, Stefan Sperling a écrit :
>>> What do you really need this feature for? I'm interested in learning
>>> more about your actual use case, the actual problem you're trying to
>>> solve, rather than what git's solution to your problem is. Maybe if
>>> I knew more about the problem itself I could give you a better answer.
>>> And maybe we can add some feature to svn to solve your problem, once we
>>> understand the problem.
>> Sure, you are right. At present time, my use cases are :
>> 1) Automatic coding style refactoring so that developpers remain happy with what they are used to : indentation, naming, layout, etc.
> The usual Subversion solution for this is to either
>
> a) write a pre-commit hook that verifies that the code conforms to the style guidelines, and rejects the commit if it does not; or
Our policies are : "developpers feel like at home" and "tools shall 
adapt to developpers". I recognise it is unusual and dissonant. So 
developpers use their coding style they are efficient with and I 
translate that to something standard. I induce the rules to provide the 
developper with something he could have written himself when check-out.

I could find examples of pre-commit hooks, for example to convert 
indentation from spaces to tabs or the opposite. Then I could find the 
lists of available hooks in the documentation. If I am not mistaken, 
there is no hook available in the opposite direction, say in the 
check-out process ?
> b) write a post-commit hook that converts code to the required style and commits this in a second revision
>
>> 2) Automatic doxygen comments generation.
> Here too the usual Subversion solution is to have a post-commit hook generate this and then commit it. Another common answer is that things that can be generated should not be stored in the repository.
> I agree with you, except in the case of a heavy process, though another shared storega place can be found instead of the svn repository. But keeping it in svn anable to apply patches of manual updates from versions to versions. I mean the documentation is only around 75% automatized.
>
>> 3) Add information in the code in the workspace, from tags included in the repository
> I don't understand... could you give an example?
>
Please, cf my other answer in a separate message.

Re: Git smudge / Clean / Filter alike in Subversion ?

[Ryan Schmidt <su...@ryandesign.com>]

On Sep 11, 2012, at 09:48, Laurent Alebarde wrote:

> Le 11/09/2012 15:49, Stefan Sperling a écrit :
>> 
>> What do you really need this feature for? I'm interested in learning
>> more about your actual use case, the actual problem you're trying to
>> solve, rather than what git's solution to your problem is. Maybe if
>> I knew more about the problem itself I could give you a better answer.
>> And maybe we can add some feature to svn to solve your problem, once we
>> understand the problem.
> Sure, you are right. At present time, my use cases are :
> 1) Automatic coding style refactoring so that developpers remain happy with what they are used to : indentation, naming, layout, etc.

The usual Subversion solution for this is to either

a) write a pre-commit hook that verifies that the code conforms to the style guidelines, and rejects the commit if it does not; or
b) write a post-commit hook that converts code to the required style and commits this in a second revision

> 2) Automatic doxygen comments generation.

Here too the usual Subversion solution is to have a post-commit hook generate this and then commit it. Another common answer is that things that can be generated should not be stored in the repository.


> 3) Add information in the code in the workspace, from tags included in the repository

I don't understand... could you give an example?

Re: Git smudge / Clean / Filter alike in Subversion ?

[Laurent Alebarde <l....@free.fr>]

Le 12/09/2012 13:30, Stefan Sperling a écrit :
> On Wed, Sep 12, 2012 at 11:40:58AM +0200, Laurent Alebarde wrote:
>> So the best way for me woudl be to use a "dummy" workspace copy
>> before commiting, and another one after check-outing :
>>
>> WS ---pre-commit-treatment-(git-clean)--->  dummy WS ---commit--->  Repository
>>
>> Repository ---check-out--->  another dummy WS
>> ---post-check-out-treatment-(git-smudge)--->  WS
> You might be able to do this kind of thing with git.
> But I doubt you'll get it working nicely with Subversion.
>
> Why don't you use git? Now that I know more details, it seems
> that git's solution fits your problem much better. It is not
> something we can easily add to svn because we don't have a
> separate stage between the working copy and the repository.
> It seems something like that would need to be added to svn first.
I do use Git, but I have to use SVN too. The idea above with 
"pre-commit-treatment-(git-clean)" and 
"post-check-out-treatment-(git-smudge)" is something outside SVN, or 
something that encapsulate SVN.
>> As code in each developper WS is not standard, I cannot diff code
>> from a WS and the repository directly. Files have to be in the same
>> referential. So either I "smudge" the file coming from the
>> repository before diff-ing it with the file of the WS, either I
>> "clean" the WS file, before diff-ing it with the repository file.
> I think that's a horrible solution to the problem.
> IMO, developers should work as a team, and thus also agree on
> common coding and quality standards. No technical solution should
> be used to resolve differences in matters of taste, or worse, cover
> up mistakes made due to sloppiness.
It is a question of taste and point of view. I agree my opinion is not 
widespread, but I support it.
>
>>> Is this information meant for interactive use by developers while
>>> developing? Or is it some extended information of the sort that
>>> svn:keywords supports, which is embedded in files you ship to
>>> customers (outside the version control system)?
>> Yes, for my 3rd use case, svn:keywords does the trick. Thanks. But
>> svn:keywords are not enough though to solve my diff paradigm above.
> It does. Since you have a way to automatically change coding
> style in files, you can perform that transformation in a diff
> tool wrapper script. You could write an external diff tool that
> takes the files from svn, normalises them, and then diffs them.
Yes, I have to encapsulate everything.

Thanks a lot for your help.

Re: Git smudge / Clean / Filter alike in Subversion ?

[Stefan Sperling <st...@elego.de>]

On Wed, Sep 12, 2012 at 11:40:58AM +0200, Laurent Alebarde wrote:
> So the best way for me woudl be to use a "dummy" workspace copy
> before commiting, and another one after check-outing :
> 
> WS ---pre-commit-treatment-(git-clean)---> dummy WS ---commit---> Repository
> 
> Repository ---check-out---> another dummy WS
> ---post-check-out-treatment-(git-smudge)---> WS

You might be able to do this kind of thing with git.
But I doubt you'll get it working nicely with Subversion.

Why don't you use git? Now that I know more details, it seems
that git's solution fits your problem much better. It is not
something we can easily add to svn because we don't have a
separate stage between the working copy and the repository.
It seems something like that would need to be added to svn first.

> As code in each developper WS is not standard, I cannot diff code
> from a WS and the repository directly. Files have to be in the same
> referential. So either I "smudge" the file coming from the
> repository before diff-ing it with the file of the WS, either I
> "clean" the WS file, before diff-ing it with the repository file.

I think that's a horrible solution to the problem.
IMO, developers should work as a team, and thus also agree on
common coding and quality standards. No technical solution should
be used to resolve differences in matters of taste, or worse, cover
up mistakes made due to sloppiness.

> >Is this information meant for interactive use by developers while
> >developing? Or is it some extended information of the sort that
> >svn:keywords supports, which is embedded in files you ship to
> >customers (outside the version control system)?
> Yes, for my 3rd use case, svn:keywords does the trick. Thanks. But
> svn:keywords are not enough though to solve my diff paradigm above.

It does. Since you have a way to automatically change coding
style in files, you can perform that transformation in a diff
tool wrapper script. You could write an external diff tool that
takes the files from svn, normalises them, and then diffs them.

Re: Git smudge / Clean / Filter alike in Subversion ?

[Laurent Alebarde <l....@free.fr>]

Le 11/09/2012 17:58, Stefan Sperling a écrit :
> On Tue, Sep 11, 2012 at 04:48:12PM +0200, Laurent Alebarde wrote:
>> Le 11/09/2012 15:49, Stefan Sperling a écrit :
>>> On Tue, Sep 11, 2012 at 02:45:49PM +0200, Laurent Alebarde wrote:
>>>> Thanks for your answer Stefan,
>>>>
>>>> Unfortunatly, no :
>>>>
>>>> The first link says Subversion is smart with binary files. That's
>>>> good to hear, but do not provide a filter or filter hook between the
>>>> workspace and the repository.
>>> Apart from built-in properties such as svn:keywords and svn:eol-style,
>>> there is no way to make changes to files during checkout or commit.
>>> You can probably use a wrapper script for this purpose, however,
>>> that wraps svn checkout, svn update, and svn commit.
>>>
>>> What do you really need this feature for? I'm interested in learning
>>> more about your actual use case, the actual problem you're trying to
>>> solve, rather than what git's solution to your problem is. Maybe if
>>> I knew more about the problem itself I could give you a better answer.
>>> And maybe we can add some feature to svn to solve your problem, once we
>>> understand the problem.
>> Sure, you are right. At present time, my use cases are :
>> 1) Automatic coding style refactoring so that developpers remain
>> happy with what they are used to : indentation, naming, layout, etc.
> In Subversion you can block commits that do not conform to policy
> by creating a pre-commit hook that evaluates changes about to be
> committed, and allows or rejects the commit based on that.
>
> This then requires developers to e.g. heed coding style to be able
> to commit. They could also use tools to modify files in their
> working copy before committing. I.e. the only difference is that
> there is no way to plug the "fix-up tooling" into svn itself, but
> that people are required to use this tooling in _addition_ to svn,
> before committing.

So the best way for me woudl be to use a "dummy" workspace copy before 
commiting, and another one after check-outing :

WS ---pre-commit-treatment-(git-clean)---> dummy WS ---commit---> Repository

Repository ---check-out---> another dummy WS 
---post-check-out-treatment-(git-smudge)---> WS
> However, if your pre-commit checks are too strict and there is no
> way to bypass them, people might end up not committing at all for
> longer than necessary, which is also bad. You should at least provide
> a way to bypass these checks (e.g. by putting a special marker into
> the log message) to allow for unforeseen circumstances where developers
> need to override these checks.
One idea behind all that is I don't want to block at all.
>> 2) Automatic doxygen comments generation.
> If this can be done in an automated way, why bother developer working
> copies with it at all? You could create an automated job that has
> its own working copy, updates to get new changes, makes any necessary
> doxygen modifications, and commits the result. svn can be scripted for
> these kinds of purposes.  See the --non-interactive and --xml options
> that many svn commands support.
Please, cf my other answer in a separate message.
>
>> 3) Add information in the code in the workspace, from tags included
>> in the repository
>> The 2 first use cases need a filter between the workspace and the
>> repository. The 3rd one needs a filter between the workspace and the
>> internal or external diff.
> I don't think I understand 3), especially what diff has got to do
> with it. What is this for?

As code in each developper WS is not standard, I cannot diff code from a 
WS and the repository directly. Files have to be in the same 
referential. So either I "smudge" the file coming from the repository 
before diff-ing it with the file of the WS, either I "clean" the WS 
file, before diff-ing it with the repository file.
>
> Is this information meant for interactive use by developers while
> developing? Or is it some extended information of the sort that
> svn:keywords supports, which is embedded in files you ship to
> customers (outside the version control system)?
Yes, for my 3rd use case, svn:keywords does the trick. Thanks. But 
svn:keywords are not enough though to solve my diff paradigm above.

>
> In the latter case, you might as well use a separate working copy
> to add this extra information, and commit it from there, instead
> of forcing unrelated changes into developer working copies.
>
> I realise that git can easily hide modifications by automatically
> making them in the index instead of the working tree. So if you're
> adding information which developers don't really need to see on
> they fly while files are being added to the index, this doesn't
> bother them much.
>
> However, Subversion does not have a concept of an "index" like git does.
> There is only a working copy and a repository. So you're really talking
> about making edits to peoples' working files, which may or may not be
> desirable, depending on the use case. And keep in mind that developers
> might have their files opened up in editors while they run 'svn update'
> or 'svn commit', and you would be making changes to the files concurrently.
> I'd imagine that could lead to unexpected results.
>
> My advice would be to use automated jobs that use separate working
> copies, if at all possible, and leave developer working copies alone.
> They'll get to see modifications made by automated commits just like
> any other commits when they run 'svn update'.
Yes, but I am afraid to be obliged to encapsulate many svn commands, at 
least diff, to integrate the whole "smudge/clean"process.

Re: Git smudge / Clean / Filter alike in Subversion ?

[Stefan Sperling <st...@elego.de>]

On Tue, Sep 11, 2012 at 04:48:12PM +0200, Laurent Alebarde wrote:
> Le 11/09/2012 15:49, Stefan Sperling a écrit :
> >On Tue, Sep 11, 2012 at 02:45:49PM +0200, Laurent Alebarde wrote:
> >>Thanks for your answer Stefan,
> >>
> >>Unfortunatly, no :
> >>
> >>The first link says Subversion is smart with binary files. That's
> >>good to hear, but do not provide a filter or filter hook between the
> >>workspace and the repository.
> >Apart from built-in properties such as svn:keywords and svn:eol-style,
> >there is no way to make changes to files during checkout or commit.
> >You can probably use a wrapper script for this purpose, however,
> >that wraps svn checkout, svn update, and svn commit.
> >
> >What do you really need this feature for? I'm interested in learning
> >more about your actual use case, the actual problem you're trying to
> >solve, rather than what git's solution to your problem is. Maybe if
> >I knew more about the problem itself I could give you a better answer.
> >And maybe we can add some feature to svn to solve your problem, once we
> >understand the problem.
> Sure, you are right. At present time, my use cases are :
> 1) Automatic coding style refactoring so that developpers remain
> happy with what they are used to : indentation, naming, layout, etc.

In Subversion you can block commits that do not conform to policy
by creating a pre-commit hook that evaluates changes about to be
committed, and allows or rejects the commit based on that.

This then requires developers to e.g. heed coding style to be able
to commit. They could also use tools to modify files in their
working copy before committing. I.e. the only difference is that
there is no way to plug the "fix-up tooling" into svn itself, but
that people are required to use this tooling in _addition_ to svn,
before committing.

However, if your pre-commit checks are too strict and there is no
way to bypass them, people might end up not committing at all for
longer than necessary, which is also bad. You should at least provide
a way to bypass these checks (e.g. by putting a special marker into
the log message) to allow for unforeseen circumstances where developers
need to override these checks.

> 2) Automatic doxygen comments generation.

If this can be done in an automated way, why bother developer working
copies with it at all? You could create an automated job that has
its own working copy, updates to get new changes, makes any necessary
doxygen modifications, and commits the result. svn can be scripted for
these kinds of purposes.  See the --non-interactive and --xml options
that many svn commands support.

> 3) Add information in the code in the workspace, from tags included
> in the repository
> The 2 first use cases need a filter between the workspace and the
> repository. The 3rd one needs a filter between the workspace and the
> internal or external diff.

I don't think I understand 3), especially what diff has got to do
with it. What is this for?

Is this information meant for interactive use by developers while
developing? Or is it some extended information of the sort that
svn:keywords supports, which is embedded in files you ship to
customers (outside the version control system)?

In the latter case, you might as well use a separate working copy
to add this extra information, and commit it from there, instead
of forcing unrelated changes into developer working copies.

I realise that git can easily hide modifications by automatically
making them in the index instead of the working tree. So if you're
adding information which developers don't really need to see on
they fly while files are being added to the index, this doesn't
bother them much.

However, Subversion does not have a concept of an "index" like git does.
There is only a working copy and a repository. So you're really talking
about making edits to peoples' working files, which may or may not be
desirable, depending on the use case. And keep in mind that developers
might have their files opened up in editors while they run 'svn update'
or 'svn commit', and you would be making changes to the files concurrently.
I'd imagine that could lead to unexpected results.

My advice would be to use automated jobs that use separate working
copies, if at all possible, and leave developer working copies alone.
They'll get to see modifications made by automated commits just like
any other commits when they run 'svn update'.

Re: Git smudge / Clean / Filter alike in Subversion ?

[Laurent Alebarde <l....@free.fr>]

Le 11/09/2012 15:49, Stefan Sperling a écrit :
> On Tue, Sep 11, 2012 at 02:45:49PM +0200, Laurent Alebarde wrote:
>> Thanks for your answer Stefan,
>>
>> Unfortunatly, no :
>>
>> The first link says Subversion is smart with binary files. That's
>> good to hear, but do not provide a filter or filter hook between the
>> workspace and the repository.
> Apart from built-in properties such as svn:keywords and svn:eol-style,
> there is no way to make changes to files during checkout or commit.
> You can probably use a wrapper script for this purpose, however,
> that wraps svn checkout, svn update, and svn commit.
>
> What do you really need this feature for? I'm interested in learning
> more about your actual use case, the actual problem you're trying to
> solve, rather than what git's solution to your problem is. Maybe if
> I knew more about the problem itself I could give you a better answer.
> And maybe we can add some feature to svn to solve your problem, once we
> understand the problem.
Sure, you are right. At present time, my use cases are :
1) Automatic coding style refactoring so that developpers remain happy 
with what they are used to : indentation, naming, layout, etc.
2) Automatic doxygen comments generation.
3) Add information in the code in the workspace, from tags included in 
the repository
The 2 first use cases need a filter between the workspace and the 
repository. The 3rd one needs a filter between the workspace and the 
internal or external diff.

Hope the clarification is right.
>> The second link says it can use external diff. But that's not what I
>> want to do. I just want to put a filter in the middle between the
>> files to diff and the internal diff.
> This is not supported. A workaround might be to use an external diff
> tool and filter its output instead.
No, from my use case 3, this is not a solution.

Re: Git smudge / Clean / Filter alike in Subversion ?

[Stefan Sperling <st...@elego.de>]

On Tue, Sep 11, 2012 at 02:45:49PM +0200, Laurent Alebarde wrote:
> Thanks for your answer Stefan,
> 
> Unfortunatly, no :
> 
> The first link says Subversion is smart with binary files. That's
> good to hear, but do not provide a filter or filter hook between the
> workspace and the repository.

Apart from built-in properties such as svn:keywords and svn:eol-style,
there is no way to make changes to files during checkout or commit.
You can probably use a wrapper script for this purpose, however,
that wraps svn checkout, svn update, and svn commit.

What do you really need this feature for? I'm interested in learning
more about your actual use case, the actual problem you're trying to
solve, rather than what git's solution to your problem is. Maybe if
I knew more about the problem itself I could give you a better answer.
And maybe we can add some feature to svn to solve your problem, once we
understand the problem.

> The second link says it can use external diff. But that's not what I
> want to do. I just want to put a filter in the middle between the
> files to diff and the internal diff.

This is not supported. A workaround might be to use an external diff
tool and filter its output instead.

Re: Git smudge / Clean / Filter alike in Subversion ?

[Laurent Alebarde <l....@free.fr>]

Thanks for your answer Stefan,

Unfortunatly, no :

The first link says Subversion is smart with binary files. That's good 
to hear, but do not provide a filter or filter hook between the 
workspace and the repository.

The second link says it can use external diff. But that's not what I 
want to do. I just want to put a filter in the middle between the files 
to diff and the internal diff.


Le 10/09/2012 16:02, Stefan Sperling a écrit :
> On Mon, Sep 10, 2012 at 03:38:10PM +0200, Laurent Alebarde wrote:
>> Hi all,
>>
>> I am looking for some Git smudge / clean capabilities in subversion
>> to be able to "clean" code before pushing it into the repository, as
>> long as filters do be able to diff binary files with for example
>> some embedded tags.
>>
>> Here is FYI the Git documentation page that explain it exactly :
>> http://git-scm.com/book/en/Customizing-Git-Git-Attributes
>>
>> Any help please ?
>>
>> Laurent.
> See http://svnbook.red-bean.com/en/1.7/svn.forcvs.binary-and-trans.html
> and http://svnbook.red-bean.com/en/1.7/svn.advanced.externaldifftools.html
> Does that do what you want?

Re: Git smudge / Clean / Filter alike in Subversion ?

[Stefan Sperling <st...@elego.de>]

On Mon, Sep 10, 2012 at 03:38:10PM +0200, Laurent Alebarde wrote:
> Hi all,
> 
> I am looking for some Git smudge / clean capabilities in subversion
> to be able to "clean" code before pushing it into the repository, as
> long as filters do be able to diff binary files with for example
> some embedded tags.
> 
> Here is FYI the Git documentation page that explain it exactly :
> http://git-scm.com/book/en/Customizing-Git-Git-Attributes
> 
> Any help please ?
> 
> Laurent.

See http://svnbook.red-bean.com/en/1.7/svn.forcvs.binary-and-trans.html
and http://svnbook.red-bean.com/en/1.7/svn.advanced.externaldifftools.html
Does that do what you want?