You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Nico Kadel-Garcia <nk...@gmail.com> on 2014/02/01 05:00:56 UTC
Re: expanding custom keywords in dump
This is why you don't use keywords if you can avoid it, especially for
multi-environment projects. Frankly, you will often have diffs with
keywords and "svn:eol-style": Don't evey try to pretend that anything
with keywords is going to be byte for byte consistent between working
environments.
And this is *especially* why you don't use "custom keywords". They're
not porable.
On Fri, Jan 31, 2014 at 8:27 AM, Thorsten Schöning
<ts...@am-soft.de> wrote:
> Guten Tag Julian Elischer,
> am Freitag, 31. Januar 2014 um 11:07 schrieben Sie:
>
>> [...]"$FreeBSD$" keyword is left unexpanded on export.
>
> This doesn't seem to be a keyword which svn clients expand by default,
> therefore you would have two building lots.
>
> http://svnbook.red-bean.com/en/1.8/svn.advanced.props.special.keywords.html
>
> Mit freundlichen Grüßen,
>
> Thorsten Schöning
>
> --
> Thorsten Schöning E-Mail:Thorsten.Schoening@AM-SoFT.de
> AM-SoFT IT-Systeme http://www.AM-SoFT.de/
>
> Telefon...........05151- 9468- 55
> Fax...............05151- 9468- 88
> Mobil..............0178-8 9468- 04
>
> AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
> AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
>
Re: expanding custom keywords in dump
Posted by Thorsten Schöning <ts...@am-soft.de>.
Guten Tag Ben Reser,
am Montag, 3. Februar 2014 um 17:49 schrieben Sie:
> Sorry, I thought you realized they'd added it because the link you provided
> documented how to use custom keywords.
No, I totally missed that feature and therefore didn't even read the
documentation again. Thanks for the hint.
Mit freundlichen Grüßen,
Thorsten Schöning
--
Thorsten Schöning E-Mail:Thorsten.Schoening@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/
Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04
AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Re: expanding custom keywords in dump
Posted by Ben Reser <be...@reser.org>.
On 2/1/14, 1:24 AM, Thorsten Schöning wrote:
> I wasn't criticizing anything. Just thought that expanding $FreeBSD$
> may have been a feature the FreeBSD guys patched into Subversion on
> their own, because some days ago it has been mentioned that FreeBSD
> patches subversion to get some features they need. If that is the
> case the questioner would need more than just svndump expanding
> keywords, because it would likely not support anything else than what
> svn clients do now. It was just a hint coming into my mind reading
> about the problem, no need to overestimate.
Sorry, I thought you realized they'd added it because the link you provided
documented how to use custom keywords.
Re: expanding custom keywords in dump
Posted by Ben Reser <be...@reser.org>.
On 2/3/14, 4:19 AM, Branko Čibej wrote:
> The patch isn't needed for 1.8, the feature is now standard in Subversion.
http://subversion.apache.org/docs/release-notes/1.8.html#custom-keywords
Re: expanding custom keywords in dump
Posted by Branko Čibej <br...@wandisco.com>.
On 03.02.2014 13:00, Tony Sweeney wrote:
> It certainly appears to be the case that one of the FreeBSD patches is to expand this keyword in Subversion 1.6 and 1.7.
> The actual patches are available here:
>
> svn://svn0.us-east.freebsd.org/ports/head/devel/subversion16
> svn://svn0.us-east.freebsd.org/ports/head/devel/subversion17
>
> They don't seem to have gotten round to putting this patch into the 1.8 port, though.
>
> svn://svn0.us-east.freebsd.org/ports/head/devel/subversion
The patch isn't needed for 1.8, the feature is now standard in Subversion.
-- Brane
--
Branko Čibej | Director of Subversion
WANdisco // Non-Stop Data
e. brane@wandisco.com
RE: expanding custom keywords in dump
Posted by Tony Sweeney <ts...@omnifone.com>.
> -----Original Message-----
> From: Thorsten Schöning [mailto:tschoening@am-soft.de]
> Sent: 01 February 2014 09:24
> To: Subversion
> Subject: Re: expanding custom keywords in dump
>
> Guten Tag Ben Reser,
> am Samstag, 1. Februar 2014 um 07:07 schrieben Sie:
>
> > So if you're going to critique their technique[...]
>
> I wasn't criticizing anything. Just thought that expanding
> $FreeBSD$ may have been a feature the FreeBSD guys patched
> into Subversion on their own, because some days ago it has
> been mentioned that FreeBSD patches subversion to get some
> features they need. If that is the case the questioner would
> need more than just svndump expanding keywords, because it
> would likely not support anything else than what svn clients
> do now.
It certainly appears to be the case that one of the FreeBSD patches is to expand this keyword in Subversion 1.6 and 1.7.
The actual patches are available here:
svn://svn0.us-east.freebsd.org/ports/head/devel/subversion16
svn://svn0.us-east.freebsd.org/ports/head/devel/subversion17
They don't seem to have gotten round to putting this patch into the 1.8 port, though.
svn://svn0.us-east.freebsd.org/ports/head/devel/subversion
There is enough information at those locations to let you manually recreate their patch for your own platform, should you wish. You probably don't need all of the patches, since most are specific to the FreeBSD ports tree build infrastructure, and you're unlikely to want their logging changes which appear to be intended to support their internal change management systems.
Tony.
> It was just a hint coming into my mind reading about
> the problem, no need to overestimate.
>
> Mit freundlichen Grüßen,
>
> Thorsten Schöning
>
> --
> Thorsten Schöning E-Mail:Thorsten.Schoening@AM-SoFT.de
> AM-SoFT IT-Systeme http://www.AM-SoFT.de/
>
> Telefon...........05151- 9468- 55
> Fax...............05151- 9468- 88
> Mobil..............0178-8 9468- 04
>
> AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
> AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email
> Security.cloud service.
> For more information please visit
> http://www.symanteccloud.com
> ______________________________________________________________________
>
> -----
> No virus found in this message.
> Checked by AVG - www.avg.com
> Version: 2014.0.4259 / Virus Database: 3681/7023 - Release
> Date: 01/21/14 Internal Virus Database is out of date.
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________
Re: expanding custom keywords in dump
Posted by Thorsten Schöning <ts...@am-soft.de>.
Guten Tag Ben Reser,
am Samstag, 1. Februar 2014 um 07:07 schrieben Sie:
> So if you're going to critique their technique[...]
I wasn't criticizing anything. Just thought that expanding $FreeBSD$
may have been a feature the FreeBSD guys patched into Subversion on
their own, because some days ago it has been mentioned that FreeBSD
patches subversion to get some features they need. If that is the
case the questioner would need more than just svndump expanding
keywords, because it would likely not support anything else than what
svn clients do now. It was just a hint coming into my mind reading
about the problem, no need to overestimate.
Mit freundlichen Grüßen,
Thorsten Schöning
--
Thorsten Schöning E-Mail:Thorsten.Schoening@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/
Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04
AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Re: expanding custom keywords in dump
Posted by Nico Kadel-Garcia <nk...@gmail.com>.
Branko, have I ever mentioned how nice it is to see employees of the
companies producing the core source code doing thoughtful posts? It's
one of the main reasons I've appreciated Subversion over the years.
On Sun, Feb 2, 2014 at 1:41 AM, Branko Čibej <br...@wandisco.com> wrote:
> On 02.02.2014 04:14, Nico Kadel-Garcia wrote:
>> On Sat, Feb 1, 2014 at 1:07 AM, Ben Reser <be...@reser.org> wrote:
>>
>>> Branko gave a perfectly reasonable answer. Beyond that I honestly don't get
>>> the point of these two emails. FreeBSD uses keywords because as an open source
>>> project they ship source. Even more importantly they have downstream projects
>>> (e.g. Apple uses their find command
>>> http://opensource.apple.com/source/shell_cmds/shell_cmds-175/find/main.c ). I
>>> can't think of a better way of tracking versioning for them once the source
>>> leaves their version control system and potentially goes into another. Yes
>>> there are all sorts of annoying bits about this.
>> If your build system has to rely on source control based fields,
>> process the code in your build system. Putting the keyword processing
>> in the source control itself certainly dates back ti RCS and CVS, and
>> has been the bane of comparing source trees in working copies, and of
>> of actually reviewing the source control that will be used for
>> building software. It profoundly interferes with generating
>> replicatable code in multiple build or test environments.
>
> You're totally missing the point of this thread. No-one said anything
> about build systems; the original poster's requirement is tracking
> upstream versions of files, not complete source trees. No amount of
> *.h.in magic is going to do that.
Build systems are the strongest reason, I think, to *want* the
keywords. If you're in a working copy of a source tree, or a source
controlled website, bulky nest of configuration files, or other such
environment, you've usually got access to the source control logs. If
you then need to include the "Author" or "Date" or "URL" for the
source, that's understandable. I've worked with that. I applied that
in particular to a build system for "kickstart" files, where I wanted
the "HeadURL" and "Revision" for each script added to the "pre" and
"post" sections to have that keyword content, and the entire generated
kickstart files to themselves be timestamped and have their own,
individual "HeadURL" and "Revison"
I wound up tweaking the system to do just what I said: prepend the
relevant information to each ""pre" and "post" script when it was
added to the master file, and put a header on top of the kickstart
files to denote the assembled file build and SVN Revision and HeadURL.
Then I committed *that*. That way, no keywords, and when I created new
branches and variants of the original code, I didn't have to remember
to hand-set "svn:keywords" or to inject a "pre-commit" file to ensure
consistent use in new files.
Doing it as a post-checkout process frees up the developer to use more
arbitrary processing. It does take time to learn and to write tools to
match.
> Obviously, in an ideal world, one would be able to migrate these kind of
> metadata between different VCS. Likewise obviously, the world is not
> ideal, and in-file keywords are a reasonable alternative if no other
> tooling can be devised. In the case of Subversion->Perforce migration,
> one could argue that it's Perforce's fault for not having an equivalent
> to Subversion's properties that could store the source repo metadata.
The in-file keywords are an easy first approach, and I admit they're
common. The problems occur over time, and with advanced use. It's
fairly unreasonable to expect Perforce tooling to be able to support a
3rd party add-on, such as the "FreeBSD" keyword, that is not in
Subversion's main codeline.
> Compared to inventing a separate-but-parallel database for maintaining
> these metadata, and all the surrounding tooling that this implies,
> expanded keywords in the files themselves appear positively benign,
> especially when they're not going to change except from further upstream
> imports.
Wll, that's just what the FreeBSD people did by adding a new keyword.
They used the Subversion repository metadata. That data is clearly
derivable at working copy creation time. That's how the keywords get
filled in anyway by the keyword expansion, so it's not a metadata
invention or storage problem there. By generating those keyword
fillins as a post-export or post checkout process, arbitrary keywords
can be created and inserted on a consistent basis *without* the
confusion and inconsistencies inevitable in directly keyword expanding
the source control file content. The developer is already using a
"separate database", namely the Subversion metadata that is not
inherent to the file content, that means a gain in flexibility and in
maintenance of "these files are keyword enabled" and "these files are
not.
Wha'ts lost is the "automagic" nature of the keyword expansion in the
primary working copy. And that's always been problematic when
stressed, even if it's made some code easier to read for new users.
I'm a strong promoter of the same approach to end-of-line handling.
Working environment processing and re-normalization of upstream source
is safer done on a case by case basis, as a build or deployment
process, and preserving the contents of the actual repository in a
pristine state. Keywords demand checkout post-processing, anyway.
Keeping it in the build or deployment system, as part of generating
the modified files *after* checkout, can allow far more flexibility.
> Last but not least ... Subversion does not expand keywords unless
> explicitly told to. This was a conscious decision we made to discourage
> exactly the kind of abuse you're griping against. But you'll have a hard
> time to find a single VCS glove that fits all potential users' feet.
Amen!!! I'm very, very glad that Subversion did *not* enable keywords
by default. Some of the worst difficulties I've had were when people
started enabling keywords erratically. Files get duplicated or added
without keywords enabled, then later keywords are enabled, and the use
of the "svn diff" command wound up burdened by spurious keyword
differences. And the "diffs" between two developers' working copies of
the same code lead to complex and unstable editing of local "diff"
commands and diff outputs, and truly horrendous nightmares of "patch"
changes to negotiate the inconsistent use.
I've got a long tale of woe about Perforce source code management of a
Linux kernel that I'll tell sometime, with a different story, if
people like.
RE: expanding custom keywords in dump
Posted by Tony Sweeney <ts...@omnifone.com>.
> -----Original Message-----
> From: Branko Čibej [mailto:brane@wandisco.com]
> Sent: 02 February 2014 06:42
> To: users@subversion.apache.org
> Subject: Re: expanding custom keywords in dump
>
> On 02.02.2014 04:14, Nico Kadel-Garcia wrote:
> > On Sat, Feb 1, 2014 at 1:07 AM, Ben Reser <be...@reser.org> wrote:
> >
> >> Branko gave a perfectly reasonable answer. Beyond that I honestly
> >> don't get the point of these two emails. FreeBSD uses keywords
> >> because as an open source project they ship source. Even more
> >> importantly they have downstream projects (e.g. Apple uses
> their find
> >> command
> >>
> http://opensource.apple.com/source/shell_cmds/shell_cmds-175/find/mai
> >> n.c ). I can't think of a better way of tracking
> versioning for them
> >> once the source leaves their version control system and
> potentially goes into another. Yes there are all sorts of
> annoying bits about this.
> > If your build system has to rely on source control based fields,
> > process the code in your build system. Putting the keyword
> processing
> > in the source control itself certainly dates back ti RCS
> and CVS, and
> > has been the bane of comparing source trees in working
> copies, and of
> > of actually reviewing the source control that will be used for
> > building software. It profoundly interferes with generating
> > replicatable code in multiple build or test environments.
>
> You're totally missing the point of this thread. No-one said
> anything about build systems; the original poster's
> requirement is tracking upstream versions of files, not
> complete source trees. No amount of *.h.in magic is going to do that.
>
> Obviously, in an ideal world, one would be able to migrate
> these kind of metadata between different VCS. Likewise
> obviously, the world is not ideal, and in-file keywords are a
> reasonable alternative if no other tooling can be devised. In
> the case of Subversion->Perforce migration, one could argue
> that it's Perforce's fault for not having an equivalent to
> Subversion's properties that could store the source repo metadata.
http://www.perforce.com/perforce/r12.1/manuals/cmdref/attribute.html
> Compared to inventing a separate-but-parallel database for
> maintaining these metadata, and all the surrounding tooling
> that this implies, expanded keywords in the files themselves
> appear positively benign, especially when they're not going
> to change except from further upstream imports.
>
> Last but not least ... Subversion does not expand keywords
> unless explicitly told to. This was a conscious decision we
> made to discourage exactly the kind of abuse you're griping
> against. But you'll have a hard time to find a single VCS
> glove that fits all potential users' feet.
>
> -- Brane
>
>
> --
> Branko Čibej | Director of Subversion
> WANdisco // Non-Stop Data
> e. brane@wandisco.com
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email
> Security.cloud service.
> For more information please visit
> http://www.symanteccloud.com
> ______________________________________________________________________
>
> -----
> No virus found in this message.
> Checked by AVG - www.avg.com
> Version: 2014.0.4259 / Virus Database: 3681/7023 - Release
> Date: 01/21/14 Internal Virus Database is out of date.
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________
Re: expanding custom keywords in dump
Posted by Branko Čibej <br...@wandisco.com>.
On 02.02.2014 04:14, Nico Kadel-Garcia wrote:
> On Sat, Feb 1, 2014 at 1:07 AM, Ben Reser <be...@reser.org> wrote:
>
>> Branko gave a perfectly reasonable answer. Beyond that I honestly don't get
>> the point of these two emails. FreeBSD uses keywords because as an open source
>> project they ship source. Even more importantly they have downstream projects
>> (e.g. Apple uses their find command
>> http://opensource.apple.com/source/shell_cmds/shell_cmds-175/find/main.c ). I
>> can't think of a better way of tracking versioning for them once the source
>> leaves their version control system and potentially goes into another. Yes
>> there are all sorts of annoying bits about this.
> If your build system has to rely on source control based fields,
> process the code in your build system. Putting the keyword processing
> in the source control itself certainly dates back ti RCS and CVS, and
> has been the bane of comparing source trees in working copies, and of
> of actually reviewing the source control that will be used for
> building software. It profoundly interferes with generating
> replicatable code in multiple build or test environments.
You're totally missing the point of this thread. No-one said anything
about build systems; the original poster's requirement is tracking
upstream versions of files, not complete source trees. No amount of
*.h.in magic is going to do that.
Obviously, in an ideal world, one would be able to migrate these kind of
metadata between different VCS. Likewise obviously, the world is not
ideal, and in-file keywords are a reasonable alternative if no other
tooling can be devised. In the case of Subversion->Perforce migration,
one could argue that it's Perforce's fault for not having an equivalent
to Subversion's properties that could store the source repo metadata.
Compared to inventing a separate-but-parallel database for maintaining
these metadata, and all the surrounding tooling that this implies,
expanded keywords in the files themselves appear positively benign,
especially when they're not going to change except from further upstream
imports.
Last but not least ... Subversion does not expand keywords unless
explicitly told to. This was a conscious decision we made to discourage
exactly the kind of abuse you're griping against. But you'll have a hard
time to find a single VCS glove that fits all potential users' feet.
-- Brane
--
Branko Čibej | Director of Subversion
WANdisco // Non-Stop Data
e. brane@wandisco.com
Re: expanding custom keywords in dump
Posted by Nico Kadel-Garcia <nk...@gmail.com>.
On Sat, Feb 1, 2014 at 1:07 AM, Ben Reser <be...@reser.org> wrote:
> Branko gave a perfectly reasonable answer. Beyond that I honestly don't get
> the point of these two emails. FreeBSD uses keywords because as an open source
> project they ship source. Even more importantly they have downstream projects
> (e.g. Apple uses their find command
> http://opensource.apple.com/source/shell_cmds/shell_cmds-175/find/main.c ). I
> can't think of a better way of tracking versioning for them once the source
> leaves their version control system and potentially goes into another. Yes
> there are all sorts of annoying bits about this.
If your build system has to rely on source control based fields,
process the code in your build system. Putting the keyword processing
in the source control itself certainly dates back ti RCS and CVS, and
has been the bane of comparing source trees in working copies, and of
of actually reviewing the source control that will be used for
building software. It profoundly interferes with generating
replicatable code in multiple build or test environments.
> Custom properties don't really make anything more difficult. Since custom
> properties are repository dictated configuration. In fact the custom property
> feature was written by the FreeBSD guys and then passed along upstream to ease
> this burden. They're not expecting Perforce to ever expand these. They stay
> literal in the source to show the base from upstream. The fact that Perforce
> doesn't expand it can be considered a feature.
I'd agree that Perforce's handling is correct. I'd also agree that
refusing the use of *any* keywords is, generally, correct.
> So if you're going to critique their technique, how about making a suggestion
> of a better technique. It's like a couple guys snickering in their Ferrari as
> a lorry goes by because he could have gotten there much faster in a Ferrari,
> even though the driver of the lorry only has the goal to get 10 tons of freight
> there not go fast.
Sorry if I sound cranky, I've been through this debate from various
angles. There are, indeed, short term conveniences in the legibility
of code when using keywords. But the long term consequences to stable
repositories of erratic disabling and re-enabling of keyword handling,
of cleaning up the accidental "diff" spew when it's handled
erratically, and the difficulty of code comparison in multiple platfom
projects continues to demonstrate its its awkwardness.
If you need to inject localized data in your source code, such as the
lat author or the upstream repository, that's what "filename.c.in" or
"filename.h.in" files are for. Process the files with local build or
copyright or whatever information as part of your software build
process, not in the basic source code. A commented "prepend" line, in
the relevant comment syntax, can be a savior and prevent internal
processing of text fields that happen to match the keyword
demarcations/
Re: expanding custom keywords in dump
Posted by Ben Reser <be...@reser.org>.
On 1/31/14, 8:00 PM, Nico Kadel-Garcia wrote:
> This is why you don't use keywords if you can avoid it, especially for
> multi-environment projects. Frankly, you will often have diffs with
> keywords and "svn:eol-style": Don't evey try to pretend that anything
> with keywords is going to be byte for byte consistent between working
> environments.
>
> And this is *especially* why you don't use "custom keywords". They're
> not porable.
>
> On Fri, Jan 31, 2014 at 8:27 AM, Thorsten Schöning
>> This doesn't seem to be a keyword which svn clients expand by default,
>> therefore you would have two building lots.
>>
>> http://svnbook.red-bean.com/en/1.8/svn.advanced.props.special.keywords.html
Branko gave a perfectly reasonable answer. Beyond that I honestly don't get
the point of these two emails. FreeBSD uses keywords because as an open source
project they ship source. Even more importantly they have downstream projects
(e.g. Apple uses their find command
http://opensource.apple.com/source/shell_cmds/shell_cmds-175/find/main.c ). I
can't think of a better way of tracking versioning for them once the source
leaves their version control system and potentially goes into another. Yes
there are all sorts of annoying bits about this.
Custom properties don't really make anything more difficult. Since custom
properties are repository dictated configuration. In fact the custom property
feature was written by the FreeBSD guys and then passed along upstream to ease
this burden. They're not expecting Perforce to ever expand these. They stay
literal in the source to show the base from upstream. The fact that Perforce
doesn't expand it can be considered a feature.
So if you're going to critique their technique, how about making a suggestion
of a better technique. It's like a couple guys snickering in their Ferrari as
a lorry goes by because he could have gotten there much faster in a Ferrari,
even though the driver of the lorry only has the goal to get 10 tons of freight
there not go fast.