You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by kf...@collab.net on 2005/05/27 19:23:45 UTC
Repository-defined autoprops.
This mail is about propagating repository-defined autoprops to
clients, a feature we don't have, but wish we did.
The Current Situation:
======================
Subversion's autoprop support is documented at:
http://svnbook.red-bean.com/en/1.1/svn-book.html#svn-ch-7-sect-2.4
For convenience, I will summarize it here.
Autoprops are properties that get automatically set on a file when it
is added or imported. (Thus, the only operations under discussion in
this mail are 'add' and 'import'.) Each client defines its autoprop
mapping independently, even if it's the same mapping used by all the
other clients talking to a given repository. The repository has no
way to broadcast autoprops; it all happens client-side.
A client specifies autoprops by setting 'enable-auto-props=yes' in the
'[miscellany]' section of ~/.subversion/config. This makes Subversion
pay attention to the '[autoprops]' section of the same file. In that
section, each line is of the form "PATTERN = PROPNAME=PROPVALUE",
where PATTERN is a file-globbing pattern, and PROPNAME and PROPVALUE
are what you think they are. If a file matches multiple PATTERNS,
it's random which one gets used -- the order in the config file does
not matter. (I suspect this is because they get hashed internally?)
As far as I can tell, autoprops work only on files, not on
directories. In typical use, this probably doesn't matter, since
people mostly use it for mime-types, eol-styles, and keywords.
Also note that we currently cannot have multiline property values.
Problems:
=========
This design makes it difficult for a repository (or a project) to
specify autoprop policy. The admin has to notify all users what the
autoprops should be, out-of-band, and the users have to manually make
the right adjustments to their ~/.subversion/config files. If the
policy is updated later, there is no automated way to transmit the
update. The admin has to send out another email, and everyone has to
make the adjustments manually again.
What Should A Solution Look Like?:
==================================
Before we can design a solution (and especially, before we can know
whether it might share any mechanism with log message templates), we
need to know what we want.
Q: Is disconnected operation important?
A: Yes, because 'svn add' is one of the actions.
Q: Is operation in the absence of a working copy important?
A: Yes, because 'svn import' is one of the actions.
Q: Is autoprops for directory paths important?
A: No, I don't think so (but I'm not positive about that).
Q: Is it enough to specify autoprops per-repository, or do they need
to be per-project, which in Subversion means per-directory?
A: I think per-directory is significantly better.
I didn't always think this. At first I thought per-repository
would be good enough. After all, the most common use cases are
svn:mime-type, svn:eol-style, and svn:keywords. I would think
that these three properties rarely need different mapping rules
for different projects in the same repository. Well, different
projects might have different policies about svn:keywords, but
probably not for svn:mime-type or svn:eol-style.
But anyway, projects may have other properties (not even
necessarily "svn:foo" properties) that they need to map.
Although I can't come up with a concrete example right now of
when different projects might need incompatible mappings, still,
this "feels like" a plausible use case. This is also why I'm
not sure whether directory support is important.
So there you have it. We want:
* Disconnected operation with a working copy.
* Connected operation without a working copy.
* Per-directory instead of just per-repository (maybe -- I at least
am not sure about this).
* It's an open question whether mapping directoryglobs<->props is
important. Right now my guess is it's not.
This might be a tall order.
Frankly, the only good solutions that come to mind involve inherited
properties. I don't, however, have a good specification for how
inherited properties should behave, nor how to implement them.
I just started to write up a non-iprop solution, and then erased it
after realizing the complexity of this problem.
So I guess I'll just post this now and at least see if we generally
agree on the problem & solution space. If anyone has any bright ideas
for how to do autoprops right, please post.
Confidential to Brane in Ljubljana: I now have a retroactive
motivation for why I started the log message template discussion first
(aside from the fact that it's more important to CollabNet, which I
certainly don't want to hide). It's that log message templates are a
simpler problem, and therefore lower-hanging fruit, because we don't
really need to support disconnected operation (which we've discussed
in another thread, so I won't repeat why here).
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository-defined autoprops.
Posted by steveking <st...@gmx.ch>.
kfogel@collab.net wrote:
> This mail is about propagating repository-defined autoprops to
> clients, a feature we don't have, but wish we did.
>
> The Current Situation:
> ======================
>
> Subversion's autoprop support is documented at:
>
> http://svnbook.red-bean.com/en/1.1/svn-book.html#svn-ch-7-sect-2.4
>
> For convenience, I will summarize it here.
[snip]
> As far as I can tell, autoprops work only on files, not on
> directories. In typical use, this probably doesn't matter, since
> people mostly use it for mime-types, eol-styles, and keywords.
Yes, they only work on files (at least, last time I checked). That's the
reason why svn:ignore can't be used with autoprops.
> Also note that we currently cannot have multiline property values.
Would also be a problem for svn:ignore.
> Problems:
> =========
>
> This design makes it difficult for a repository (or a project) to
> specify autoprop policy. The admin has to notify all users what the
> autoprops should be, out-of-band, and the users have to manually make
> the right adjustments to their ~/.subversion/config files. If the
> policy is updated later, there is no automated way to transmit the
> update. The admin has to send out another email, and everyone has to
> make the adjustments manually again.
>
> What Should A Solution Look Like?:
> ==================================
>
> Before we can design a solution (and especially, before we can know
> whether it might share any mechanism with log message templates), we
> need to know what we want.
>
> Q: Is disconnected operation important?
>
> A: Yes, because 'svn add' is one of the actions.
>
> Q: Is operation in the absence of a working copy important?
>
> A: Yes, because 'svn import' is one of the actions.
IMHO this isn't important. Usually, you use import when you start a
project. Once you've imported all the files, you then start setting up
policies (or am I completely wrong here?).
> Q: Is autoprops for directory paths important?
>
> A: No, I don't think so (but I'm not positive about that).
I think that one is important. At least, if this not only has to work
for the above mentioned Subversion properties. TSVN for example uses
properties on folders as a replacement for certain server dictated settings.
> Q: Is it enough to specify autoprops per-repository, or do they need
> to be per-project, which in Subversion means per-directory?
>
> A: I think per-directory is significantly better.
>
> I didn't always think this. At first I thought per-repository
> would be good enough. After all, the most common use cases are
> svn:mime-type, svn:eol-style, and svn:keywords. I would think
> that these three properties rarely need different mapping rules
> for different projects in the same repository. Well, different
> projects might have different policies about svn:keywords, but
> probably not for svn:mime-type or svn:eol-style.
>
> But anyway, projects may have other properties (not even
> necessarily "svn:foo" properties) that they need to map.
> Although I can't come up with a concrete example right now of
> when different projects might need incompatible mappings, still,
> this "feels like" a plausible use case. This is also why I'm
> not sure whether directory support is important.
If you look at the properties TSVN already has defined to use as project
settings, I think per-directory would be great.
> So there you have it. We want:
>
> * Disconnected operation with a working copy.
>
> * Connected operation without a working copy.
>
> * Per-directory instead of just per-repository (maybe -- I at least
> am not sure about this).
>
> * It's an open question whether mapping directoryglobs<->props is
> important. Right now my guess is it's not.
>
> This might be a tall order.
>
> Frankly, the only good solutions that come to mind involve inherited
> properties. I don't, however, have a good specification for how
> inherited properties should behave, nor how to implement them.
I don't know how this would be implemented, but this is what I think
inherited properties should behave:
- props get inherited from the directory tree downwards (parent to children)
- if a child dir/file has the same property set as the parent, the child
has priority (this so that subprojects can override the settings of
parent projects, i.e. define their own props/rules/settings)
- if a child has the same property set, but the first line of it's value
is e.g. '*', the childs props don't overwrite the parent props but add
to them (useful for svn:ignore, svn:mime-type, ...)
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository-defined autoprops.
Posted by Jani Averbach <ja...@jaa.iki.fi>.
On 2005-05-27 15:29-0500, kfogel@collab.net wrote:
>
> No need to have sections, it can just use paths.
>
> However, one problem is that now everyone needs write access to a
> global file. What if the projects do not trust each other, but still
> share a repository?
One option is that we have a that special directory (defined by
property), and under it we have subdirectories, one per project. This
way different projects under same repository can protect their own
config file. Logic to look for the spesific config file could be:
for path /project_1/trunk/foo.c:
if /config/project_1:
# This is editable by project_1
use /config/project_1/global.cfg
elif /config/global.cfg:
# this is set up by repository admin
use /config/global.cfg
else:
use default
BR, Jani
--
Jani Averbach
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository-defined autoprops.
Posted by Greg Hudson <gh...@MIT.EDU>.
On Sat, 2005-05-28 at 09:21 -0500, kfogel@collab.net wrote:
> I'd like to avoid having Subversion impose political structure, as
> much as possible. If people *choose* to put access control on who can
> set up log message templates or autoprops, that's fine, but we
> shouldn't force that choice on them.
I also prefer a solution which does not involve a centrally located file
containing path entries, for this reason.
I also feel like any solution which puts a config file into the
repository is a little redundant. A config file is a way of marshalling
a <string,string> -> string mapping into a byte stream; we already have
a string -> string mapping in the repository in the form of properties.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository-defined autoprops.
Posted by kf...@collab.net.
SteveKing <st...@gmx.ch> writes:
> No need to give everyone write access to that file. Only project
> admins would really need write access. And if the project admins don't
> trust each other, the repo admin would have to do the job. Basically
> the same as with hook scripts - users don't usually have access to
> those either.
What's a "project admin"? :-)
I'd like to avoid having Subversion impose political structure, as
much as possible. If people *choose* to put access control on who can
set up log message templates or autoprops, that's fine, but we
shouldn't force that choice on them.
Note that there is no project admin in the Subversion project itself,
for example. We have a few people who have root on the repository
box, but there's absolutely no reason those people should have special
control over log message template or autoprops. If we had those
features in our repository, they'd probably be editable by all full
committers.
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository-defined autoprops.
Posted by SteveKing <st...@gmx.ch>.
kfogel@collab.net wrote:
> SteveKing <st...@gmx.ch> writes:
>
>>Another idea comes to mind. Instead of using inherited properties, why
>>not use a specially named config file stored in the repository?
>>If per-directory isn't needed, a somewhat simpler solution could be:
>>
>>- if a file named e.g. .repoconfig is in stored in the repository
>>root, then use this as the repository config file (same format as the
>>config file now has).
>>- After every update/checkout/... (operations that update the working
>>copy) also fetch the config file
>>- the config file is stored in a subfolder in the Subversion config
>>folder (%APPDATA%\Subversion\repoconfigs\{repo-uuid} on windows for
>>example).
>
>
> This was exactly the proposal I wrote and then erased :-), yeah.
I've also documented this in issue 1974
http://subversion.tigris.org/issues/show_bug.cgi?id=1974
Just curious: why did you erase the proposal?
>>Advantage:
>>- easyer to implement (I think)
>>- same format as the normal config file, so user already know how it works
>>- only one copy of the configuration stored, not on each and every
>>file/folder in the working copy as with inherited properties
>>
>>Disadvantage:
>>- only per-repository, not per-folder (even though the config file
>>could have sections for subfolders)
>
>
> No need to have sections, it can just use paths.
>
> However, one problem is that now everyone needs write access to a
> global file. What if the projects do not trust each other, but still
> share a repository?
No need to give everyone write access to that file. Only project admins
would really need write access. And if the project admins don't trust
each other, the repo admin would have to do the job. Basically the same
as with hook scripts - users don't usually have access to those either.
>>- to edit that file, one has to check out the repository root
>>(non-recursively), edit the file and commit it
>
> Better to have it in a subdirectory of the root. Greg Stein has a
> great solution for this:
>
> You put it in a subfolder named "/config". But, that's configurable,
> via an svn:config-dir property set on the root directory of the
> repository! So if people need to use "/config" for something else,
> they can just set that property. Subversion would always look where
> the property points; if no property, then default is "/config".
That's even better!
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository-defined autoprops.
Posted by Charles Bailey <ba...@gmail.com>.
On 5/27/05, Chris Foote <cf...@v21.me.uk> wrote:
> ----- Original Message -----
> From: <kf...@collab.net>
> To: "SteveKing" <st...@gmx.ch>
> Sent: Friday, May 27, 2005 9:29 PM
>
>
> > > Disadvantage:
> > > - only per-repository, not per-folder (even though the config file
> > > could have sections for subfolders)
> >
> > No need to have sections, it can just use paths.
> >
> Using sections would make the file easier to read/use IMO.
>
> Suppose you want to apply eol=native in all directories except those
> under the /projA/trunk directory.
>
> e.g. no sections
> [auto-props]
> *.c = svn:eol-style=native
> *.h = svn:eol-style=native
> /projA/trunk/*.c = svn:eol-style=CRLF
> /projA/trunk/*.h = svn:eol-style=CRLF
This also has the (previously noted by implication) problem that order
of application is not guaranteed now; this would have to change to
permit "overriding" of props. I can see reasonable arguments for
either (reverse-)order-appearing-in-config-file or length-of-match
precedence.
--
Regards,
Charles Bailey
Lists: bailey _dot_ charles _at_ gmail _dot_ com
Other: bailey _at_ newman _dot_ upenn _dot_ edu
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository-defined autoprops.
Posted by Chris Foote <cf...@v21.me.uk>.
----- Original Message -----
From: <kf...@collab.net>
To: "SteveKing" <st...@gmx.ch>
Sent: Friday, May 27, 2005 9:29 PM
> > Disadvantage:
> > - only per-repository, not per-folder (even though the config file
> > could have sections for subfolders)
>
> No need to have sections, it can just use paths.
>
Using sections would make the file easier to read/use IMO.
Suppose you want to apply eol=native in all directories except those
under the /projA/trunk directory.
e.g. no sections
[auto-props]
*.c = svn:eol-style=native
*.h = svn:eol-style=native
/projA/trunk/*.c = svn:eol-style=CRLF
/projA/trunk/*.h = svn:eol-style=CRLF
e.g. using sections.
[auto-props]
*.c = svn:eol-style=native
*.h = svn:eol-style=native
[ap:/projA/trunk]
*.c = svn:eol-style=CRLF
*.h = svn:eol-style=CRLF
Without using sections the [auto-props] list would soon get very long
and with a lot of repetition of paths.
> Better to have it in a subdirectory of the root. Greg Stein has a
> great solution for this:
>
> You put it in a subfolder named "/config". But, that's configurable,
> via an svn:config-dir property set on the root directory of the
> repository! So if people need to use "/config" for something else,
> they can just set that property. Subversion would always look where
> the property points; if no property, then default is "/config".
>
Is there any reason that this config couldn't be just a file in the repos
conf directory (where svnserve.conf is stored)?
As there is no need for the repository config to be versioned, is there?
--
Chris
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository-defined autoprops.
Posted by kf...@collab.net.
SteveKing <st...@gmx.ch> writes:
> Another idea comes to mind. Instead of using inherited properties, why
> not use a specially named config file stored in the repository?
> If per-directory isn't needed, a somewhat simpler solution could be:
>
> - if a file named e.g. .repoconfig is in stored in the repository
> root, then use this as the repository config file (same format as the
> config file now has).
> - After every update/checkout/... (operations that update the working
> copy) also fetch the config file
> - the config file is stored in a subfolder in the Subversion config
> folder (%APPDATA%\Subversion\repoconfigs\{repo-uuid} on windows for
> example).
This was exactly the proposal I wrote and then erased :-), yeah.
> Advantage:
> - easyer to implement (I think)
> - same format as the normal config file, so user already know how it works
> - only one copy of the configuration stored, not on each and every
> file/folder in the working copy as with inherited properties
>
> Disadvantage:
> - only per-repository, not per-folder (even though the config file
> could have sections for subfolders)
No need to have sections, it can just use paths.
However, one problem is that now everyone needs write access to a
global file. What if the projects do not trust each other, but still
share a repository?
> - to edit that file, one has to check out the repository root
> (non-recursively), edit the file and commit it
Better to have it in a subdirectory of the root. Greg Stein has a
great solution for this:
You put it in a subfolder named "/config". But, that's configurable,
via an svn:config-dir property set on the root directory of the
repository! So if people need to use "/config" for something else,
they can just set that property. Subversion would always look where
the property points; if no property, then default is "/config".
> - if a project already has such a file in the repository root stored,
> this change would break. Although I don't think that any project out
> there really has such a file stored there. If in doubt, the file could
> be named with a UUID.
See above about using a special property to redirect.
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository-defined autoprops.
Posted by SteveKing <st...@gmx.ch>.
kfogel@collab.net wrote:
> This mail is about propagating repository-defined autoprops to
> clients, a feature we don't have, but wish we did.
[snip]
> So there you have it. We want:
>
> * Disconnected operation with a working copy.
>
> * Connected operation without a working copy.
>
> * Per-directory instead of just per-repository (maybe -- I at least
> am not sure about this).
>
> * It's an open question whether mapping directoryglobs<->props is
> important. Right now my guess is it's not.
>
> This might be a tall order.
>
> Frankly, the only good solutions that come to mind involve inherited
> properties. I don't, however, have a good specification for how
> inherited properties should behave, nor how to implement them.
Another idea comes to mind. Instead of using inherited properties, why
not use a specially named config file stored in the repository?
If per-directory isn't needed, a somewhat simpler solution could be:
- if a file named e.g. .repoconfig is in stored in the repository root,
then use this as the repository config file (same format as the config
file now has).
- After every update/checkout/... (operations that update the working
copy) also fetch the config file
- the config file is stored in a subfolder in the Subversion config
folder (%APPDATA%\Subversion\repoconfigs\{repo-uuid} on windows for
example).
Advantage:
- easyer to implement (I think)
- same format as the normal config file, so user already know how it works
- only one copy of the configuration stored, not on each and every
file/folder in the working copy as with inherited properties
Disadvantage:
- only per-repository, not per-folder (even though the config file could
have sections for subfolders)
- to edit that file, one has to check out the repository root
(non-recursively), edit the file and commit it
- if a project already has such a file in the repository root stored,
this change would break. Although I don't think that any project out
there really has such a file stored there. If in doubt, the file could
be named with a UUID.
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository-defined autoprops.
Posted by David Wilson <dw...@botanicus.net>.
kfogel@collab.net wrote:
> So I guess I'll just post this now and at least see if we generally
> agree on the problem & solution space. If anyone has any bright ideas
> for how to do autoprops right, please post.
I'm not entirely sure whether this is a bright idea, but it may solve
the problem of storage and disconnected operation:
As Greg Hudson mentioned, autoprops are simple (<key>, <value>) lists
already, and AFAICS treated as apr_hashes in the client. What issues
would be introduced if the server itself was solely responsible for
storing and applying the autoprops during a commit operation?
In that scheme, the Subversion client need not be modified to support
server-side autoprops, and backwards compatibility with older clients is
maintained by default. The actual list of autoprops could be stored as
an svn_hash somewhere in the repository directory, and possibly edited
via the svnadmin tool.
Thinking over this, after a commit WCs could be in an inconsistent state
as the WC props for a given file may not reflect the state on the
server, since the server added extra props that the client did not send.
I don't understand enough about Subversions internals to know if this is
a problem.
Another problem with this scheme would be how it interacts with older
client accesses via file:// schemes. Actually, I'll stop attempting to
contribute there.. This is too complex a problem for my rookie knowledge. ;)
Just some thoughts for the fire,
David.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org