You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Robbie Gibson <ro...@jenniandrobbie.com> on 2006/06/27 10:29:17 UTC
Complex tag creation performance
Hi,
We have a simple SVN setup where everybody works in the trunk. When we want
to put our work into the production system we do a selective update of the
files concerned - that is, we work on a file-by-file basis and hardly ever
do a global "svn up" on the production systems. To keep track of production
versions we create a complex tag with svn cp WC URL. This works fine,
except that after a few months we started noticing a slowdown in the tag
creation.
Investigations showed that the "svn cp" was checking the version of every
file to find the oldest revision and then using that as the basis for the
tag, plus changes since. As we never do a global update and some files
seldom change, the oldest revision could be 6 months ago - even if every
file is completely up to date!
To get over the problem I have written a small script which "updates" files
with no pending changes to the latest version (i.e. my script updates the
metadata for up-to-date files and directories). It's a bit slow but it
seems to help. I have three questions:
1) Is this behaviour described by design, or is it a bug?
2) Does anybody have a better suggestion for how to resolve my problem?
Thanks,
R
Re: Complex tag creation performance
Posted by Robbie Gibson <ro...@gmail.com>.
On 27/06/06, John Allen <jo...@moyville.net> wrote:
>
> Do the tagging using server side URLs not WC URLs.
> This will make it almost instanteous
Can't do this, since our WC is a mix of revisions.
R
Re: Complex tag creation performance
Posted by John Allen <jo...@moyville.net>.
Do the tagging using server side URLs not WC URLs.
This will make it almost instanteous
On Tuesday 27 June 2006 11:29, Robbie Gibson wrote:
> Hi,
> We have a simple SVN setup where everybody works in the trunk. When we
> want to put our work into the production system we do a selective update of
> the files concerned - that is, we work on a file-by-file basis and hardly
> ever do a global "svn up" on the production systems. To keep track of
> production versions we create a complex tag with svn cp WC URL. This works
> fine, except that after a few months we started noticing a slowdown in the
> tag creation.
>
> Investigations showed that the "svn cp" was checking the version of every
> file to find the oldest revision and then using that as the basis for the
> tag, plus changes since. As we never do a global update and some files
> seldom change, the oldest revision could be 6 months ago - even if every
> file is completely up to date!
>
> To get over the problem I have written a small script which "updates" files
> with no pending changes to the latest version (i.e. my script updates the
> metadata for up-to-date files and directories). It's a bit slow but it
> seems to help. I have three questions:
> 1) Is this behaviour described by design, or is it a bug?
> 2) Does anybody have a better suggestion for how to resolve my problem?
>
> Thanks,
> R
--
John Allen, mailto:john.allen@moyville.net
MandrakeClub Silver Member. http://www.moyville.net
Site of the day: http://www.kde.ie/
Mandriva Linux release 2006.0 (Official) for i586, kernel 2.6.12-12mdk
18:27:30 up 1:33, 4 users, load average: 0.09, 0.24, 0.24
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Complex tag creation performance
Posted by Robbie Gibson <ro...@gmail.com>.
On 27/06/06, Robbie Gibson <ro...@gmail.com> wrote:
>
> Hi,
> We have a simple SVN setup where everybody works in the trunk. When we
> want to put our work into the production system we do a selective update of
> the files concerned - that is, we work on a file-by-file basis and hardly
> ever do a global "svn up" on the production systems. To keep track of
> production versions we create a complex tag with svn cp WC URL. This works
> fine, except that after a few months we started noticing a slowdown in the
> tag creation.
>
> Investigations showed that the "svn cp" was checking the version of every
> file to find the oldest revision and then using that as the basis for the
> tag, plus changes since. As we never do a global update and some files
> seldom change, the oldest revision could be 6 months ago - even if every
> file is completely up to date!
>
> To get over the problem I have written a small script which "updates"
> files with no pending changes to the latest version (i.e. my script
> updates the metadata for up-to-date files and directories). It's a bit slow
> but it seems to help. I have three questions:
> 1) Is this behaviour described by design, or is it a bug?
> 2) Does anybody have a better suggestion for how to resolve my problem?
>
> Thanks,
> R
>
Forgot the third question!
3) In order for my script to have any effect, it needs to update the
meta-data for directories as well as files. But updating a directory always
seems to update the files within it, even if the -N option is specified. Is
there any way round this?
Thanks,
R
Re: Complex tag creation performance
Posted by Robbie Gibson <ro...@gmail.com>.
On 27/06/06, John Rouillard <ro...@renesys.com> wrote:
>
> This leaves each file in the production tree at a different merge
>
revision making merge tracking difficult at best.
>
> I would be interested in your mechanisms for tracking this sort of
> thing.
Well we don't merge very often - if a developer wants to put a revision into
production he has to put all files for that revision into production. If
there is a "blocking" revision then he either has to take that revision as
well, or do a merge, and he usually opts for taking the blocking version.
So if I have a directory at rev. 10 and a developer commits rev. 20 where he
touches files A & B then he updates files A and B to version 20. This works
fine, as long as there are no other changes between rev. 10 and rev. 20.
If, for example, rev 15 involves files B and C then he has to explicitly
take version 15 as well (and check that file C isn't touched between rev 10
and rev 15). That means that svn info is usually enough to tell what
version of file we're looking at.
What I ended up doing was writing a small program that scans the working
copy for files later than a certain version (that version having been
certified by QC) and remembers them. Then I do a global svn up -r whatever,
followed by updates of the individual files that had later revisions.
I'm not sure if this corresponds to your scenario, doesn't really sound like
it ...
Thanks for your input,
R
PS It seems that the performance hit for tag creation comes when you have
lots of mismatches between a parent directory and its children, if anybody's
interested
Re: Complex tag creation performance
Posted by John Rouillard <ro...@renesys.com>.
On Tue, Jun 27, 2006 at 12:29:17PM +0200, Robbie Gibson wrote:
> We have a simple SVN setup where everybody works in the trunk. When we want
> to put our work into the production system we do a selective update of the
> files concerned - that is, we work on a file-by-file basis and hardly ever
> do a global "svn up" on the production systems. To keep track of production
> versions we create a complex tag with svn cp WC URL.
I do something similar. I am using svn for tracking system
configuration files. Since there are multiple independent subsystems in
the tree, and multiple people working in the tree, the production tree
is almost never the head revision of the working tree. This leaves
each file in the production tree at a different merge revision making
merge tracking difficult at best.
I would be interested in your mechanisms for tracking this sort of
thing. I have used properties to track the merge revisions for each
file, but it's far from automated. Also the lack of a single command
to search all revisions of a file/files for a property leads to a lot
of:
for i in `svn log -q $1 | sed -ne 's/^r\([0-9]*\).*/\1/p'`
do
property=`svn propget --revprop -r $i mergepoint $1`
[ -n "$property" ] && echo "$i $property"
done
which puts out lines like:
571 work/Config/bin/Rdist:490
which means that revision 571 of Rdist was the result of the merge
from revision 490 of work/Config/bin/Rdist.
> This works fine, except that after a few months we started noticing
> a slowdown in the tag creation.
>
> Investigations showed that the "svn cp" was checking the version of every
> file to find the oldest revision and then using that as the basis for the
> tag, plus changes since. As we never do a global update and some files
> seldom change, the oldest revision could be 6 months ago - even if every
> file is completely up to date!
>
> To get over the problem I have written a small script which "updates" files
> with no pending changes to the latest version (i.e. my script updates the
> metadata for up-to-date files and directories). It's a bit slow but it
> seems to help. I have three questions:
> 1) Is this behaviour described by design, or is it a bug?
> 2) Does anybody have a better suggestion for how to resolve my problem?
Would merging into and committing a separate production branch and then
tagging that branch speed things up? (At the cost of having to do the
merge). At the very least it should have fewer revisions in it than
your trunk tree. With a production branch you aren't building a tag
tree from scratch each time. Instead you are building it from only
released copies of your files.
Also some form of delete updated files from the production branch,
copy the files for the release into the production branch, tag may
work as well. I submitted a question on that a bit ago. The list
archives should have the thread.
--
-- rouilj
John Rouillard
System Administrator
Renesys Corporation
603-643-9300 x 111
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org