You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Greg Hudson <gh...@MIT.EDU> on 2002/10/18 04:38:05 UTC
Our DAV version-urls aren't very stable
Unmodified files in tag or branch directories do not have stable DAV
version-urls. This could affect the performance of HTTP caches and
other DeltaV-based tools, although it does not affect Subversion
directly. Read on for details.
Consider the following repository:
% svnadmin create /repos
% svn mkdir -m '' file:///repos/foo # Rev 1
% svn mkdir -m '' file:///repos/foo/bar # Rev 2
% svn co file:///repos/foo/bar bar
% cd bar && touch quux && svn add quux && svn ci -m '' quux # Rev 3
% cd .. && rm -rf bar
% svn cp -m '' file:///repos/foo/bar file:///repos/foo/baz # Rev 4
Here's a list of what's in the repository, with created_revs:
% svn ls -Rv file:///repos
_ 4 ghudson 0 Oct 17 21:46 foo/
_ 3 ghudson 0 Oct 17 21:46 foo/bar/
_ 3 ghudson 0 Oct 17 21:46 foo/bar/quux
_ 4 ghudson 0 Oct 17 21:46 foo/baz/
_ 3 ghudson 0 Oct 17 21:46 foo/baz/quux
Notice that the CR of foo/baz/quux is 3, even though foo/baz/quux
doesn't exist as of revision 3. This is expected due to lazy copying,
but it has some implications for DAV. Observe what happens on a DAV
checkout:
% svn co http://hostname/repos/foo/baz baz
% grep -A 2 'name="quux"' baz/.svn/entries
<entry
committed-rev="3"
name="quux"
% grep '\!svn' baz/.svn/wcprops/quux.svn-work
/svn/repos/!svn/ver/4/foo/baz/quux
Notice that even though the committed-rev is 3, the version-url
mentions revision 4. This is because of
mod_dav_svn/util.c:dav_svn_safe_get_cr(), which makes sure that the
version-url actually works by falling back to the version of the root
(that is, the version you are checking out or updating to) if it finds
that a lookup of (CR,path) doesn't yield the same node-revision as
(checkout-rev,path).
The resulting version-url is unstable. For instance:
% rm -rf baz
% svn mkdir -m '' file:///repos/unrelated # Rev 5
% svn co http://hostname/repos/foo/baz baz
% grep '\!svn' baz/.svn/wcprops/quux.svn-work
/svn/repos/!svn/ver/5/foo/baz/quux
What's the impact? If many people are checking out a branch or tag of
some popular source tree through an HTTP cache, the cache won't be
able to do much good if there is unrelated commit activity in the
repository. I'll check out revision 1952 and get one set of
version-urls; you'll check out revision 1955 and get a different set.
The cache will see different URLs and have to fetch the files over
again.
For a bit of history,
http://subversion.tigris.org/servlets/ReadMsg?list=dev&msgNo=4011&list=dev
is a discussion about this problem from a couple of years ago. (For
"DAV-ID" read "version-url".) There have been other discussions; we
eventually settled on (CR, path) as the ID, but that obviously doesn't
work for unmodified files inside copied directories.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Our DAV version-urls aren't very stable
Posted by Jeff Stuart <js...@computer-city.net>.
On Fri, 2002-10-18 at 10:29, cmpilato@collab.net wrote:
> Is that any clearer?
>
AHH yes.. thank you! :)
--
Jeff Stuart <js...@computer-city.net>
Re: Our DAV version-urls aren't very stable
Posted by cm...@collab.net.
Jeff Stuart <js...@computer-city.net> writes:
> On Fri, 2002-10-18 at 00:38, Greg Hudson wrote:
> > Unmodified files in tag or branch directories do not have stable DAV
> > version-urls. This could affect the performance of HTTP caches and
> > other DeltaV-based tools, although it does not affect Subversion
> > directly. Read on for details.
>
> Is this REALLY a problem? IE let's assume for a moment the default
> repos structure. IE /Trunk, /Tag, /Branch. I can see caching /Tag.
> But I wouldn't want a cache of /Trunk as by definition, it'll be
> changing. Possibly quickly! The same possibly with /Branch. IE I'd
> always want to make sure I got the latest code from each.
I think your misunderstanding Greg's complaint, Jeff. Subversion's
update mechanism doesn't really have a notion "get me the latest
code". When the update begins, Subversion asks for the youngest
revision, and from there on out it requests the files and directories
at that youngest revision. Now, this is (we think) a good thing,
because it means that if someone commits a new revision during your
update, it won't cause parts of your update to get a different
revision than what you'd grabbed already before the commit happened.
Greg's concern is that there is redundancy in the set of our URLs for
a given object. That is, if a file was added in revision 3, and
untouched for the next 6 revisions, it would be completely correct to
ask for that file as:
file URL @ rev 3
file URL @ rev 4
file URL @ rev 5
file URL @ rev 6
file URL @ rev 7
file URL @ rev 8
file URL @ rev 9
All of these are correct. And would actually happen if, say, 7 people
each ran 'svn update' after a different one of the above revisions was
added. What you'd like to happen is that the first guy doing the
updating (after revision 3) would get the contents of the file the
hard way, via the filesystem. But you'd want the next 6 guys and
their updates to simply hit the cache -- after all, the file *didn't
change* in those next 6 revisions. But our URL schema prevents a
caching server from pulling that off, since the cache 'keys' are
different -- they differ by exactly the revision number portion of the
URL, in this case.
Is that any clearer?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Our DAV version-urls aren't very stable
Posted by Ben Collins-Sussman <su...@collab.net>.
Jeff Stuart <js...@computer-city.net> writes:
> On Fri, 2002-10-18 at 00:38, Greg Hudson wrote:
> > Unmodified files in tag or branch directories do not have stable DAV
> > version-urls. This could affect the performance of HTTP caches and
> > other DeltaV-based tools, although it does not affect Subversion
> > directly. Read on for details.
>
> Is this REALLY a problem? IE let's assume for a moment the default
> repos structure. IE /Trunk, /Tag, /Branch. I can see caching /Tag.
> But I wouldn't want a cache of /Trunk as by definition, it'll be
> changing. Possibly quickly! The same possibly with /Branch. IE I'd
> always want to make sure I got the latest code from each.
>
> Furthermore, didn't one of the devel team say earlier that
> using/depending on the internal URLs of svn is a BAD thing?
>
> Or am I completely missing the point? :)
Yeah, you are. :)
When you do an update or checkout, your svn client does a GET on a
(revision, path) pair. But *really*, our DAV protocol represents each
such pair with a a special opaque url that represents... well, let's
say it's kinda like the equivalent of an 'inode' in the repository.
So suppose Joe checks out all of (5, /branch/foo). Some caching http
proxy notices all of the inode GET requests and caches the data. Now
Jane comes along and checks out (6, /branch/foo). Perhaps only 3
files actually changed between revision 5 and 6 in that branch;
therefore, Jane's checkout should be GETting the exact same set of
inodes, except for the 3 new ones. This is where the HTTP caching
proxy really speeds things up.
But Greg is pointing out that the 'stable' inode-like identities
aren't really stable when you copy whole subtrees from /trunk to
/branch... which kinda defeats the point of having them at all. If
caching doesn't work, we may as well have people simply GET the (rev
path) pairs directly.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Our DAV version-urls aren't very stable
Posted by Jeff Stuart <js...@computer-city.net>.
On Fri, 2002-10-18 at 00:38, Greg Hudson wrote:
> Unmodified files in tag or branch directories do not have stable DAV
> version-urls. This could affect the performance of HTTP caches and
> other DeltaV-based tools, although it does not affect Subversion
> directly. Read on for details.
Is this REALLY a problem? IE let's assume for a moment the default
repos structure. IE /Trunk, /Tag, /Branch. I can see caching /Tag.
But I wouldn't want a cache of /Trunk as by definition, it'll be
changing. Possibly quickly! The same possibly with /Branch. IE I'd
always want to make sure I got the latest code from each.
Furthermore, didn't one of the devel team say earlier that
using/depending on the internal URLs of svn is a BAD thing?
Or am I completely missing the point? :)
--
Jeff Stuart <js...@computer-city.net>
RE: Our DAV version-urls aren't very stable
Posted by Bill Tutt <ra...@lyra.org>.
If the file system had the new code to give you the following
information for filesystem directory entries, would this be enough to
ensure that the generated version-url for foo/baz/quux included a
version URL of 4 instead of 5?
* First visible on path to me at Revision XYZ.
* First visible on path to me at Revision XYZ via CopyID ABC.
The first bit of information should be enough for your needs, and the
second bit of information would be extra info for log/status reports so
that the output for svn log foo/baz/quuz could now include a "branched
from A/B/C at revision XYZ" output on lazily copied repository paths.
Bill
> From: Greg Hudson [mailto:ghudson@MIT.EDU]
>
> Unmodified files in tag or branch directories do not have stable DAV
> version-urls. This could affect the performance of HTTP caches and
> other DeltaV-based tools, although it does not affect Subversion
> directly. Read on for details.
>
> Consider the following repository:
>
> % svnadmin create /repos
> % svn mkdir -m '' file:///repos/foo # Rev 1
> % svn mkdir -m '' file:///repos/foo/bar # Rev 2
> % svn co file:///repos/foo/bar bar
> % cd bar && touch quux && svn add quux && svn ci -m '' quux # Rev 3
> % cd .. && rm -rf bar
> % svn cp -m '' file:///repos/foo/bar file:///repos/foo/baz # Rev 4
>
> Here's a list of what's in the repository, with created_revs:
>
> % svn ls -Rv file:///repos
> _ 4 ghudson 0 Oct 17 21:46 foo/
> _ 3 ghudson 0 Oct 17 21:46 foo/bar/
> _ 3 ghudson 0 Oct 17 21:46 foo/bar/quux
> _ 4 ghudson 0 Oct 17 21:46 foo/baz/
> _ 3 ghudson 0 Oct 17 21:46 foo/baz/quux
>
> Notice that the CR of foo/baz/quux is 3, even though foo/baz/quux
> doesn't exist as of revision 3. This is expected due to lazy copying,
> but it has some implications for DAV. Observe what happens on a DAV
> checkout:
>
> % svn co http://hostname/repos/foo/baz baz
> % grep -A 2 'name="quux"' baz/.svn/entries
> <entry
> committed-rev="3"
> name="quux"
> % grep '\!svn' baz/.svn/wcprops/quux.svn-work
> /svn/repos/!svn/ver/4/foo/baz/quux
>
> Notice that even though the committed-rev is 3, the version-url
> mentions revision 4. This is because of
> mod_dav_svn/util.c:dav_svn_safe_get_cr(), which makes sure that the
> version-url actually works by falling back to the version of the root
> (that is, the version you are checking out or updating to) if it finds
> that a lookup of (CR,path) doesn't yield the same node-revision as
> (checkout-rev,path).
>
> The resulting version-url is unstable. For instance:
>
> % rm -rf baz
> % svn mkdir -m '' file:///repos/unrelated # Rev 5
> % svn co http://hostname/repos/foo/baz baz
> % grep '\!svn' baz/.svn/wcprops/quux.svn-work
> /svn/repos/!svn/ver/5/foo/baz/quux
>
> What's the impact? If many people are checking out a branch or tag of
> some popular source tree through an HTTP cache, the cache won't be
> able to do much good if there is unrelated commit activity in the
> repository. I'll check out revision 1952 and get one set of
> version-urls; you'll check out revision 1955 and get a different set.
> The cache will see different URLs and have to fetch the files over
> again.
>
> For a bit of history,
>
http://subversion.tigris.org/servlets/ReadMsg?list=dev&msgNo=4011&list=d
ev
> is a discussion about this problem from a couple of years ago. (For
> "DAV-ID" read "version-url".) There have been other discussions; we
> eventually settled on (CR, path) as the ID, but that obviously doesn't
> work for unmodified files inside copied directories.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org