You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by cm...@collab.net on 2002/06/17 14:55:02 UTC
svn.collab.net is going down...
...for a Subversion upgrade and repository conversion.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn.collab.net is going down...
Posted by Branko Čibej <br...@xbc.nu>.
Karl Fogel wrote:
>Branko Čibej <br...@xbc.nu> writes:
>
>
>>Would it be very horrible if I called a halt to that conversion?
>>
>>
>
>Not horrible, but is it necessary?
>
>Ahh, I think I see. Your change implies another repository dump/load
>conversion, to eliminate overlapping windows in the data store. So
>waiting is good, because we'd rather convert the repository once, not
>twice.
>
Yes, exactly.
>BUT: Mike Pilato points out that we're going to need another
>dump/reload sequence soon anyway, after he finishes implementing the
>changed paths cache for 'svn log'. But he's saving the dumpfile he
>made this time, so _next_ time he only needs to dump the revisions
>made since this upgrade :-).
>
>So don't worry that the repository is getting upgraded now. It's
>going to happen again very soon anyway.
>
>
Well, that's O.K. then. Mike, go ahead and do your worst. :-)
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn.collab.net is going down...
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Branko Čibej <br...@xbc.nu> writes:
> Would it be very horrible if I called a halt to that conversion?
Not horrible, but is it necessary?
Ahh, I think I see. Your change implies another repository dump/load
conversion, to eliminate overlapping windows in the data store. So
waiting is good, because we'd rather convert the repository once, not
twice.
BUT: Mike Pilato points out that we're going to need another
dump/reload sequence soon anyway, after he finishes implementing the
changed paths cache for 'svn log'. But he's saving the dumpfile he
made this time, so _next_ time he only needs to dump the revisions
made since this upgrade :-).
So don't worry that the repository is getting upgraded now. It's
going to happen again very soon anyway.
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn.collab.net is going down...
Posted by Branko Čibej <br...@xbc.nu>.
Branko Čibej wrote:
[...]
> My suggestion would be:
>
> * rip out that space optimization
> * apply Daniel Berlin's patch
>
> (http://subversion.tigris.org/servlets/ReadMsg?list=dev&msgId=163165)
> that
> o introduces an address cache for the svndiff
> generator/parser, and
> o changes the number representation in svndiff be128 to le128
> o changes the svndiff format version, with backward
> compatibility
> * then convert the repository
> * then integrate the delta combiner
>
> The first two steps are fairly trivial, and I could get them checked
> in (and unit tested, of course) tonight.
And the second step isn't absolutely necessary before the conversion;
but the first most certainly is.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn.collab.net is going down...
Posted by Branko Čibej <br...@xbc.nu>.
Greg Hudson wrote:
>On Mon, 2002-06-17 at 13:37, Branko Čibej wrote:
>
>
>>It makes the delta combiner waaay simpler if I can assume that, when
>>combining two windows A and B, the length of the target view produced by
>>A is the same as the length of the source view expected by B, and that
>>A's target view starts at B'a source view offset.
>>
>>
>
>Hm. I see. Otherwise the source windows can grow very large. In the
>current scheme of things, if you apply ten deltas in sequence to get
>from A to K, then in theory the first block of K might depend on data
>from anywhere in the first 11*winsize bytes of A. (You have to produce
>two windows of J to build the source window for K; you have to produce
>three windows of I to build two windows of J; etc.) So I'm not sure
>this problem is solvable even in the long term, at least with reasonably
>streamy behavior.
>
Something like that, yes.
>On the other hand, it's kind of a bummer not to have any slop in the
>source windows. With no slop, if I add a 300-byte paragraph to the
>beginning of a large text file, there will be roughly 300 bytes of stuff
>to insert in each window of the diff. Maybe not so big a deal as long
>as the window size is reasonably large.
>
That, and the fact that whatever data is shifted gets compressed by
vdelta, too. And Daniel's patch will compress the svndiff even more, so
we probably end up even in general.
We can always bump the delta window from 100k to 1M if it becomes a
problem, but i don't think it will. Sure, anybody using subversion as a
backing store for her video post-processing lab is in for a shock. :-)
>[About Daniel's work:]
>
>
>>I think it should go in.
>>
>>
>
>I like it too; I just didn't think he thought it was ready yet.
>
I'll ask him before I touch it.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn.collab.net is going down...
Posted by Greg Hudson <gh...@MIT.EDU>.
On Mon, 2002-06-17 at 13:37, Branko Čibej wrote:
> It makes the delta combiner waaay simpler if I can assume that, when
> combining two windows A and B, the length of the target view produced by
> A is the same as the length of the source view expected by B, and that
> A's target view starts at B'a source view offset.
Hm. I see. Otherwise the source windows can grow very large. In the
current scheme of things, if you apply ten deltas in sequence to get
from A to K, then in theory the first block of K might depend on data
from anywhere in the first 11*winsize bytes of A. (You have to produce
two windows of J to build the source window for K; you have to produce
three windows of I to build two windows of J; etc.) So I'm not sure
this problem is solvable even in the long term, at least with reasonably
streamy behavior.
On the other hand, it's kind of a bummer not to have any slop in the
source windows. With no slop, if I add a 300-byte paragraph to the
beginning of a large text file, there will be roughly 300 bytes of stuff
to insert in each window of the diff. Maybe not so big a deal as long
as the window size is reasonably large.
[About Daniel's work:]
> I think it should go in.
I like it too; I just didn't think he thought it was ready yet.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn.collab.net is going down...
Posted by Branko Čibej <br...@xbc.nu>.
Greg Hudson wrote:
>On Mon, 2002-06-17 at 13:11, Branko Čibej wrote:
>
>
>>The result is that the combiner would have to
>>handle overlapping delta windows, which is definitely more than I'd want
>>to do for Alpha -- it is, in fact, a slightly huge amount of work.
>>
>>
>
>I'm not sure I understand what's difficult about handling overlapping
>source windows. What benefit do you gain from assuming they don't
>overlap?
>
It makes the delta combiner waaay simpler if I can assume that, when
combining two windows A and B, the length of the target view produced by
A is the same as the length of the source view expected by B, and that
A's target view starts at B'a source view offset.
Otherwise, I'd have to concatenate windows in the combiner, etc. Doable,
but moch more work. I actually plan to do it, but there's no way it
could be finished for Alpha, therefore not before 1.0.
>> * apply Daniel Berlin's patch
>>
>>
>
>I've been kind of wondering what happened to this work. The message you
>cited says "There are still a few things I need to do before I'd like to
>see this officially reviewed," so I assumed got to the 98% mark and
>stalled there.
>
Well, he didn't get it officially reviewed, which is our (my) fault,
really. I looked at it again now, and it looks very nice indeed. I think
it should go in.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn.collab.net is going down...
Posted by Greg Hudson <gh...@MIT.EDU>.
On Mon, 2002-06-17 at 13:11, Branko Čibej wrote:
> The result is that the combiner would have to
> handle overlapping delta windows, which is definitely more than I'd want
> to do for Alpha -- it is, in fact, a slightly huge amount of work.
I'm not sure I understand what's difficult about handling overlapping
source windows. What benefit do you gain from assuming they don't
overlap?
> * apply Daniel Berlin's patch
I've been kind of wondering what happened to this work. The message you
cited says "There are still a few things I need to do before I'd like to
see this officially reviewed," so I assumed got to the 98% mark and
stalled there.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn.collab.net is going down...
Posted by Branko Čibej <br...@xbc.nu>.
cmpilato@collab.net wrote:
>...for a Subversion upgrade and repository conversion.
>
>
Would it be very horrible if I called a halt to that conversion?
It turns out that, if I want to finish the delta combiner in time for
Alpha, I have to change the way we create deltas. Specifically, I have
to rip out Greg Hudson's space optimization in
text_delta.c:svn_txdelta_next_window(), which intentionally makes the
source windows overlap. The result is that the combiner would have to
handle overlapping delta windows, which is definitely more than I'd want
to do for Alpha -- it is, in fact, a slightly huge amount of work.
My suggestion would be:
* rip out that space optimization
* apply Daniel Berlin's patch
(http://subversion.tigris.org/servlets/ReadMsg?list=dev&msgId=163165)
that
o introduces an address cache for the svndiff
generator/parser, and
o changes the number representation in svndiff be128 to le128
o changes the svndiff format version, with backward compatibility
* then convert the repository
* then integrate the delta combiner
The first two steps are fairly trivial, and I could get them checked in
(and unit tested, of course) tonight.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository back up, but working copy re-checkout needed
Posted by Ben Collins-Sussman <su...@collab.net>.
Karl Fogel <kf...@newton.ch.collab.net> writes:
> Ben Collins-Sussman <su...@collab.net> writes:
> > The bug we're talking about fixing is in the svn client: making the
> > svn client automatically detect-and-regenerate invalidated opaque url
> > caches, so that an old working copy can be updated by a newer
> > repository.
>
> But -- it turns out (Greg Stein just walked in the door, straight from
> the airport) that there's a way to fix this server-side, and we're
> working on it right now. More in a bit...
Yes... here is gstein's explanation:
When the client does a GET and provides a 'base' custom request header
that contains an (old-style) opaque url, it's telling the server:
"hey, I want this file, and here's the version I already have."
At this point, the server has the option of either returning svndiff
data against the base file, or just returning fulltext. It specifies
what it's doing in the GET response.
Greg's server-side solution (which we're testing now) is that if the
'base' header looks like junk, then just return fulltext. It's always
had the right to do so.
[And also: the opaque url cache will be automatically fixed anyway,
because it's already embedded in the update-report-response.]
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository back up, but working copy re-checkout needed
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Ben Collins-Sussman <su...@collab.net> writes:
> The bug we're talking about fixing is in the svn client: making the
> svn client automatically detect-and-regenerate invalidated opaque url
> caches, so that an old working copy can be updated by a newer
> repository.
But -- it turns out (Greg Stein just walked in the door, straight from
the airport) that there's a way to fix this server-side, and we're
working on it right now. More in a bit...
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository back up, but working copy re-checkout needed
Posted by Ben Collins-Sussman <su...@collab.net>.
"Ronald V. Simmons" <Va...@thesimmonses.net> writes:
> actually my windows clients were upgraded to R2196 on Friday (post
> the dump/load conversion you helped me with last week, Ben) and some
> of them have been complaining today of this problem. I just hadn't
> had time to look closely into it and put it down to their standard
> whingeing. It's pretty amusing to be this bleeding edge. :^)
> Anyway, I suspect that fixes both post and R2092 will be needed on
> the client side and that might make the branching structure a bit
> more complicated..
Huh? Why would we need to put the fix on the pre-2092 branch? That
branch is only for fixing bugs in 'svnadmin dump'.
The bug we're talking about fixing is in the svn client: making the
svn client automatically detect-and-regenerate invalidated opaque url
caches, so that an old working copy can be updated by a newer
repository.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository back up, but working copy re-checkout needed
Posted by "Ronald V. Simmons" <Va...@thesimmonses.net>.
actually my windows clients were upgraded to R2196 on Friday (post the dump/load conversion you helped me with last week, Ben) and some of them have been complaining today of this problem. I just hadn't had time to look closely into it and put it down to their standard whingeing. It's pretty amusing to be this bleeding edge. :^) Anyway, I suspect that fixes both post and R2092 will be needed on the client side and that might make the branching structure a bit more complicated..
rvs
----- Original Message -----
From: "Ben Collins-Sussman" <su...@collab.net>
To: "Ronald V. Simmons" <Va...@thesimmonses.net>
Cc: "Subversion Dev list" <de...@subversion.tigris.org>
Sent: Monday, June 17, 2002 5:45 PM
Subject: Re: Repository back up, but working copy re-checkout needed
> "Ronald V. Simmons" <Va...@thesimmonses.net> writes:
>
> > It would still be extremely helpful to go ahead and commit the fix.
> > Many of us have repositories and clients which are suffering from
> > the exact same problem. I can build and deploy an svn client (since
> > I don't have modified svn code in my local tree I don't mind the new
> > svn co so much) and that will alleviate the bigger problem for me.
>
> Yes, I suspect gstein will read this, and quickly write a followup
> change to r1863. Just guessing.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository back up, but working copy re-checkout needed
Posted by Ben Collins-Sussman <su...@collab.net>.
"Ronald V. Simmons" <Va...@thesimmonses.net> writes:
> It would still be extremely helpful to go ahead and commit the fix.
> Many of us have repositories and clients which are suffering from
> the exact same problem. I can build and deploy an svn client (since
> I don't have modified svn code in my local tree I don't mind the new
> svn co so much) and that will alleviate the bigger problem for me.
Yes, I suspect gstein will read this, and quickly write a followup
change to r1863. Just guessing.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Repository back up, but working copy re-checkout needed
Posted by "Ronald V. Simmons" <Va...@thesimmonses.net>.
It would still be extremely helpful to go ahead and commit the fix. Many of us have repositories and clients which are suffering from the exact same problem. I can build and deploy an svn client (since I don't have modified svn code in my local tree I don't mind the new svn co so much) and that will alleviate the bigger problem for me.
thanks
rvs
----- Original Message -----
From: "Karl Fogel" <kf...@newton.ch.collab.net>
To: "Subversion Dev list" <de...@subversion.tigris.org>
Cc: "Glenn A. Thompson" <gt...@cdr.net>
Sent: Monday, June 17, 2002 5:15 PM
Subject: Repository back up, but working copy re-checkout needed
> Okay folks, the repository is back up again, but with a catch:
>
> You'll need to check out working copies from svn.collab.net again, due
> to the versioned-resource url format change. Here is the command
> line, so you won't even have to think:
>
> svn co http://svn.collab.net/repos/svn/trunk -d YOUR_WC_NAME
>
> You're probably saying to yourself "But wait, I thought they
> anticipated this a while back and put compatibility code in the client
> so it would automatically expire old-style cached urls?"
>
> Yes, we did, see revision 1863 for the bulk of that change. But it
> looks like we forgot the case where the client sends an opaque vsn
> rsrc url to provide the *base* for an update -- the base against which
> the server should send svndiff data, that is. So now, when you try to
> update an old working copy, this GET request...
>
> GET /repos/svn/!svn/ver/2248/trunk/subversion/include/svn_wc.h HTTP/1.1
> User-Agent: neon/0.21.1 SVN/0.13.0 (dev build)
> Keep-Alive:
> Connection: TE, Keep-Alive
> TE: trailers
> X-SVN-VR-Base: /repos/svn/$svn/ver/295.14.2.96.1.2/trunk/subversion/\
> include/svn_wc.h
> Host: svn.collab.net
>
> ...results in this server response:
>
> HTTP/1.1 400 Bad Request
> Date: Mon, 17 Jun 2002 20:43:46 GMT
> Server: Apache/2.0.39-dev (Unix) DAV/2 SVN/0.13.0 (dev build)
> Vary: accept-language
> Accept-Ranges: bytes
> Content-Length: 741
> Connection: close
> Content-Type: text/html; charset=ISO-8859-1
> X-Pad: avoid browser bug
>
> ... but unfortunately, the client doesn't have any special code to
> notice that response and simply invalidate the cached vsn-rsrc-url.
> It only has such code for the commit case.
>
> Ooops :-). Sorry, folks.
>
> We could, I suppose, commit a fix for this. But you still wouldn't be
> able to update to get the fix :-), so we'd have to mail it out as a
> patch, which people would apply and then rebuild... Not a very smooth
> upgrade path at all.
>
> So given that the smooth upgrade path is already not an option, the
> easiest thing is probably just for everyone to check out again. Note
> that it's okay to check out using a client that is not bleeding edge:
> the initial vsn rsrc urls come from the server, not the client.
>
> -Karl (& Mike & Ben)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Repository back up, but working copy re-checkout needed
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Okay folks, the repository is back up again, but with a catch:
You'll need to check out working copies from svn.collab.net again, due
to the versioned-resource url format change. Here is the command
line, so you won't even have to think:
svn co http://svn.collab.net/repos/svn/trunk -d YOUR_WC_NAME
You're probably saying to yourself "But wait, I thought they
anticipated this a while back and put compatibility code in the client
so it would automatically expire old-style cached urls?"
Yes, we did, see revision 1863 for the bulk of that change. But it
looks like we forgot the case where the client sends an opaque vsn
rsrc url to provide the *base* for an update -- the base against which
the server should send svndiff data, that is. So now, when you try to
update an old working copy, this GET request...
GET /repos/svn/!svn/ver/2248/trunk/subversion/include/svn_wc.h HTTP/1.1
User-Agent: neon/0.21.1 SVN/0.13.0 (dev build)
Keep-Alive:
Connection: TE, Keep-Alive
TE: trailers
X-SVN-VR-Base: /repos/svn/$svn/ver/295.14.2.96.1.2/trunk/subversion/\
include/svn_wc.h
Host: svn.collab.net
...results in this server response:
HTTP/1.1 400 Bad Request
Date: Mon, 17 Jun 2002 20:43:46 GMT
Server: Apache/2.0.39-dev (Unix) DAV/2 SVN/0.13.0 (dev build)
Vary: accept-language
Accept-Ranges: bytes
Content-Length: 741
Connection: close
Content-Type: text/html; charset=ISO-8859-1
X-Pad: avoid browser bug
... but unfortunately, the client doesn't have any special code to
notice that response and simply invalidate the cached vsn-rsrc-url.
It only has such code for the commit case.
Ooops :-). Sorry, folks.
We could, I suppose, commit a fix for this. But you still wouldn't be
able to update to get the fix :-), so we'd have to mail it out as a
patch, which people would apply and then rebuild... Not a very smooth
upgrade path at all.
So given that the smooth upgrade path is already not an option, the
easiest thing is probably just for everyone to check out again. Note
that it's okay to check out using a client that is not bleeding edge:
the initial vsn rsrc urls come from the server, not the client.
-Karl (& Mike & Ben)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
svn.collab.net undergoing debugging
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
If you're having problems updating, you're not alone :-).
Mike Pilato and I are debugging a problem in the new repository.
Somehow Berkeley is returning EINVAL when we try to close the new
`copies' as part of cleanup_fs_apr(). Our repro recipe is: on the
svn.collab.net repos, run 'svnadmin youngest' once. It succeeds. Run
it again, see it fail. Pretty simple.
Watch this space for more details; we're deep in it now, so not going
to write a dev-list novel at this point.
Glenn: I hope the above answers your post, too :-).
-K
"Glenn A. Thompson" <gt...@cdr.net> writes:
> Hey:
>
> When?
> Right now?
>
> I'm in computer hell. I lost the dev box I was using to work on
> Subversion.
> While I was was recovering that box, I lost a production Sun Box which
> diverted me.
> So, now I have my last backup but want to update with the repos.
> Am I screwed, and for how long?
>
> gat
>
> PS Anyone know an exorcist? I appear to have pissed off the computer
> gods:-)
>
> cmpilato@collab.net wrote:
>
> > ...for a Subversion upgrade and repository conversion.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> > For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn.collab.net is going down...
Posted by "Glenn A. Thompson" <gt...@cdr.net>.
Hey:
When?
Right now?
I'm in computer hell. I lost the dev box I was using to work on
Subversion.
While I was was recovering that box, I lost a production Sun Box which
diverted me.
So, now I have my last backup but want to update with the repos.
Am I screwed, and for how long?
gat
PS Anyone know an exorcist? I appear to have pissed off the computer
gods:-)
cmpilato@collab.net wrote:
> ...for a Subversion upgrade and repository conversion.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org