You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by David Summers <da...@summersoft.fay.ar.us> on 2007/05/20 17:23:15 UTC
Re: New (1.5) feature questions -- things we need to address before
a release can happen
On Sun, 20 May 2007, Mark Phippard wrote:
> DAV write-through proxy
> This has the potential to be a killer feature, but it is another one
> that is not heavily documented. There is a notes file, but it looks
> like it pre-dates the implementation in some of the details. This one
> looks fairly easy to turn on but could benefit from some more
> explanation about what it wants to accomplish with some real-world
> implementation scenarios. For example, it seems fairly important that
> the read-only "slaves" are updated after a commit. Presumably svnsync
> would be a good candidate for doing this. An example hook-based
> implementation would be useful. Also, some discussion on the
> ramifications would be useful. What if I do a commit, followed by an
> update and my mirror has not been updated yet? Likewise, what if
> another user does a commit, and then I do a commit by mirror has not
> been updated yet. Will that effect me?
>
I have been using Subversion 1.4.2 on a Win2K3R2 master server with
Subversion 1.5 r???? (I forget, updated last sometime in January)
pre-release on a Linux Redhat 4 server since mid-December.
I have a post-commit script that does svnsync from the master to the
server and do web-dav proyxing (is that the correct term?) on the slave
server so we can commit to the slave and it re-directs the commit to the
master.
Generally it works well, however when the commit is "large" (I'm not sure,
several megabytes or more or at least the commit time of a few minutes
(???)) then we usually have problems where the slave has not synced up
with
the master yet and the client that committed it to the slave gets confused
and starts giving errors. It has been possible but problematic to get
things straightened out, and sometimes I've had a svnsync not go through
and had to manually kick it and/or delete the svn:lock (I think that is
the property) to get things going again.
Is there a way to do a synchronous sync with the slave or some code magic
behind the scenes that could be added in the client or slave server or
master server to tell the client to wait a little longer to be synced
up with the slave server correctly?
I think what is happening is that the slave says "I've committed a certain
version" and then at the end of the commit it checks that the slave server
has that version which hasn't been synced yet and it gets confused.
Thankfully this has not happened a whole lot but it has happened on
multiple occasions. I'm telling my users that when they have a "large"
commit (size or time is hard to determine) then to switch to the master to
commit it and then switch back to the slave again.
Any help, ideas, pointers, would be appreciated.
It is a VERY useful feature and it generally works well.
Thanks!
--
David Wayne Summers "Linux: Because reboots are for hardware upgrades!"
david@summersoft.fay.ar.us PGP Key: http://summersoft.fay.ar.us/~david/pgp.txt
PGP Key fingerprint = 0B44 B118 85CC F4EC 7021 1ED4 1516 5B78 E320 2001
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: New (1.5) feature questions -- things we need to address before a release can happen
Posted by Karl Fogel <kf...@red-bean.com>.
"Mark Phippard" <ma...@gmail.com> writes:
> It seems like if the server can seamlessly re-direct a commit to the
> master server, it could potentially do the same for an update, when it
> detected the need.
>
> That being said, doing so might cover up that there is a problem with
> the slave server that no one realizes exists. It seems like if the
> client got an error, but was no worse for it once the slave was synced
> would be the ideal.
That would certainly be the easiest thing to do; it might be a good
way to start, then we can see how annoying it is and make a more
sophisticated behavior if necessary.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: New (1.5) feature questions -- things we need to address before a release can happen
Posted by Mark Phippard <ma...@gmail.com>.
On 5/20/07, Karl Fogel <kf...@red-bean.com> wrote:
> "Mark Phippard" <ma...@gmail.com> writes:
> >> Generally it works well, however when the commit is "large" (I'm not sure,
> >> several megabytes or more or at least the commit time of a few minutes
> >> (???)) then we usually have problems where the slave has not synced up
> >> with
> >> the master yet and the client that committed it to the slave gets confused
> >> and starts giving errors. It has been possible but problematic to get
> >> things straightened out, and sometimes I've had a svnsync not go through
> >> and had to manually kick it and/or delete the svn:lock (I think that is
> >> the property) to get things going again.
> >>
> >> Is there a way to do a synchronous sync with the slave or some code magic
> >> behind the scenes that could be added in the client or slave server or
> >> master server to tell the client to wait a little longer to be synced
> >> up with the slave server correctly?
> >>
> >> I think what is happening is that the slave says "I've committed a certain
> >> version" and then at the end of the commit it checks that the slave server
> >> has that version which hasn't been synced yet and it gets confused.
> >>
> >> Thankfully this has not happened a whole lot but it has happened on
> >> multiple occasions. I'm telling my users that when they have a "large"
> >> commit (size or time is hard to determine) then to switch to the master to
> >> commit it and then switch back to the slave again.
> >>
> >> Any help, ideas, pointers, would be appreciated.
> >
> > Seems like a problem that should be identified and fixed before we
> > release. Now that we have svnsync it would be super-cool if it could
> > be formally integrated into this so that you did not need a hook at
> > all.
>
> A client-side solution might work. Take a client that's reading from
> a slave repository S, but whose commits go to a master M.
>
> 1. S is at revision R, M is at revision R
>
> 2. Now the client commits. Afterwards, M is at revision R+1, but S
> is still at revision R. (Assume the commit is large, so S can't
> sync up immediately.)
>
> 3. If the client knows that its commit went through to M, it can
> obey a rule that says: "Until S is at R+1 or higher, do all
> reads from M".
>
> I'm hand-waving on some details, especially since write-throughs are
> supposed to be transparent to the client (is that right?), and this
> plan would mean they aren't quite transparent. One possibility:
>
> We could have a new server error code, used only by read-only mirrors,
> that says: "You tried to update to a revision higher than my youngest
> revision; maybe you should try reading from this other repository over
> here (the master)." The client would redirect its communications just
> for this RA session, and not remember the master permanently, since
> presumably the slave will update soon anyway.
>
> That would, in effect, implement the above-proposed rule.
It seems like if the server can seamlessly re-direct a commit to the
master server, it could potentially do the same for an update, when it
detected the need.
That being said, doing so might cover up that there is a problem with
the slave server that no one realizes exists. It seems like if the
client got an error, but was no worse for it once the slave was synced
would be the ideal.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: New (1.5) feature questions -- things we need to address before a release can happen
Posted by Karl Fogel <kf...@red-bean.com>.
Peter McNab <mc...@optushome.com.au> writes:
> So long as R+1 is guaranteed to be the result of this clients commit
> and not from another clients simultaneous action.
That shouldn't be an issue. I should have said "R+X", where X is the
difference between R and whatever revision the client created in the
master. Usually, X will be 1, but it might be higher. It doesn't
matter, since whatever value X has, the client will know it.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: New (1.5) feature questions -- things we need to address before
a release can happen
Posted by Peter McNab <mc...@optushome.com.au>.
Karl Fogel wrote:
> "Mark Phippard" <ma...@gmail.com> writes:
>
>>> Generally it works well, however when the commit is "large" (I'm not sure,
>>> several megabytes or more or at least the commit time of a few minutes
>>> (???)) then we usually have problems where the slave has not synced up
>>> with
>>> the master yet and the client that committed it to the slave gets confused
>>> and starts giving errors. It has been possible but problematic to get
>>> things straightened out, and sometimes I've had a svnsync not go through
>>> and had to manually kick it and/or delete the svn:lock (I think that is
>>> the property) to get things going again.
>>>
>>> Is there a way to do a synchronous sync with the slave or some code magic
>>> behind the scenes that could be added in the client or slave server or
>>> master server to tell the client to wait a little longer to be synced
>>> up with the slave server correctly?
>>>
>>> I think what is happening is that the slave says "I've committed a certain
>>> version" and then at the end of the commit it checks that the slave server
>>> has that version which hasn't been synced yet and it gets confused.
>>>
>>> Thankfully this has not happened a whole lot but it has happened on
>>> multiple occasions. I'm telling my users that when they have a "large"
>>> commit (size or time is hard to determine) then to switch to the master to
>>> commit it and then switch back to the slave again.
>>>
>>> Any help, ideas, pointers, would be appreciated.
>>>
>> Seems like a problem that should be identified and fixed before we
>> release. Now that we have svnsync it would be super-cool if it could
>> be formally integrated into this so that you did not need a hook at
>> all.
>>
>
> A client-side solution might work. Take a client that's reading from
> a slave repository S, but whose commits go to a master M.
>
> 1. S is at revision R, M is at revision R
>
> 2. Now the client commits. Afterwards, M is at revision R+1, but S
> is still at revision R. (Assume the commit is large, so S can't
> sync up immediately.)
>
> 3. If the client knows that its commit went through to M, it can
> obey a rule that says: "Until S is at R+1 or higher, do all
> reads from M".
>
>
So long as R+1 is guaranteed to be the result of this clients commit and
not from another clients simultaneous action.
Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: New (1.5) feature questions -- things we need to address before a release can happen
Posted by Karl Fogel <kf...@red-bean.com>.
"Mark Phippard" <ma...@gmail.com> writes:
>> Generally it works well, however when the commit is "large" (I'm not sure,
>> several megabytes or more or at least the commit time of a few minutes
>> (???)) then we usually have problems where the slave has not synced up
>> with
>> the master yet and the client that committed it to the slave gets confused
>> and starts giving errors. It has been possible but problematic to get
>> things straightened out, and sometimes I've had a svnsync not go through
>> and had to manually kick it and/or delete the svn:lock (I think that is
>> the property) to get things going again.
>>
>> Is there a way to do a synchronous sync with the slave or some code magic
>> behind the scenes that could be added in the client or slave server or
>> master server to tell the client to wait a little longer to be synced
>> up with the slave server correctly?
>>
>> I think what is happening is that the slave says "I've committed a certain
>> version" and then at the end of the commit it checks that the slave server
>> has that version which hasn't been synced yet and it gets confused.
>>
>> Thankfully this has not happened a whole lot but it has happened on
>> multiple occasions. I'm telling my users that when they have a "large"
>> commit (size or time is hard to determine) then to switch to the master to
>> commit it and then switch back to the slave again.
>>
>> Any help, ideas, pointers, would be appreciated.
>
> Seems like a problem that should be identified and fixed before we
> release. Now that we have svnsync it would be super-cool if it could
> be formally integrated into this so that you did not need a hook at
> all.
A client-side solution might work. Take a client that's reading from
a slave repository S, but whose commits go to a master M.
1. S is at revision R, M is at revision R
2. Now the client commits. Afterwards, M is at revision R+1, but S
is still at revision R. (Assume the commit is large, so S can't
sync up immediately.)
3. If the client knows that its commit went through to M, it can
obey a rule that says: "Until S is at R+1 or higher, do all
reads from M".
I'm hand-waving on some details, especially since write-throughs are
supposed to be transparent to the client (is that right?), and this
plan would mean they aren't quite transparent. One possibility:
We could have a new server error code, used only by read-only mirrors,
that says: "You tried to update to a revision higher than my youngest
revision; maybe you should try reading from this other repository over
here (the master)." The client would redirect its communications just
for this RA session, and not remember the master permanently, since
presumably the slave will update soon anyway.
That would, in effect, implement the above-proposed rule.
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: New (1.5) feature questions -- things we need to address before a release can happen
Posted by Mark Phippard <ma...@gmail.com>.
On 5/20/07, David Summers <da...@summersoft.fay.ar.us> wrote:
>
> On Sun, 20 May 2007, Mark Phippard wrote:
>
> > DAV write-through proxy
> > This has the potential to be a killer feature, but it is another one
> > that is not heavily documented. There is a notes file, but it looks
> > like it pre-dates the implementation in some of the details. This one
> > looks fairly easy to turn on but could benefit from some more
> > explanation about what it wants to accomplish with some real-world
> > implementation scenarios. For example, it seems fairly important that
> > the read-only "slaves" are updated after a commit. Presumably svnsync
> > would be a good candidate for doing this. An example hook-based
> > implementation would be useful. Also, some discussion on the
> > ramifications would be useful. What if I do a commit, followed by an
> > update and my mirror has not been updated yet? Likewise, what if
> > another user does a commit, and then I do a commit by mirror has not
> > been updated yet. Will that effect me?
> >
>
> I have been using Subversion 1.4.2 on a Win2K3R2 master server with
> Subversion 1.5 r???? (I forget, updated last sometime in January)
> pre-release on a Linux Redhat 4 server since mid-December.
I was hoping you would reply since AFAIK, you are one of the few/only
that have been using it in advance of the release.
> I have a post-commit script that does svnsync from the master to the
> server and do web-dav proyxing (is that the correct term?) on the slave
> server so we can commit to the slave and it re-directs the commit to the
> master.
How does the script work? Could you make a sample version of it and
add it to the file in notes?
> Generally it works well, however when the commit is "large" (I'm not sure,
> several megabytes or more or at least the commit time of a few minutes
> (???)) then we usually have problems where the slave has not synced up
> with
> the master yet and the client that committed it to the slave gets confused
> and starts giving errors. It has been possible but problematic to get
> things straightened out, and sometimes I've had a svnsync not go through
> and had to manually kick it and/or delete the svn:lock (I think that is
> the property) to get things going again.
>
> Is there a way to do a synchronous sync with the slave or some code magic
> behind the scenes that could be added in the client or slave server or
> master server to tell the client to wait a little longer to be synced
> up with the slave server correctly?
>
> I think what is happening is that the slave says "I've committed a certain
> version" and then at the end of the commit it checks that the slave server
> has that version which hasn't been synced yet and it gets confused.
>
> Thankfully this has not happened a whole lot but it has happened on
> multiple occasions. I'm telling my users that when they have a "large"
> commit (size or time is hard to determine) then to switch to the master to
> commit it and then switch back to the slave again.
>
> Any help, ideas, pointers, would be appreciated.
Seems like a problem that should be identified and fixed before we
release. Now that we have svnsync it would be super-cool if it could
be formally integrated into this so that you did not need a hook at
all.
> It is a VERY useful feature and it generally works well.
I agree that it could be a very important feature. I'd like to see
someone pick it up and try to polish it up some more so that we could
really promote people to use it when the need exists.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org