many anonsvn replicas...

["Perry E. Metzger" <pe...@piermont.com>]

"Perry E. Metzger" <pe...@piermont.com> writes:
>    The two strategies that have come to mind for me are to do an
>    incremental dump after every commit, copying that to the anon
>    repository -- but that would miss property changes (another reason
>    I'm bringing up issue #1). The other possibility that came to mind
>    is somehow sending the db transaction log files over to the copy
>    and replaying them there, but I must confess I have little or no
>    idea how to do that.

By the way, this brings to mind my favorite unimplemented project
that I've always wanted to do with CVS and now would like to do with
SVN -- I'm mentioning it here because I may never actually get enough
round tuits and other people might like the idea.

Lets say you have an open source project where you want to have a lot
of anon repositories. Have the main repository run an nntp server, and
post PGP signed versions of all incremental changes to a special
newsgroup. Run nntp on a tree of anon servers... and watch as the
updates flood fill to all of them in seconds. Add some code which
checks the signatures and applies the changes, and something to do a
periodic last ditch consistency check (say posting cryptographic
hashes of the whole repository every day or so) and you have an
anonsvn infrastructure that scales to thousands of copies of the
repository that are all nearly in sync in real time...

If you find nntp too disgusting, some custom service could work the
same way, but there's lots of good nntp code with high performance out
there already, I figure...

Perry

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: many anonsvn replicas...

["Perry E. Metzger" <pe...@piermont.com>]

Holger Krekel <py...@devel.trillke.net> writes:
>> If you find nntp too disgusting, some custom service could work the
>> same way, but there's lots of good nntp code with high performance out
>> there already, I figure...
>
> For smaller setups maybe just triggering the peers to grab incremental 
> dumps from the master repo might be enough. This sounds simple enough
> if we are dealing with three or four repository mirrors. 
>
> For massively distributed environments, like every developer
> running his own repo, using bittorrents for wider distribution also 
> sounds worthwhile. 

I think NNTP is easier -- it is built for flood filling files already
between a bunch of servers. It is easy to build trees of NNTP hosts,
too. A custom protocol might also work, but I don't think Bit Torrent
is really built for the task...

> Then again, really interesting would be if you can actually commit to any 
> of the many repositories.

No. It provides you with read only replicas -- committing to them
will not work. Presumably, though, one could enhance "commit" so that
it knew which was the main repository, checked that your copy was at
the same revision number as the main one, and let you commit to the main...


-- 
Perry E. Metzger		perry@piermont.com

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: many anonsvn replicas...

[Holger Krekel <py...@devel.trillke.net>]

Hi Perry,

Perry E. Metzger wrote:
> Lets say you have an open source project where you want to have a lot
> of anon repositories. Have the main repository run an nntp server, and
> post PGP signed versions of all incremental changes to a special
> newsgroup. Run nntp on a tree of anon servers... and watch as the
> updates flood fill to all of them in seconds. Add some code which
> checks the signatures and applies the changes, and something to do a
> periodic last ditch consistency check (say posting cryptographic
> hashes of the whole repository every day or so) and you have an
> anonsvn infrastructure that scales to thousands of copies of the
> repository that are all nearly in sync in real time...

I like the basic idea (but could only be bothered to help when doing
it in python :-). 

> If you find nntp too disgusting, some custom service could work the
> same way, but there's lots of good nntp code with high performance out
> there already, I figure...

For smaller setups maybe just triggering the peers to grab incremental 
dumps from the master repo might be enough. This sounds simple enough
if we are dealing with three or four repository mirrors. 

For massively distributed environments, like every developer
running his own repo, using bittorrents for wider distribution also 
sounds worthwhile. 

Then again, really interesting would be if you can actually commit to any 
of the many repositories.  ASFAIK the possible resulting merge conflicts 
are not trivial to handle or to avoid unless you revert to some centralized 
locking mechanism which defeats the massively distributed scenario and
leads to single point of failures.  Also the uniqueness of revision numbers 
will suffer and what not ...  (someone will surely point out that 
the documentation explains all this already :-) 

cheers,

    holger

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: many anonsvn replicas...

["Perry E. Metzger" <pe...@piermont.com>]

Craig Peterein <cr...@peterein.org> writes:
> On Wed, Feb 18, 2004 at 04:42:31PM -0500, Perry E. Metzger wrote:
>> Lets say you have an open source project where you want to have a lot
>> of anon repositories. Have the main repository run an nntp server, and
>> post PGP signed versions of all incremental changes to a special
>> newsgroup. Run nntp on a tree of anon servers... and watch as the
>> updates flood fill to all of them in seconds. Add some code which
>> checks the signatures and applies the changes, and something to do a
>> periodic last ditch consistency check (say posting cryptographic
>> hashes of the whole repository every day or so) and you have an
>> anonsvn infrastructure that scales to thousands of copies of the
>> repository that are all nearly in sync in real time...
>
> This sounds like monotone: http://www.venge.net/monotone/

Not really, no. In fact, not at all so far as I can tell.

I'm talking about a way of doing lots of read-only replicas of a
repository (or indeed of any set of data) -- not a source control
system.

Perry

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: many anonsvn replicas...

[Craig Peterein <cr...@peterein.org>]

On Wed, Feb 18, 2004 at 04:42:31PM -0500, Perry E. Metzger wrote:
> Lets say you have an open source project where you want to have a lot
> of anon repositories. Have the main repository run an nntp server, and
> post PGP signed versions of all incremental changes to a special
> newsgroup. Run nntp on a tree of anon servers... and watch as the
> updates flood fill to all of them in seconds. Add some code which
> checks the signatures and applies the changes, and something to do a
> periodic last ditch consistency check (say posting cryptographic
> hashes of the whole repository every day or so) and you have an
> anonsvn infrastructure that scales to thousands of copies of the
> repository that are all nearly in sync in real time...

This sounds like monotone: http://www.venge.net/monotone/

Craig


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org