You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Travis P <sv...@castle.fastmail.fm> on 2005/02/24 00:16:35 UTC
Transaction building with bindings, do I need a WC on disk
I've a couple question regarding what's generally possible that I think
some developers could probably answer pretty quickly.
If I'm using the Python (or Perl I imagine; thus far I just built
Python) bindings, can I build a transaction on the server-side
in-memory. I essentially want to merge a change into another location
automatically iff it is conflict-free because the original files are
identical (guaranteed conflict-free merge).
In this instance, each release generally contains about 8000 files and
is about 35 Mb. Many commits to trunk (usually of <= 20 files) will
automatically need to commit to 2 or 3 releases.
One idea would be to only checkout some things -- i.e. use
non-recursive checkouts -- which could be a huge savings in most
instances, but from what I've seen reported to the mailing lists here,
non-r checkouts suffer some significant bugs.
Maybe if I build a very selective WC myself with the bindings, the
non-r checkout-related bugs won't rear themselves?
Any ideas about all this?
Btw: this is all part of my effort to simulate the useful
functionality that allows 'linked' files in CMVC across releases. I
think this is similar to what some people have discussed wrt some
shared files in VSS too. In this case, we are going to define our
repository layout so we can make some assumptions and write tools to
those assumptions.
-Travis
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Transaction building with bindings, do I need a WC on disk
Posted by Travis P <sv...@castle.fastmail.fm>.
On Feb 24, 2005, at 3:09 AM, Chia-liang Kao wrote:
> You want to look at the delta editor api. And Perl is the only
> bindings support
> driving and consuming the delta editor calls for modifying trees.
>
> and svk happens to be such a client written with the perl bindings
> that allows
> you to merge without a wc. you can port changes painlessly with:
>
> svk merge -c234 //project/trunk //project/releng/1.0
>
> If the merge has conflicts you will be prompted to accept theirs or
> yours, or
> launch external merge tools.
Thanks very much for this information, particularly about the need
for using exactly the Perl bindings and why.
-Travis
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Transaction building with bindings, do I need a WC on disk
Posted by Chia-liang Kao <cl...@clkao.org>.
Travis P <svn <at> castle.fastmail.fm> writes:
> If I'm using the Python (or Perl I imagine; thus far I just built
> Python) bindings, can I build a transaction on the server-side
> in-memory. I essentially want to merge a change into another location
> automatically iff it is conflict-free because the original files are
> identical (guaranteed conflict-free merge).
>
> In this instance, each release generally contains about 8000 files and
> is about 35 Mb. Many commits to trunk (usually of <= 20 files) will
> automatically need to commit to 2 or 3 releases.
You want to look at the delta editor api. And Perl is the only bindings support
driving and consuming the delta editor calls for modifying trees.
and svk happens to be such a client written with the perl bindings that allows
you to merge without a wc. you can port changes painlessly with:
svk merge -c234 //project/trunk //project/releng/1.0
If the merge has conflicts you will be prompted to accept theirs or yours, or
launch external merge tools.
HTH,
ClK
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Transaction building with bindings, do I need a WC on disk
Posted by Daniel Patterson <da...@danpat.net>.
Ben Collins-Sussman wrote:
>
> On Feb 23, 2005, at 10:59 PM, Daniel Patterson wrote:
>
>>
>> They have the situation where they would like to let users upload zip
>> files to update portions of the tree, via a web interface.
>
>
> Haven't you just described the mod_dav_svn 'autoversioning' feature?
Yes, for simple changes. However, our end users have no DAV clients
(Windows NT 4.0 is their desktop environment) and they probably won't
be direcly exposed to the DAV interface anyway (Subversion is being
used by the web application as a back end).
When one of our users uploads a collection of files, it'd be nice
to commit them in one go (and not depend on Apache+DAV to be able
to do it, and not need a working copy).
daniel
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Transaction building with bindings, do I need a WC on disk
Posted by Ben Collins-Sussman <su...@collab.net>.
On Feb 23, 2005, at 10:59 PM, Daniel Patterson wrote:
>
> They have the situation where they would like to let users upload zip
> files to update portions of the tree, via a web interface.
Haven't you just described the mod_dav_svn 'autoversioning' feature?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Transaction building with bindings, do I need a WC on disk
Posted by Daniel Patterson <da...@danpat.net>.
Then Ben Collins-Sussman wrote:
> My reaction is: I don't understand the bigger picture, and why you're
> trying to do such complex things.
>
> AFAICT, you need to port a 'set of changes' from the trunk to N
> different release branches. Why can't you do it the normal way? Keep
> a working copy of every release branch. Run 'svn merge' on each one,
> and then 'svn commit'.
>
> That's how changes get ported from one branch to another in Subversion.
> I don't see what -N checkouts have to do with anything, or why you're
> trying to copy files around between branches. I'm wondering if you're
> trying to artifically transfer a paradigm from some other SCM system to
> Subversion.
I've actually had a user who's playing with the javahl bindings ask
me a kind of similar question. He's *very* interested in using Subversion
as the filesystem backend for a web application. The history information
that it provides is a million dollar feature from his point of view
(until I pointed out Subversion, they were going to keep files on disc
with .1, .2, .3 extensions).
They have the situation where they would like to let users upload zip
files to update portions of the tree, via a web interface. The entire
tree is huge (many gigabytes), so keeping a working copy on disc is
a bit of a pain. In addition, most queries to the website will hit
the HEAD revision to retrieve content. When an upload happens, he
wants to be able to construct a commit_editor and commit changes
without having to have a working copy. From what I can tell, the
java bindings fall a touch short of being able to do that just now.
Another context where the "merge without WC" might be useful is in
simulating a promotional model like some other CM tools provide
(CA's Harvest is one that I've used like this). If changes can
only be made on /trunk (enforced with pre-commit hooks), then it's
always safe to merge to branches, no conflicts will arise (assuming
you ensure that dependent revisions are always merged together).
This is nice for people who want to "promote to test" without a
massive working copy.
daniel
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Transaction building with bindings, do I need a WC on disk
Posted by Travis P <sv...@castle.fastmail.fm>.
On Feb 23, 2005, at 9:20 PM, Ben Collins-Sussman wrote:
> My reaction is: I don't understand the bigger picture, and why you're
> trying to do such complex things.
>
> AFAICT, you need to port a 'set of changes' from the trunk to N
> different release branches. Why can't you do it the normal way? Keep
> a working copy of every release branch. Run 'svn merge' on each one,
> and then 'svn commit'.
Yes, that is fine, good, and the way that I operate projects generally.
Automating the checkout/merge/commit to various releases when it is
conflict-free because the changed files are identical would match a
currently enjoyed function of the current SCM system and would help me
convince the group to change.
Yes, the group does not find out until the nightly build if these
changes actually broke the thing: the reality is that it works well
enough and breakages because of the multi-release checkin are rare
enough (platform differences account for most failures: AIX, Linux,
Sun, and HP, 32-bit and 64-bit of each, on platform 'native'
compilers).
> That's how changes get ported from one branch to another in
> Subversion. I don't see what -N checkouts have to do with anything,
> or why you're trying to copy files around between branches. I'm
> wondering if you're trying to artifically transfer a paradigm from
> some other SCM system to Subversion.
I think this is pretty generic actually.
Assume something like:
repos/trunk/
repos/release/branch/
11.x/
10.x/
9.x/
8.x/
repos/release/tag/<many tags>
Let's say a bug is found and fixed with two sourcefile changes:
src/libclient/init.c
src/util/idsort.c
Assume that those two files have been unchanged for many years and so
are identical from version 8 until trunk. Let's assume that each
release generates a 55 Mb working copy. Rather than requiring 220 Mb
of additional disk space for the branches (we already have a full trunk
WC) and transferring that all across the network (cross-country), what
if we could, for each release, just download a couple Mb to get the
src/libclient/ and src/util/ directories in our working copy (rooted
at their common release directory of course so the commit of the two
changes is one atomic commit) and avoid downloading, for example, the
25 Mb of documentation that each release contains. We'd then merge
into the limited WC which contains everything necessary for the merge.
The vast majority of changes involve < 20 files in just a few
directories (kilobytes of change). That's the common case I'd like to
try and optimize.
From a Linux server (svn 1.1.3) to AIX (svn 1.0.1 client--yes, this'll
be upgraded shortly) both within the same building, https checkout to
/tmp of the 55 Mb working copy of trunk takes ~6 minutes. (The users
I'm concerned about will be doing cross-country checkouts.)
Actually, relatedly, I'd like to be able to checkout our trunk and have
multiple WCs, each containing different changes-in-progress AND, I'd
like to exclude the documentation directory from most of them. Is
there a way to do this if the documentation is at the same level as the
directories of source? (trunk/doc/ trunk/libclient/*.c) It's nice
having the docs tightly tied to the releases like that, except for the
size of the WCs when making doc-change-free alterations.
I hope this email clarifies my questions and doesn't just heighten the
confusion. ;-)
I do admit that I changed the questions a bit from my original ones.
-Travis
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Transaction building with bindings, do I need a WC on disk
Posted by Ben Collins-Sussman <su...@collab.net>.
On Feb 23, 2005, at 6:16 PM, Travis P wrote:
>
> I've a couple question regarding what's generally possible that I
> think some developers could probably answer pretty quickly.
>
> If I'm using the Python (or Perl I imagine; thus far I just built
> Python) bindings, can I build a transaction on the server-side
> in-memory. I essentially want to merge a change into another location
> automatically iff it is conflict-free because the original files are
> identical (guaranteed conflict-free merge).
>
> In this instance, each release generally contains about 8000 files and
> is about 35 Mb. Many commits to trunk (usually of <= 20 files) will
> automatically need to commit to 2 or 3 releases.
>
> One idea would be to only checkout some things -- i.e. use
> non-recursive checkouts -- which could be a huge savings in most
> instances, but from what I've seen reported to the mailing lists here,
> non-r checkouts suffer some significant bugs.
>
> Maybe if I build a very selective WC myself with the bindings, the
> non-r checkout-related bugs won't rear themselves?
>
> Any ideas about all this?
>
My reaction is: I don't understand the bigger picture, and why you're
trying to do such complex things.
AFAICT, you need to port a 'set of changes' from the trunk to N
different release branches. Why can't you do it the normal way? Keep
a working copy of every release branch. Run 'svn merge' on each one,
and then 'svn commit'.
That's how changes get ported from one branch to another in Subversion.
I don't see what -N checkouts have to do with anything, or why you're
trying to copy files around between branches. I'm wondering if you're
trying to artifically transfer a paradigm from some other SCM system to
Subversion.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org