You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by David Summers <da...@summersoft.fay.ar.us> on 2003/02/01 05:46:12 UTC
SVNSERVE Tests Failing
For the last several days, a whole slew of SVNSERVE tests have been
failing on both RedHat 7.3 and 8.0.
Has anyone else seen this?
--
David Wayne Summers "Linux: Because reboots are for hardware upgrades!"
david@summersoft.fay.ar.us PGP Key: http://summersoft.fay.ar.us/~david/pgp.txt
PGP Key fingerprint = C0 E0 4F 50 DD A9 B6 2B 60 A1 31 7E D2 28 6D A8
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Eric Gillespie <ep...@pretzelnet.org>.
Greg Hudson <gh...@MIT.EDU> writes:
> So, I'm feeling like ra_svn is being treated as a second-class citizen
> by some developers. Before and during its development, I was assured
> that it would be well-received as a pre-1.0 feature, but now people seem
> to feel free to check in stuff which is known to leave ra_svn broken or
> which implements features in the other ra layers but not in ra_svn.
I agree completely. Today i went to finish off my patch for
svn_wc_is_ignored and found that ra_svn was busted. I was not
happy. Unfortunately, i spent time tracking down which revision
broke things before checking the list. Stupid, stupid.
> Without a lot of effort from me (which can't happen; I have to put too
> much effort into my job right now), this will be a self-reinforcing
> attitude. ra_svn will be the access layer which doesn't require Apache
> but also doesn't work so well.
While i agree, that others need to test new code with ra_svn, i
can say that even if they don't, i will be around to keep ra_svn
working. I have no intentions of letting it become the layer
that doesn't work. I am working on this current problem now.
BTW, the specific revision that broke it is 4648.
--
Eric Gillespie <*> epg@pretzelnet.org
Build a fire for a man, and he'll be warm for a day. Set a man on
fire, and he'll be warm for the rest of his life. -Terry Pratchett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Kevin Pilch-Bisson <ke...@pilch-bisson.net> writes:
> Can I take this opportunity to promote something that I have recently started
> doing at work?
>
> On my team at work, each developer has a private branch of the code we work
> on. We're free to do whatever we want on that branch, and use it to
> checkpoint however we choose.
>
> However, in order to merge our changes into the main development branch (which
> we do about once per week), we have to make sure that we can build both debug
> and release versions (not really applicable here), and pass a large suite of
> tests on both debug and release builds. Of course we also have an automated
> system that we submit patches to, and it performs all these checks for us,
> then checks in the change if they all pass.
>
> This allows us to check in small changes, yet maintain quality on the main
> branch.
>
> Maybe svn should try something similar.
Personally, I think our breakages are not frequent enough to warrant
the inconvenience this method would cause (I've worked this way on
other projects, and it does impose some extra burden on the
developers.)
Just my $.02,
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Kevin Pilch-Bisson <ke...@pilch-bisson.net>.
On Sat, Feb 01, 2003 at 04:35:56PM -0600, Karl Fogel wrote:
> The more general issue is harder to solve. Our test suite takes (on
> my development box) about a half-hour to run over any given RA layer.
> That means that each RA layer we add adds a half-hour of testing time.
> I can run the tests over ra_svn more often, but that will mean
> sometimes *not* running them over dav, or local. More likely, I'll
> run some subset of the tests over each of the ra layers, as a sanity
> check (I should really have started doing that earlier). It's not
> ideal, but an 1.5 hour test run for every change is not sustainable.
> Development would slow down too much.
Can I take this opportunity to promote something that I have recently started
doing at work?
On my team at work, each developer has a private branch of the code we work
on. We're free to do whatever we want on that branch, and use it to
checkpoint however we choose.
However, in order to merge our changes into the main development branch (which
we do about once per week), we have to make sure that we can build both debug
and release versions (not really applicable here), and pass a large suite of
tests on both debug and release builds. Of course we also have an automated
system that we submit patches to, and it performs all these checks for us,
then checks in the change if they all pass.
This allows us to check in small changes, yet maintain quality on the main
branch.
Maybe svn should try something similar.
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Kevin Pilch-Bisson http://www.pilch-bisson.net
"Historically speaking, the presences of wheels in Unix
has never precluded their reinvention." - Larry Wall
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Re: SVNSERVE Tests Failing
Posted by Philip Martin <ph...@codematters.co.uk>.
Karl Fogel <kf...@newton.ch.collab.net> writes:
> The more general issue is harder to solve. Our test suite takes (on
> my development box) about a half-hour to run over any given RA layer.
> That means that each RA layer we add adds a half-hour of testing time.
> I can run the tests over ra_svn more often, but that will mean
> sometimes *not* running them over dav, or local. More likely, I'll
> run some subset of the tests over each of the ra layers, as a sanity
> check (I should really have started doing that earlier). It's not
> ideal, but an 1.5 hour test run for every change is not sustainable.
> Development would slow down too much.
Like Karl, I don't run the regression tests over all the RA layers for
each change I commit, it does take a long time (although not as long
on Linux as the times I've seen reported for running the tests on
Windows). There are however things that can be done to "optimise" the
procedure. The following applies to a Linux machine as that's what I
use, other OSs probably allow something similar.
I can run all the ra_local, ra_dav and ra_svn tests in half an hour on
this machine, which is three or four years old. To achieve that I use
two techniques. First, I use tmpfs filesystems to make the tests
faster. Second, I use three builds outside the source directory, so I
can run the three tests in parallel. These two techniques complement
one another, the out-of-source build makes using tmpfs easy, and using
tmpfs reduces disk IO allowing parallel testing. If I were to run the
regression tests sequentially, without tmpfs, it would take over two
hours on this machine.
I configure and build using something like
$ svn co http://.../trunk svn
$ mkdir obj
$ cd obj
$ ../svn/configure
$ make
$ make check
About the only difference compared to an in-source build is that when
I want to run a single python test I cannot do
$ cd subversion/tests/clients/cmdline
$ ./basic_tests.py 1
instead I need to specify the path to the python file
$ cd subversion/tests/clients/cmdline
$ /source/dir/svn/subversion/tests/clients/cmdline/basic_tests.py 1
Building outside the source directory does not on its own make the
tests faster -- to do that I use a tmpfs filesystem. This requires a
kernel with tmpfs support, I am using 2.4.20 built with CONFIG_TMPFS=y.
I have lines in my /etc/fstab like
tmpfs /build/dir/subversion/tests tmpfs defaults,users,noauto,exec,size=300m 0 0
where /build/dir is the path to the obj directory, and I use it like
$ mount /build/dir/subversion/tests
$ make mkdir-init
$ make
which mounts the tmpfs filesystem over the tests directory, creates
the tests directory hierarchy, and rebuilds the test stuff. That
doesn't take very long. Now running "make check" it is twice as fast
as before, and the amount of disk activity is much reduced.
Using tmpfs speeds up the tests, but it still doesn't use all the
available CPU. So I can use a second, parallel, build directory
configured to use the same source directory as the first. I build it,
mount a second tmpfs filesystem, and now I can run a second instance
of the regression tests using a different RA layer. I can then go on
to a third directory and run all three RA layers.
This machine does not have 3x300MB of memory so three tmpfs sets of
the regression test output will force it to swap. That's not a big
problem, writing to swap is much less disk intensive than writing to
active BDB databases on disk.
Note, any files created within the tmpfs mount will be lost when it is
unmounted. That's not really a problem since all the files are
generated objects, executables or test output. The test source code
remains on the normal filesystem, so any changes you make are
preserved.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Mix <mi...@acm.org>.
Karl Fogel wrote:
> Mix <mi...@acm.org> writes:
>
>>FAIL: prop_tests.py 11: set and get a revprop change
>>END: prop_tests.py
>>
>>The script is still running as I write this so you should see the full
>>svn-breakage email in a few minutes.
>
> This is the one test that requires setting up a repository hook, as I
> recall -- because revprop changes require
>
> REPOS/hooks/pre-revprop-change
>
> to exist, be executable, and exit with success, or the revprop change
> will not go through.
>
> The test is supposed to set up this situation, but is there some
> reason why what it's doing wouldn't work on your environment?
Well, it passed 4701 and I didn't do anything special.
I'll look into it.
Mix.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Eric Gillespie <ep...@pretzelnet.org>.
Karl Fogel <kf...@newton.ch.collab.net> writes:
> This is the one test that requires setting up a repository hook, as I
> recall -- because revprop changes require
>
> REPOS/hooks/pre-revprop-change
>
> to exist, be executable, and exit with success, or the revprop change
> will not go through.
>
> The test is supposed to set up this situation, but is there some
> reason why what it's doing wouldn't work on your environment?
No, that isn't it. See my other mail in this thread.
--
Eric Gillespie <*> epg@pretzelnet.org
Build a fire for a man, and he'll be warm for a day. Set a man on
fire, and he'll be warm for the rest of his life. -Terry Pratchett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Mix <mi...@acm.org> writes:
> FAIL: prop_tests.py 11: set and get a revprop change
> END: prop_tests.py
>
> The script is still running as I write this so you should see the full
> svn-breakage email in a few minutes.
This is the one test that requires setting up a repository hook, as I
recall -- because revprop changes require
REPOS/hooks/pre-revprop-change
to exist, be executable, and exit with success, or the revprop change
will not go through.
The test is supposed to set up this situation, but is there some
reason why what it's doing wouldn't work on your environment?
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Eric Gillespie <ep...@pretzelnet.org>.
Mix <mi...@acm.org> writes:
> Well, I was testing the script today (without causing a failure) and
> this showed up in the output:
>
> CMD: svnadmin "create" "repositories/prop_tests-11"
> CMD: svnadmin dump "local_tmp/repos" | svnadmin load
> "repositories/prop_tests-11"
> CMD: svn "co" "--username" "jrandom" "--password" "rayjandom"
> "svn://localhost/repositories/prop_tests-11" "working_copies/prop_tests-11"
> CMD: svn "propset" "--revprop" "-r" "0" "cash-sound" "cha-ching!"
> "working_copies/prop_tests-11"
> svn: A repository hook failed.
> svn: pre-revprop-change hook failed with error output:
>
> FAIL: prop_tests.py 11: set and get a revprop change
> END: prop_tests.py
Yep, that's caused by the apr problem i reported. Not sure why
only some people see it. Apply this patch to apr
threadproc/unix/proc.c:
Index: proc.c
===================================================================
RCS file: /home/cvspublic/apr/threadproc/unix/proc.c,v
retrieving revision 1.62
diff -a -u -r1.62 proc.c
--- proc.c 6 Jan 2003 23:44:38 -0000 1.62
+++ proc.c 24 Jan 2003 01:43:42 -0000
@@ -474,7 +474,11 @@
waitpid_options |= WNOHANG;
}
- if ((pstatus = waitpid(proc->pid, &exit_int, waitpid_options)) > 0) {
+ do {
+ pstatus = waitpid(proc->pid, &exit_int, waitpid_options);
+ } while (pstatus < 0 && errno == EINTR);
+
+ if (pstatus > 0) {
proc->pid = pstatus;
if (WIFEXITED(exit_int)) {
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Mix <mi...@acm.org>.
Karl Fogel wrote:
> Mix <mi...@acm.org> writes:
>>Ok. Everything should now be setup on my end for a nightly svncheck.
>
> Okay. If you want to deliberately cause one failure, just to test
> that a mail with the expected format goes to svn-breakage, feel free
> (just include a note somewhere in the body that it's not a real
> failure).
>
> Thanks for setting this up.
Well, I was testing the script today (without causing a failure) and
this showed up in the output:
CMD: svnadmin "create" "repositories/prop_tests-11"
CMD: svnadmin dump "local_tmp/repos" | svnadmin load
"repositories/prop_tests-11"
CMD: svn "co" "--username" "jrandom" "--password" "rayjandom"
"svn://localhost/repositories/prop_tests-11" "working_copies/prop_tests-11"
CMD: svn "propset" "--revprop" "-r" "0" "cash-sound" "cha-ching!"
"working_copies/prop_tests-11"
svn: A repository hook failed.
svn: pre-revprop-change hook failed with error output:
FAIL: prop_tests.py 11: set and get a revprop change
END: prop_tests.py
The script is still running as I write this so you should see the full
svn-breakage email in a few minutes.
Mix.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Mix <mi...@acm.org> writes:
> Ok. Everything should now be setup on my end for a nightly svncheck.
Okay. If you want to deliberately cause one failure, just to test
that a mail with the expected format goes to svn-breakage, feel free
(just include a note somewhere in the body that it's not a real
failure).
Thanks for setting this up.
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Mix <mi...@acm.org>.
Karl Fogel wrote:
> Mix <mi...@acm.org> writes:
>>>Is anyone doing a regular ra_svn check? Does anyone want to?
>>
>>I can setup a nightly ra_svn check.
>
> That's great -- much thanks.
>
> I'd say, set it up to email svn-breakage if and only if 'make check'
> fails, with the subject line something like this, so it's clear at a
> glance what happened:
>
> Subject: svn rev 5000: FAIL (ra_svn, i686-pc-linux-gnu shared)
>
> And for the body, just the 'make check' output followed by the
> contents of tests.log, I guess.
Ok. Everything should now be setup on my end for a nightly svncheck.
Mix.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Mix <mi...@acm.org> writes:
> > the `svn-breakage' list.
> > Is anyone doing a regular ra_svn check? Does anyone want to?
>
> I can setup a nightly ra_svn check.
That's great -- much thanks.
I'd say, set it up to email svn-breakage if and only if 'make check'
fails, with the subject line something like this, so it's clear at a
glance what happened:
Subject: svn rev 5000: FAIL (ra_svn, i686-pc-linux-gnu shared)
And for the body, just the 'make check' output followed by the
contents of tests.log, I guess.
I hope not to be the person who causes a "FAIL" mail to be sent :-).
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Mix <mi...@acm.org>.
Karl Fogel wrote:
> Wow, you are so right. That really shouldn't be so hard to set up,
> either. Some people are already running autobuilds, sending output to
> the `svn-breakage' list.
>
> Is anyone doing a regular ra_svn check? Does anyone want to?
I can setup a nightly ra_svn check.
Mix.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Branko Čibej <br...@xbc.nu>.
Karl Fogel wrote:
>Eric Gillespie <ep...@pretzelnet.org> writes:
>
>
>>I have no doubt that Karl actually is that busy. I know i am.
>>What the project needs is an auto-builder running the test suite
>>over all ra layers every night.
>>
>>
>
>Wow, you are so right. That really shouldn't be so hard to set up,
>either. Some people are already running autobuilds, sending output to
>the `svn-breakage' list.
>
>Is anyone doing a regular ra_svn check? Does anyone want to?
>
>
Hey, getting my autotesters to run ra_svn in addition to ra_local
shouldn't be that much work. I'll ger right on it. (I've been putting
off adding ra_dav tests because that _is_ a bit of a bitch to
automate... oh, well.)
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Eric Gillespie <ep...@pretzelnet.org> writes:
> I have no doubt that Karl actually is that busy. I know i am.
> What the project needs is an auto-builder running the test suite
> over all ra layers every night.
Wow, you are so right. That really shouldn't be so hard to set up,
either. Some people are already running autobuilds, sending output to
the `svn-breakage' list.
Is anyone doing a regular ra_svn check? Does anyone want to?
Somewhat orthogonally:
What if we should change `svn-breakage's to be "don't email if
everything passes, only email if something went wrong"? (Of course,
recipients could do such filtering with their mail clients, but it's
easier not to have to.) If silence is golden, then we'll all notice
when something makes a noise.
Also, the subject lines of `svn-breakage' mails don't indicate which
RA layer passed or failed 'make check'. Right now, they look like:
Subject: svn rev 28790: FAIL (i986-pc-linux-gnu shared)
But perhaps this:
Subject: svn rev 28790: FAIL (ra_svn, i986-pc-linux-gnu shared)
...would be more useful?
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Eric Gillespie <ep...@pretzelnet.org>.
Mix <mi...@acm.org> writes:
> Busy guy if you didn't have 30 minutes in 4 days.
I have no doubt that Karl actually is that busy. I know i am.
What the project needs is an auto-builder running the test suite
over all ra layers every night.
--
Eric Gillespie <*> epg@pretzelnet.org
Build a fire for a man, and he'll be warm for a day. Set a man on
fire, and he'll be warm for the rest of his life. -Terry Pratchett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Mix <mi...@acm.org>.
Karl Fogel wrote:
> The more general issue is harder to solve. Our test suite takes (on
> my development box) about a half-hour to run over any given RA layer.
> That means that each RA layer we add adds a half-hour of testing time.
> I can run the tests over ra_svn more often, but that will mean
> sometimes *not* running them over dav, or local. More likely, I'll
> run some subset of the tests over each of the ra layers, as a sanity
> check (I should really have started doing that earlier). It's not
> ideal, but an 1.5 hour test run for every change is not sustainable.
> Development would slow down too much.
$ svn log -r 4648 | grep rev
rev 4648: kfogel | 2003-01-29 00:32:33 -0500 (Wed, 29 Jan 2003) | 61 lines
$ date
Sat Feb 1 18:25:11 EST 2003
Busy guy if you didn't have 30 minutes in 4 days.
Mix.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Greg Hudson <gh...@MIT.EDU> writes:
> So, I'm feeling like ra_svn is being treated as a second-class citizen
> by some developers. Before and during its development, I was assured
> that it would be well-received as a pre-1.0 feature, but now people seem
> to feel free to check in stuff which is known to leave ra_svn broken or
> which implements features in the other ra layers but not in ra_svn.
>
> Without a lot of effort from me (which can't happen; I have to put too
> much effort into my job right now), this will be a self-reinforcing
> attitude. ra_svn will be the access layer which doesn't require Apache
> but also doesn't work so well.
>
> I am not pleased.
There are several points here; I'll try to address them individually.
First of all, I'm sorry -- the breakage in this case is my fault, and
of course I'll fix it. I didn't know the changes would break ra_svn
when I checked them in. Not sure what makes you claim the stuff was
"known to leave ra_svn broken". Maybe it was known to someone, but
not me :-).
The more general issue is harder to solve. Our test suite takes (on
my development box) about a half-hour to run over any given RA layer.
That means that each RA layer we add adds a half-hour of testing time.
I can run the tests over ra_svn more often, but that will mean
sometimes *not* running them over dav, or local. More likely, I'll
run some subset of the tests over each of the ra layers, as a sanity
check (I should really have started doing that earlier). It's not
ideal, but an 1.5 hour test run for every change is not sustainable.
Development would slow down too much.
Although I can understand why you're frustrated, your characterization
of the situation is frankly unfair:
> I was assured
> that it would be well-received as a pre-1.0 feature, but now people seem
> to feel free to check in stuff which is known to leave ra_svn broken or
> which implements features in the other ra layers but not in ra_svn.
Yes, and the assurances are still true, temporary breakage
notwithstanding. However, you can't just drop in a new RA layer, and
then largely step away from its development and maintenance for N
weeks, without seeing some negative side effects. There's going to be
some lag while others learn how to pick up the slack.
I mean, what did you expect, Greg?
Adding a new RA layer means there's another maintenance path in the
project, whenever someone implements a new feature (say, checksums)
that affects the RA layers. You are under no obligation to maintain
it, but what are others supposed to do? The checksums issue was filed
long before ra_svn existed; should the addition of the new layer
suddenly magnify the work needed to resolve that issue? Should
schedules suddenly be increased by .5 because of of the new code?
You have a complaint, but you don't offer a solution (except for me to
spend more time in front of a computer than I already do, thanks very
much). Well, I agree with your complaint, and I don't have a solution
either. So this situation looks symmetrical to me; the only
difference is I didn't mail the list about my displeasure.
It would be nice if resources were infinite. As you know, they're
not. You don't have to help ra_svn keep abreast of interface changes,
even though you're the most qualified person to do so. But don't
complain because we can't always keep up. I spend enough hours in
front of a computer editing Subversion code as it is; I don't need to
be told I'm obligated to maintain a whole new library even for issues
that were filed & estimated before that library existed -- nor do the
other volunteers.
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Eric Gillespie <ep...@pretzelnet.org> writes:
> Greg Hudson set me straight; he told me you were counting on the
> code that confused me not being executed. Ah, OK, now i get it.
> I got the answer to "please tell me it hasn't been intentionally
> broken" there.
>
> I was not trying to be antagonistic.
Thanks -- I'm glad to hear that.
There's a very long-standing policy of not breaking trunk (uh, on
purpose, anyway :-) ). So if you see commit that breaks trunk, you
can safely assume it is not what the committer intended; and that if
they did intend it, they probably shouldn't have. (With cheap
branches, it's hard to imagine a situation in which someone would
actually need to break trunk to get something done.)
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Eric Gillespie <ep...@pretzelnet.org> writes:
> Also, i think there's some confusion over what i mean by
> "intentional". No one intentionally creates a bug. But some
> projects operate under the HEAD-must-always-build rule (the only
> sane way to go, IMNSHO) but others allow HEAD to be broken
> sometimes. I have had co-workers who insisted that it was OK for
> HEAD to be broken for a few days while they worked at a problem
> iteratively. The idea of a branch was foreign to these people.
> *That* is what i thought was happening here, and that's what i
> mean by "intentionally broken".
Gotcha.
Fortunately, we're one of the sane projects, in theory at least.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Eric Gillespie <ep...@pretzelnet.org>.
Eric Gillespie <ep...@pretzelnet.org> writes:
> Greg Hudson set me straight; he told me you were counting on the
> code that confused me not being executed. Ah, OK, now i get it.
> I got the answer to "please tell me it hasn't been intentionally
> broken" there.
I still had it wrong. Greg Hudson set me straight yet again,
and i think i finally understand. This is the crucial bit i
missed:
+ svn_delta_editor_t *ra_svn_editor = svn_delta_default_editor (pool);
So when i saw libsvn_client/commit.c:66 trying to call apply_text
(which was a NULL pointer), and saw that it was intentionally
NULL, i checked ra_dav. And over there, apply_text was *never*
set. I completely overlooked that ra_dav was doing calling this
svn_delta_default_editor function. I thought ra_svn had a NULL
in apply_text and ra_dav just had a random value. *That* looked
intentional. I said so, and said please tell me i'm missing
something. Well, i was.
Also, i think there's some confusion over what i mean by
"intentional". No one intentionally creates a bug. But some
projects operate under the HEAD-must-always-build rule (the only
sane way to go, IMNSHO) but others allow HEAD to be broken
sometimes. I have had co-workers who insisted that it was OK for
HEAD to be broken for a few days while they worked at a problem
iteratively. The idea of a branch was foreign to these people.
*That* is what i thought was happening here, and that's what i
mean by "intentionally broken".
--
Eric Gillespie <*> epg@pretzelnet.org
Build a fire for a man, and he'll be warm for a day. Set a man on
fire, and he'll be warm for the rest of his life. -Terry Pratchett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Eric Gillespie <ep...@pretzelnet.org>.
Karl Fogel <kf...@newton.ch.collab.net> writes:
> It has, however, been *unintentially* broken since Wednesday. Please
> don't use such needlessly accusatory phrasing -- what do you gain by
> it?
This isn't the first time you've thought i was being
antagonistic. I seem to have trouble communicating with you, and
for that i apologize.
I saw that the code was calling a function pointer that would not
be set, and it was *known* that it would not be set; heck, that
same revision commented on that fact. It looked like you had
committed broken code with the intent to come back and fix it
later. I asked if that was true, and hoped it wasn't.
Greg Hudson set me straight; he told me you were counting on the
code that confused me not being executed. Ah, OK, now i get it.
I got the answer to "please tell me it hasn't been intentionally
broken" there.
I was not trying to be antagonistic.
--
Eric Gillespie <*> epg@pretzelnet.org
Build a fire for a man, and he'll be warm for a day. Set a man on
fire, and he'll be warm for the rest of his life. -Terry Pratchett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Eric Gillespie <ep...@pretzelnet.org> writes:
> Again, am i missing something? Please tell me this hasn't been
> intentionally broken since Wednesday.
"This hasn't been intentionally broken since Wednesday."
There. Happy?
It has, however, been *unintentially* broken since Wednesday. Please
don't use such needlessly accusatory phrasing -- what do you gain by
it?
Anyone can write a bug (heck, I can even write more than one). But no
one here intentionally breaks code, at least without announcing it
first and waiting to hear what others say.
This bug is my fault. I should have tested more carefully (like, at
least ran some subset of the tests over ra_svn), and I didn't. My
apologies; I'll change the way I test, so this is less likely to
happen in the future.
However, as someone who recently submitted a patch that *didn't even
compile*, Eric, I think you could be a little less rancorous in
pointing out breakage. After I wasted my own time discovering that
your patch didn't build, I don't recall casting doubts on your
intentions in my rather amicable followups.
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Whither apply_textdelta
Posted by Greg Hudson <gh...@MIT.EDU>.
On Wed, 2003-02-12 at 22:27, Greg Stein wrote:
> [ (was Re: SVNSERVE Tests Failing) ... no wonder I missed it :-) ]
Yeah, well, I changed the subject eventually. :)
> On Tue, Feb 11, 2003 at 05:51:05PM -0500, Greg Hudson wrote:
> P5: The editor is in the best position to determine the format of the
> delta output (e.g. svndiff vs. unidiff) based on requirements or
> environmental/cmdline switches or whatever.
That may be true for the erstwhile XML editor, but I'm not sure it's
true in general.
> > S1 variant #1: Keep apply_textdelta but also add apply_fulltext. This
> > would let S1P[2-4] be more elegant, though not more efficient.
> Is the implication here, that all editors support both code paths, and the
> driver chooses?
I didn't describe any schemes where the editor doesn't have to support
both code paths. Editor composition becomes difficult when you give
editors that much leeway.
[Regarding S1 variant #2, where apply_textdelta can return a flag
saying "please use full text":]
> I'm not sure about the "gives up on P4". Couldn't the editor read the
> necessary config to decide what flag value to return?
Hm, or the editors could take a flag on creation which they pass to the
drivers in turn. (That way the configuration could be read by
libsvn_client or whatever.) You're right.
> > Inconsistent with the rest of the editor
> > interface, which uses a push model.
> The notion of inconsistency is a bit of a red herring, I think. There are no
> other streams in the editor interface. We "push" values at the editor in the
> form of arguments, and you could also say that we push streams at the
> editor. Sure, the editor happens to read from the stream, so it can be
> called "pulling" the data, but that isn't inconsistent with anything else in
> the editor.
I think it is very different. For every other part of the editor, we
provide a chunk of information when we have it. Passing an input stream
to apply_text or apply_textdelta means that for file bodies we suddenly
switch gears and have the editor request chunks of information when it
wants them.
> > The ra_svn receiving editor uses a callback to get at base
> > texts from the FS or WC as appropriate. ra_dav uses a callback to get
> > at base texts from the WC for update/switch/diff operations
> I disagree with this part of the solution. Since the base is optional, the
> driver simply passes NULL to the editor.
But we don't have the target stream either; all we have is some diffs we
got from the network. So we need the base text in order to reproduce
the target stream.
> Hmm. I keep thinking that apply_text should have a mime type for the target
> stream to indicate that the stream is a diff rather than a fulltext.
Uh, full stop. No description of apply_text has ever allowed the target
stream to be a diff. Certainly, what Karl started implementing did not
have that property. What you're describing is functionally similar to
S1 variant #1 (possibly combined with variant #3) as I described things.
> Oh: the guys convinced me that "heck, the driver should just rebuild the
> fulltext and pass that in [along with the base] rather than complicating the
> interface with a possibly-diff/possibly-full stream argument." And now we
> see the results of that: the driver may have to reach into some other system
> to fetch the base, when it doesn't necessarily have it. Thus, it is best to
> allow for diffs to flow through the system. If somebody *wants* a full text,
> then assume they have access to the base.
> [ and that assumption holds water in all of our cases right now ]
I'm not sure what this means.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Whither apply_textdelta
Posted by Greg Stein <gs...@lyra.org>.
[ (was Re: SVNSERVE Tests Failing) ... no wonder I missed it :-) ]
On Tue, Feb 11, 2003 at 05:51:05PM -0500, Greg Hudson wrote:
>...
> P1: Neon doesn't have a push interface for request bodies, so right now
> ra_dav buffers text deltas on commits. There's a legitimate question of
> whether solving this problem is allowing the tail to wag the dog.
>
> P2: We send self-compressed deltas on checkout and import. This saves
> bandwidth but takes time, and there might be more efficient ways to save
> time and space using commodity compression code (in the ra layers).
> More measurement is required to see if we should stop self-compressing,
> but I'll assume for the sake of argument that we want to send full texts
> for imports and checkouts.
>
> P3: We use deltas when it makes no sense: over ra_local and over
> ra_svn/ra_dav connections to loopback. The editor is in the best
> position to notice these cases, since editor implementations live inside
> ra_lib and driver implementations do not.
>
> P4: We use deltas when it makes very little sense: over fast LANs. We
> shouldn't try to guess when bandwidth is cheaper than CPU time, but in
> an ideal world the user might be able to make that decision in
> .subversion/servers or on the command line.
P5: The editor is in the best position to determine the format of the
delta output (e.g. svndiff vs. unidiff) based on requirements or
environmental/cmdline switches or whatever.
(this consideration came up with the XML editor stuff; it is preferable to
insert "plain text" diffs rather than base64-encoded svndiffs)
P6: Developer-only consideration: we always send deltas, which precludes
the option of fulltexts during development of new subsystems.
(not sure if this is a problem today, but this was one of my original
motivators)
[ and yes, I agree with P1..P4 ]
> Here are the solutions people have proposed, and how they solve the
> above problems:
Agreed, except for the following points:
> S1: Leave apply_textdelta alone.
>...
> S1P2: Instead of self-compressing when no base is available, fake up
> delta windows which just say "insert this blob of new text." This
> is just as efficient as any other proposed scheme for sending full
> texts.
Here, the checkout and import drivers are assuming that fulltext is always
the right thing to do. The editors may decide that self-compressing is the
right thing (think 56k modem). This is sort of a converse of P2..P4 where
the problem is that we want to *avoid* deltas. In certain cases, we also
*want* the deltas. As you point out, typically, the [RA] editor is in the
best position to know the requirements.
> S1P3: Teach the drivers to send full texts (using faked-up delta
> windows) according to a flag. Add a new function to ra_lib to ask
> whether deltification is desirable, which can be by libsvn_client to
> inform svn_wc_adm_crawl. On the server side, svnserve or
> mod_dav_svn can provide the flag to libsvn_repos.
Right.
> S1P4: Leverage S1P3; simply pass the full text flag to drivers if
> configuration says it's appropriate.
Probably a tri-state: always-full-text, never-full-text, default.
S1P5: no solution? This would be a transformation from windows to a
full text to a GNU diff somehow.
S1P6: leverage S1P3/S1P4; the developer just munges the flag to ensure
that only fulltexts are used.
> S1 variant #1: Keep apply_textdelta but also add apply_fulltext. This
> would let S1P[2-4] be more elegant, though not more efficient.
Is the implication here, that all editors support both code paths, and the
driver chooses?
> S1 variant #2: Modify apply_textdelta so that it can return a flag
> saying "you can send me deltas if you want, but I think bandwidth is
> cheap; you should send me full text instead for better performance."
> Eliminates the need to pass flags to drivers to solve P3 (though that
> mostly gives up on P4). Easiest if we don't use variant #1.
I'm not sure about the "gives up on P4". Couldn't the editor read the
necessary config to decide what flag value to return?
> S1 variant #3: Invert apply_textdelta without worrying about full
> texts. (By "invert" I mean that the driver provides a callback which
> allows apply_textdelta to gather delta windows, instead of the editor
> providing a callback which the driver pushes delta windows onto.)
> Replace or agument svn_txdelta_to_svndiff with a function that pushes in
> the opposite direction. Solves P1, doesn't affect P[2-4]. Compatible
> with variant #1 or #2. Inconsistent with the rest of the editor
> interface, which uses a push model.
The notion of inconsistency is a bit of a red herring, I think. There are no
other streams in the editor interface. We "push" values at the editor in the
form of arguments, and you could also say that we push streams at the
editor. Sure, the editor happens to read from the stream, so it can be
called "pulling" the data, but that isn't inconsistent with anything else in
the editor.
Except for that line, I agree with the commentary.
> S2: Replace apply_textdelta with apply_text([base stream], target
> stream).
Note the optional base.
> The ra_svn receiving editor uses a callback to get at base
> texts from the FS or WC as appropriate. ra_dav uses a callback to get
> at base texts from the WC for update/switch/diff operations
I disagree with this part of the solution. Since the base is optional, the
driver simply passes NULL to the editor. It is assumed that the editor (the
FS or WC) has the base. In this case, yes: it does. The original design also
stated that the editor is free to raise an error if a base is not provided,
it requires one, and it cannot find the base through other means.
Thus, callbacks are not required.
> (no, the
> pain is not limited to ra_svn, you just won't notice it as much inside
> ra_dav because of the background pain level).
I expect that the "background pain" you refer to will diminish. Per previous
emails, I think that is mostly due to slowly building insanity in ra_dav
rather than anything truly central.
> Inconsistent with the
> rest of the editor interface which uses a push model.
Ahem. :-)
> S2P1: As with S1 variant #3, svn_txdelta_to_svndiff will need to be
> agumented or replaced with a function that pushes in the opposite
> direction. After that is done, it is easy enough to make ra_dav's
> apply_text supply svndiff data on demand from Neon.
The core functionality is a read-stream which serializes windows into the
svndiff format. The stream would be initialized with an svn_txdelta_stream_t
for reading the windows. Presumably, some kind of internal buffering would
happen for unread, serialized windows. The logic is something like:
def svndiff_formatter_read(self, buf, buflen):
if not self.buffer:
window = svn_txdelta_next_window(self.txdelta)
self.buffer = svndiff_serialize(window)
result = self.buffer[:buflen]
self.buffer = self.buffer[buflen:]
return result
ra_dav can then read from the formatter svn_stream to satisfy Neon's
callbacks for more data.
> S2P2: For checkout and import, pass NULL for the base stream. The
> editor can still self-compress if it wants to, or it can send a full
> text.
>
> S2P3: The editor can choose to ignore the base stream and send a
> full text if it believes bandwidth to be cheap.
>
> S2P4: Editors could be passed a flag at creation time to use full
> text instead of deltas, in which case they would ignore the bsae
> stream.
Agreed on all three. I would also augment S2P4 with the fact that an editor
can figure out the flag by itself, if it has access to the right bits.
S2P5: the editor generates the appropriate output from the base and
target, based on whatever configuration/setting.
S2P6: some #ifdef in the editor controls its behavior
> S2 variant #1: Replace apply_textdelta with apply_text(target stream);
> the base text becomes implicit. ra_dav uses a callback to get at base
> texts from the WC for commit as well as update/switch/diff operations.
> S2P2 changes a little bit, in that the callback will say "there is no
> base text" and then the editor decide whether to self-compress or send
> full text.
ra_dav expects a callback to get base texts for when it retrieves files.
This could leverage that callback. However, I prefer an editor interface
that has the base stream explicit so that these callbacks are not required.
Hmm. I keep thinking that apply_text should have a mime type for the target
stream to indicate that the stream is a diff rather than a fulltext.
Oh: the guys convinced me that "heck, the driver should just rebuild the
fulltext and pass that in [along with the base] rather than complicating the
interface with a possibly-diff/possibly-full stream argument." And now we
see the results of that: the driver may have to reach into some other system
to fetch the base, when it doesn't necessarily have it. Thus, it is best to
allow for diffs to flow through the system. If somebody *wants* a full text,
then assume they have access to the base.
[ and that assumption holds water in all of our cases right now ]
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Whither apply_textdelta (was Re: SVNSERVE Tests Failing)
Posted by Greg Hudson <gh...@MIT.EDU>.
On Tue, 2003-02-11 at 11:11, Karl Fogel wrote:
> I don't know the answer yet; would love some feedback. Are the
> problems identified here the same ones you were thinking of?
Yes. By moving the "implicit base" requirement inward, you require the
addition of callbacks to allow the connecting pipes to produce and
consume deltas. That's doable, but complicated; I think the information
provided to the editor should match the information we expect to see
going over the net; none of this clever judo stuff which moves
complexity from the places you'd expect to find it to the places you
wouldn't.
Re: SVNSERVE Tests Failing
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Regarding the `apply_text' change, I wrote:
> Hmmm. I won't be doing more coding on this before Monday, at which
> time I'll be flying out to CA to be in the CollabNet office for a
> week, where Greg Stein is. We've been talking a lot about this
> change, he's very familiar with it; will ponder and talk with him.
> (Not trying to avoid list discussion, it's just that I won't be on the
> list again until sometime late Monday or early Tuesday.)
Greg Stein and I did talk about this some more. He pointed out that
the editor will be carrying more information, so naturally the
connection between two editors must grow (in complexity or
"bandwidth") to carry that information too. In other words, the
change doesn't destroy the piping property, it just means that the
connectors have to get a bit more sophisticated, in order to send all
the information across the pipe.
That said, I've been thinking some more about this. The rest of this
mail retraces those thoughts; sorry for the length, but thoroughness
is important here.
First of all, things would probably be clearer without the `base'
stream into the interface. It's not strictly necessary, and perhaps
should go away. Consider:
OLD INTERFACE:
--------------
driver of apply_textdelta()
deltifies according to some
BASE, then pushes deltas
at the editor =========================> driver on the other side
receives windows, pushes
them directly at the
editor over here, which
uses the same implied
BASE to recover fulltext.
NEW INTERFACE:
--------------
driver of apply_text()
hands BASE and TARGET
streams to editor, which
optionally deltifies TARGET
and sends that across to ==============> driver the other side,
which can use the same
implied BASE to recover
the fulltext of TARGET,
which it then hands to
the editor over here.
(Strictly speaking, the driver on the right side of the new-style
interface could also hand BASE to that editor, but it's usually
useless to do so, which is why the apply_text() interface explicitly
says that BASE is always optional.)
The point I'm trying to bring out with this comparison is that the
base stream just happens to be explicit in the new interface, but it's
not necessary that it be so. It could be implicit, as it was in the
old interface. The documentation for both apply_text() and
apply_textdelta() contains the following:
/*
* @a file_baton indicates the file we're creating or updating, and the
* ancestor file on which it is based; it is the baton set by some
* prior @c add_file or @c open_file callback.
*/
...which kind of implies that the base stream always produces not just
some random text, but specifically the text of the ancestor -- if it
were anything else, the driver on the right-hand side would probably
guess wrong when it tried to obtain the base stream locally, and would
therefore undeltify against the wrong data.
Since the source of the base stream is implied anyway, that makes me
think we shouldn't be passing in the base stream as a parameter.
Instead, apply_text() should just take the target_stream. If it
*chooses* to deltify that target stream according to an implied
ancestor, that's fine, because the driver on the other side will know
how to undeltify according to the same ancestor -- that driver is the
first editor's partner, after all. It's not driving the editor, it's
being driven by it, thus bits from the editor's state (such as the
implied ancestor) are available to it.
Then the new interface wouldn't be carrying any more information than
the old interface. It would be carrying the same information, just in
a different way. Deltification would still become strictly a decision
of the connecting pipe editor, not a requirement of the interface, and
that's a good thing IMHO. But there would be no need to name the base
stream when calling apply_text(). If the connection is to be
efficient, the base's identity is implied anyway, so we should just
use that base.
Oh. But I think I see a concern... Maybe the same concern you were
getting at before?
The problem with "implied base" is that it shifts data use away from
data source. In the diagram below, data flow is from left to right.
For example, in a commit, the "source" data on the far left is the
text base plus modified working file, and the "dest" on the far right
is the filesystem, the base node plus the new node resulting from this
edit. The "X" marks indicate which component is responsible for
deltifying/undeltifying using base data:
s d e ------------------> d e d
o r d ------------------> r d e
u i i ------------------> i i s
r v t ------------------> v t t
c e o ------------------> e o
e r r ------------------> r r
.................................................................
old api: X X
new api: X X
As the diagram shows, we've shifted the need for base text inward,
away from the sources of base texts.
So the trouble with the new api is that that the components that use
the base text don't directly "touch" the base's source (that is, the
working copy or the repository).
Now, on the left side, we've gotten around that problem by having the
driver create a stream to supply the base text to the editor. But how
will we get around it on the right side? I suppose the driver could
reach into "dest" to get a base by which to undeltify data... which it
would then pass to the rightmost editor, who writes it into the dest!
Is that weird, though? Should the rightmost driver be independent of
dest, and get everything it needs solely from the information coming
from its left?
I don't know the answer yet; would love some feedback. Are the
problems identified here the same ones you were thinking of?
In any case, it's clear to me that this change cannot continue to be
designed in phone calls and whiteboard sessions between me, Greg
Stein, and the Chicago mafia. It needs to be discussed on the list,
until we're sure we've got all bases covered.
Issue #510 does not block any pre-1.0 issues; it's pre-1.0 only
because it's an API finalization issue. Since it doesn't block
anything, I'm going to move it out of the 0.18 milestone while we
discuss. (If we don't figure something out soon, I can move some of
the changes already made over to a branch.)
-Karl
> Greg Hudson <gh...@MIT.EDU> writes:
> > Before apply_text, the editor interface worked like plumbing. You could
> > take an arbitary driver and plug it into an arbitrary editor, more or
> > less. More interestingly, you could insert a connecting pipe between
> > driver and editor:
> >
> > driver --> [editor --> network --> driver] --> editor
> >
> > libsvn_ra_svn/editor.c is such a connecting pipe. So was the XML
> > editor, before we got rid of it.
> >
> > With apply_text, there is no way to build such a connecting pipe without
> > sending full texts over the wire. The receiving end of the connecting
> > pipe no longer has enough information to make editing calls.
> >
> > I can see why apply_text is appealing. It moves a little bit of code
> > from driver to editor, and it lets us avoid deltifying over ra_local for
> > free (by omitting code, rather than by adding conditionals). But it
> > destroys an important property of the editor interface and, as a result,
> > will significantly complicate the ra_svn code if it stands.
>
> Yah. It wasn't just that, it was also that it made it easy to not
> deltify for new imports (which we formerly had to do). It also
> changes the interface from driver-push (pushing to window handlers,
> that is) to editor-pull, although that could probably be accomplished
> without changing the input type.
>
> I was aware we were losing this property, but assumed (perhaps rashly)
> that it wasn't a real loss, because the driver could reconstruct the
> correct stream.
>
> Hmmm. I won't be doing more coding on this before Monday, at which
> time I'll be flying out to CA to be in the CollabNet office for a
> week, where Greg Stein is. We've been talking a lot about this
> change, he's very familiar with it; will ponder and talk with him.
> (Not trying to avoid list discussion, it's just that I won't be on the
> list again until sometime late Monday or early Tuesday.)
>
> > If absolutely no one else agrees with me, I'll withdraw my veto. I
> > think I have a strong argument here, though.
>
> You do. Need to think; I grok the objection, I just don't fully grok
> the solution space yet.
>
> I guess the summary of this email is: "Hmmm. Objection understood.
> Off to think now." :-)
>
> If we do end up trying to press forward with the change, I will make
> sure that ra_svn's editor is the first converted, so that the
> complications become apparent as early as possible.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Greg Hudson <gh...@MIT.EDU>.
On Sat, 2003-02-01 at 20:24, Karl Fogel wrote:
> Yah. It wasn't just that, it was also that it made it easy to not
> deltify for new imports (which we formerly had to do).
If, for speed reasons, we want to send full texts of new files
(sacrificing the bandwidth benefit of self-compressed deltas), we can do
that without modifying the apply_textdelta interface. The sender just
has to make up fake delta windows. Each window contains one
instruction, saying, "insert this large block of new data." These fake
deltas would have minimal time and negligible space overhead.
The same technique can be used to avoid deltifying over ra_svn, only the
driver needs to have some way to know that the consumer doesn't want
real deltas. (Not as elegant as the current apply_text, where the
consumer just has to omit the deletification code.) We could do that
with a function or flag in the editor interface (a function would give
us the flexibility of asking "do you want deltas" over the network; a
flag would be simpler; I'm not sure whether we would want the
functionality).
There may be more elegant solutions which don't break the
connecting-pipe property, but this should certainly be considered as
part of the solution space.
> It also
> changes the interface from driver-push (pushing to window handlers,
> that is) to editor-pull, although that could probably be accomplished
> without changing the input type.
Right; changing the direction and changing the stream types are
orthogonal.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Greg Hudson <gh...@MIT.EDU> writes:
> Before apply_text, the editor interface worked like plumbing. You could
> take an arbitary driver and plug it into an arbitrary editor, more or
> less. More interestingly, you could insert a connecting pipe between
> driver and editor:
>
> driver --> [editor --> network --> driver] --> editor
>
> libsvn_ra_svn/editor.c is such a connecting pipe. So was the XML
> editor, before we got rid of it.
>
> With apply_text, there is no way to build such a connecting pipe without
> sending full texts over the wire. The receiving end of the connecting
> pipe no longer has enough information to make editing calls.
>
> I can see why apply_text is appealing. It moves a little bit of code
> from driver to editor, and it lets us avoid deltifying over ra_local for
> free (by omitting code, rather than by adding conditionals). But it
> destroys an important property of the editor interface and, as a result,
> will significantly complicate the ra_svn code if it stands.
Yah. It wasn't just that, it was also that it made it easy to not
deltify for new imports (which we formerly had to do). It also
changes the interface from driver-push (pushing to window handlers,
that is) to editor-pull, although that could probably be accomplished
without changing the input type.
I was aware we were losing this property, but assumed (perhaps rashly)
that it wasn't a real loss, because the driver could reconstruct the
correct stream.
Hmmm. I won't be doing more coding on this before Monday, at which
time I'll be flying out to CA to be in the CollabNet office for a
week, where Greg Stein is. We've been talking a lot about this
change, he's very familiar with it; will ponder and talk with him.
(Not trying to avoid list discussion, it's just that I won't be on the
list again until sometime late Monday or early Tuesday.)
> If absolutely no one else agrees with me, I'll withdraw my veto. I
> think I have a strong argument here, though.
You do. Need to think; I grok the objection, I just don't fully grok
the solution space yet.
I guess the summary of this email is: "Hmmm. Objection understood.
Off to think now." :-)
If we do end up trying to press forward with the change, I will make
sure that ra_svn's editor is the first converted, so that the
complications become apparent as early as possible.
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Greg Hudson <gh...@MIT.EDU>.
On Sat, 2003-02-01 at 18:12, Karl Fogel wrote:
> The idea is that the editor driver on the other side knows the base
> against which to apply the incoming svndiff stream -- so it uses that
> base to reconstruct the fulltext.
I understand the idea. Let's see if I can better explain the objection.
Before apply_text, the editor interface worked like plumbing. You could
take an arbitary driver and plug it into an arbitrary editor, more or
less. More interestingly, you could insert a connecting pipe between
driver and editor:
driver --> [editor --> network --> driver] --> editor
libsvn_ra_svn/editor.c is such a connecting pipe. So was the XML
editor, before we got rid of it.
With apply_text, there is no way to build such a connecting pipe without
sending full texts over the wire. The receiving end of the connecting
pipe no longer has enough information to make editing calls.
I can see why apply_text is appealing. It moves a little bit of code
from driver to editor, and it lets us avoid deltifying over ra_local for
free (by omitting code, rather than by adding conditionals). But it
destroys an important property of the editor interface and, as a result,
will significantly complicate the ra_svn code if it stands.
If absolutely no one else agrees with me, I'll withdraw my veto. I
think I have a strong argument here, though.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Greg Hudson <gh...@MIT.EDU> writes:
> I think the committer assumed that all editor providers were using the
> default editor, which isn't the case.
>
> However, I have a much bigger problem to bring up. apply_text cannot be
> efficiently marshalled over the network as an editor operation. The
> sender looks at the input streams and constructs a diff, sends diff
> windows over the network, and the receiver is stuck; it can't make
> another apply_text call with just the diffs. That means
> libsvn_ra_svn/editor.c would have to be broken out into N different
> editor implementations, each of which knows how to handle diffs.
>
> So, -1 on apply_text. The editor interface shouldn't work that way.
That's a mighty quick veto you've got there, Tex :-).
The idea is that the editor driver on the other side knows the base
against which to apply the incoming svndiff stream -- so it uses that
base to reconstruct the fulltext.
One purpose of this change is to make svndiff transmission an internal
implementation aspect of an RA layer, so the editor interface doesn't
have to concern itself with the details of transmission.
The "source" editor's apply_text() is free to use base and target to
construct svndiff, GNU diff, or anything else, as long as it's worked
out the details with the receiving driver. The editors themselves
won't need to worry about it.
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Greg Hudson <gh...@MIT.EDU>.
I think the committer assumed that all editor providers were using the
default editor, which isn't the case.
However, I have a much bigger problem to bring up. apply_text cannot be
efficiently marshalled over the network as an editor operation. The
sender looks at the input streams and constructs a diff, sends diff
windows over the network, and the receiver is stuck; it can't make
another apply_text call with just the diffs. That means
libsvn_ra_svn/editor.c would have to be broken out into N different
editor implementations, each of which knows how to handle diffs.
So, -1 on apply_text. The editor interface shouldn't work that way.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Eric Gillespie <ep...@pretzelnet.org>.
Am i missing something, or is this known to be broken on both
ra_svn and ra_dav? Is it the general policy of this project for
the trunk to be broken for days? If this is a mistake, that's
one thing, but it looks like this is known breakage, which makes
it hard for people working on other things. Shouldn't this kind
of thing happen on a branch?
What i see is libsvn_client/commit.c:66 doing this:
SVN_ERR (editor->apply_text (file_baton, NULL, NULL, NULL, contents,
editor, pool));
But apply_text is NULL:
static const svn_delta_editor_t ra_svn_editor = {
ra_svn_target_rev,
ra_svn_open_root,
ra_svn_delete_entry,
ra_svn_add_dir,
ra_svn_open_dir,
ra_svn_change_dir_prop,
ra_svn_close_dir,
ra_svn_add_file,
ra_svn_open_file,
ra_svn_apply_textdelta,
NULL, /* ### todo#510: implement ra_svn_apply_text() */
ra_svn_change_file_prop,
ra_svn_close_file,
ra_svn_close_edit,
ra_svn_abort_edit
};
I look in svn_ra_dav__get_commit_editor and it isn't even initializing
it to NULL, which means that for ra_dav apply_text contains a random
value!
Again, am i missing something? Please tell me this hasn't been
intentionally broken since Wednesday.
--
Eric Gillespie <*> epg@pretzelnet.org
Build a fire for a man, and he'll be warm for a day. Set a man on
fire, and he'll be warm for the rest of his life. -Terry Pratchett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Greg Hudson <gh...@MIT.EDU>.
On Sat, 2003-02-01 at 08:58, Philip Martin wrote:
> Yes, ra_svn is broken. I believe it is fallout from the work done for
> issue 510, see "### todo#510" in subversion/libsvn_ra_svn/editor.c.
So, I'm feeling like ra_svn is being treated as a second-class citizen
by some developers. Before and during its development, I was assured
that it would be well-received as a pre-1.0 feature, but now people seem
to feel free to check in stuff which is known to leave ra_svn broken or
which implements features in the other ra layers but not in ra_svn.
Without a lot of effort from me (which can't happen; I have to put too
much effort into my job right now), this will be a self-reinforcing
attitude. ra_svn will be the access layer which doesn't require Apache
but also doesn't work so well.
I am not pleased.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: SVNSERVE Tests Failing
Posted by Philip Martin <ph...@codematters.co.uk>.
David Summers <da...@summersoft.fay.ar.us> writes:
> For the last several days, a whole slew of SVNSERVE tests have been
> failing on both RedHat 7.3 and 8.0.
>
> Has anyone else seen this?
Yes, ra_svn is broken. I believe it is fallout from the work done for
issue 510, see "### todo#510" in subversion/libsvn_ra_svn/editor.c.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org