You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Jan Hendrik <ja...@bigfoot.com> on 2003/10/10 11:13:14 UTC
Repos corruption continued
Hi all out there!
In regard of the repos corruption reported before there has appeared
something that might be simple coincidence or not:
A newly created repos (SVN .30) worked for some time. Today it
became corrupted, too, just as the other one originally created
under .27 and continuing to becoming corrupted again and again.
The coincidence is that it happened to both repos with revisions
being in their 50s.
Otherwise their history is relatively similar:
- Repos created, folder structure imported (direct import did not
work, timed out, ended in error);
- 5300 files (4500 HTML, 700 JPG, rest GIF, JS, CSS etc.)
committed in chunks of 100 to 500 files;
- several minor commits;
- after site-wide changes all HTML files were committed again, split
up as before, commits this time stretched over three days, the
change was really minor this time, one line of approximately 100
characters added compared to several partly larger changes with
the first repos (no regret, server side includes would not have
helped this time);
- some more minor commits, revisions reached the 50s and the
repos became corrupted.
Different from the first repos I have not touched the unused log files.
The strings file is 100 MB (first repos: 130 BM), the web project
itself 70 MB, repos are on NTFS partition with ample space,
system W2K SP2, access per http.
Does this by chance ring any bells? Otherwise I'll lay aside SVN
for the time being till I get more RAM for the server machine
(currently 128, then 512, according to Dell the maximum for the
P4).
At least from following the list I got the impression that SVN is
perhaps not particularly happy with large imports/commits. It
seems to me that irrespective of the system there are latent
problems. Not with the final size of the repos or the number of files,
but with commits.
Jan Hendrik
---------------------------------------
Freedom quote:
You need only reflect that one of the best ways
to get yourself a reputation as a dangerous citizen these days
is to go about repeating the very phrases
which our founding fathers used in the great struggle for independence.
-- Charles Austin Beard (1874-1948)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Repos corruption continued
Posted by Jan Hendrik <ja...@bigfoot.com>.
Concerning Re: Repos corruption continued
Ben Collins-Sussman wrote on 10 Oct 2003, 7:38, at least in part:
> Well, yes, we have heard of "latent problems with commits", but none
> of the developers seems able to reproduce them yet.
>
> However, a bunch of us have started scalability tests this week.
> Maybe we'll reveal something. For example, yesterday I started doing
> repeated imports of the mozilla source tree (340M, 42k files), looking
> for problems. The only problem we found was a memory leak, no timeout
> problems.
Good news you have, Ben!
The memory leak reminds me that usually the server machine (P4,
128MB, 1GB pagefile) displays a message about too little virtual
memory when the corruption happens in the course of committing
via HTTP over the LAN. This is independant of both size of commit
and other load of the server. However, though I could reproduce the
corruption vice versa with my machine being the server (P200,
144MB, 460MB pagefile), only in one case I got that or a similar
warning about virtual memory. And then it sounded reasonable as it
was a big commit (300-500 files perhaps) and I run Word97,
HomeSite 5, Pegasus Mail, Opera 5 and some other things at the
moment.
Our web projects can't compare with Mozilla though <G>: 5300
files, 4500 HTML, the rest mostly JPG. All in all about 70MB. I
would not have suggested, but as you say you and others are
testing in this department - if you care to try the troublemakers
yourself you find them at
(1) http://www.marine-niemeyer.com
(2) http://www.marine-niemeyer.com/artbooks
(3) http://www.ridinger.de
(4) http://www.ridinger-niemeyer.com
The local project structure is
d:\internet\marine (1)
d:\internet\artbooks (2)
d:\internet\thienemann (3)
d:\internet\ridinger (4)
The repos structure:
H:\SVNRepos\repos1\trunk\internet\marine etc. Branch and tag
folders are present at the trunk level, but empty.
Between (1) and (2) there are barely links, so a grabber might fail
on (2) from the top level. Or I could give you an FTP password off-
list.
As import of d:\internet failed (dunno anymore if for memory or time
out) I copied the stuff into a bare bones checkout and committed,
1, 2, 4 in one operation each, (3) folder by folder (OK, folders with
fewer files in a joint commit, maximum number of files was at about
500). The same after my recent site-wide changes.
Jan Hendrik
---------------------------------------
Freedom quote:
The power which a multiple millionaire,
who may be my neighbor and perhaps my employer has over me
is far less than that which the smallest functionary
who wields the coercive power of the state,
and on whose discretion it depends
whether and how I am allowed to live or to work.
-- F. A. Hayek
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Repos corruption continued
Posted by Ben Collins-Sussman <su...@collab.net>.
"Jan Hendrik" <ja...@bigfoot.com> writes:
> At least from following the list I got the impression that SVN is
> perhaps not particularly happy with large imports/commits. It
> seems to me that irrespective of the system there are latent
> problems. Not with the final size of the repos or the number of files,
> but with commits.
Well, yes, we have heard of "latent problems with commits", but none
of the developers seems able to reproduce them yet.
However, a bunch of us have started scalability tests this week.
Maybe we'll reveal something. For example, yesterday I started doing
repeated imports of the mozilla source tree (340M, 42k files), looking
for problems. The only problem we found was a memory leak, no
timeout problems.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org