clue for repos corruption?

[Jan Hendrik <ja...@bigfoot.com>]

Hi all out there!

It could be I found a clue for the constant repos corruption here. It 
seems possible that changed timestamps without files being 
actually changed causes memory not being released and furtheron 
leads to the desaster:

Starting once more with the notoriously corrupted repository from 
which I dumpfiltered all binary files, and two working copies, all on 
my local machine, both accessing the repos through http, things 
went fine for some time. Later I added GIF and JPG files step by 
step. Then troubles began again.

But only with *one* working copy! The other working copy 
continued operating fine, commits & updates would slightly raise 
memory consumption and then fall back to the level before. But 
that other working copy always raises memory usage to the ceiling 
and corrupts the repos. And even after SVN has terminated with 
the timeout error memory consumption goes up and up and up ...

Finally I managed to complete update on a minor directory before 
the memory ceiling was reached. But then memory was *not* 
released. Apache keeps every bit it has collected during update. 
Only if stopped this memory is released.

Checking out a further working copy was no problem, it operates 
fine, just as wc2. The three working copies are just about 2 
revisions apart, perhaps 15-20 files different/new.

There is, however, one difference that may give a clue:

Some files have the timestamp changed due to uploading to the 
webserver. It should be less than 50 files, but I can't say exactly. 
We discussed this before. With dial up having a working copy on 
the webserver is no alternative, nor is using export as it would 
result in complete upload every time. Someone suggested rsynch, 
however, I cannot setup an rsynch on the server and as far as I 
understood clients for Windows are more or less experimental. 
Anyway, having computed checksums over dial up does not sound 
different from having a working copy online. The tool I currently use 
compares filesize and timestamp and if at least one is different it 
considers the files being different. After upload it sets the 
timestamp of the local file to that on the server. That's why I get 
timestamps changed in the working copy.

I won't say this could explain everything. But it seems to explain 
why troubles were nearly guaranteed when the repos was on 
another machine - it is always my working copy that gets 
timestamps changed. There still might be an underlying memory 
problem, but not with "normal" SVN operation for else corruption 
should happen with the other local working copies, too.

Best regards

Jan Hendrik


---------------------------------------
Freedom quote:

     Capitalism is not an 'ism.'
     It is closer to being the opposite of an 'ism,'
     because it is simply the freedom of ordinary people
     to make whatever economic transactions they can mutually agree to.
                -- Dr. Thomas Sowell


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: clue for repos corruption?

[Jan Hendrik <ja...@bigfoot.com>]

Concerning Re: clue for repos corruption?
John Peacock wrote on 7 Nov 2003, 6:59, at least in part:

> FWIW, rsync on Win32 works just fine (using Cygwin tools at least). 
> We have a massive set of files which we use rsync daily to update (and
> I am talking thousands of files of various sizes).  We aren't using a
> dialup line, but it is over a WAN link (fractional T1) and is much
> faster than using FTP or drive maps.

John,
thanks once more for the input. At second glance I got some rsync 
packages that might not be too difficult to set up. Unfortunately our 
hoster does not allow running programs and as far as I have found 
out yet they do not offer rsync as a service on their own, but I am 
still confirming this with a technician. It is obvious that rsync draws 
a good deal of speed from running on both machines. But as it 
seems I have to wait and hope that SVN some day updates 
checksums if timestamps have changed. And it is good to know 
that SVN at least reads my email - for how else it should have 
known and started corruption on the other repos just after posting? 
<G>

Best regards

Jan Hendrik


---------------------------------------
Freedom quote:

     Capitalism is not an 'ism.'
     It is closer to being the opposite of an 'ism,'
     because it is simply the freedom of ordinary people
     to make whatever economic transactions they can mutually agree to.
                -- Dr. Thomas Sowell


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: clue for repos corruption?

[Jan Hendrik <ja...@bigfoot.com>]

Concerning Re: clue for repos corruption?
John Peacock wrote on 7 Nov 2003, 6:59, at least in part:

> FWIW, rsync on Win32 works just fine (using Cygwin tools at least). 
> We have a massive set of files which we use rsync daily to update (and
> I am talking thousands of files of various sizes).  We aren't using a
> dialup line, but it is over a WAN link (fractional T1) and is much
> faster than using FTP or drive maps.

John,

thank you for your comment. I'll look at that again, though I have 
only FTP access. It was a university site describing setup of rsync 
for their student's private pages and the admin requested Linux for 
mentioned reasons. However, I have the feeling Cygwin would be 
more than I could bear currently ... this SVN thing has drained me 
quite a lot - much work and trouble for weeks and no result, except 
that it would be fine if only thing would start running fine.

Best regards

Jan Hendrik

---------------------------------------
Freedom quote:

     American history has proven when people live in freedom
     the vast majority of sovereign individuals believe in,
     and practice, the golden rule.
                -- Rick Gaber


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: clue for repos corruption?

[John Peacock <jp...@rowman.com>]

Jan Hendrik wrote:
> result in complete upload every time. Someone suggested rsynch, 
> however, I cannot setup an rsynch on the server and as far as I 
> understood clients for Windows are more or less experimental. 

FWIW, rsync on Win32 works just fine (using Cygwin tools at least).  We have a 
massive set of files which we use rsync daily to update (and I am talking 
thousands of files of various sizes).  We aren't using a dialup line, but it is 
over a WAN link (fractional T1) and is much faster than using FTP or drive maps.

You would have to install Cygwin and the tools you need to connect to the server 
(we use ssh as a service).  It's not too hard, but you do have to install and 
run the one service.

HTH

John

-- 
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4720 Boston Way
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5747


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: clue for repos corruption?

[Jan Hendrik <ja...@bigfoot.com>]

Sorry for having bothered you with that stuff again. For two days it 
was as described, just after I had posted that report working copy 2 
joined the corruption game. Think we can finally close these 
threads till the brandnew P10 with autogenerating terabytes of 
RAM comes out. Or till Linux becomes something non-techies can 
setup and use (what I expect never to happen, it will always take 
programming and making and compiling before, stuff only people 
with a university degree understand). One or more of 
SVN/Apache/Berkeley are not meant to run in the Win2K 
environment here.

With appropriate sarcasm - Jan Hendrik

Concerning clue for repos corruption?
Jan Hendrik wrote on 7 Nov 2003, 11:53, at least in part:

> Hi all out there!
> 
> It could be I found a clue for the constant repos corruption here. It
> seems possible that changed timestamps without files being actually
> changed causes memory not being released and furtheron leads to the
> desaster:
> 
> Starting once more with the notoriously corrupted repository from
> which I dumpfiltered all binary files, and two working copies, all on
> my local machine, both accessing the repos through http, things went
> fine for some time. Later I added GIF and JPG files step by step. Then
> troubles began again.
> 
> But only with *one* working copy! The other working copy 
> continued operating fine, commits & updates would slightly raise
> memory consumption and then fall back to the level before. But that
> other working copy always raises memory usage to the ceiling and
> corrupts the repos. And even after SVN has terminated with the timeout
> error memory consumption goes up and up and up ...
> 
> Finally I managed to complete update on a minor directory before the
> memory ceiling was reached. But then memory was *not* released. Apache
> keeps every bit it has collected during update. Only if stopped this
> memory is released.
> 
> Checking out a further working copy was no problem, it operates 
> fine, just as wc2. The three working copies are just about 2 
> revisions apart, perhaps 15-20 files different/new.
> 
> There is, however, one difference that may give a clue:
> 
> Some files have the timestamp changed due to uploading to the 
> webserver. It should be less than 50 files, but I can't say exactly.
> We discussed this before. With dial up having a working copy on the
> webserver is no alternative, nor is using export as it would result in
> complete upload every time. Someone suggested rsynch, however, I
> cannot setup an rsynch on the server and as far as I understood
> clients for Windows are more or less experimental. Anyway, having
> computed checksums over dial up does not sound different from having a
> working copy online. The tool I currently use compares filesize and
> timestamp and if at least one is different it considers the files
> being different. After upload it sets the timestamp of the local file
> to that on the server. That's why I get timestamps changed in the
> working copy.
> 
> I won't say this could explain everything. But it seems to explain why
> troubles were nearly guaranteed when the repos was on another machine
> - it is always my working copy that gets timestamps changed. There
> still might be an underlying memory problem, but not with "normal" SVN
> operation for else corruption should happen with the other local
> working copies, too.
> 
> Best regards
> 
> Jan Hendrik
> 
> 
> ---------------------------------------
> Freedom quote:
> 
>      Capitalism is not an 'ism.'
>      It is closer to being the opposite of an 'ism,'
>      because it is simply the freedom of ordinary people
>      to make whatever economic transactions they can mutually agree
>      to.
>                 -- Dr. Thomas Sowell
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org For
> additional commands, e-mail: users-help@subversion.tigris.org
> 


---------------------------------------
Freedom quote:

     They that can give up essential liberty
     to obtain a little temporary safety
     deserve neither liberty nor safety.
                -- Benjamin Franklin


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org