You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Ron Bieber <ro...@bieberlabs.com> on 2004/02/01 13:56:53 UTC
Re: cvs2svn.py takes long but sleeps?
I've had this happen to me as well. It will sit on a particular commit
for an extended period of time and the process is sleeping.
My environment is a 1.2ghz Dell Workgroup Server running RedHat 7.3.
No NFS shares or anything like that were used -- all disks were local.
I've had the conversion running for up to 2 weeks without the process
finishing. .
-- Ron
On Fri, 2004-01-30 at 07:34, Kai Bartels wrote:
> Hi *!
>
> (sorry for the provocating subject, it's short that way ;^) )
>
> cvs2svn.py is running for about 24h now on a P4 2.4G here on our tiny
> 2.1G cvs repository :^) . I don't know which pass it is doing right now,
> output lines look like this:
> committing: Tue Jan 15 13:50:45 2002, over 3 seconds
>
> As it is still in Jan 2002 I expect some more days of work. :^(
>
> What appears strange to me is that the process is using nearly no CPU
> time (~2%) nor does it seem to do disk or net IO (cvs repository is mounted
> via nfs). In other words: I wonder why cvs2svn.py isn't using the ressources
> of the machine...
>
> Any thoughts on this? What is the fact I forgot about?
>
> regards, Kai
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: [w] Re: cvs2svn.py takes long but sleeps?
Posted by Kai Bartels <ka...@picturesafe.de>.
Hi *!
I had another try with revision 8547 of cvs2svn.py and the behaviour is
basicly the same. :^(
I think it's pass 5 that takes up the time. The first commits are fast,
but it has reached commit 19xxx now and each file in a commit, even the
small ones take about 30 seconds where the machine seems to do
nothing at all. (NB: I think the nfs mount is not used in pass 5.)
Maybe it's the bigger number of commits that slow down cvs2svn, or
subversion itself??
I get kind of a stack trace when I cancel the process; I append both
(the 7921 and the 8547) stack trace below, maybe someone has an idea
when looking over them!?
Regards, Kai
On Mon, Feb 02, 2004 at 11:34:26AM -0600, David Summers wrote:
>
> I'd like to report my (finally) succesful conversion of my 10.5 Gigabyte
> CVS repository to Subversion. I tried again about 1 or 2 weeks ago and
> got cvs2svn errors, but evidently some changes in cvs2svn this last week
> fixed the remaining problems (that I was having) and I finally got my
> repository to convert and load this last Friday. I've tried about 4-5
> times beginning this last summer and ran into various problems each time.
>
> Statistics:
> ===========
>
> CVS Repository: ~10.5 Gigabytes
> SVN Repository: ~10 Gigabytes
> Total Commits: 7754 (spanning from 2000-04-05 to present)
> Working Copy: ~6 Gigabytes
> Conversion Time: ~10 hours
>
> My supervisor has told me he won't consider Subversion until it hits 1.0
> so I'm getting everything in place to present him with the
> advantages/disadvantages of converting to Subversion and showing him a
> working demonstration from a snapshot of our CVS repository taken last
> week.
>
> Thanks to Karl and everyone else on the team for the hard work and great
> software!
>
> --
> David Wayne Summers "Linux: Because reboots are for hardware upgrades!"
> david@summersoft.fay.ar.us PGP Key: http://summersoft.fay.ar.us/~david/pgp.txt
> PGP Key fingerprint = C0 E0 4F 50 DD A9 B6 2B 60 A1 31 7E D2 28 6D A8
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
On Mon, Feb 02, 2004 at 10:42:23AM +0100, Kai Bartels wrote:
> Hi Karl, hi Ron!
>
>
> On Fri, Jan 30, 2004 at 08:38:15AM -0600, kfogel@collab.net wrote:
> > Kai Bartels <ka...@picturesafe.de> writes:
> > > As it is still in Jan 2002 I expect some more days of work. :^(
>
> (advanced to Mar 1 2002 by now :^) )
>
> > Wow, no idea -- never seen this one before. You don't normally have
> > performance problems over this NFS share?
>
> I set up this ro-share just for use in the conversion to protect the
> cvs-repository from ... well, you never know... ;^)
>
> > What version of cvs2svn is it? Check for a line like:
>
> It's 7921 from the 0.37.1 release.
>
> > Is the directory in which you're running cvs2svn also mounted via NFS?
> > cvs2svn uses many, many tmp files as it works, including some dbm
> > files. Could that be relevant?
>
> No, temp files and cvs svn-repository are on lokal disks.
>
> And (as the point was raised in some other mails) the BDB version
> installed here seems to be 4.2 .
>
> > -Karl
>
> On Sun, Feb 01, 2004 at 07:56:53AM -0600, Ron Bieber wrote:
> > I've had this happen to me as well. It will sit on a particular commit
> > for an extended period of time and the process is sleeping.
> >
> >
> > My environment is a 1.2ghz Dell Workgroup Server running RedHat 7.3.
> > No NFS shares or anything like that were used -- all disks were local.
> > I've had the conversion running for up to 2 weeks without the process
> > finishing. .
>
> Wow!! I think I'll stop my process today ... ;^)
>
> I also tried refinecvs.pl , but it cannot find Cvs/Repository/Exception.pm
> in my local perl 5.8.2 installation. Maybe I have to search for that
> perl package and give this tool another try ...
>
> > -- Ron
>
>
> greetings, Kai
>
> --
> "Was Menschen in der Rangordnung der Nahrungszuteilung
> hinter die Schweine stellt, ist die Tatsache, dass sie
> weder Geld haben, noch einen Besitzer." ~ <N3 9411242325>
> KBartels@picturesafe.de + www.picturesafe.de
> picturesafe GmbH, Lueerstr. 3, D-30175 Hannover fon:+49 511 85620 56
> ++ PGP Key fingerprint 1324 98A8 3BF8 4166 05E1 82D3 E168 6849 6022 5D32 ++
> -----BEGIN GEEK CODE BLOCK-----
> VERSION: 3.12
> GCS d- s: a C++$ UL/S++$ P++ L++ E W-- N w--- !O !M PS+ PE Y+ PGP+ t R@ tv+>-
> b+>++ DI++ G e+++ h* y?
> ------END GEEK CODE BLOCK------
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
8547:
----------------------------------- schnipp -----------------------------------
loading revision 19883 into '../repository/'
adding or changing 1.1 : 'trunk/s4/mybrand/site/mybrand/quattro/common/img/de/teaser_inhalt.gif'
new revision: 19884
committing: Thu Jan 24 11:27:10 2002, over 0 seconds
loading revision 19884 into '../repository/'
adding or changing 1.1 : 'trunk/s4/mybrand/site/mybrand/quattro/backoffice/img/de/um1_arbeitskataloge.gif'
adding or changing 1.1 : 'trunk/s4/mybrand/site/mybrand/quattro/backoffice/img/de/um1_arbeitskataloge_slc.gif'
Traceback (most recent call last):
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2384, in ?
main()
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2380, in main
convert(ctx, start_pass=start_pass)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2224, in convert
_passes[i](ctx)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2159, in pass4
c.commit(dumper, ctx, sym_tracker)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 1912, in commit
sym_tracker.enroot_tags(svn_path, svn_rev, tags)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 1449, in enroot_tags
self.enroot_names(svn_path, svn_rev, tags, self.tags_opening_revs_key)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 1438, in enroot_names
this_entry_val = marshal.loads(self.db[this_entry_key])
File "/usr/lib/python2.3/bsddb/__init__.py", line 116, in __getitem__
return self.db[key]
KeyboardInterrupt
svn@wintermute:~/trans-tmp$
----------------------------------- schnapp -----------------------------------
7921:
----------------------------------- schnipp -----------------------------------
adding or changing 1.1 : trunk/psinternet/en/images/imageiio.pe4
adding or changing 1.1 : trunk/psinternet/en/images/imaginfo.pe4
adding or changing 1.1 : trunk/psinternet/en/images/jobs_hl.gif
Traceback (most recent call last):
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2277, in ?
main()
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2273, in main
convert(ctx, start_pass=start_pass)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2129, in convert
_passes[i](ctx)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2065, in pass4
c.commit(dumper, ctx, sym_tracker)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 1818, in commit
sym_tracker.enroot_tags(svn_path, svn_rev, tags)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 1362, in enroot_tags
self.enroot_names(svn_path, svn_rev, tags, self.tags_opening_revs_key)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 1342, in enroot_names
self.bump_rev_count(parent_key, svn_rev, opening_key)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 1322, in bump_rev_count
self.db[item_key] = marshal.dumps(entry_val)
File "/usr/lib/python2.3/bsddb/__init__.py", line 120, in __setitem__
self.db[key] = value
KeyboardInterrupt
----------------------------------- schnapp -----------------------------------
--
"You will still be here tomorrow but your dreams may not" <Cat Stevens>
KBartels@picturesafe.de + www.picturesafe.de
picturesafe GmbH, Lueerstr. 3, D-30175 Hannover fon:+49 511 85620 56
++ PGP Key fingerprint 1324 98A8 3BF8 4166 05E1 82D3 E168 6849 6022 5D32 ++
-----BEGIN GEEK CODE BLOCK-----
VERSION: 3.12
GCS d- s: a C++$ UL/S++$ P++ L++ E W-- N w--- !O !M PS+ PE Y+ PGP+ t R@ tv+>-
b+>++ DI++ G e+++ h* y?
------END GEEK CODE BLOCK------
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [w] Re: cvs2svn.py takes long but sleeps?
Posted by Kai Bartels <ka...@picturesafe.de>.
Hi *!
I had another try with revision 8547 of cvs2svn.py and the behaviour is
basicly the same. :^(
I think it's pass 5 that takes up the time. The first commits are fast,
but it has reached commit 19xxx now and each file in a commit, even the
small ones take about 30 seconds where the machine seems to do
nothing at all. (NB: I think the nfs mount is not used in pass 5.)
Maybe it's the bigger number of commits that slow down cvs2svn, or
subversion itself??
I get kind of a stack trace when I cancel the process; I append both
(the 7921 and the 8547) stack trace below, maybe someone has an idea
when looking over them!?
Regards, Kai
On Mon, Feb 02, 2004 at 11:34:26AM -0600, David Summers wrote:
>
> I'd like to report my (finally) succesful conversion of my 10.5 Gigabyte
> CVS repository to Subversion. I tried again about 1 or 2 weeks ago and
> got cvs2svn errors, but evidently some changes in cvs2svn this last week
> fixed the remaining problems (that I was having) and I finally got my
> repository to convert and load this last Friday. I've tried about 4-5
> times beginning this last summer and ran into various problems each time.
>
> Statistics:
> ===========
>
> CVS Repository: ~10.5 Gigabytes
> SVN Repository: ~10 Gigabytes
> Total Commits: 7754 (spanning from 2000-04-05 to present)
> Working Copy: ~6 Gigabytes
> Conversion Time: ~10 hours
>
> My supervisor has told me he won't consider Subversion until it hits 1.0
> so I'm getting everything in place to present him with the
> advantages/disadvantages of converting to Subversion and showing him a
> working demonstration from a snapshot of our CVS repository taken last
> week.
>
> Thanks to Karl and everyone else on the team for the hard work and great
> software!
>
> --
> David Wayne Summers "Linux: Because reboots are for hardware upgrades!"
> david@summersoft.fay.ar.us PGP Key: http://summersoft.fay.ar.us/~david/pgp.txt
> PGP Key fingerprint = C0 E0 4F 50 DD A9 B6 2B 60 A1 31 7E D2 28 6D A8
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
On Mon, Feb 02, 2004 at 10:42:23AM +0100, Kai Bartels wrote:
> Hi Karl, hi Ron!
>
>
> On Fri, Jan 30, 2004 at 08:38:15AM -0600, kfogel@collab.net wrote:
> > Kai Bartels <ka...@picturesafe.de> writes:
> > > As it is still in Jan 2002 I expect some more days of work. :^(
>
> (advanced to Mar 1 2002 by now :^) )
>
> > Wow, no idea -- never seen this one before. You don't normally have
> > performance problems over this NFS share?
>
> I set up this ro-share just for use in the conversion to protect the
> cvs-repository from ... well, you never know... ;^)
>
> > What version of cvs2svn is it? Check for a line like:
>
> It's 7921 from the 0.37.1 release.
>
> > Is the directory in which you're running cvs2svn also mounted via NFS?
> > cvs2svn uses many, many tmp files as it works, including some dbm
> > files. Could that be relevant?
>
> No, temp files and cvs svn-repository are on lokal disks.
>
> And (as the point was raised in some other mails) the BDB version
> installed here seems to be 4.2 .
>
> > -Karl
>
> On Sun, Feb 01, 2004 at 07:56:53AM -0600, Ron Bieber wrote:
> > I've had this happen to me as well. It will sit on a particular commit
> > for an extended period of time and the process is sleeping.
> >
> >
> > My environment is a 1.2ghz Dell Workgroup Server running RedHat 7.3.
> > No NFS shares or anything like that were used -- all disks were local.
> > I've had the conversion running for up to 2 weeks without the process
> > finishing. .
>
> Wow!! I think I'll stop my process today ... ;^)
>
> I also tried refinecvs.pl , but it cannot find Cvs/Repository/Exception.pm
> in my local perl 5.8.2 installation. Maybe I have to search for that
> perl package and give this tool another try ...
>
> > -- Ron
>
>
> greetings, Kai
>
> --
> "Was Menschen in der Rangordnung der Nahrungszuteilung
> hinter die Schweine stellt, ist die Tatsache, dass sie
> weder Geld haben, noch einen Besitzer." ~ <N3 9411242325>
> KBartels@picturesafe.de + www.picturesafe.de
> picturesafe GmbH, Lueerstr. 3, D-30175 Hannover fon:+49 511 85620 56
> ++ PGP Key fingerprint 1324 98A8 3BF8 4166 05E1 82D3 E168 6849 6022 5D32 ++
> -----BEGIN GEEK CODE BLOCK-----
> VERSION: 3.12
> GCS d- s: a C++$ UL/S++$ P++ L++ E W-- N w--- !O !M PS+ PE Y+ PGP+ t R@ tv+>-
> b+>++ DI++ G e+++ h* y?
> ------END GEEK CODE BLOCK------
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
8547:
----------------------------------- schnipp -----------------------------------
loading revision 19883 into '../repository/'
adding or changing 1.1 : 'trunk/s4/mybrand/site/mybrand/quattro/common/img/de/teaser_inhalt.gif'
new revision: 19884
committing: Thu Jan 24 11:27:10 2002, over 0 seconds
loading revision 19884 into '../repository/'
adding or changing 1.1 : 'trunk/s4/mybrand/site/mybrand/quattro/backoffice/img/de/um1_arbeitskataloge.gif'
adding or changing 1.1 : 'trunk/s4/mybrand/site/mybrand/quattro/backoffice/img/de/um1_arbeitskataloge_slc.gif'
Traceback (most recent call last):
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2384, in ?
main()
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2380, in main
convert(ctx, start_pass=start_pass)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2224, in convert
_passes[i](ctx)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2159, in pass4
c.commit(dumper, ctx, sym_tracker)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 1912, in commit
sym_tracker.enroot_tags(svn_path, svn_rev, tags)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 1449, in enroot_tags
self.enroot_names(svn_path, svn_rev, tags, self.tags_opening_revs_key)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 1438, in enroot_names
this_entry_val = marshal.loads(self.db[this_entry_key])
File "/usr/lib/python2.3/bsddb/__init__.py", line 116, in __getitem__
return self.db[key]
KeyboardInterrupt
svn@wintermute:~/trans-tmp$
----------------------------------- schnapp -----------------------------------
7921:
----------------------------------- schnipp -----------------------------------
adding or changing 1.1 : trunk/psinternet/en/images/imageiio.pe4
adding or changing 1.1 : trunk/psinternet/en/images/imaginfo.pe4
adding or changing 1.1 : trunk/psinternet/en/images/jobs_hl.gif
Traceback (most recent call last):
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2277, in ?
main()
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2273, in main
convert(ctx, start_pass=start_pass)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2129, in convert
_passes[i](ctx)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 2065, in pass4
c.commit(dumper, ctx, sym_tracker)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 1818, in commit
sym_tracker.enroot_tags(svn_path, svn_rev, tags)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 1362, in enroot_tags
self.enroot_names(svn_path, svn_rev, tags, self.tags_opening_revs_key)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 1342, in enroot_names
self.bump_rev_count(parent_key, svn_rev, opening_key)
File "/home/kaba/tmp/subversion/subversion-0.37.0/tools/cvs2svn/cvs2svn.py", line 1322, in bump_rev_count
self.db[item_key] = marshal.dumps(entry_val)
File "/usr/lib/python2.3/bsddb/__init__.py", line 120, in __setitem__
self.db[key] = value
KeyboardInterrupt
----------------------------------- schnapp -----------------------------------
--
"You will still be here tomorrow but your dreams may not" <Cat Stevens>
KBartels@picturesafe.de + www.picturesafe.de
picturesafe GmbH, Lueerstr. 3, D-30175 Hannover fon:+49 511 85620 56
++ PGP Key fingerprint 1324 98A8 3BF8 4166 05E1 82D3 E168 6849 6022 5D32 ++
-----BEGIN GEEK CODE BLOCK-----
VERSION: 3.12
GCS d- s: a C++$ UL/S++$ P++ L++ E W-- N w--- !O !M PS+ PE Y+ PGP+ t R@ tv+>-
b+>++ DI++ G e+++ h* y?
------END GEEK CODE BLOCK------
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: [w] Re: cvs2svn.py takes long but sleeps?
Posted by Kai Bartels <ka...@picturesafe.de>.
Hi Karl, hi Ron!
On Fri, Jan 30, 2004 at 08:38:15AM -0600, kfogel@collab.net wrote:
> Kai Bartels <ka...@picturesafe.de> writes:
> > As it is still in Jan 2002 I expect some more days of work. :^(
(advanced to Mar 1 2002 by now :^) )
> Wow, no idea -- never seen this one before. You don't normally have
> performance problems over this NFS share?
I set up this ro-share just for use in the conversion to protect the
cvs-repository from ... well, you never know... ;^)
> What version of cvs2svn is it? Check for a line like:
It's 7921 from the 0.37.1 release.
> Is the directory in which you're running cvs2svn also mounted via NFS?
> cvs2svn uses many, many tmp files as it works, including some dbm
> files. Could that be relevant?
No, temp files and cvs svn-repository are on lokal disks.
And (as the point was raised in some other mails) the BDB version
installed here seems to be 4.2 .
> -Karl
On Sun, Feb 01, 2004 at 07:56:53AM -0600, Ron Bieber wrote:
> I've had this happen to me as well. It will sit on a particular commit
> for an extended period of time and the process is sleeping.
>
>
> My environment is a 1.2ghz Dell Workgroup Server running RedHat 7.3.
> No NFS shares or anything like that were used -- all disks were local.
> I've had the conversion running for up to 2 weeks without the process
> finishing. .
Wow!! I think I'll stop my process today ... ;^)
I also tried refinecvs.pl , but it cannot find Cvs/Repository/Exception.pm
in my local perl 5.8.2 installation. Maybe I have to search for that
perl package and give this tool another try ...
> -- Ron
greetings, Kai
--
"Was Menschen in der Rangordnung der Nahrungszuteilung
hinter die Schweine stellt, ist die Tatsache, dass sie
weder Geld haben, noch einen Besitzer." ~ <N3 9411242325>
KBartels@picturesafe.de + www.picturesafe.de
picturesafe GmbH, Lueerstr. 3, D-30175 Hannover fon:+49 511 85620 56
++ PGP Key fingerprint 1324 98A8 3BF8 4166 05E1 82D3 E168 6849 6022 5D32 ++
-----BEGIN GEEK CODE BLOCK-----
VERSION: 3.12
GCS d- s: a C++$ UL/S++$ P++ L++ E W-- N w--- !O !M PS+ PE Y+ PGP+ t R@ tv+>-
b+>++ DI++ G e+++ h* y?
------END GEEK CODE BLOCK------
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org