You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Joe Stump <jo...@joestump.net> on 2005/03/22 20:16:23 UTC
svn update slow / post-commit issues
Hey all,
Just set up subversion and I'm noticing a few problems. I work for a
web company with a typical dev/live setup. I'd like post-commit to
update our dev sites, but I'm having problems - the email sends out
just fine, but this fails:
# set up our charset/locales
export LANG=en_US.ISO-8859-1
export LC_CTYPE=en_US.ISO-8859-1
cd /var/www && /usr/bin/svn update > /tmp/svn.log
Any ideas?
Also, svn update on the top level of our working directory takes
*forever* (it's a large repository at 224MB). Is there anything to
speed up this process? I'd hate to have to wait minutes to check for
updates on my dev server.
Thanks!
--Joe
--
Joseph C. Stump
joe@joestump.net
http://www.joestump.net
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn update slow / post-commit issues
Posted by John Szakmeister <jo...@szakmeister.net>.
On Tuesday 22 March 2005 19:20, Joe Stump wrote:
> Well, I did two things to cut the update times down to nothing (well 3
> seconds via HTTP and 2 seconds using file://).
>
> 1.) Removed a directory that was filled with dynamically created
> content (ie. a new folder for each record is created in a single
> directory - legacy code - don't yell at me). This had 2500'ish
> directories each with 1-N images (about 400MB on disk).
Was the directory that you removed versioned? I'm curious because we
should skip over an unversioned directory and avoid checking the status
on those 2500 files. However, if the directory is versioned, there's
probably no clever way to avoid that. Just out of curiosity, how long
does it take to do an 'ls' in that directory?
-John
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn update slow / post-commit issues
Posted by Joe Stump <jo...@joestump.net>.
Well, I did two things to cut the update times down to nothing (well 3
seconds via HTTP and 2 seconds using file://).
1.) Removed a directory that was filled with dynamically created
content (ie. a new folder for each record is created in a single
directory - legacy code - don't yell at me). This had 2500'ish
directories each with 1-N images (about 400MB on disk).
2.) Switched from using my.domain.com to 192.168.1.25
I'm sure #1 had more to do with it than anything else.
--Joe
--
Joseph C. Stump
joe@joestump.net
http://www.joestump.net
On Mar 22, 2005, at 4:11 PM, Ben Collins-Sussman wrote:
>
> On Mar 22, 2005, at 5:44 PM, Joe Stump wrote:
>
>> Thanks for the comparison numbers, but I'm not sure what could be
>> "seriously wrong" with my setup. I have everything set up per the
>> book and I'm on a local network. I'm able to reproduce the delay
>> using file:// as well.
>>
>
> Is your working copy really large in terms of the number of files and
> directories?
>
> 'svn update' needs to crawl the working copy and read every single
> .svn/entries file in every directory. Then it reports the
> working-revs to the server. Can you do tests to figure out what's
> taking so long? For example, are there many minutes of client disk
> I/O before the server begins to send a network response? Where is the
> delay happening?
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn update slow / post-commit issues
Posted by Ben Collins-Sussman <su...@collab.net>.
On Mar 22, 2005, at 5:44 PM, Joe Stump wrote:
> Thanks for the comparison numbers, but I'm not sure what could be
> "seriously wrong" with my setup. I have everything set up per the book
> and I'm on a local network. I'm able to reproduce the delay using
> file:// as well.
>
Is your working copy really large in terms of the number of files and
directories?
'svn update' needs to crawl the working copy and read every single
.svn/entries file in every directory. Then it reports the working-revs
to the server. Can you do tests to figure out what's taking so long?
For example, are there many minutes of client disk I/O before the
server begins to send a network response? Where is the delay
happening?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn update slow / post-commit issues
Posted by Joe Stump <jo...@joestump.net>.
Thanks for the comparison numbers, but I'm not sure what could be
"seriously wrong" with my setup. I have everything set up per the book
and I'm on a local network. I'm able to reproduce the delay using
file:// as well.
--Joe
--
Joseph C. Stump
joe@joestump.net
http://www.joestump.net
On Mar 22, 2005, at 3:19 PM, Ben Collins-Sussman wrote:
>
> On Mar 22, 2005, at 3:24 PM, Joe Stump wrote:
>
>> OK, I've fixed my post-commit issues, but I'm still seeing HORRIFIC
>> update times over DAV. In fact, they're timing out now. Is anyone
>> using svn over DAV with a repository as big as mine (~220MB)? How
>> long do your updates take?
>
> Sounds like there's something seriously wrong with your setup.
>
> Subversion's own working copy is 23 megs of source (56 megs on disk),
> and takes me about 4 seconds to update over http.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn update slow / post-commit issues
Posted by Ben Collins-Sussman <su...@collab.net>.
On Mar 22, 2005, at 3:24 PM, Joe Stump wrote:
> OK, I've fixed my post-commit issues, but I'm still seeing HORRIFIC
> update times over DAV. In fact, they're timing out now. Is anyone
> using svn over DAV with a repository as big as mine (~220MB)? How long
> do your updates take?
Sounds like there's something seriously wrong with your setup.
Subversion's own working copy is 23 megs of source (56 megs on disk),
and takes me about 4 seconds to update over http.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn update slow / post-commit issues
Posted by Saulius Grazulis <gr...@akl.lt>.
On Tuesday 22 March 2005 23:24, Joe Stump wrote:
> OK, I've fixed my post-commit issues, but I'm still seeing HORRIFIC
> update times over DAV. In fact, they're timing out now. Is anyone using
> svn over DAV with a repository as big as mine (~220MB)? How long do
> your updates take?
xray@delfinas: xray/ > du -hs svn-repositories/*
868M svn-repositories/project1
24M svn-repositories/project2
271M svn-repositories/project3
3.6M svn-repositories/project4
52M svn-repositories/project5
104M svn-repositories/project6
932k svn-repositories/test
# the 271M project3:
saulius@kolibris: project3/ > time svn up
[ nothing to transfer ]
real 0m17.022suser 0m0.160s
sys 0m0.130s
#the 868M project1:
saulius@kolibris: project1/ > time svn update
[ transfered 2017 files, some large binary ones, ~1.4G in unpacked tree]
real 5m40.638s
user 0m38.570s
sys 1m27.310s
#the 868M project1:
saulius@kolibris: project1/ > time svn update
[ nothing to transfer ]
real 0m9.403s
user 0m0.950s
sys 0m0.090s
Does not seem that slow...
CPU:
on delfinas: AMD Athlon(tm) XP 1700+
on kolibris: Intel(R) Pentium(R) M processor 1700MHz
HDD: IDE, DMA on
Repository protocol: svn+ssh
Nothing special...
Are you sure your name server is set up correctly? Ssh, in my hands, has
*huge* timeouts when I have a default route set up and can not reach
nameserver...
--
Saulius Gražulis
Visuomeninė organizacija "Atviras Kodas Lietuvai"
P.Vileišio g. 18
LT-10306 Vilnius
Lietuva (Lithuania)
tel/fax: (+370-5)-210 40 05
mobilus: (+370-684)-49802, (+370-614)-36366
Re: svn update slow / post-commit issues
Posted by Joe Stump <jo...@joestump.net>.
OK, I've fixed my post-commit issues, but I'm still seeing HORRIFIC
update times over DAV. In fact, they're timing out now. Is anyone using
svn over DAV with a repository as big as mine (~220MB)? How long do
your updates take?
Thanks!
--Joe
--
Joseph C. Stump
joe@joestump.net
http://www.joestump.net
On Mar 22, 2005, at 12:32 PM, Robert Mohr wrote:
> On Tue, 22 Mar 2005 12:16:23 -0800, Joe Stump <jo...@joestump.net> wrote:
>> Hey all,
>>
>> Just set up subversion and I'm noticing a few problems. I work for a
>> web company with a typical dev/live setup. I'd like post-commit to
>> update our dev sites, but I'm having problems - the email sends out
>> just fine, but this fails:
>>
>> # set up our charset/locales
>> export LANG=en_US.ISO-8859-1
>> export LC_CTYPE=en_US.ISO-8859-1
>> cd /var/www && /usr/bin/svn update > /tmp/svn.log
>>
>> Any ideas?
>
> I assume /tmp/svn.log isn't showing anything useful, so try running
> this command instead:
>
> cd /var/www && /usr/bin/svn update > /tmp/svn.log 2&>1 #redirects
> STDERR to STDOUT
>
> Also see <http://subversion.tigris.org/faq.html#website-auto-update>.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn update slow / post-commit issues
Posted by Robert Mohr <mo...@gmail.com>.
On Tue, 22 Mar 2005 12:16:23 -0800, Joe Stump <jo...@joestump.net> wrote:
> Hey all,
>
> Just set up subversion and I'm noticing a few problems. I work for a
> web company with a typical dev/live setup. I'd like post-commit to
> update our dev sites, but I'm having problems - the email sends out
> just fine, but this fails:
>
> # set up our charset/locales
> export LANG=en_US.ISO-8859-1
> export LC_CTYPE=en_US.ISO-8859-1
> cd /var/www && /usr/bin/svn update > /tmp/svn.log
>
> Any ideas?
I assume /tmp/svn.log isn't showing anything useful, so try running
this command instead:
cd /var/www && /usr/bin/svn update > /tmp/svn.log 2&>1 #redirects
STDERR to STDOUT
Also see <http://subversion.tigris.org/faq.html#website-auto-update>.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org