You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Barry Scott <ba...@barrys-emacs.org> on 2006/02/17 15:10:08 UTC
Efficient propget/proplist of one directories entries?
It would help speed up my WorkBench GUI if there was an efficient way
to do a proplist
of all the entries in a directory. This was discussed in the past and
I wonder if solving this
problem is likely and it so when?
I use svn_client_status to get all the status info for one directory
in an efficient way but
looping over each file and doing a svn_client_proplist is very slow
compared to the
speed of status.
Here is the output of a pysvn test program:
status() elapsed time: 0.071s for 166 entries 0.000426s/entry
propget() elapsed time: 2.988s for 166 entries 0.018000s/entry
The 2.9s of the 166 propget calls dwarfs the status call.
The test code is below
Barry
import sys
import pysvn
import time
c = pysvn.Client()
time_status_start = time.time()
all_status = c.status( sys.argv[1], recurse=False, ignore=True )
time_status_end = time.time()
time_propget_start = time.time()
all_proplists = []
for status in all_status:
if status.text_status != pysvn.wc_status_kind.unversioned:
all_proplists.append( c.proplist( status.path ) )
time_propget_end = time.time()
print (' status() elapsed time: %.3fs for %d entries %.6fs/entry' %
(time_status_end-time_status_start
,len(all_status)
,(time_status_end-time_status_start) /len(all_status)))
print ('propget() elapsed time: %.3fs for %d entries %.6fs/entry' %
(time_propget_end-time_propget_start
,len(all_proplists)
,(time_propget_end-time_propget_start) / len(all_proplists)))
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Efficient propget/proplist of one directories entries?
Posted by Philip Martin <ph...@codematters.co.uk>.
Barry Scott <ba...@barrys-emacs.org> writes:
> Anyone have a comment on svn ever helping with this problem?
>
> Barry
>
> On Feb 17, 2006, at 17:51, Barry Scott wrote:
>
>>
>> On Feb 17, 2006, at 15:10, Barry Scott wrote:
>>
>>> It would help speed up my WorkBench GUI if there was an efficient
>>> way to do a proplist of all the entries in a directory. This was
>>> discussed in the past and I wonder if solving this problem is
>>> likely and it so when?
This is the "replace the boolean recurse flag with an integer depth
parameter" enhancement. It's been addressed in one or two places in
wc layer but not yet in client layer, I think you will find an issue
in the tracker.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Efficient propget/proplist of one directories entries?
Posted by Barry Scott <ba...@barrys-emacs.org>.
Anyone have a comment on svn ever helping with this problem?
Barry
On Feb 17, 2006, at 17:51, Barry Scott wrote:
>
> On Feb 17, 2006, at 15:10, Barry Scott wrote:
>
>> It would help speed up my WorkBench GUI if there was an efficient
>> way to do a proplist
>> of all the entries in a directory. This was discussed in the past
>> and I wonder if solving this
>> problem is likely and it so when?
>>
>> I use svn_client_status to get all the status info for one
>> directory in an efficient way but
>> looping over each file and doing a svn_client_proplist is very
>> slow compared to the
>> speed of status.
>>
>> Here is the output of a pysvn test program:
>>
>> status() elapsed time: 0.071s for 166 entries 0.000426s/entry
>> propget() elapsed time: 2.988s for 166 entries 0.018000s/entry
>
> These values where obtained on Mac OS X 10.4 on a 1GHz G4
> running svn 1.3.0.
>
> I coded a pure python proplist function and added it to the test
> here is the
> results:
>
> status() elapsed time: 0.074s for 166 entries 0.000446s/entry
> propget() elapsed time: 3.025s for 157 entries 0.019265s/entry
> propget2() elapsed time: 0.056s for 157 entries 0.000359s/entry
>
> Its 54 times faster. Of course I'm not locking, which I could do
> once for
> all 157 entries which I'd guess would not slow things down by much.
>
> And here is myproplist:
>
> def myproplist( path ):
> if os.path.isdir( path ):
> prop_file = os.path.join( path, '.svn', 'dir-props' )
> else:
> dirname, basename = os.path.split( status.path )
> prop_file = os.path.join( dirname, '.svn', 'props',
> basename + '.svn-work' )
>
> result = {}
> try:
> f = file( prop_file )
> except EnvironmentError:
> return []
>
> while True:
> line = f.readline()
> if line == '':
> break
> if line == 'END\n':
> break
> code, length = line.split()
> body = f.read( int(length)+1 )
> if code == 'K':
> key = body[:-1]
> elif code == 'V':
> result[ key ] = body[:-1]
> else:
> raise ValueError( 'Unparsed line %s' % line )
> f.close()
> if len(result) > 0:
> return [(str(path), result)]
> else:
> return []
>
> Barry
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Efficient propget/proplist of one directories entries?
Posted by Barry Scott <ba...@barrys-emacs.org>.
On Feb 17, 2006, at 15:10, Barry Scott wrote:
> It would help speed up my WorkBench GUI if there was an efficient
> way to do a proplist
> of all the entries in a directory. This was discussed in the past
> and I wonder if solving this
> problem is likely and it so when?
>
> I use svn_client_status to get all the status info for one
> directory in an efficient way but
> looping over each file and doing a svn_client_proplist is very slow
> compared to the
> speed of status.
>
> Here is the output of a pysvn test program:
>
> status() elapsed time: 0.071s for 166 entries 0.000426s/entry
> propget() elapsed time: 2.988s for 166 entries 0.018000s/entry
These values where obtained on Mac OS X 10.4 on a 1GHz G4
running svn 1.3.0.
I coded a pure python proplist function and added it to the test here
is the
results:
status() elapsed time: 0.074s for 166 entries 0.000446s/entry
propget() elapsed time: 3.025s for 157 entries 0.019265s/entry
propget2() elapsed time: 0.056s for 157 entries 0.000359s/entry
Its 54 times faster. Of course I'm not locking, which I could do once
for
all 157 entries which I'd guess would not slow things down by much.
And here is myproplist:
def myproplist( path ):
if os.path.isdir( path ):
prop_file = os.path.join( path, '.svn', 'dir-props' )
else:
dirname, basename = os.path.split( status.path )
prop_file = os.path.join( dirname, '.svn', 'props', basename
+ '.svn-work' )
result = {}
try:
f = file( prop_file )
except EnvironmentError:
return []
while True:
line = f.readline()
if line == '':
break
if line == 'END\n':
break
code, length = line.split()
body = f.read( int(length)+1 )
if code == 'K':
key = body[:-1]
elif code == 'V':
result[ key ] = body[:-1]
else:
raise ValueError( 'Unparsed line %s' % line )
f.close()
if len(result) > 0:
return [(str(path), result)]
else:
return []
Barry
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org