You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by David James <dj...@collab.net> on 2006/01/20 18:52:26 UTC
Re: Subversion python bindings ignoring pool arguments?
On 1/20/06, Christian Boos <cb...@neuf.fr> wrote:
> We have another issue in Trac concerning the python bindings
> for SVN 1.3.0.
>
> May I ask you to have a look at
> http://projects.edgewall.com/trac/ticket/2620
> and tell me what you think?
Hi Christian,
This looks like a Subversion 1.3 bug. In order to fix this I'll need a
short script which reproduces the problem. Does the script you
attached to the issue (check_svn_repos.py) do the trick if I run it on
a really big repository? See
http://projects.edgewall.com/trac/attachment/ticket/2620/check_svn_repos.py
List of Trac bugs identified so far in Subversion 1.3:
- 2472 (assertion failures). http://projects.edgewall.com/trac/ticket/2472
- 2611 (mod python pool problems).
http://projects.edgewall.com/trac/ticket/2611
- 2620 (memory leak). http://projects.edgewall.com/trac/ticket/2620
2472 has been fixed in Subversion trunk, but I have not figured out
2611 and 2620 yet.
Cheers,
David
--
David James -- http://www.cs.toronto.edu/~james
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Subversion python bindings ignoring pool arguments?
Posted by David James <dj...@collab.net>.
On 22 Jan 2006 14:28:28 +0100, Lars Gullik Bjønnes <la...@gullik.net> wrote:
> I did that, but only setup the script to check the first 8000 revs.
> Same behaviour. Memory used grows very fast. Note that
> svn_repos_replay is commented out and also my earlier test shows that
> it is the delta.make_editor that 'leaks' the memory.
Hi Lars,
With your script, it was very easy for me to reproduce the problem.
This issue should now be fixed in r18230 -- I've proposed the fix for
Subversion 1.3.1.
This memory leak has actually been present in Subversion since the
creation of the Python bindings, but it only became obvious in
Subversion 1.3.0 thanks to our new automatic memory management scheme.
Thanks for your help!
David
--
David James -- http://www.cs.toronto.edu/~james
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Subversion python bindings ignoring pool arguments?
Posted by Lars Gullik Bjønnes <la...@gullik.net>.
larsbj@gullik.net (Lars Gullik Bjønnes) writes:
| David James <dj...@collab.net> writes:
|
| | On 1/20/06, Christian Boos <cb...@neuf.fr> wrote:
| | > We have another issue in Trac concerning the python bindings
| | > for SVN 1.3.0.
| | >
| | > May I ask you to have a look at
| | > http://projects.edgewall.com/trac/ticket/2620
| | > and tell me what you think?
| |
| | Hi Christian,
| |
| | This looks like a Subversion 1.3 bug. In order to fix this I'll need a
| | short script which reproduces the problem. Does the script you
| | attached to the issue (check_svn_repos.py) do the trick if I run it on
| | a really big repository? See
| | http://projects.edgewall.com/trac/attachment/ticket/2620/check_svn_repos.py
|
| I am pretty sure that it does not since it only checks a single
| revision. I'll modify the script to run on a range of revisions and
| test that on the big repo that I have (78000 revs, 12GB)
I did that, but only setup the script to check the first 8000 revs.
Same behaviour. Memory used grows very fast. Note that
svn_repos_replay is commented out and also my earlier test shows that
it is the delta.make_editor that 'leaks' the memory.
This is what my modified script looks like:
from trac.versioncontrol.svn_fs import Pool
from trac.util import TracError
from svn import repos, fs, delta
import sys
import os
class ChangeCollectorChecker(object):
def __init__(self, path):
self.pool = Pool()
# Remove any trailing slash or else subversion might abort
path = os.path.normpath(path).replace('\\', '/')
self.path = repos.svn_repos_find_root_path(path, self.pool())
if self.path is None:
raise TracError, \
"%s does not appear to be a Subversion repository." % path
self.repos = repos.svn_repos_open(self.path, self.pool())
self.fs_ptr = repos.svn_repos_fs(self.repos)
def check(self, rev):
pool = Pool(self.pool)
root = fs.revision_root(self.fs_ptr, rev, pool())
editor = repos.RevisionChangeCollector(self.fs_ptr, rev, pool())
e_ptr, e_baton = delta.make_editor(editor, pool())
#repos.svn_repos_replay(root, e_ptr, e_baton, pool())
if __name__ == '__main__':
try:
path = sys.argv[1]
#rev = sys.argv[2]
collector = ChangeCollectorChecker(path)
for rev in range(8000):
print "Checking revision %s" % rev
collector.check(int(rev))
import gc
print "Checked revision %s (%d objects unreachable)" % (rev, gc.collect())
except:
import traceback
traceback.print_exc(file=sys.stderr)
print>>sys.stderr, "Usage: python check_repos.py <repository_path> <revision>"
sys.exit(2)
--
Lgb
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Subversion python bindings ignoring pool arguments?
Posted by Lars Gullik Bjønnes <la...@gullik.net>.
David James <dj...@collab.net> writes:
| On 1/20/06, Christian Boos <cb...@neuf.fr> wrote:
| > We have another issue in Trac concerning the python bindings
| > for SVN 1.3.0.
| >
| > May I ask you to have a look at
| > http://projects.edgewall.com/trac/ticket/2620
| > and tell me what you think?
|
| Hi Christian,
|
| This looks like a Subversion 1.3 bug. In order to fix this I'll need a
| short script which reproduces the problem. Does the script you
| attached to the issue (check_svn_repos.py) do the trick if I run it on
| a really big repository? See
| http://projects.edgewall.com/trac/attachment/ticket/2620/check_svn_repos.py
I am pretty sure that it does not since it only checks a single
revision. I'll modify the script to run on a range of revisions and
test that on the big repo that I have (78000 revs, 12GB)
--
Lgb
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org