You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Kim Sparrow <ks...@lightpointe.com> on 2004/12/21 01:02:03 UTC
Working around presumed svndumpfilter memory leak
First, a little background: a coworker of mine put half a gig of static
data into our repository (twice!) and now the repository is about 1.2 GB
instead of ~100MB. Doh. This might not be a problem, except that
the repository gets a nightly backup to our corporate server, and three
weeks of backups are kept. You do the math, it's pretty big.
So what I want to do is to use svndumpfilter to exclude the offending
files, exept it bombs out at the first of the two commits after a couple
minutes, simply saying "abnormal program termination". I found bug 1899
http://subversion.tigris.org/issues/show_bug.cgi?id=1899
which may be the culprit. In this bug report, it's a single large file
used with 'include'. In my situation it's hundreds of 2-5MB files that
I want to exclude, while there are other files in the same commit that I
want to keep.
Regardless of the source of the problem, does anybody have any
suggestions to work around this? I suppose I could manually edit the
dumpfile, but I don't relish the thought of removing several hundred
large entries from a 1.2 gig dumpfile! I don't even have an editor that
will handle a file of that size!
Cheers,
Kim
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Working around presumed svndumpfilter memory leak
Posted by "Alex R. Mosteo" <al...@mosteo.com>.
Max Bowsher wrote:
>>> So what I want to do is to use svndumpfilter to exclude the offending
>>> files, exept it bombs out at the first of the two commits after a couple
>>> minutes, simply saying "abnormal program termination". I found bug 1899
>>> http://subversion.tigris.org/issues/show_bug.cgi?id=1899
>>> which may be the culprit. In this bug report, it's a single large file
>>> used with 'include'. In my situation it's hundreds of 2-5MB files that
>>> I want to exclude, while there are other files in the same commit that I
>>> want to keep.
> I have fixed this issue 1899. You can either build from trunk, or simply
> replace the svndumpfilter/main.c file in 1.1.2 with the one from:
> http://svn.collab.net/repos/svn/branches/1.1.x-svndumpfilter/subversion/svndumpfilter/main.c
Thanks!
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Working around presumed svndumpfilter memory leak
Posted by Max Bowsher <ma...@ukf.net>.
Alex R. Mosteo wrote:
> Kim Sparrow wrote:
>> First, a little background: a coworker of mine put half a gig of static
>> data into our repository (twice!) and now the repository is about 1.2 GB
>> instead of ~100MB. Doh. This might not be a problem, except that
>> the repository gets a nightly backup to our corporate server, and three
>> weeks of backups are kept. You do the math, it's pretty big.
>>
>> So what I want to do is to use svndumpfilter to exclude the offending
>> files, exept it bombs out at the first of the two commits after a couple
>> minutes, simply saying "abnormal program termination". I found bug 1899
>> http://subversion.tigris.org/issues/show_bug.cgi?id=1899
>> which may be the culprit. In this bug report, it's a single large file
>> used with 'include'. In my situation it's hundreds of 2-5MB files that
>> I want to exclude, while there are other files in the same commit that I
>> want to keep.
>>
>> Regardless of the source of the problem, does anybody have any
>> suggestions to work around this? I suppose I could manually edit the
>> dumpfile, but I don't relish the thought of removing several hundred
>> large entries from a 1.2 gig dumpfile! I don't even have an editor that
>> will handle a file of that size!
>
> We are on the same boat, but sadly I can't offer any advice. I've been
> suggested to take a build of svndumpfilter with debug symbols and
> posting a gdb backtrace at the point of the abortion. I've been unable
> to do it for various reasons, but if you want to try...
>
> You can also look for recent posts about svndumpfilter to get the
> specifics of my problem. I would put a pointer but the subversion server
> seems to be having problems with tomcat at the moment.
I have fixed this issue 1899. You can either build from trunk, or simply
replace the svndumpfilter/main.c file in 1.1.2 with the one from:
http://svn.collab.net/repos/svn/branches/1.1.x-svndumpfilter/subversion/svndumpfilter/main.c
Max.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Working around presumed svndumpfilter memory leak
Posted by "Alex R. Mosteo" <al...@mosteo.com>.
Kim Sparrow wrote:
> First, a little background: a coworker of mine put half a gig of static
> data into our repository (twice!) and now the repository is about 1.2 GB
> instead of ~100MB. Doh. This might not be a problem, except that
> the repository gets a nightly backup to our corporate server, and three
> weeks of backups are kept. You do the math, it's pretty big.
>
> So what I want to do is to use svndumpfilter to exclude the offending
> files, exept it bombs out at the first of the two commits after a couple
> minutes, simply saying "abnormal program termination". I found bug 1899
> http://subversion.tigris.org/issues/show_bug.cgi?id=1899
> which may be the culprit. In this bug report, it's a single large file
> used with 'include'. In my situation it's hundreds of 2-5MB files that
> I want to exclude, while there are other files in the same commit that I
> want to keep.
>
> Regardless of the source of the problem, does anybody have any
> suggestions to work around this? I suppose I could manually edit the
> dumpfile, but I don't relish the thought of removing several hundred
> large entries from a 1.2 gig dumpfile! I don't even have an editor that
> will handle a file of that size!
We are on the same boat, but sadly I can't offer any advice. I've been
suggested to take a build of svndumpfilter with debug symbols and
posting a gdb backtrace at the point of the abortion. I've been unable
to do it for various reasons, but if you want to try...
You can also look for recent posts about svndumpfilter to get the
specifics of my problem. I would put a pointer but the subversion server
seems to be having problems with tomcat at the moment.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org