Commit with huge log message

[Senthil Kumaran S <se...@collab.net>]

Hi,

I tried committing a small change to http://svn.collab.net/repos/svn/trunk but
got irrelevant errors, since the log message was 14MB in size (no I didn't
intent to put such a big log message, but it revealed this issue). This was
observed both over neon and serf.

Here is what I got:

For Neon,
<snip>
$ ~/subversion-dev/builds/pure-trunk-neon/bin/svn ci -F /tmp/log.txt
../subversion/svn/commit-cmd.c:142: (apr_err=175008)
../subversion/libsvn_client/commit.c:854: (apr_err=175008)
svn: Commit failed (details follow):
../subversion/libsvn_ra_neon/props.c:1158: (apr_err=175008)
svn: At least one property change failed; repository is unchanged
../subversion/libsvn_ra_neon/util.c:613: (apr_err=175002)
svn: PROPPATCH of
'/repos/svn/!svn/wbl/927ed5ba-a9c3-49a4-9772-5e64216fd89e/38669': Could not
send request body: connection was closed by server (https://svn.collab.net)
</snip>

For serf,
<snip>
$ svn ci -F /tmp/log.txt
../subversion/svn/commit-cmd.c:142: (apr_err=20014)
../subversion/libsvn_client/commit.c:854: (apr_err=20014)
svn: Commit failed (details follow):
../subversion/libsvn_client/commit_util.c:1614: (apr_err=20014)
../subversion/libsvn_delta/path_driver.c:173: (apr_err=20014)
../subversion/libsvn_ra_serf/commit.c:1264: (apr_err=20014)
../subversion/libsvn_ra_serf/commit.c:762: (apr_err=20014)
../subversion/libsvn_ra_serf/util.c:568: (apr_err=20014)
svn: Error running context: Internal error
Segmentation fault
</snip>

At a first glance I couldn't make out the reason for the error. So shouldn't we
give a more pleasing error message?

Also, the above shows that there is a limit for the size of log message, do
anyone know what is the maximum size of log message that can be committed. If
we do have a limit, then IMO it should be caught in libsvn_client and errored
out before we even attempt to contact the server as in the above case.

Thoughts?
-- 
Senthil Kumaran S
http://www.stylesen.org/

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2382390

Re: Commit with huge log message

[Paul Burba <pt...@gmail.com>]

On Tue, Aug 11, 2009 at 5:55 AM, Senthil Kumaran S<se...@collab.net> wrote:
> Senthil Kumaran S wrote:
>> Also, the above shows that there is a limit for the size of log message, do
>> anyone know what is the maximum size of log message that can be committed. If
>
> Further look at this revealed that, this is not a limitation with subversion
> per se, it is a limitation with apache (may be not a limitation but for good?)

Hi Senthil,

Regardless of the Apache limitations, there are some effective
limitations to svn:log (or any other property) sizes in Subversion's
basic design.  As "The Subversion Book" notes:

"While Subversion places few restrictions on the names and values you
use for properties, it has not been designed to optimally carry large
property values or large sets of properties on a given file or
directory. Subversion commonly holds all the property names and values
associated with a single item in memory at the same time, which can
cause detrimental performance or failed operations when extremely
large property sets are used."

Log messages get passed around as char * | svn_string_t * |
svn_stringbuf_t * and stored in hashes and arrays as such.
Unfortunately the memory management around these allocations is pretty
"loose" and we seem to copy the log message into a long lived pool
every chance we get :-(  I made a slight improvement in r37868, but I
suspect further improvements are going to be a lot more work.

Paul

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2382485

Re: Commit with huge log message

[Senthil Kumaran S <se...@collab.net>]

Senthil Kumaran S wrote:
> Also, the above shows that there is a limit for the size of log message, do
> anyone know what is the maximum size of log message that can be committed. If

Further look at this revealed that, this is not a limitation with subversion
per se, it is a limitation with apache (may be not a limitation but for good?)
which is obvious from the following run,

<snip>
$ svnadmin create repos1
$ svn co file:///tmp/repos1 wc
Checked out revision 0.
$ cd wc/
$ touch file1.txt
$ svn add file1.txt
A         file1.txt
$ svn ci -F /tmp/log.txt
Adding         file1.txt
Transmitting file data .
Committed revision 1.
</snip>

Over dav,
<snip>
$ svn co http://localhost/svn/repos1 wc-http
A    wc-http/file1.txt
Checked out revision 1.
$ cd wc-http/
$ touch file2.txt
$ svn add file2.txt
A         file2.txt
$ svn ci -F /tmp/log.txt
../subversion/svn/commit-cmd.c:142: (apr_err=20014)
../subversion/libsvn_client/commit.c:854: (apr_err=20014)
svn: Commit failed (details follow):
../subversion/libsvn_client/commit_util.c:1614: (apr_err=20014)
../subversion/libsvn_delta/path_driver.c:173: (apr_err=20014)
../subversion/libsvn_ra_serf/commit.c:1264: (apr_err=20014)
../subversion/libsvn_ra_serf/commit.c:762: (apr_err=20014)
../subversion/libsvn_ra_serf/util.c:568: (apr_err=20014)
svn: Error running context: Internal error
</snip>

Hmmm, that is probably a known issue ;) the default value for
"LimitXMLRequestBody" is 1 million bytes (approx 1mb), so obviously my commit
wont succeed.

Again the segfault for this in the case of serf is not acceptable and also the
irrelevant error message by neon should be improved.

Should we avoid this by getting this info via server capabilities (mod_dav_svn
can advertise this?) and error out before sending the entire log message xml?

-- 
Senthil Kumaran S
http://www.stylesen.org/

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2382400