You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Attila <at...@gmx.net> on 2021/07/13 21:48:48 UTC
svn log xml hangs and produces too many logentry closing tags
Hi
I have a problem getting the svn log in a branch after sync-merging a commit from trunk.
This commit in trunk is a merge of an old and complex branch with many commits.
The client accessing the repository over svn:// url.
(paths and text is redacted)
The head revision is: 10801
When I run the following command on the client (in the working copy), it prints a long partial xml-log output, then hangs.
/usr/bin/svn log --xml -g -v -r 10701:HEAD /path/to/branch-wc
When observing in "top", the command uses no visible CPU resources on hang. (I waited ca. 2 minutes)
The hanging command does mot exits on CTRL-c, it does not exits on "kill -TERM pid", I have to send "kill -KILL pid" to terminate it.
When I run the command with strace it hangs at read(4,
...SNIP...
read(4, " ( ) ( 4:file true false ) ) ( 1"..., 16384) = 16384
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d83000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d81000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7f000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7d000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7b000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d79000
read(4, "***/****-***_redacted_*****_****"..., 16384) = 14773
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d77000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d75000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d73000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d71000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d6f000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d6d000
read(4,
When I observe the server, there is a CPU activity at the begin, but when the client hangs, the server seems to be in idle.
Just a corresponding svnserve process is there with no visible cpu usage.
In svnserve.log is nothing relevant to see.
The svnserve command is:
svnserve -d -r /path/to/repositories \
--log-file=/var/log/svnserve.log \
--memory-cache-size 1024 \
--cache-txdeltas yes \
--cache-fulltexts yes
When I try to get the xml-log on the server with the corresponding file:// repository URL:
/usr/bin/svn log --xml -g -v -r 10701:HEAD file://path/to/local/repositories/project/branch
The command finishes in ca 5-10 seconds and I get the xml output, but the output has a way too many </logentry> lines.
There are 1217 occurrences of the string “<logentry" and 33191 occurrences of the string "</logentry>" in the output xml.
There are several thousand lines of </logentry> in a row in many places in repeated blocks.
Details:
Client and Server OS:
Linux 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1+deb10u1 (2020-04-27) x86_64 GNU/Linux
The repository is ca. 4 GB.
Running "svnadmin verify" on the server founds no errors.
I have no other problems with the server, checkout and commit works normal.
svn --version
svn, version 1.10.4 (r1850624)
compiled Feb 10 2021, 20:15:45 on x86_64-pc-linux-gnu
Copyright (C) 2019 The Apache Software Foundation.
This software consists of contributions made by many people;
see the NOTICE file for more information.
Subversion is open source software, see http://subversion.apache.org/
The following repository access (RA) modules are available:
* ra_svn : Module for accessing a repository using the svn network protocol.
- with Cyrus SASL authentication
- handles 'svn' scheme
* ra_local : Module for accessing a repository on local disk.
- handles 'file' scheme
* ra_serf : Module for accessing a repository via WebDAV protocol using serf.
- using serf 1.3.9 (compiled with 1.3.9)
- handles 'http' scheme
- handles 'https' scheme
The following authentication credential caches are available:
* Plaintext cache in /home/username/.subversion
* Gnome Keyring
* GPG-Agent
* KWallet (KDE)
Googling around gives me these two somewhat related hits:
https://issues.apache.org/jira/browse/SVN-4856
https://issues.apache.org/jira/browse/SVN-4711
But I do'nt use the --search parameter.
Is this a bug or are there any suggestions how to solve this problem?
Thanks,
Attila
Re: svn log xml hangs and produces too many logentry closing tags
Posted by Attila <at...@gmx.net>.
> On 9 Aug 2021, at 11:48, Daniel Sahlberg <da...@gmail.com> wrote:
>
> Den lör 17 juli 2021 kl 15:02 skrev Attila <atiware@gmx.net <ma...@gmx.net>>:
> On 14 Jul 2021, at 21:42, Nathan Hartman <hartman.nathan@gmail.com <ma...@gmail.com>> wrote:
>>
>> On Tue, Jul 13, 2021 at 5:49 PM Attila <atiware@gmx.net <ma...@gmx.net>> wrote:
>>>
>>> Hi
>>>
>>> I have a problem getting the svn log in a branch after sync-merging a commit from trunk.
>>> This commit in trunk is a merge of an old and complex branch with many commits.
>>>
>>> The client accessing the repository over svn:// url.
>>> (paths and text is redacted)
>>> The head revision is: 10801
>>>
>>> When I run the following command on the client (in the working copy), it prints a long partial xml-log output, then hangs.
>>> /usr/bin/svn log --xml -g -v -r 10701:HEAD /path/to/branch-wc
>>>
>>> When observing in "top", the command uses no visible CPU resources on hang. (I waited ca. 2 minutes)
>>> The hanging command does mot exits on CTRL-c, it does not exits on "kill -TERM pid", I have to send "kill -KILL pid" to terminate it.
>>>
>>> When I run the command with strace it hangs at read(4,
>>> ...SNIP...
>>> read(4, " ( ) ( 4:file true false ) ) ( 1"..., 16384) = 16384
>>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d83000
>>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d81000
>>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7f000
>>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7d000
>>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7b000
>>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d79000
>>> read(4, "***/****-***_redacted_*****_****"..., 16384) = 14773
>>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d77000
>>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d75000
>>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d73000
>>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d71000
>>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d6f000
>>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d6d000
>>> read(4,
>>>
>>> When I observe the server, there is a CPU activity at the begin, but when the client hangs, the server seems to be in idle.
>>> Just a corresponding svnserve process is there with no visible cpu usage.
>>> In svnserve.log is nothing relevant to see.
>>>
>>> The svnserve command is:
>>> svnserve -d -r /path/to/repositories \
>>> --log-file=/var/log/svnserve.log \
>>> --memory-cache-size 1024 \
>>> --cache-txdeltas yes \
>>> --cache-fulltexts yes
>>>
>>> When I try to get the xml-log on the server with the corresponding file:// repository URL:
>>> /usr/bin/svn log --xml -g -v -r 10701:HEAD file://path/to/local/repositories/project/branch <>
>>> The command finishes in ca 5-10 seconds and I get the xml output, but the output has a way too many </logentry> lines.
>>>
>>> There are 1217 occurrences of the string “<logentry" and 33191 occurrences of the string "</logentry>" in the output xml.
>>> There are several thousand lines of </logentry> in a row in many places in repeated blocks.
>>>
>>> Details:
>>> Client and Server OS:
>>> Linux 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1+deb10u1 (2020-04-27) x86_64 GNU/Linux
>>>
>>> The repository is ca. 4 GB.
>>> Running "svnadmin verify" on the server founds no errors.
>>> I have no other problems with the server, checkout and commit works normal.
>>>
>>> svn --version
>>> svn, version 1.10.4 (r1850624)
>>> compiled Feb 10 2021, 20:15:45 on x86_64-pc-linux-gnu
>>>
>>> Copyright (C) 2019 The Apache Software Foundation.
>>> This software consists of contributions made by many people;
>>> see the NOTICE file for more information.
>>> Subversion is open source software, see http://subversion.apache.org/ <http://subversion.apache.org/>
>>>
>>> The following repository access (RA) modules are available:
>>>
>>> * ra_svn : Module for accessing a repository using the svn network protocol.
>>> - with Cyrus SASL authentication
>>> - handles 'svn' scheme
>>> * ra_local : Module for accessing a repository on local disk.
>>> - handles 'file' scheme
>>> * ra_serf : Module for accessing a repository via WebDAV protocol using serf.
>>> - using serf 1.3.9 (compiled with 1.3.9)
>>> - handles 'http' scheme
>>> - handles 'https' scheme
>>>
>>> The following authentication credential caches are available:
>>>
>>> * Plaintext cache in /home/username/.subversion
>>> * Gnome Keyring
>>> * GPG-Agent
>>> * KWallet (KDE)
>>
>>
>> Hello Attila,
>>
>> Thanks for this detailed explanation. There are quite a few important
>> clues here.
>>
>> To help narrow down the search for the culprit: Are you able to run
>> the same 'svn log' command against the same working copy, but without
>> the '--xml', and get a correct output in a reasonable amount of time?
>
> Hi Nathan
>
> thanks for the suggestions.
>
> The command without the --xml parameter hangs on the client and works on the server.
> The text output on the server seems be ok.
> This suggests that the hang and the closing xml tags are two unrelated problems.
>
> Hi Attila,
>
> Thanks for the detailed explaination!
>
> I agree these are two different problems. More below
>
>> If log without '--xml' works and log with '--xml' gets stuck, is it
>> feasible to try to bisect the revision range, e.g., run log on the
>> range r10701:10751, and in a separate invocation on r10751:10801, and
>
> I tried that, it seems that the problem was not the last sync-merge commit.
> The command succeeds when there is no sync-merge commit in the range.
> There are two sync-merges in the range the first one is a small merge of 3-4 changed files. These 3-4 files are unmodified in the branch.
> Whenever i include one of these sync-merges in the range, the command hangs.
>
> What if you run the command on the whole range but without '-g'? I'm expecting that should work, both with and without '--xml'.
The command runs within milliseconds error free and produces the right output with and without the —xml parameter. The logentry tags are balanced in the produced xml output.
>
>> try to see if there's a revision in particular where SVN is getting
>> confused? If so, then perhaps knowing a little more about that
>> revision could help narrow it down.
>>
>> By the way, is this (1.10.4) the SVN version on the server, the
>> client, or both?
>>
>
> both.
> The server and client is Debian Buster and I have updated both to the latest version hoping it will help.
> The installed subversion package is:
> subversion/stable,stable,now 1.10.4-1+deb10u2 amd64 [installed]
>
> The client is a private vagrant vm running on macOS in VMware Fusion pro, the server is an ESXi vm.
>
>>> Googling around gives me these two somewhat related hits:
>>> https://issues.apache.org/jira/browse/SVN-4856 <https://issues.apache.org/jira/browse/SVN-4856>
>>> https://issues.apache.org/jira/browse/SVN-4711 <https://issues.apache.org/jira/browse/SVN-4711>
>>>
>>> But I do'nt use the --search parameter.
>>
>> It could nevertheless be related. The spurious XML closing tags seems
>> to ring a bell.
>
>
> I have been chasing svn log --xml for the last two days.
>
> As far as I understand, both issues are caused by the -g (or --use-merge-history) argument. This forces the code to recursively consider the log of the merge history.
>
> In #4711 it is the --search filtering that causes the client to emit a </logentry> tag even when it didn't emit the corresponding <logentry> tag. I believe I have found a solution., details in http://mail-archives.apache.org/mod_mbox/subversion-dev/202108.mbox/%3C8215e3c9-2b81-c703-eaf9-4199ada8e485%40gmail.com%3E <http://mail-archives.apache.org/mod_mbox/subversion-dev/202108.mbox/%3C8215e3c9-2b81-c703-eaf9-4199ada8e485%40gmail.com%3E>. When this is committed, I'm planning to go after 4856.
>
> In #4856 it seems that the library is issuing the callback for the closing tag (</logentry>) even when it shouldn't. The same issue would affects the log in plain text format, but since there is no closing tag, it is not visible in the output. It is fairly easy to produce the issue of merging in multiple steps.
>
>>> Is this a bug or are there any suggestions how to solve this problem?
>>
>> It sounds like a bug. My suggestions above.
>
> Here is a preliminary backtrace of the hanging client (without debug symbols for now)
> I need some time to spin up a copy of the server and the client and re-compile svn with debug symbols.
> I will come back with more details as soon as i have the time to do more tests.
>
> # gdb /usr/bin/svn 1399
> GNU gdb (Debian 8.2.1-2+b3) 8.2.1
> Copyright (C) 2018 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html <http://gnu.org/licenses/gpl.html>>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> Type "show copying" and "show warranty" for details.
> This GDB was configured as "x86_64-linux-gnu".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/ <http://www.gnu.org/software/gdb/bugs/>>.
> Find the GDB manual and other documentation resources online at:
> <http://www.gnu.org/software/gdb/documentation/ <http://www.gnu.org/software/gdb/documentation/>>.
>
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> Reading symbols from /usr/bin/svn...(no debugging symbols found)...done.
> Attaching to program: /usr/bin/svn, process 1399
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_client-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_wc-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_delta-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_diff-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_subr-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libaprutil-1.so.0...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libapr-1.so.0...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading symbols from /usr/lib/debug/.build-id/e9/1114987a0147bd050addbd591eb8994b29f4b3.debug...done.
> done.
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
> Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from /usr/lib/debug/.build-id/18/b9a9a8c523e5cfe5b5d946d605d09242f09798.debug...done.
> done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_local-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_repos-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_serf-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libz.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libexpat.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsqlite3.so.0...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/liblz4.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libutf8proc.so.2...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libuuid.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/librt.so.1...Reading symbols from /usr/lib/debug/.build-id/5d/cf98ad684962be494af28a1051793fd39e4ebc.debug...done.
> done.
> Reading symbols from /lib/x86_64-linux-gnu/libcrypt.so.1...Reading symbols from /usr/lib/debug/.build-id/c6/966ba263bacba7f67c23643ba373aa51a162e9.debug...done.
> done.
> Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from /usr/lib/debug/.build-id/d3/583c742dd47aaa860c5ae0c0c5bdbcd2d54f61.debug...done.
> done.
> Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug/.build-id/f2/5dfd7b95be4ba386fd71080accae8c0732b711.debug...done.
> done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_fs-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_x-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_base-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_util-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsasl2.so.2...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libserf-1.so.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...Reading symbols from /usr/lib/debug/.build-id/88/5dda4b4a5cea600e7b5b98c1ad86996c8d2299.debug...done.
> done.
> Reading symbols from /lib/x86_64-linux-gnu/libdb-5.3.so...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libresolv.so.2...Reading symbols from /usr/lib/debug/.build-id/02/6c3ba167f64f631eb8781fca2269fbc2ee7ca5.debug...done.
> done.
> Reading symbols from /lib/x86_64-linux-gnu/libssl.so.1.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libcrypto.so.1.1...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libgssapi_krb5.so.2...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libkrb5.so.3...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libk5crypto.so.3...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libcom_err.so.2...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libkrb5support.so.0...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libkeyutils.so.1...(no debugging symbols found)...done.
> Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libsasldb.so...(no debugging symbols found)...done.
> Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libanonymous.so...(no debugging symbols found)...done.
> Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libcrammd5.so...(no debugging symbols found)...done.
> Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so...(no debugging symbols found)...done.
> Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/liblogin.so...(no debugging symbols found)...done.
> Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libntlm.so...(no debugging symbols found)...done.
> Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libplain.so...(no debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libnss_files.so.2...Reading symbols from /usr/lib/debug/.build-id/4b/ff8b782e1602c596e856bdef06e642e50e7fa7.debug...done.
> done.
> Reading symbols from /lib/x86_64-linux-gnu/libnss_dns.so.2...Reading symbols from /usr/lib/debug/.build-id/bb/8b756d3859e8fe20d7aa6d8255f27942589f47.debug...done.
> done.
> 0x00007f276a1fc50e in __libc_read (fd=4, buf=0x7f2768e19000, nbytes=16384) at ../sysdeps/unix/sysv/linux/read.c:26
> 26 ../sysdeps/unix/sysv/linux/read.c: No such file or directory.
> (gdb) bt
> #0 0x00007f276a1fc50e in __libc_read (fd=4, buf=0x7f2768e19000, nbytes=16384) at ../sysdeps/unix/sysv/linux/read.c:26
> #1 0x00007f276a22f37f in apr_socket_recv () from /lib/x86_64-linux-gnu/libapr-1.so.0
> #2 0x00007f2769fc1353 in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #3 0x00007f2769fc158c in svn_ra_svn.stream_read () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #4 0x00007f2769fbb242 in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #5 0x00007f2769fbb61a in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #6 0x00007f2769fbb68b in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #7 0x00007f2769fbbc7f in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #8 0x00007f2769fbbcbe in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #9 0x00007f2769fbde67 in svn_ra_svn.read_item () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #10 0x00007f2769fb583a in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #11 0x00007f276a46033a in svn_client_log5 () from /lib/x86_64-linux-gnu/libsvn_client-1.so.1
> #12 0x000055f9507b6779 in ?? ()
> #13 0x000055f9507c2541 in ?? ()
> #14 0x000055f9507a7513 in ?? ()
> #15 0x00007f276a04e09b in __libc_start_main (main=0x55f9507a7490, argc=7, argv=0x7ffee0fc23c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffee0fc23b8) at ../csu/libc-start.c:308
> #16 0x000055f9507a759a in ?? ()
> (gdb)
>
> Any chance to do the same on the server?
>
> Can you attach to the repository in any other way (http/https or file) to verify if the error is in svnserve or in the repository code?
>
> Kind regards
> Daniel Sahlberg
I now have two vm's running in VMWare Fusion for testing. One vm with the server and one with the client. I have subversion installed on both with debug symbols.
I have tried the backtrace several times and it always produces the same output on hang.
Below the backtrace of the server and the client with debug symbols. All repository paths in the backtraces are redacted.
======================
Below the server backtrace:
root@svntest1:~# which svnserve
/usr/bin/svnserve
root@svntest1:~# file /usr/bin/svnserve
/usr/bin/svnserve: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=690982730405d811d152ce9bce79b5725590ca7b, with debug_info, not stripped
root@svntest1:~# /usr/bin/svnserve --version
svnserve, version 1.10.4 (r1850624)
compiled Feb 10 2021, 20:15:45 on x86_64-pc-linux-gnu
Copyright (C) 2019 The Apache Software Foundation.
This software consists of contributions made by many people;
see the NOTICE file for more information.
Subversion is open source software, see http://subversion.apache.org/
The following repository back-end (FS) modules are available:
* fs_fs : Module for working with a plain file (FSFS) repository.
* fs_x : Module for working with an experimental (FSX) repository.
* fs_base : Module for working with a Berkeley DB repository.
Cyrus SASL authentication is available.
root@svntest1:~# uname -a
Linux svntest1 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18) x86_64 GNU/Linux
root@svntest1:~# ps ax | grep svn
449 ? Ss 0:00 /usr/bin/svnserve --daemon --pid-file /run/svnserve/svnserve.pid --root /data/subversion --log-file /var/log/svnserve/svnserve.log --memory-cache-size 1024 --cache-txdeltas yes --cache-fulltexts yes
1016 ? S 0:12 /usr/bin/svnserve --daemon --pid-file /run/svnserve/svnserve.pid --root /data/subversion --log-file /var/log/svnserve/svnserve.log --memory-cache-size 1024 --cache-txdeltas yes --cache-fulltexts yes
1018 pts/0 S+ 0:00 grep svn
root@svntest1:~# gdb /usr/bin/svnserve 1016
GNU gdb (Debian 8.2.1-2+b3) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/svnserve...done.
Attaching to program: /usr/bin/svnserve, process 1016
Reading symbols from /lib/x86_64-linux-gnu/libsvn_repos-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_delta-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_subr-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libaprutil-1.so.0...Reading symbols from /usr/lib/debug/.build-id/9b/d595ade2a5191a525571848e604912d0d4d9f2.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libapr-1.so.0...Reading symbols from /usr/lib/debug/.build-id/2b/d8bdd5984b745e54e0144c6cb3a52319149669.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libsasl2.so.2...Reading symbols from /usr/lib/debug/.build-id/99/bf5a225908fd4124228d4f3e19c67d7138144f.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading symbols from /usr/lib/debug/.build-id/e9/1114987a0147bd050addbd591eb8994b29f4b3.debug...done.
done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from /usr/lib/debug/.build-id/18/b9a9a8c523e5cfe5b5d946d605d09242f09798.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_fs-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_x-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_base-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_util-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libz.so.1...Reading symbols from /usr/lib/debug/.build-id/3a/f7c4bceb19b6c83f76e2822b9a23041d85f6d1.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libexpat.so.1...Reading symbols from /usr/lib/debug/.build-id/9a/27bd3a88235268c02ed62ce01274a059fbb41d.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libsqlite3.so.0...Reading symbols from /usr/lib/debug/.build-id/de/5b665b816f5fee97547a83709c017f13265bc6.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/liblz4.so.1...Reading symbols from /usr/lib/debug/.build-id/85/51af4090d0b66af3a66e3848557aa3aa789ae2.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libutf8proc.so.2...Reading symbols from /usr/lib/debug/.build-id/fe/46dc620bf7046edea84ac41a4a20f56e31c877.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libuuid.so.1...Reading symbols from /usr/lib/debug/.build-id/09/6caf62fb91c7beffc5f53e0b315229a5b22c20.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/librt.so.1...Reading symbols from /usr/lib/debug/.build-id/5d/cf98ad684962be494af28a1051793fd39e4ebc.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libcrypt.so.1...Reading symbols from /usr/lib/debug/.build-id/c6/966ba263bacba7f67c23643ba373aa51a162e9.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from /usr/lib/debug/.build-id/d3/583c742dd47aaa860c5ae0c0c5bdbcd2d54f61.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libresolv.so.2...Reading symbols from /usr/lib/debug/.build-id/02/6c3ba167f64f631eb8781fca2269fbc2ee7ca5.debug...done.
done.
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug/.build-id/f2/5dfd7b95be4ba386fd71080accae8c0732b711.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libdb-5.3.so...Reading symbols from /usr/lib/debug/.build-id/67/4e2dc8ff2d056a41d25065ec098d81392952a3.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...Reading symbols from /usr/lib/debug/.build-id/88/5dda4b4a5cea600e7b5b98c1ad86996c8d2299.debug...done.
done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libsasldb.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libanonymous.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libcrammd5.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libcrypto.so.1.1...Reading symbols from /usr/lib/debug/.build-id/74/9142d5ee728a76e7cdc61fd79d2311a77405a2.debug...done.
done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/liblogin.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libntlm.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libplain.so...(no debugging symbols found)...done.
0x00007fa8e55e150e in __libc_read (fd=5, buf=buf@entry=0x7fa8e4538000, nbytes=16384) at ../sysdeps/unix/sysv/linux/read.c:26
26 ../sysdeps/unix/sysv/linux/read.c: No such file or directory.
(gdb) bt
#0 0x00007fa8e55e150e in __libc_read (fd=5, buf=buf@entry=0x7fa8e4538000, nbytes=16384) at ../sysdeps/unix/sysv/linux/read.c:26
#1 0x00007fa8e563137f in read (__nbytes=<optimized out>, __buf=0x7fa8e4538000, __fd=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/unistd.h:44
#2 apr_socket_recv (sock=0x7fa8e453f0d0,
buf=0x7fa8e4538000 "( log ( ( 0: ) ( 10701 ) ( 10801 ) true false 0 true revprops ( 10:svn:author 8:svn:date 7:svn:log ) ) ) 168.116.222/Repository/Project/branches/feature-12 32:SVN/1.10.4 (x86_64-pc-linux-gnu) ( ) ) ", len=0x7fffec9fbfb8) at ./network_io/unix/sendrecv.c:81
#3 0x00007fa8e569a19d in sock_read_cb (baton=0x7fa8e453c1b8,
buffer=0x7fa8e4538000 "( log ( ( 0: ) ( 10701 ) ( 10801 ) true false 0 true revprops ( 10:svn:author 8:svn:date 7:svn:log ) ) ) 168.116.222/Repository/Project/branches/feature-12 32:SVN/1.10.4 (x86_64-pc-linux-gnu) ( ) ) ", len=0x7fffec9fbfb8) at ./subversion/libsvn_ra_svn/streams.c:119
#4 0x00007fa8e570b6dc in svn_stream_read2 (stream=0x7fa8e453c1c8,
buffer=0x7fa8e4538000 "( log ( ( 0: ) ( 10701 ) ( 10801 ) true false 0 true revprops ( 10:svn:author 8:svn:date 7:svn:log ) ) ) 168.116.222/Repository/Project/branches/feature-12 32:SVN/1.10.4 (x86_64-pc-linux-gnu) ( ) ) ", len=0x7fffec9fbfb8) at ./subversion/libsvn_subr/stream.c:185
#5 0x00007fa8e569a4b8 in svn_ra_svn__stream_read (stream=0x7fa8e453c220,
data=0x7fa8e4538000 "( log ( ( 0: ) ( 10701 ) ( 10801 ) true false 0 true revprops ( 10:svn:author 8:svn:date 7:svn:log ) ) ) 168.116.222/Repository/Project/branches/feature-12 32:SVN/1.10.4 (x86_64-pc-linux-gnu) ( ) ) ", len=0x7fffec9fbfb8) at ./subversion/libsvn_ra_svn/streams.c:210
#6 0x00007fa8e56914e7 in readbuf_input (conn=0x7fa8e4534000,
data=0x7fa8e4538000 "( log ( ( 0: ) ( 10701 ) ( 10801 ) true false 0 true revprops ( 10:svn:author 8:svn:date 7:svn:log ) ) ) 168.116.222/Repository/Project/branches/feature-12 32:SVN/1.10.4 (x86_64-pc-linux-gnu) ( ) ) ", len=0x7fffec9fbfb8, pool=0x7fa8e453d028) at ./subversion/libsvn_ra_svn/marshal.c:508
#7 0x00007fa8e56917be in readbuf_fill (conn=0x7fa8e4534000, pool=0x7fa8e453d028) at ./subversion/libsvn_ra_svn/marshal.c:565
#8 0x00007fa8e5691877 in readbuf_getchar (result=0x7fffec9fc07f "", pool=0x7fa8e453d028, conn=0x7fa8e4534000) at ./subversion/libsvn_ra_svn/marshal.c:579
#9 readbuf_getchar_skip_whitespace (conn=0x7fa8e4534000, pool=0x7fa8e453d028, result=0x7fffec9fc07f "") at ./subversion/libsvn_ra_svn/marshal.c:589
#10 0x00007fa8e569477c in svn_ra_svn__read_item (conn=0x7fa8e4534000, pool=0x7fa8e453d028, item=0x7fffec9fc0c8) at ./subversion/libsvn_ra_svn/marshal.c:1562
#11 0x00007fa8e569549a in svn_ra_svn__read_tuple (conn=0x7fa8e4534000, pool=0x7fa8e453d028, fmt=0x7fa8e569c884 "wl") at ./subversion/libsvn_ra_svn/marshal.c:1742
#12 0x00007fa8e5695c30 in svn_ra_svn__handle_command (terminate=0x7fffec9fc27c, cmd_hash=0x7fa8e453f2c0, baton=0x7fa8e453c250, conn=0x7fa8e4534000, error_on_disconnect=0, pool=0x7fa8e453d028)
at ./subversion/libsvn_ra_svn/marshal.c:1924
#13 0x00005652e290b045 in serve_interruptable (terminate_p=0x0, connection=0x7fa8e453f0a0, is_busy=0x0, pool=0x7fa8e453f028) at ./subversion/svnserve/serve.c:4410
#14 0x00005652e290b615 in serve_socket (connection=0x7fa8e453f0a0, pool=0x7fa8e453f028) at ./subversion/svnserve/svnserve.c:599
#15 0x00005652e290ce9b in sub_main (exit_code=0x7fffec9fc584, argc=14, argv=0x7fffec9fc688, pool=0x7fa8e57d8028) at ./subversion/svnserve/svnserve.c:1356
#16 0x00005652e290d062 in main (argc=14, argv=0x7fffec9fc688) at ./subversion/svnserve/svnserve.c:1411
(gdb)
=====================
Below the client backtrace:
root@svnwc1:/home/vagrant# which svn
/usr/bin/svn
root@svnwc1:/home/vagrant# file /usr/bin/svn
/usr/bin/svn: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=6c4d8ac322b27d2da5807a1eb045342cced23973, with debug_info, not stripped
root@svnwc1:/home/vagrant# svn --version
svn, version 1.10.4 (r1850624)
compiled Feb 10 2021, 20:15:45 on x86_64-pc-linux-gnu
Copyright (C) 2019 The Apache Software Foundation.
This software consists of contributions made by many people;
see the NOTICE file for more information.
Subversion is open source software, see http://subversion.apache.org/
The following repository access (RA) modules are available:
* ra_svn : Module for accessing a repository using the svn network protocol.
- with Cyrus SASL authentication
- handles 'svn' scheme
* ra_local : Module for accessing a repository on local disk.
- handles 'file' scheme
* ra_serf : Module for accessing a repository via WebDAV protocol using serf.
- using serf 1.3.9 (compiled with 1.3.9)
- handles 'http' scheme
- handles 'https' scheme
The following authentication credential caches are available:
* Plaintext cache in /root/.subversion
* Gnome Keyring
* GPG-Agent
* KWallet (KDE)
root@svnwc1:/home/vagrant# uname -a
Linux svnwc1 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1+deb10u1 (2020-04-27) x86_64 GNU/Linux
root@svnwc1:/home/vagrant# ps ax | grep svn
109472 pts/0 S+ 0:01 /usr/bin/svn log -g -v -r 10701:10801 /path/to/workingcopy/of/feature-12
109475 pts/1 S+ 0:00 grep svn
root@svnwc1:/home/vagrant# gdb /usr/bin/svn 109472
GNU gdb (Debian 8.2.1-2+b3) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/svn...done.
Attaching to program: /usr/bin/svn, process 109472
Reading symbols from /lib/x86_64-linux-gnu/libsvn_client-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_wc-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_delta-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_diff-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_subr-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libaprutil-1.so.0...Reading symbols from /usr/lib/debug/.build-id/9b/d595ade2a5191a525571848e604912d0d4d9f2.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libapr-1.so.0...Reading symbols from /usr/lib/debug/.build-id/2b/d8bdd5984b745e54e0144c6cb3a52319149669.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading symbols from /usr/lib/debug/.build-id/e9/1114987a0147bd050addbd591eb8994b29f4b3.debug...done.
done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from /usr/lib/debug/.build-id/18/b9a9a8c523e5cfe5b5d946d605d09242f09798.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_local-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_repos-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_serf-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libz.so.1...Reading symbols from /usr/lib/debug/.build-id/3a/f7c4bceb19b6c83f76e2822b9a23041d85f6d1.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libexpat.so.1...Reading symbols from /usr/lib/debug/.build-id/9a/27bd3a88235268c02ed62ce01274a059fbb41d.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libsqlite3.so.0...Reading symbols from /usr/lib/debug/.build-id/de/5b665b816f5fee97547a83709c017f13265bc6.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/liblz4.so.1...Reading symbols from /usr/lib/debug/.build-id/85/51af4090d0b66af3a66e3848557aa3aa789ae2.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libutf8proc.so.2...Reading symbols from /usr/lib/debug/.build-id/fe/46dc620bf7046edea84ac41a4a20f56e31c877.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libuuid.so.1...Reading symbols from /usr/lib/debug/.build-id/09/6caf62fb91c7beffc5f53e0b315229a5b22c20.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/librt.so.1...Reading symbols from /usr/lib/debug/.build-id/5d/cf98ad684962be494af28a1051793fd39e4ebc.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libcrypt.so.1...Reading symbols from /usr/lib/debug/.build-id/c6/966ba263bacba7f67c23643ba373aa51a162e9.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from /usr/lib/debug/.build-id/d3/583c742dd47aaa860c5ae0c0c5bdbcd2d54f61.debug...done.
done.
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug/.build-id/f2/5dfd7b95be4ba386fd71080accae8c0732b711.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_fs-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_x-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_base-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_util-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsasl2.so.2...Reading symbols from /usr/lib/debug/.build-id/99/bf5a225908fd4124228d4f3e19c67d7138144f.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libserf-1.so.1...Reading symbols from /usr/lib/debug/.build-id/eb/bed04d3e14fd0204e244f6621ee3ce1dfd84f9.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...Reading symbols from /usr/lib/debug/.build-id/88/5dda4b4a5cea600e7b5b98c1ad86996c8d2299.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libdb-5.3.so...Reading symbols from /usr/lib/debug/.build-id/67/4e2dc8ff2d056a41d25065ec098d81392952a3.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libresolv.so.2...Reading symbols from /usr/lib/debug/.build-id/02/6c3ba167f64f631eb8781fca2269fbc2ee7ca5.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libssl.so.1.1...Reading symbols from /usr/lib/debug/.build-id/7d/f8c588a7ca055565fe8bd49a604746fb2eb1ae.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libcrypto.so.1.1...Reading symbols from /usr/lib/debug/.build-id/74/9142d5ee728a76e7cdc61fd79d2311a77405a2.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libgssapi_krb5.so.2...Reading symbols from /usr/lib/debug/.build-id/fc/25787cdec113023f7f3ef1a85c44c2b96e7de5.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libkrb5.so.3...Reading symbols from /usr/lib/debug/.build-id/a8/78f36044ab31951f1a275f2134757c5854b87c.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libk5crypto.so.3...Reading symbols from /usr/lib/debug/.build-id/be/410728ca057034fc08978fc156b9f0553d7d71.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libcom_err.so.2...Reading symbols from /usr/lib/debug/.build-id/df/fd546cdf7248805473c118886139f88bf01415.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libkrb5support.so.0...Reading symbols from /usr/lib/debug/.build-id/c5/4ef3584f013660152971ae32e58d4078bb80be.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libkeyutils.so.1...Reading symbols from /usr/lib/debug/.build-id/b3/3b7f30aea5d2bc14a939fa750862d09a4ac80e.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_auth_gnome_keyring-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libsecret-1.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgio-2.0.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgobject-2.0.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libglib-2.0.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgcrypt.so.20...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgmodule-2.0.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libmount.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libselinux.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libffi.so.6...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libpcre.so.3...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgpg-error.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libblkid.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_auth_kwallet-1.so.1...done.
Reading symbols from /lib/x86_64-linux-gnu/libdbus-1.so.3...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libKF5Wallet.so.5...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libKF5I18n.so.5...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libKF5CoreAddons.so.5...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libQt5Gui.so.5...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libQt5DBus.so.5...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libQt5Core.so.5...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libstdc++.so.6...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgcc_s.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsystemd.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libKF5WindowSystem.so.5...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libKF5ConfigCore.so.5...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libfam.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libGL.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libpng16.so.16...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libharfbuzz.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libicui18n.so.63...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libicuuc.so.63...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libpcre2-16.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libdouble-conversion.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/liblzma.so.5...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libQt5Widgets.so.5...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libQt5X11Extras.so.5...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libX11.so.6...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libxcb.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libxcb-keysyms.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libGLX.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libGLdispatch.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libfreetype.so.6...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgraphite2.so.3...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libicudata.so.63...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libXau.so.6...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libXdmcp.so.6...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libXext.so.6...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libbsd.so.0...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libsasldb.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libanonymous.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libcrammd5.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/liblogin.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libntlm.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libplain.so...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libnss_files.so.2...Reading symbols from /usr/lib/debug/.build-id/4b/ff8b782e1602c596e856bdef06e642e50e7fa7.debug...done.
done.
0x00007fc597ff550e in __libc_read (fd=4, buf=buf@entry=0x7fc592111000, nbytes=16384) at ../sysdeps/unix/sysv/linux/read.c:26
26 ../sysdeps/unix/sysv/linux/read.c: No such file or directory.
(gdb) bt
#0 0x00007fc597ff550e in __libc_read (fd=4, buf=buf@entry=0x7fc592111000, nbytes=16384) at ../sysdeps/unix/sysv/linux/read.c:26
#1 0x00007fc59802837f in read (__nbytes=<optimized out>, __buf=0x7fc592111000, __fd=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/unistd.h:44
#2 0x00007fc59802837f in apr_socket_recv
(sock=0x7fc596bfb9a0, buf=0x7fc592111000 "76:/Project/trunk/somefolder-a/redacted-filename-path/redacted-filename-abc.txt A ( ) ( 4:file true true ) ) ( 53:/Project/trunk/somefolder-b/redacted-filename-def.txt M ( ) ( 4:file true false ) ) ( "..., len=0x7ffe2b2eee78) at ./network_io/unix/sendrecv.c:81
#3 0x00007fc597dac19d in sock_read_cb
(baton=0x7fc5921151b8, buffer=0x7fc592111000 "76:/Project/trunk/somefolder-a/redacted-filename-path/redacted-filename-abc.txt A ( ) ( 4:file true true ) ) ( 53:/Project/trunk/somefolder-b/redacted-filename-def.txt M ( ) ( 4:file true false ) ) ( "..., len=0x7ffe2b2eee78) at ./subversion/libsvn_ra_svn/streams.c:119
#4 0x00007fc5980d56dc in svn_stream_read2
(stream=0x7fc5921151c8, buffer=0x7fc592111000 "76:/Project/trunk/somefolder-a/redacted-filename-path/redacted-filename-abc.txt A ( ) ( 4:file true true ) ) ( 53:/Project/trunk/somefolder-b/redacted-filename-def.txt M ( ) ( 4:file true false ) ) ( "..., len=0x7ffe2b2eee78) at ./subversion/libsvn_subr/stream.c:185
#5 0x00007fc597dac4b8 in svn_ra_svn__stream_read
(stream=0x7fc592115220, data=0x7fc592111000 "76:/Project/trunk/somefolder-a/redacted-filename-path/redacted-filename-abc.txt A ( ) ( 4:file true true ) ) ( 53:/Project/trunk/somefolder-b/redacted-filename-def.txt M ( ) ( 4:file true false ) ) ( "..., len=0x7ffe2b2eee78) at ./subversion/libsvn_ra_svn/streams.c:210
#6 0x00007fc597da34e7 in readbuf_input
(conn=0x7fc59210d000, data=0x7fc592111000 "76:/Project/trunk/somefolder-a/redacted-filename-path/redacted-filename-abc.txt A ( ) ( 4:file true true ) ) ( 53:/Project/trunk/somefolder-b/redacted-filename-def.txt M ( ) ( 4:file true false ) ) ( "..., len=0x7ffe2b2eee78, pool=0x7fc592108028)
at ./subversion/libsvn_ra_svn/marshal.c:508
#7 0x00007fc597da37be in readbuf_fill (conn=0x7fc59210d000, pool=0x7fc592108028) at ./subversion/libsvn_ra_svn/marshal.c:565
#8 0x00007fc597da3877 in readbuf_getchar (result=0x7ffe2b2eef3f "(", pool=0x7fc592108028, conn=0x7fc59210d000) at ./subversion/libsvn_ra_svn/marshal.c:579
#9 0x00007fc597da3877 in readbuf_getchar_skip_whitespace (conn=0x7fc59210d000, pool=0x7fc592108028, result=0x7ffe2b2eef3f "(")
at ./subversion/libsvn_ra_svn/marshal.c:589
#10 0x00007fc597da5ccc in read_item (conn=0x7fc59210d000, pool=0x7fc592108028, item=0x7ffe2b2ef2b0, first_char=40 '(', level=2)
at ./subversion/libsvn_ra_svn/marshal.c:1416
#11 0x00007fc597da5de1 in read_item (conn=0x7fc59210d000, pool=0x7fc592108028, item=0x7fc5921080a0, first_char=40 '(', level=1)
at ./subversion/libsvn_ra_svn/marshal.c:1433
#12 0x00007fc597da67b1 in svn_ra_svn__read_item (conn=0x7fc59210d000, pool=0x7fc592108028, item=0x7ffe2b2ef500) at ./subversion/libsvn_ra_svn/marshal.c:1563
#13 0x00007fc597d96fd2 in perform_ra_svn_log
--Type <RET> for more, q to quit, c to continue without paging--
(outer_error=0x7ffe2b2ef6c0, session=0x7fc596c0a238, paths=0x7fc596c08190, start=10701, end=10801, limit=0, discover_changed_paths=1, strict_node_history=0, include_merged_revisions=1, revprops=0x7fc596c0c5a8, receiver=0x557e345734d3 <svn_cl__log_entry_receiver>, receiver_baton=0x7ffe2b2efb60, pool=0x7fc596c08028)
at ./subversion/libsvn_ra_svn/client.c:1929
#14 0x00007fc597d9777a in ra_svn_log
(session=0x7fc596c0a238, paths=0x7fc596c08190, start=10701, end=10801, limit=0, discover_changed_paths=1, strict_node_history=0, include_merged_revisions=1, revprops=0x7fc596c0c5a8, receiver=0x557e345734d3 <svn_cl__log_entry_receiver>, receiver_baton=0x7ffe2b2efb60, pool=0x7fc596c08028)
at ./subversion/libsvn_ra_svn/client.c:2090
#15 0x00007fc59816a8ea in svn_ra_get_log2
(session=0x7fc596c0a238, paths=0x7fc596c0c5e0, start=10701, end=10801, limit=0, discover_changed_paths=1, strict_node_history=0, include_merged_revisions=1, revprops=0x7fc596c0c5a8, receiver=0x557e345734d3 <svn_cl__log_entry_receiver>, receiver_baton=0x7ffe2b2efb60, pool=0x7fc596c08028) at ./subversion/libsvn_ra/ra_loader.c:822
#16 0x00007fc5982b631f in run_ra_get_log
(revision_ranges=0x7fc592105500, paths=0x7fc596c0c5e0, log_segments=0x7fc5921056c0, actual_loc=0x7fc5921055b8, ra_session=0x7fc596c0a238, targets=0x7fc596c0c3b8, limit=0, discover_changed_paths=1, strict_node_history=0, include_merged_revisions=1, revprops=0x7fc596c0c5a8, real_receiver=0x557e345734d3 <svn_cl__log_entry_receiver>, real_receiver_baton=0x7ffe2b2efb60, ctx=0x7fc598313440, scratch_pool=0x7fc598312028) at ./subversion/libsvn_client/log.c:809
#17 0x00007fc5982b6811 in svn_client_log5
(targets=0x7fc596c0c3b8, peg_revision=0x7ffe2b2efb70, opt_rev_ranges=0x7fc5983124f8, limit=0, discover_changed_paths=1, strict_node_history=0, include_merged_revisions=1, revprops=0x7fc596c0c5a8, real_receiver=0x557e345734d3 <svn_cl__log_entry_receiver>, real_receiver_baton=0x7ffe2b2efb60, ctx=0x7fc598313440, pool=0x7fc598312028) at ./subversion/libsvn_client/log.c:943
#18 0x0000557e345751a9 in svn_cl__log (os=0x7fc598312520, baton=0x7ffe2b2efe30, pool=0x7fc598312028) at ./subversion/svn/log-cmd.c:885
#19 0x0000557e345872fd in sub_main (exit_code=0x7ffe2b2f0144, argc=7, argv=0x7ffe2b2f0248, pool=0x7fc598312028) at ./subversion/svn/svn.c:3247
#20 0x0000557e345875c2 in main (argc=7, argv=0x7ffe2b2f0248) at ./subversion/svn/svn.c:3332
(gdb)
Cheers,
Attila
Re: svn log xml hangs and produces too many logentry closing tags
Posted by Daniel Sahlberg <da...@gmail.com>.
Den lör 17 juli 2021 kl 15:02 skrev Attila <at...@gmx.net>:
> On 14 Jul 2021, at 21:42, Nathan Hartman <ha...@gmail.com> wrote:
>
>
> On Tue, Jul 13, 2021 at 5:49 PM Attila <at...@gmx.net> wrote:
>
>
> Hi
>
> I have a problem getting the svn log in a branch after sync-merging a
> commit from trunk.
> This commit in trunk is a merge of an old and complex branch with many
> commits.
>
> The client accessing the repository over svn:// url.
> (paths and text is redacted)
> The head revision is: 10801
>
> When I run the following command on the client (in the working copy), it
> prints a long partial xml-log output, then hangs.
> /usr/bin/svn log --xml -g -v -r 10701:HEAD /path/to/branch-wc
>
> When observing in "top", the command uses no visible CPU resources on
> hang. (I waited ca. 2 minutes)
> The hanging command does mot exits on CTRL-c, it does not exits on "kill
> -TERM pid", I have to send "kill -KILL pid" to terminate it.
>
> When I run the command with strace it hangs at read(4,
> ...SNIP...
> read(4, " ( ) ( 4:file true false ) ) ( 1"..., 16384) = 16384
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f63f1d83000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f63f1d81000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f63f1d7f000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f63f1d7d000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f63f1d7b000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f63f1d79000
> read(4, "***/****-***_redacted_*****_****"..., 16384) = 14773
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f63f1d77000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f63f1d75000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f63f1d73000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f63f1d71000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f63f1d6f000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f63f1d6d000
> read(4,
>
> When I observe the server, there is a CPU activity at the begin, but when
> the client hangs, the server seems to be in idle.
> Just a corresponding svnserve process is there with no visible cpu usage.
> In svnserve.log is nothing relevant to see.
>
> The svnserve command is:
> svnserve -d -r /path/to/repositories \
> --log-file=/var/log/svnserve.log \
> --memory-cache-size 1024 \
> --cache-txdeltas yes \
> --cache-fulltexts yes
>
> When I try to get the xml-log on the server with the corresponding file://
> repository URL:
> /usr/bin/svn log --xml -g -v -r 10701:HEAD
> file://path/to/local/repositories/project/branch
> The command finishes in ca 5-10 seconds and I get the xml output, but the
> output has a way too many </logentry> lines.
>
> There are 1217 occurrences of the string “<logentry" and 33191 occurrences
> of the string "</logentry>" in the output xml.
> There are several thousand lines of </logentry> in a row in many places in
> repeated blocks.
>
> Details:
> Client and Server OS:
> Linux 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1+deb10u1 (2020-04-27) x86_64
> GNU/Linux
>
> The repository is ca. 4 GB.
> Running "svnadmin verify" on the server founds no errors.
> I have no other problems with the server, checkout and commit works normal.
>
> svn --version
> svn, version 1.10.4 (r1850624)
> compiled Feb 10 2021, 20:15:45 on x86_64-pc-linux-gnu
>
> Copyright (C) 2019 The Apache Software Foundation.
> This software consists of contributions made by many people;
> see the NOTICE file for more information.
> Subversion is open source software, see http://subversion.apache.org/
>
> The following repository access (RA) modules are available:
>
> * ra_svn : Module for accessing a repository using the svn network
> protocol.
> - with Cyrus SASL authentication
> - handles 'svn' scheme
> * ra_local : Module for accessing a repository on local disk.
> - handles 'file' scheme
> * ra_serf : Module for accessing a repository via WebDAV protocol using
> serf.
> - using serf 1.3.9 (compiled with 1.3.9)
> - handles 'http' scheme
> - handles 'https' scheme
>
> The following authentication credential caches are available:
>
> * Plaintext cache in /home/username/.subversion
> * Gnome Keyring
> * GPG-Agent
> * KWallet (KDE)
>
>
>
> Hello Attila,
>
> Thanks for this detailed explanation. There are quite a few important
> clues here.
>
> To help narrow down the search for the culprit: Are you able to run
> the same 'svn log' command against the same working copy, but without
> the '--xml', and get a correct output in a reasonable amount of time?
>
>
> Hi Nathan
>
> thanks for the suggestions.
>
> The command without the --xml parameter hangs on the client and works on
> the server.
> The text output on the server seems be ok.
> This suggests that the hang and the closing xml tags are two unrelated
> problems.
>
Hi Attila,
Thanks for the detailed explaination!
I agree these are two different problems. More below
> If log without '--xml' works and log with '--xml' gets stuck, is it
> feasible to try to bisect the revision range, e.g., run log on the
> range r10701:10751, and in a separate invocation on r10751:10801, and
>
>
> I tried that, it seems that the problem was not the last sync-merge commit.
> The command succeeds when there is no sync-merge commit in the range.
> There are two sync-merges in the range the first one is a small merge of
> 3-4 changed files. These 3-4 files are unmodified in the branch.
> Whenever i include one of these sync-merges in the range, the command
> hangs.
>
What if you run the command on the whole range but without '-g'? I'm
expecting that should work, both with and without '--xml'.
try to see if there's a revision in particular where SVN is getting
> confused? If so, then perhaps knowing a little more about that
> revision could help narrow it down.
>
> By the way, is this (1.10.4) the SVN version on the server, the
> client, or both?
>
>
> both.
> The server and client is Debian Buster and I have updated both to the
> latest version hoping it will help.
> The installed subversion package is:
> subversion/stable,stable,now 1.10.4-1+deb10u2 amd64 [installed]
>
> The client is a private vagrant vm running on macOS in VMware Fusion pro,
> the server is an ESXi vm.
>
> Googling around gives me these two somewhat related hits:
> https://issues.apache.org/jira/browse/SVN-4856
> https://issues.apache.org/jira/browse/SVN-4711
>
> But I do'nt use the --search parameter.
>
>
> It could nevertheless be related. The spurious XML closing tags seems
> to ring a bell.
>
>
I have been chasing svn log --xml for the last two days.
As far as I understand, both issues are caused by the -g (or
--use-merge-history) argument. This forces the code to recursively consider
the log of the merge history.
In #4711 it is the --search filtering that causes the client to emit a
</logentry> tag even when it didn't emit the corresponding <logentry> tag.
I believe I have found a solution., details in
http://mail-archives.apache.org/mod_mbox/subversion-dev/202108.mbox/%3C8215e3c9-2b81-c703-eaf9-4199ada8e485%40gmail.com%3E.
When this is committed, I'm planning to go after 4856.
In #4856 it seems that the library is issuing the callback for the closing
tag (</logentry>) even when it shouldn't. The same issue would affects the
log in plain text format, but since there is no closing tag, it is not
visible in the output. It is fairly easy to produce the issue of merging in
multiple steps.
Is this a bug or are there any suggestions how to solve this problem?
>
>
> It sounds like a bug. My suggestions above.
>
>
> Here is a preliminary backtrace of the hanging client (without debug
> symbols for now)
> I need some time to spin up a copy of the server and the client and
> re-compile svn with debug symbols.
> I will come back with more details as soon as i have the time to do more
> tests.
>
> # gdb /usr/bin/svn 1399
> GNU gdb (Debian 8.2.1-2+b3) 8.2.1
> Copyright (C) 2018 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> Type "show copying" and "show warranty" for details.
> This GDB was configured as "x86_64-linux-gnu".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>.
> Find the GDB manual and other documentation resources online at:
> <http://www.gnu.org/software/gdb/documentation/>.
>
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> Reading symbols from /usr/bin/svn...(no debugging symbols found)...done.
> Attaching to program: /usr/bin/svn, process 1399
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_client-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_wc-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_delta-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_diff-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_subr-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libaprutil-1.so.0...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libapr-1.so.0...(no debugging
> symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading
> symbols from
> /usr/lib/debug/.build-id/e9/1114987a0147bd050addbd591eb8994b29f4b3.debug...done.
> done.
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
> Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols
> from
> /usr/lib/debug/.build-id/18/b9a9a8c523e5cfe5b5d946d605d09242f09798.debug...done.
> done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_local-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_repos-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_serf-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libz.so.1...(no debugging
> symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libexpat.so.1...(no debugging
> symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsqlite3.so.0...(no debugging
> symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/liblz4.so.1...(no debugging
> symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libutf8proc.so.2...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libuuid.so.1...(no debugging
> symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/librt.so.1...Reading symbols
> from
> /usr/lib/debug/.build-id/5d/cf98ad684962be494af28a1051793fd39e4ebc.debug...done.
> done.
> Reading symbols from /lib/x86_64-linux-gnu/libcrypt.so.1...Reading symbols
> from
> /usr/lib/debug/.build-id/c6/966ba263bacba7f67c23643ba373aa51a162e9.debug...done.
> done.
> Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols
> from
> /usr/lib/debug/.build-id/d3/583c742dd47aaa860c5ae0c0c5bdbcd2d54f61.debug...done.
> done.
> Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from
> /usr/lib/debug/.build-id/f2/5dfd7b95be4ba386fd71080accae8c0732b711.debug...done.
> done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_fs-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_x-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_base-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_util-1.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libsasl2.so.2...(no debugging
> symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libserf-1.so.1...(no debugging
> symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...Reading symbols
> from
> /usr/lib/debug/.build-id/88/5dda4b4a5cea600e7b5b98c1ad86996c8d2299.debug...done.
> done.
> Reading symbols from /lib/x86_64-linux-gnu/libdb-5.3.so...(no debugging
> symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libresolv.so.2...Reading
> symbols from
> /usr/lib/debug/.build-id/02/6c3ba167f64f631eb8781fca2269fbc2ee7ca5.debug...done.
> done.
> Reading symbols from /lib/x86_64-linux-gnu/libssl.so.1.1...(no debugging
> symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libcrypto.so.1.1...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libgssapi_krb5.so.2...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libkrb5.so.3...(no debugging
> symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libk5crypto.so.3...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libcom_err.so.2...(no debugging
> symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libkrb5support.so.0...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libkeyutils.so.1...(no
> debugging symbols found)...done.
> Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libsasldb.so...(no
> debugging symbols found)...done.
> Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libanonymous.so...(no
> debugging symbols found)...done.
> Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libcrammd5.so...(no
> debugging symbols found)...done.
> Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so...(no
> debugging symbols found)...done.
> Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/liblogin.so...(no
> debugging symbols found)...done.
> Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libntlm.so...(no
> debugging symbols found)...done.
> Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libplain.so...(no
> debugging symbols found)...done.
> Reading symbols from /lib/x86_64-linux-gnu/libnss_files.so.2...Reading
> symbols from
> /usr/lib/debug/.build-id/4b/ff8b782e1602c596e856bdef06e642e50e7fa7.debug...done.
> done.
> Reading symbols from /lib/x86_64-linux-gnu/libnss_dns.so.2...Reading
> symbols from
> /usr/lib/debug/.build-id/bb/8b756d3859e8fe20d7aa6d8255f27942589f47.debug...done.
> done.
> 0x00007f276a1fc50e in __libc_read (fd=4, buf=0x7f2768e19000, nbytes=16384)
> at ../sysdeps/unix/sysv/linux/read.c:26
> 26 ../sysdeps/unix/sysv/linux/read.c: No such file or directory.
> (gdb) bt
> #0 0x00007f276a1fc50e in __libc_read (fd=4, buf=0x7f2768e19000,
> nbytes=16384) at ../sysdeps/unix/sysv/linux/read.c:26
> #1 0x00007f276a22f37f in apr_socket_recv () from
> /lib/x86_64-linux-gnu/libapr-1.so.0
> #2 0x00007f2769fc1353 in ?? () from
> /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #3 0x00007f2769fc158c in svn_ra_svn.stream_read () from
> /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #4 0x00007f2769fbb242 in ?? () from
> /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #5 0x00007f2769fbb61a in ?? () from
> /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #6 0x00007f2769fbb68b in ?? () from
> /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #7 0x00007f2769fbbc7f in ?? () from
> /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #8 0x00007f2769fbbcbe in ?? () from
> /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #9 0x00007f2769fbde67 in svn_ra_svn.read_item () from
> /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #10 0x00007f2769fb583a in ?? () from
> /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
> #11 0x00007f276a46033a in svn_client_log5 () from
> /lib/x86_64-linux-gnu/libsvn_client-1.so.1
> #12 0x000055f9507b6779 in ?? ()
> #13 0x000055f9507c2541 in ?? ()
> #14 0x000055f9507a7513 in ?? ()
> #15 0x00007f276a04e09b in __libc_start_main (main=0x55f9507a7490, argc=7,
> argv=0x7ffee0fc23c8, init=<optimized out>, fini=<optimized out>,
> rtld_fini=<optimized out>, stack_end=0x7ffee0fc23b8) at
> ../csu/libc-start.c:308
> #16 0x000055f9507a759a in ?? ()
> (gdb)
>
Any chance to do the same on the server?
Can you attach to the repository in any other way (http/https or file) to
verify if the error is in svnserve or in the repository code?
Kind regards
Daniel Sahlberg
Re: svn log xml hangs and produces too many logentry closing tags
Posted by Attila <at...@gmx.net>.
On 14 Jul 2021, at 21:42, Nathan Hartman <ha...@gmail.com> wrote:
>
> On Tue, Jul 13, 2021 at 5:49 PM Attila <atiware@gmx.net <ma...@gmx.net>> wrote:
>>
>> Hi
>>
>> I have a problem getting the svn log in a branch after sync-merging a commit from trunk.
>> This commit in trunk is a merge of an old and complex branch with many commits.
>>
>> The client accessing the repository over svn:// url.
>> (paths and text is redacted)
>> The head revision is: 10801
>>
>> When I run the following command on the client (in the working copy), it prints a long partial xml-log output, then hangs.
>> /usr/bin/svn log --xml -g -v -r 10701:HEAD /path/to/branch-wc
>>
>> When observing in "top", the command uses no visible CPU resources on hang. (I waited ca. 2 minutes)
>> The hanging command does mot exits on CTRL-c, it does not exits on "kill -TERM pid", I have to send "kill -KILL pid" to terminate it.
>>
>> When I run the command with strace it hangs at read(4,
>> ...SNIP...
>> read(4, " ( ) ( 4:file true false ) ) ( 1"..., 16384) = 16384
>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d83000
>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d81000
>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7f000
>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7d000
>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7b000
>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d79000
>> read(4, "***/****-***_redacted_*****_****"..., 16384) = 14773
>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d77000
>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d75000
>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d73000
>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d71000
>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d6f000
>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d6d000
>> read(4,
>>
>> When I observe the server, there is a CPU activity at the begin, but when the client hangs, the server seems to be in idle.
>> Just a corresponding svnserve process is there with no visible cpu usage.
>> In svnserve.log is nothing relevant to see.
>>
>> The svnserve command is:
>> svnserve -d -r /path/to/repositories \
>> --log-file=/var/log/svnserve.log \
>> --memory-cache-size 1024 \
>> --cache-txdeltas yes \
>> --cache-fulltexts yes
>>
>> When I try to get the xml-log on the server with the corresponding file:// repository URL:
>> /usr/bin/svn log --xml -g -v -r 10701:HEAD file://path/to/local/repositories/project/branch
>> The command finishes in ca 5-10 seconds and I get the xml output, but the output has a way too many </logentry> lines.
>>
>> There are 1217 occurrences of the string “<logentry" and 33191 occurrences of the string "</logentry>" in the output xml.
>> There are several thousand lines of </logentry> in a row in many places in repeated blocks.
>>
>> Details:
>> Client and Server OS:
>> Linux 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1+deb10u1 (2020-04-27) x86_64 GNU/Linux
>>
>> The repository is ca. 4 GB.
>> Running "svnadmin verify" on the server founds no errors.
>> I have no other problems with the server, checkout and commit works normal.
>>
>> svn --version
>> svn, version 1.10.4 (r1850624)
>> compiled Feb 10 2021, 20:15:45 on x86_64-pc-linux-gnu
>>
>> Copyright (C) 2019 The Apache Software Foundation.
>> This software consists of contributions made by many people;
>> see the NOTICE file for more information.
>> Subversion is open source software, see http://subversion.apache.org/
>>
>> The following repository access (RA) modules are available:
>>
>> * ra_svn : Module for accessing a repository using the svn network protocol.
>> - with Cyrus SASL authentication
>> - handles 'svn' scheme
>> * ra_local : Module for accessing a repository on local disk.
>> - handles 'file' scheme
>> * ra_serf : Module for accessing a repository via WebDAV protocol using serf.
>> - using serf 1.3.9 (compiled with 1.3.9)
>> - handles 'http' scheme
>> - handles 'https' scheme
>>
>> The following authentication credential caches are available:
>>
>> * Plaintext cache in /home/username/.subversion
>> * Gnome Keyring
>> * GPG-Agent
>> * KWallet (KDE)
>
>
> Hello Attila,
>
> Thanks for this detailed explanation. There are quite a few important
> clues here.
>
> To help narrow down the search for the culprit: Are you able to run
> the same 'svn log' command against the same working copy, but without
> the '--xml', and get a correct output in a reasonable amount of time?
Hi Nathan
thanks for the suggestions.
The command without the --xml parameter hangs on the client and works on the server.
The text output on the server seems be ok.
This suggests that the hang and the closing xml tags are two unrelated problems.
>
> If log without '--xml' works and log with '--xml' gets stuck, is it
> feasible to try to bisect the revision range, e.g., run log on the
> range r10701:10751, and in a separate invocation on r10751:10801, and
I tried that, it seems that the problem was not the last sync-merge commit.
The command succeeds when there is no sync-merge commit in the range.
There are two sync-merges in the range the first one is a small merge of 3-4 changed files. These 3-4 files are unmodified in the branch.
Whenever i include one of these sync-merges in the range, the command hangs.
> try to see if there's a revision in particular where SVN is getting
> confused? If so, then perhaps knowing a little more about that
> revision could help narrow it down.
>
> By the way, is this (1.10.4) the SVN version on the server, the
> client, or both?
>
both.
The server and client is Debian Buster and I have updated both to the latest version hoping it will help.
The installed subversion package is:
subversion/stable,stable,now 1.10.4-1+deb10u2 amd64 [installed]
The client is a private vagrant vm running on macOS in VMware Fusion pro, the server is an ESXi vm.
>> Googling around gives me these two somewhat related hits:
>> https://issues.apache.org/jira/browse/SVN-4856 <https://issues.apache.org/jira/browse/SVN-4856>
>> https://issues.apache.org/jira/browse/SVN-4711 <https://issues.apache.org/jira/browse/SVN-4711>
>>
>> But I do'nt use the --search parameter.
>
> It could nevertheless be related. The spurious XML closing tags seems
> to ring a bell.
>
>> Is this a bug or are there any suggestions how to solve this problem?
>
> It sounds like a bug. My suggestions above.
Here is a preliminary backtrace of the hanging client (without debug symbols for now)
I need some time to spin up a copy of the server and the client and re-compile svn with debug symbols.
I will come back with more details as soon as i have the time to do more tests.
# gdb /usr/bin/svn 1399
GNU gdb (Debian 8.2.1-2+b3) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/svn...(no debugging symbols found)...done.
Attaching to program: /usr/bin/svn, process 1399
Reading symbols from /lib/x86_64-linux-gnu/libsvn_client-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_wc-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_delta-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_diff-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_subr-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libaprutil-1.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libapr-1.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading symbols from /usr/lib/debug/.build-id/e9/1114987a0147bd050addbd591eb8994b29f4b3.debug...done.
done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from /usr/lib/debug/.build-id/18/b9a9a8c523e5cfe5b5d946d605d09242f09798.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_local-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_repos-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_ra_serf-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libz.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libexpat.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsqlite3.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/liblz4.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libutf8proc.so.2...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libuuid.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/librt.so.1...Reading symbols from /usr/lib/debug/.build-id/5d/cf98ad684962be494af28a1051793fd39e4ebc.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libcrypt.so.1...Reading symbols from /usr/lib/debug/.build-id/c6/966ba263bacba7f67c23643ba373aa51a162e9.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from /usr/lib/debug/.build-id/d3/583c742dd47aaa860c5ae0c0c5bdbcd2d54f61.debug...done.
done.
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug/.build-id/f2/5dfd7b95be4ba386fd71080accae8c0732b711.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_fs-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_x-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_base-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsvn_fs_util-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsasl2.so.2...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libserf-1.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...Reading symbols from /usr/lib/debug/.build-id/88/5dda4b4a5cea600e7b5b98c1ad86996c8d2299.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libdb-5.3.so...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libresolv.so.2...Reading symbols from /usr/lib/debug/.build-id/02/6c3ba167f64f631eb8781fca2269fbc2ee7ca5.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libssl.so.1.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libcrypto.so.1.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgssapi_krb5.so.2...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libkrb5.so.3...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libk5crypto.so.3...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libcom_err.so.2...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libkrb5support.so.0...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libkeyutils.so.1...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libsasldb.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libanonymous.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libcrammd5.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/liblogin.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libntlm.so...(no debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/sasl2/libplain.so...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libnss_files.so.2...Reading symbols from /usr/lib/debug/.build-id/4b/ff8b782e1602c596e856bdef06e642e50e7fa7.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libnss_dns.so.2...Reading symbols from /usr/lib/debug/.build-id/bb/8b756d3859e8fe20d7aa6d8255f27942589f47.debug...done.
done.
0x00007f276a1fc50e in __libc_read (fd=4, buf=0x7f2768e19000, nbytes=16384) at ../sysdeps/unix/sysv/linux/read.c:26
26 ../sysdeps/unix/sysv/linux/read.c: No such file or directory.
(gdb) bt
#0 0x00007f276a1fc50e in __libc_read (fd=4, buf=0x7f2768e19000, nbytes=16384) at ../sysdeps/unix/sysv/linux/read.c:26
#1 0x00007f276a22f37f in apr_socket_recv () from /lib/x86_64-linux-gnu/libapr-1.so.0
#2 0x00007f2769fc1353 in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
#3 0x00007f2769fc158c in svn_ra_svn.stream_read () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
#4 0x00007f2769fbb242 in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
#5 0x00007f2769fbb61a in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
#6 0x00007f2769fbb68b in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
#7 0x00007f2769fbbc7f in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
#8 0x00007f2769fbbcbe in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
#9 0x00007f2769fbde67 in svn_ra_svn.read_item () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
#10 0x00007f2769fb583a in ?? () from /lib/x86_64-linux-gnu/libsvn_ra_svn-1.so.1
#11 0x00007f276a46033a in svn_client_log5 () from /lib/x86_64-linux-gnu/libsvn_client-1.so.1
#12 0x000055f9507b6779 in ?? ()
#13 0x000055f9507c2541 in ?? ()
#14 0x000055f9507a7513 in ?? ()
#15 0x00007f276a04e09b in __libc_start_main (main=0x55f9507a7490, argc=7, argv=0x7ffee0fc23c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffee0fc23b8) at ../csu/libc-start.c:308
#16 0x000055f9507a759a in ?? ()
(gdb)
Thanks
Attila
Re: svn log xml hangs and produces too many logentry closing tags
Posted by Nathan Hartman <ha...@gmail.com>.
On Tue, Jul 13, 2021 at 5:49 PM Attila <at...@gmx.net> wrote:
>
> Hi
>
> I have a problem getting the svn log in a branch after sync-merging a commit from trunk.
> This commit in trunk is a merge of an old and complex branch with many commits.
>
> The client accessing the repository over svn:// url.
> (paths and text is redacted)
> The head revision is: 10801
>
> When I run the following command on the client (in the working copy), it prints a long partial xml-log output, then hangs.
> /usr/bin/svn log --xml -g -v -r 10701:HEAD /path/to/branch-wc
>
> When observing in "top", the command uses no visible CPU resources on hang. (I waited ca. 2 minutes)
> The hanging command does mot exits on CTRL-c, it does not exits on "kill -TERM pid", I have to send "kill -KILL pid" to terminate it.
>
> When I run the command with strace it hangs at read(4,
> ...SNIP...
> read(4, " ( ) ( 4:file true false ) ) ( 1"..., 16384) = 16384
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d83000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d81000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7f000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7d000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7b000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d79000
> read(4, "***/****-***_redacted_*****_****"..., 16384) = 14773
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d77000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d75000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d73000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d71000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d6f000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d6d000
> read(4,
>
> When I observe the server, there is a CPU activity at the begin, but when the client hangs, the server seems to be in idle.
> Just a corresponding svnserve process is there with no visible cpu usage.
> In svnserve.log is nothing relevant to see.
>
> The svnserve command is:
> svnserve -d -r /path/to/repositories \
> --log-file=/var/log/svnserve.log \
> --memory-cache-size 1024 \
> --cache-txdeltas yes \
> --cache-fulltexts yes
>
> When I try to get the xml-log on the server with the corresponding file:// repository URL:
> /usr/bin/svn log --xml -g -v -r 10701:HEAD file://path/to/local/repositories/project/branch
> The command finishes in ca 5-10 seconds and I get the xml output, but the output has a way too many </logentry> lines.
>
> There are 1217 occurrences of the string “<logentry" and 33191 occurrences of the string "</logentry>" in the output xml.
> There are several thousand lines of </logentry> in a row in many places in repeated blocks.
>
> Details:
> Client and Server OS:
> Linux 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1+deb10u1 (2020-04-27) x86_64 GNU/Linux
>
> The repository is ca. 4 GB.
> Running "svnadmin verify" on the server founds no errors.
> I have no other problems with the server, checkout and commit works normal.
>
> svn --version
> svn, version 1.10.4 (r1850624)
> compiled Feb 10 2021, 20:15:45 on x86_64-pc-linux-gnu
>
> Copyright (C) 2019 The Apache Software Foundation.
> This software consists of contributions made by many people;
> see the NOTICE file for more information.
> Subversion is open source software, see http://subversion.apache.org/
>
> The following repository access (RA) modules are available:
>
> * ra_svn : Module for accessing a repository using the svn network protocol.
> - with Cyrus SASL authentication
> - handles 'svn' scheme
> * ra_local : Module for accessing a repository on local disk.
> - handles 'file' scheme
> * ra_serf : Module for accessing a repository via WebDAV protocol using serf.
> - using serf 1.3.9 (compiled with 1.3.9)
> - handles 'http' scheme
> - handles 'https' scheme
>
> The following authentication credential caches are available:
>
> * Plaintext cache in /home/username/.subversion
> * Gnome Keyring
> * GPG-Agent
> * KWallet (KDE)
Hello Attila,
Thanks for this detailed explanation. There are quite a few important
clues here.
To help narrow down the search for the culprit: Are you able to run
the same 'svn log' command against the same working copy, but without
the '--xml', and get a correct output in a reasonable amount of time?
If log without '--xml' works and log with '--xml' gets stuck, is it
feasible to try to bisect the revision range, e.g., run log on the
range r10701:10751, and in a separate invocation on r10751:10801, and
try to see if there's a revision in particular where SVN is getting
confused? If so, then perhaps knowing a little more about that
revision could help narrow it down.
By the way, is this (1.10.4) the SVN version on the server, the
client, or both?
> Googling around gives me these two somewhat related hits:
> https://issues.apache.org/jira/browse/SVN-4856
> https://issues.apache.org/jira/browse/SVN-4711
>
> But I do'nt use the --search parameter.
It could nevertheless be related. The spurious XML closing tags seems
to ring a bell.
> Is this a bug or are there any suggestions how to solve this problem?
It sounds like a bug. My suggestions above.
Thanks for your feedback,
Nathan
Re: svn log xml hangs and produces too many logentry closing tags
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Attila wrote on Tue, Jul 13, 2021 at 23:48:48 +0200:
> Hi
>
> I have a problem getting the svn log in a branch after sync-merging a commit from trunk.
> This commit in trunk is a merge of an old and complex branch with many commits.
>
> The client accessing the repository over svn:// url.
> (paths and text is redacted)
> The head revision is: 10801
>
> When I run the following command on the client (in the working copy), it prints a long partial xml-log output, then hangs.
> /usr/bin/svn log --xml -g -v -r 10701:HEAD /path/to/branch-wc
>
> When observing in "top", the command uses no visible CPU resources on hang. (I waited ca. 2 minutes)
> The hanging command does mot exits on CTRL-c, it does not exits on "kill -TERM pid", I have to send "kill -KILL pid" to terminate it.
>
> When I run the command with strace it hangs at read(4,
> ...SNIP...
> read(4, " ( ) ( 4:file true false ) ) ( 1"..., 16384) = 16384
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d83000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d81000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7f000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7d000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d7b000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d79000
> read(4, "***/****-***_redacted_*****_****"..., 16384) = 14773
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d77000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d75000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d73000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d71000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d6f000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f63f1d6d000
> read(4,
>
> When I observe the server, there is a CPU activity at the begin, but when the client hangs, the server seems to be in idle.
> Just a corresponding svnserve process is there with no visible cpu usage.
> In svnserve.log is nothing relevant to see.
>
> The svnserve command is:
> svnserve -d -r /path/to/repositories \
> --log-file=/var/log/svnserve.log \
> --memory-cache-size 1024 \
> --cache-txdeltas yes \
> --cache-fulltexts yes
>
> When I try to get the xml-log on the server with the corresponding file:// repository URL:
> /usr/bin/svn log --xml -g -v -r 10701:HEAD file://path/to/local/repositories/project/branch
> The command finishes in ca 5-10 seconds and I get the xml output, but the output has a way too many </logentry> lines.
>
> There are 1217 occurrences of the string “<logentry" and 33191 occurrences of the string "</logentry>" in the output xml.
> There are several thousand lines of </logentry> in a row in many places in repeated blocks.
>
> Details:
> Client and Server OS:
> Linux 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1+deb10u1 (2020-04-27) x86_64 GNU/Linux
>
> The repository is ca. 4 GB.
> Running "svnadmin verify" on the server founds no errors.
> I have no other problems with the server, checkout and commit works normal.
>
> svn --version
> svn, version 1.10.4 (r1850624)
> compiled Feb 10 2021, 20:15:45 on x86_64-pc-linux-gnu
>
> Copyright (C) 2019 The Apache Software Foundation.
> This software consists of contributions made by many people;
> see the NOTICE file for more information.
> Subversion is open source software, see http://subversion.apache.org/
>
> The following repository access (RA) modules are available:
>
> * ra_svn : Module for accessing a repository using the svn network protocol.
> - with Cyrus SASL authentication
> - handles 'svn' scheme
> * ra_local : Module for accessing a repository on local disk.
> - handles 'file' scheme
> * ra_serf : Module for accessing a repository via WebDAV protocol using serf.
> - using serf 1.3.9 (compiled with 1.3.9)
> - handles 'http' scheme
> - handles 'https' scheme
>
> The following authentication credential caches are available:
>
> * Plaintext cache in /home/username/.subversion
> * Gnome Keyring
> * GPG-Agent
> * KWallet (KDE)
>
>
> Googling around gives me these two somewhat related hits:
> https://issues.apache.org/jira/browse/SVN-4856
> https://issues.apache.org/jira/browse/SVN-4711
>
> But I do'nt use the --search parameter.
>
> Is this a bug or are there any suggestions how to solve this problem?
It could be SVN-4856, which doesn't use --search either, or it could be
something else that happens whenever the output is large enough. Could
you try «svn log -v -r HEAD:0 '^/' >/dev/null» without -g? That might
not actually be large enough (considering HEAD is ≈11k and your output
had ≈33k cases of </logentry>), so you might have to create a test
repository with sufficiently many dummy commits.
Aside: I just added to SVN-4856 a Unix port of the regression test.
It would be nice to port it into tests/cmdline/ too; any volunteers?
Cheers,
Daniel