Re: Deadlock-like behaviour of svnserve in multi-threaded mode (-T)

[Tomas Bilek <to...@polarion.com>]

Hello Ivan,
I am continuing in topic, which Radek has started previously. I have created
networkdump where you can see communication on localhost. Dump was started
approx 5 s before our application started but is starts using SVN later.
There are relative time stamps.   Also I have packed process monitor result
but unfortunetely from time after svn starts hang up, because when I was
running process monitor I was not able to reproduce that problem, so I
turned it on after the system hangs.

To logs.
1) Network (network dump
https://drive.google.com/file/d/0ByHPI8jVgwP8ZHJxbHZ5Z3BScVU/view?usp=sharing)
when i filter only tcp.port = 3690 I see that communication stopped and
there is window from 245s until 637s 
and this is snippet from our app log
2016-11-07 12:54:20,753 [ObjectIndex-refreshIndex-1 | u:p] INFO 
com.polarion.platform.startup  -  -- -- 631 from 731 objects were refreshed
(86%)
2016-11-07 13:00:54,030 [ObjectIndex-refreshIndex-2 | u:p] INFO 
com.polarion.platform.startup  -  -- -- 731 from 731 objects were refreshed
(100%)
   /////Indexing take some time from 10-20s so the difference 300-400s is
corresponding.

2) Process Monitor  (dumps
https://drive.google.com/file/d/0ByHPI8jVgwP8Qm0xa1ExNEQxZFU/view?usp=sharing)
you can filter at start for svnserve.exe. at start 07.11.2016
13:00:52,5687177   there is "disconnect sequence" 

Do you think, that it is enough info to determine, where is the problem?



--
View this message in context: http://subversion.1072662.n5.nabble.com/Deadlock-like-behaviour-of-svnserve-in-multi-threaded-mode-T-tp196421p197955.html
Sent from the Subversion Dev mailing list archive at Nabble.com.

RE: Deadlock-like behaviour of svnserve in multi-threaded mode (-T)

["Stroleny, Jakub" <ja...@siemens.com>]

Hi,

we're struggling with this issue again, but we're able to reproduce it now.
We have tested issue on svn 1.9.7 and 1.10.0 and it seems it is still valid.

I wrote some simple application to reproduce the issue. I've
described it in comment, see related issue https://issues.apache.org/jira/browse/SVN-4626
The application needs to be run probably multiple times to reproduce it correctly.

Basically, it seems that it is reproducible only when svnserve is running on Windows machine.
I did some tests on svnserve running on Linux machine and issue did not occur.

The client application is stuck on socket read from svnserve and there is no response
from server. We have network dumps and process monitor dumps, but
there isn't any response or network traffic when issue occurs.
The client application is blocked on the connection and  no progress is done
until we free some other connection to svnserve.

Can someone help us with analysis for this issue? I can provide more details if needed.
I had suspicious on svnserve.c:min_thread_count property, which is by default set to 1, but
we have 5+ concurrent connection, so that pool do not have enough workers to process
incoming requests. I have tried suggestion with "--min-threads" and "--max-threads" as
described also in previous replies, but without any change - it has same behavior as before.
Is there anything more what can we do to debug/trace this issue more?


Thank you,
Jakub Stroleny


-----Original Message-----
From: Daniel Shahaf [mailto:danielsh@apache.org]
Sent: Tuesday, November 22, 2016 5:53 PM
To: dev@subversion.apache.org
Subject: Re: Deadlock-like behaviour of svnserve in multi-threaded mode (-T)

Ping.  Has anyone looked into the network dump to try and understand what's happening here?

-----------------
Siemens Industry Software, s.r.o.
Praha 4, Doudlebská 1699/5, PSČ 140 00
IČ 256 51 897
Zapsaná v obchodním rejstříku vedeném Městským soudem v Praze, oddíl C, vložka 58222

Důležité upozornění: Tato zpráva má jen informativní charakter. Obsah této zprávy odesílatele nezavazuje a odesílatel nemá v úmyslu touto zprávou uzavřít smlouvu, přijmout nabídku, potvrdit uzavření smlouvy ani nezakládá předsmluvní odpovědnost jejího odesílatele, ledaže je odesílatelem ve zprávě uvedeno výslovně jinak. Obsah této zprávy (včetně příloh) je důvěrný. Pokud nejste zamýšleným adresátem této zprávy, zpřístupnění, kopírování, distribuce nebo jiné užití obsahu zprávy je zakázáno a v takovém případě, prosím, okamžitě informujte odesílatele a poté zprávu (vč. příloh) odstraňte z Vašeho systému.

Re: Deadlock-like behaviour of svnserve in multi-threaded mode (-T)

[Daniel Shahaf <da...@apache.org>]

Ping.  Has anyone looked into the network dump to try and understand
what's happening here?