[jira] [Created] (DERBY-7123) Opening many client connections at the same time can hang

["Mattias Lundstrom (Jira)" <ji...@apache.org>]

Mattias Lundstrom created DERBY-7123:
----------------------------------------

             Summary: Opening many client connections at the same time can hang
                 Key: DERBY-7123
                 URL: https://issues.apache.org/jira/browse/DERBY-7123
             Project: Derby
          Issue Type: Bug
          Components: JDBC
    Affects Versions: 10.15.2.0, 10.14.2.0
         Environment: Test run on
$ uname -a
Linux mattiasl-laptop2020 5.8.0-63-generic #71-Ubuntu SMP Tue Jul 13 15:59:12 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

            Reporter: Mattias Lundstrom
         Attachments: DerbyTest.java

If many connections are opened at the same time towards a Derby network server using ClientConnectionPoolDataSource.getPooledConnection, then this operation may hang indefinitely. 

This has been tested with network server version 10.14.2.0 and 10.15.2.0 and the same client versions. 

Attached test client program (for 10.15.2.0) to show the behavior. It just repeatedly tries to open 1000 connections in parallel to the same server (in the production system, there were much fewer connections, but that makes it is harder to reproduce). It is generally slow and sometimes seems to hang indefinitely.

Test can be run by just having a local network server and create a database and then
{code:java}
$ javac -cp derbyclient.jar:derbytools.jar DerbyTest.java
$ java -cp .:derbyclient.jar:derbytools.jar DerbyTest localhost 1527 seconddb 
{code}
(where "seconddb" is the test database)

Testing seems to show that if the getPooledConnection operation is synchronized, the problem disappears (see commented out code in test program). We are using this workaround in our application for now.

Running the test program, sometimes the connection seems to hang indefinitely or at the very least be extremely slow. In these situations neither client nor server consumes any CPU and the client seems to wait for IO from the server.

Stacktrace from client
{quote}java.base@11.0.11/java.net.SocketInputStream.socketRead0(Native Method) 
 java.base@11.0.11/java.net.SocketInputStream.socketRead(SocketInputStream.java:115) 
 java.base@11.0.11/java.net.SocketInputStream.read(SocketInputStream.java:168) 
 java.base@11.0.11/java.net.SocketInputStream.read(SocketInputStream.java:140) 
 app//org.apache.derby.client.net.Reply.fill(Reply.java:172) 
 app//org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Reply.java:216) 
 app//org.apache.derby.client.net.Reply.readDssHeader(Reply.java:319) 
 app//org.apache.derby.client.net.Reply.startSameIdChainParse(Reply.java:1085) 
 app//org.apache.derby.client.net.NetConnectionReply.readExchangeServerAttributes(NetConnectionReply.java:56) 
 app//org.apache.derby.client.net.NetConnection.readServerAttributesAndKeyExchange(NetConnection.java:764) 
 app//org.apache.derby.client.net.NetConnection.flowServerAttributesAndKeyExchange(NetConnection.java:708) 
 app//org.apache.derby.client.net.NetConnection.flowUSRIDONLconnect(NetConnection.java:562) 
 app//org.apache.derby.client.net.NetConnection.flowConnect(NetConnection.java:359) 
 app//org.apache.derby.client.net.NetConnection.initialize(NetConnection.java:274) 
 app//org.apache.derby.client.net.NetConnection.<init>(NetConnection.java:260) 
 app//org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newNetConnection(ClientJDBCObjectFactoryImpl.java:327) 
 app//org.apache.derby.client.ClientPooledConnection.<init>(ClientPooledConnection.java:130) 
 app//org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newClientPooledConnection(ClientJDBCObjectFactoryImpl.java:85) 
 app//org.apache.derby.client.BasicClientDataSource.getPooledConnectionX(BasicClientDataSource.java:1619) 
 app//org.apache.derby.client.BasicClientDataSource.getPooledConnectionMinion(BasicClientDataSource.java:1560) 
 app//org.apache.derby.jdbc.ClientConnectionPoolDataSource.getPooledConnection(ClientConnectionPoolDataSource.java:69)
{quote}
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)