You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/11/04 11:07:03 UTC
[GitHub] [shardingsphere] xiamenmin opened a new issue, #21964: Deadlock issue in CONNECTION_STRICTLY mode
xiamenmin opened a new issue, #21964:
URL: https://github.com/apache/shardingsphere/issues/21964
## Question
When I use the cCONNECTION_STRICTLY mode, I have the problem of getting database connections and waiting for each other in a concurrent scenario.
### Business scene
- The current data is divided into 10 tables, and when I configure maxConnectionSizePerQuery = 5, and the instantaneous concurrency is > 8, the following problems will occur
### Configuration
- MaxActive = 15
- MaxWait = 60_000
- max.connections.size.per.query = 5
- DruidDataSource
- Version: apache-shardingsphere-jdbc-4.1.1
I add comments to the source code
``` java
package org.apache.shardingsphere.shardingjdbc.jdbc.adapter;
public abstract class AbstractConnectionAdapter extends AbstractUnsupportedOperationConnection {
private List<Connection> createConnections(final String dataSourceName, final DataSource dataSource, final int connectionSize) throws SQLException {
List<Connection> result = new ArrayList<>(connectionSize);
for (int i = 0; i < connectionSize; i++) {
try {
System.out.println("Connections name:" + Thread.currentThread().getName() + "dataSourceName:" + dataSourceName + " progress:" + result.size() + "/" + connectionSize);
....
}
return result;
}
}
```
### Running result log
Connections name:ForkJoinPool.commonPool-worker-9 dataSourceName:cmc progress:1/5
Connections name:ForkJoinPool.commonPool-worker-6 dataSourceName:cmc progress:1/5
Connections name:ForkJoinPool.commonPool-worker-8 dataSourceName:cmc progress:1/5
Connections name:ForkJoinPool.commonPool-worker-1 dataSourceName:cmc progress:1/5
Connections name:ForkJoinPool.commonPool-worker-11 dataSourceName:cmc progress:1/5
Connections name:ForkJoinPool.commonPool-worker-10 dataSourceName:cmc progress:1/5
Connections name:ForkJoinPool.commonPool-worker-4 dataSourceName:cmc progress:1/5
Connections name:ForkJoinPool.commonPool-worker-2 dataSourceName:cmc progress:2/5
Connections name:ForkJoinPool.commonPool-worker-15 dataSourceName:cmc progress:2/5
Connections name:ForkJoinPool.commonPool-worker-13 dataSourceName:cmc progress:1/5
### Problems
- Wait for each other to get the thread pool
- maxWait timeout
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz commented on issue #21964: Deadlock issue in CONNECTION_STRICTLY mode
Posted by "sandynz (via GitHub)" <gi...@apache.org>.
sandynz commented on issue #21964:
URL: https://github.com/apache/shardingsphere/issues/21964#issuecomment-1481014897
@xiamenmin thanks for your demo. Suggest to create a new repository to write the demo, but not in ShardingSphere repository, it might take long time to download it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] xiamenmin commented on issue #21964: Deadlock issue in CONNECTION_STRICTLY mode
Posted by "xiamenmin (via GitHub)" <gi...@apache.org>.
xiamenmin commented on issue #21964:
URL: https://github.com/apache/shardingsphere/issues/21964#issuecomment-1482338111
ok i have updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] zhaojinchao95 commented on issue #21964: Deadlock issue in CONNECTION_STRICTLY mode
Posted by GitBox <gi...@apache.org>.
zhaojinchao95 commented on issue #21964:
URL: https://github.com/apache/shardingsphere/issues/21964#issuecomment-1369509207
> Hi @xiamenmin , looks you're running version 4.1.1, could you try version 5.3.0?
Next version, thx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz commented on issue #21964: Deadlock issue in CONNECTION_STRICTLY mode
Posted by GitBox <gi...@apache.org>.
sandynz commented on issue #21964:
URL: https://github.com/apache/shardingsphere/issues/21964#issuecomment-1364933453
Hi @xiamenmin , looks you're running version 4.1.1, could you try version 5.3.0?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] zhaojinchao95 commented on issue #21964: Deadlock issue in CONNECTION_STRICTLY mode
Posted by GitBox <gi...@apache.org>.
zhaojinchao95 commented on issue #21964:
URL: https://github.com/apache/shardingsphere/issues/21964#issuecomment-1311154200
@huang714669 @xiamenmin Hi guys, thank you for your feedback , i will check it. At the same time, welcome to solve together.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] xiamenmin commented on issue #21964: Deadlock issue in CONNECTION_STRICTLY mode
Posted by "xiamenmin (via GitHub)" <gi...@apache.org>.
xiamenmin commented on issue #21964:
URL: https://github.com/apache/shardingsphere/issues/21964#issuecomment-1471933863
Ok, let me try
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] xiamenmin commented on issue #21964: Deadlock issue in CONNECTION_STRICTLY mode
Posted by "xiamenmin (via GitHub)" <gi...@apache.org>.
xiamenmin commented on issue #21964:
URL: https://github.com/apache/shardingsphere/issues/21964#issuecomment-1477596808
Steps to reproduce
1. I wrote a case at https://github.com/xiamenmin/shardingsphere/tree/deadlock
2. Start sharding--spring-boot-starter-jpa--standalone--local ExampleMain
3. Check the log
@zhaojinchao95 @sandynz @huang714669 have time to discuss
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] huang714669 commented on issue #21964: Deadlock issue in CONNECTION_STRICTLY mode
Posted by GitBox <gi...@apache.org>.
huang714669 commented on issue #21964:
URL: https://github.com/apache/shardingsphere/issues/21964#issuecomment-1311149308
I encounter the same deadlock problem, my datasource is hikari, do you fix this now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] xiamenmin commented on issue #21964: Deadlock issue in CONNECTION_STRICTLY mode
Posted by "xiamenmin (via GitHub)" <gi...@apache.org>.
xiamenmin commented on issue #21964:
URL: https://github.com/apache/shardingsphere/issues/21964#issuecomment-1477567901
This problem still exists. First of all, I see that the code is only added in the 5.1.0 + version
- https://github.com/apache/shardingsphere/blob/master/docs/blog/content/material/2022_03_18_Executor_Engine_Performance_Optimization_Showcase_with_Apache_ShardingSphere_5.1.0.en.md
But if `isNeedAggregateRewrite` is not hit, there will still be a deadlock problem, because there is no lock in the source code to obtain the connection pool.
`public final class ConnectionManager implements ExecutorJDBCConnectionManager, AutoCloseable {
private List<Connection> createConnections(final String dataSourceName, final DataSource dataSource, final int connectionSize,
final TransactionConnectionContext transactionConnectionContext) throws SQLException {
List<Connection> result = new ArrayList<>(connectionSize);
for (int i = 0; i < connectionSize; i++) {
try {
Connection connection = createConnection(dataSourceName, dataSource, transactionConnectionContext);
methodInvocationRecorder.replay(connection);
result.add(connection);
} catch (final SQLException ex) {
for (Connection each : result) {
each.close();
}
throw new OverallConnectionNotEnoughException(connectionSize, result.size()).toSQLException();
}
}
return result;
}
}`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org