You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2020/10/16 08:22:48 UTC
[GitHub] [shardingsphere] piggytough opened a new issue #7809: How to improve efficiency of batch insert ?
piggytough opened a new issue #7809:
URL: https://github.com/apache/shardingsphere/issues/7809
**My version** : Sharding-JDBC 4.1.1
sql like this: insert into pressure(userid,orderid) values(?, ?),(?,?),...
insert data size is 5000, and i've used prepare statement.
when NOT using sharding, time consumes about 40ms, but sharding takes about 400ms.
Sharding tables configuration:
Analysis by arthas, I can see that time mostly cost on function `org.apache.shardingsphere.sharding.rewrite.token.generator.impl.keygen.GeneratedKeyInsertValuesTokenGenerator:isToAddDerivedLiteralExpression`. See below:
Is there anything I can do to improve efficiency?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] piggytough commented on issue #7809: How to improve efficiency of batch insert ?
Posted by GitBox <gi...@apache.org>.
piggytough commented on issue #7809:
URL: https://github.com/apache/shardingsphere/issues/7809#issuecomment-710716294
> there has been fixed in master branch, can you try it with master branch 5.0.0.
> #7816
sure!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] yu199195 closed issue #7809: How to improve efficiency of batch insert ?
Posted by GitBox <gi...@apache.org>.
yu199195 closed issue #7809:
URL: https://github.com/apache/shardingsphere/issues/7809
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] kimmking commented on issue #7809: How to improve efficiency of batch insert ?
Posted by GitBox <gi...@apache.org>.
kimmking commented on issue #7809:
URL: https://github.com/apache/shardingsphere/issues/7809#issuecomment-710034538
there has been fixed in master branch, can you try it with master branch 5.0.0.
https://github.com/apache/shardingsphere/pull/7816
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] kimmking commented on issue #7809: How to improve efficiency of batch insert ?
Posted by GitBox <gi...@apache.org>.
kimmking commented on issue #7809:
URL: https://github.com/apache/shardingsphere/issues/7809#issuecomment-709972707
It seems isToAddDerivedLiteralExpression method will call getGroupedParameters for an empty check every time.
So we can make two improvement choices:
- cache the GroupedParameters for one sql and will called 5000 times in multiple values
- add a new check empty method and avoid new/merge parameters
```
private boolean isToAddDerivedLiteralExpression(final InsertStatementContext insertStatementContext, final int insertValueCount) {
return insertStatementContext.getGroupedParameters().get(insertValueCount).isEmpty();
}
public List<List<Object>> getGroupedParameters() {
List<List<Object>> result = new LinkedList<>();
for (InsertValueContext each : insertValueContexts) {
result.add(each.getParameters());
}
if (null != insertSelectContext) {
result.add(insertSelectContext.getParameters());
}
return result;
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org