You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2020/02/11 02:37:05 UTC
[GitHub] [incubator-shardingsphere] lklkxcxc opened a new issue #4231:
mysqldump import sharding-proxy slowly
lklkxcxc opened a new issue #4231: mysqldump import sharding-proxy slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231
Sharding-proxy version: `4.0.0-RC1`
Problems encountered：`It took 24 hours to import 10 million records into sharding-proxy.`
Sharding-proxy Config: `start.sh -Xmx8g -Xms8g -Xmn4g,server.yaml use default parm`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] nevereverever commented on issue #4231:
mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
nevereverever commented on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-595541962
> You can google or baidu PreparedStatement to find its usage.
@kimmking PreparedStatement can be use in sql scripts?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] kimmking edited a comment on issue
#4231: mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
kimmking edited a comment on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-586404965
You can google or baidu PreparedStatement to find its usage.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] kimmking commented on issue #4231:
mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
kimmking commented on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-584591912
Terry means you can use ‘insert into pku_influencing_factor VALUES (?,?,?,?)’ with PreparedStatement and put the parameters for every record. @lklkxcxc
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] lklkxcxc commented on issue #4231:
mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
lklkxcxc commented on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-584488633
@terrymanu Do you know why insert is so slow?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] lklkxcxc commented on issue #4231:
mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
lklkxcxc commented on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-584619277
@kimmking Can you give me a specific example? thank you！
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] lklkxcxc edited a comment on issue
#4231: mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
lklkxcxc edited a comment on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-584493358
@terrymanu There is not much change ,I have used param:`serverTimezone=UTC&useSSL=false&characterEncoding=utf8&useServerPrepStmts=true&cachePrepStmts=true`
The test is faster to import directly into backend mysql database；
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] lklkxcxc commented on issue #4231:
mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
lklkxcxc commented on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-584493358
@terrymanu There is not much change ,I have used param:`serverTimezone=UTC&useSSL=false&characterEncoding=utf8&useServerPrepStmts=true&cachePrepStmts=true`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] lklkxcxc edited a comment on issue
#4231: mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
lklkxcxc edited a comment on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-584619277
@kimmking Can you give me a specific example? thank you！It isnot that how use parm `useServerPrepStmts=true&cachePrepStmts=true`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] terrymanu commented on issue #4231:
mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
terrymanu commented on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-584490879
Because if you don't use PreparedStatement, it will parse SQL for every call which are very slow.
ShardingSphere will cache parse result if using PreparedStatement.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] lklkxcxc edited a comment on issue
#4231: mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
lklkxcxc edited a comment on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-584484024
@terrymanu : I didn't used PreparedStatement and ParameterMarker. What are PreparedStatement and ParameterMarker?
database0?serverTimezone=UTC&useSSL=false&characterEncoding=utf8
use sql `insert into pku_influencing_factor VALUES (0,'test','test',123.00)` normal only 1 record
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] lklkxcxc edited a comment on issue
#4231: mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
lklkxcxc edited a comment on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-584619277
@kimmking Can you give me a specific example? thank you！It isnot that how use parm `useServerPrepStmts=true&cachePrepStmts=true`?
I found add prarm `rewriteBatchedStatements=true` can to improve speed but still not very fast
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] lklkxcxc closed issue #4231: mysqldump
file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
lklkxcxc closed issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] lklkxcxc edited a comment on issue
#4231: mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
lklkxcxc edited a comment on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-584619277
@kimmking Can you give me a specific example? thank you！It isnot that how use parm `useServerPrepStmts=true&cachePrepStmts=true`?
@terrymanu I found add prarm `rewriteBatchedStatements=true` can to improve speed but still not very fast
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] lklkxcxc edited a comment on issue
#4231: mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
lklkxcxc edited a comment on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-584484024
@terrymanu : I didn't used PreparedStatement and ParameterMarker. What are PreparedStatement and ParameterMarker?
database0?serverTimezone=UTC&useSSL=false&characterEncoding=utf8
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] terrymanu edited a comment on issue
#4231: mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
terrymanu edited a comment on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-584490879
Because if you don't use PreparedStatement, it will parse SQL for every call which are very slow.
ShardingSphere will cache parse result if using PreparedStatement.
Please search `PreparedStatement` from google to get more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] terrymanu commented on issue #4231:
mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
terrymanu commented on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-584481790
Do you use PreparedStatement and ParameterMarker?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] lklkxcxc commented on issue #4231:
mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
lklkxcxc commented on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-584484024
@terrymanu : I didn't used PreparedStatement and ParameterMarker. What are PreparedStatement and ParameterMarker?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] kimmking commented on issue #4231:
mysqldump file import sharding-proxy insert speed slowly
Posted by GitBox <gi...@apache.org>.
kimmking commented on issue #4231: mysqldump file import sharding-proxy insert speed slowly
URL: https://github.com/apache/incubator-shardingsphere/issues/4231#issuecomment-586404965
You can google or baidu PreparedStatement
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services