You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/04/25 03:51:50 UTC
[GitHub] [shardingsphere] VeejaLiu opened a new issue, #17077: Hikari error occurred when i using scaling.
VeejaLiu opened a new issue, #17077:
URL: https://github.com/apache/shardingsphere/issues/17077
Two resource:
```
(Old): database_team_default
(New): database_team_new
```
Old TABLE RULE:
```
CREATE SHARDING TABLE RULE email_campaign (
DATANODES("database_team_default.EMAIL_CAMPAIGN")
);
```
Create SCALING RULE:
```
CREATE SHARDING SCALING RULE sharding_scaling(
INPUT(WORKER_THREAD=40, BATCH_SIZE=1000),
OUTPUT(WORKER_THREAD=40, BATCH_SIZE=1000),
STREAM_CHANNEL(TYPE(NAME=MEMORY, PROPERTIES("block-queue-size"=10000))),
COMPLETION_DETECTOR(TYPE(NAME=IDLE, PROPERTIES("incremental-task-idle-minute-threshold"=30))),
DATA_CONSISTENCY_CHECKER(TYPE(NAME=DATA_MATCH, PROPERTIES("chunk-size"=1000)))
);
```
New Table Rule:
```
ALTER SHARDING TABLE RULE email_campaign (
DATANODES("database_team_new.EMAIL_CAMPAIGN_TEAM_${['0','1','2']}"),
TABLE_STRATEGY(
TYPE=standard,
SHARDING_COLUMN=email_campaign_team_id,
SHARDING_ALGORITHM=email_campaign_table_sharding_strategy
),
KEY_GENERATE_STRATEGY(
COLUMN=email_campaign_id,
TYPE(NAME=snowflake)
)
);
```
Then Automatic scaling triggered...
And then:
<img width="1669" alt="image" src="https://user-images.githubusercontent.com/22064617/165017883-acc4624e-ed7e-4353-a4e9-a96e100fbbbb.png">
```
[ERROR] 2022-04-25 03:30:29.464 [ShardingSphere-Scaling-execute-0] o.a.s.d.p.c.i.AbstractImporter - flush failed 0/3 times.
java.sql.SQLException: HikariDataSource HikariDataSource (HikariPool-49) has been closed.
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:96)
at org.apache.shardingsphere.driver.jdbc.core.connection.ConnectionManager.createConnection(ConnectionManager.java:369)
at org.apache.shardingsphere.driver.jdbc.core.connection.ConnectionManager.createConnections(ConnectionManager.java:338)
at org.apache.shardingsphere.driver.jdbc.core.connection.ConnectionManager.getConnections(ConnectionManager.java:327)
at org.apache.shardingsphere.infra.executor.sql.prepare.driver.DriverExecutionPrepareEngine.group(DriverExecutionPrepareEngine.java:89)
at org.apache.shardingsphere.infra.executor.sql.prepare.AbstractExecutionPrepareEngine.prepare(AbstractExecutionPrepareEngine.java:68)
at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement.initBatchPreparedStatementExecutor(ShardingSpherePreparedStatement.java:576)
at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement.executeBatch(ShardingSpherePreparedStatement.java:557)
at org.apache.shardingsphere.data.pipeline.core.importer.AbstractImporter.executeBatchInsert(AbstractImporter.java:172)
at org.apache.shardingsphere.data.pipeline.core.importer.AbstractImporter.doFlush(AbstractImporter.java:147)
at org.apache.shardingsphere.data.pipeline.core.importer.AbstractImporter.tryFlush(AbstractImporter.java:132)
at org.apache.shardingsphere.data.pipeline.core.importer.AbstractImporter.flushInternal(AbstractImporter.java:123)
at org.apache.shardingsphere.data.pipeline.core.importer.AbstractImporter.lambda$flush$2(AbstractImporter.java:114)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1508)
at org.apache.shardingsphere.data.pipeline.core.importer.AbstractImporter.flush(AbstractImporter.java:112)
at org.apache.shardingsphere.data.pipeline.core.importer.AbstractImporter.write(AbstractImporter.java:93)
at org.apache.shardingsphere.data.pipeline.core.importer.AbstractImporter.doStart(AbstractImporter.java:79)
at org.apache.shardingsphere.data.pipeline.api.executor.AbstractLifecycleExecutor.start(AbstractLifecycleExecutor.java:41)
at org.apache.shardingsphere.data.pipeline.api.executor.AbstractLifecycleExecutor.run(AbstractLifecycleExecutor.java:61)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:830)
[INFO ] 2022-04-25 03:30:30.465 [ShardingSphere-Scaling-execute-0] o.a.s.d.p.c.i.AbstractImporter - importer write done, rowCount=28000, finishedByBreak=false
[INFO ] 2022-04-25 03:30:30.465 [ShardingSphere-Scaling-execute-1] o.a.s.d.p.core.task.InventoryTask - importer onSuccess, taskId=database_team_default.EMAIL_CAMPAIGN#0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] azexcy commented on issue #17077: Hikari error occurred when i using scaling.
Posted by GitBox <gi...@apache.org>.
azexcy commented on issue #17077:
URL: https://github.com/apache/shardingsphere/issues/17077#issuecomment-1108408574
@VeejaLiu Thank you for your feedback, I will investigate this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz commented on issue #17077: Hikari error occurred when i using scaling.
Posted by GitBox <gi...@apache.org>.
sandynz commented on issue #17077:
URL: https://github.com/apache/shardingsphere/issues/17077#issuecomment-1108096222
Hi @VeejaLiu , thanks for you feedback.
Which version or commit are you running?
Seems the used dataSource was closed by dataSourceManager, maybe there's other error occur before this.
Could you find more ERROR in log file before this stack trace?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz commented on issue #17077: Hikari error occurred when i using scaling.
Posted by GitBox <gi...@apache.org>.
sandynz commented on issue #17077:
URL: https://github.com/apache/shardingsphere/issues/17077#issuecomment-1108139416
OK, I'll try to reproduce it.
And there's another issue in UnsignedIntHandler.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] azexcy commented on issue #17077: Hikari error occurred when i using scaling.
Posted by GitBox <gi...@apache.org>.
azexcy commented on issue #17077:
URL: https://github.com/apache/shardingsphere/issues/17077#issuecomment-1110909292
@VeejaLiu Hello, I fix the problem of unsign type of MySQL not handled correctly when using scaling. But I can't reproduce the error of `HikariDataSource` , can you provide more information of it? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz closed issue #17077: Hikari error occurred when i using scaling.
Posted by GitBox <gi...@apache.org>.
sandynz closed issue #17077: Hikari error occurred when i using scaling.
URL: https://github.com/apache/shardingsphere/issues/17077
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] VeejaLiu commented on issue #17077: Hikari error occurred when i using scaling.
Posted by GitBox <gi...@apache.org>.
VeejaLiu commented on issue #17077:
URL: https://github.com/apache/shardingsphere/issues/17077#issuecomment-1108121130
@sandynz Thank you, guys.
I'm using 5.1.1
I did have another problem before that.
When I did scaling, I encountered this problem:
<img width="1669" alt="image" src="https://user-images.githubusercontent.com/22064617/165031044-8c219429-4fea-455f-b65d-17d2b4d151fc.png">
Then I deleted the old scaling configuration and recreated the task.
Then... you've seen it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org