You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/10/10 06:32:40 UTC
[GitHub] [shardingsphere] wsm12138 opened a new issue, #20458: NPE: when execute `MIGRATE TABLE source_ds.t_order INTO sharding_db.t_order;` , `select * from table` occur `ERROR 30000 (42000): Unknown exception: null`
wsm12138 opened a new issue, #20458:
URL: https://github.com/apache/shardingsphere/issues/20458
## Bug Report
### Which version of ShardingSphere did you use?
```
ShardingSphere-5.1.3-SNAPSHOT
Commit ID: 39850eb7a188b59fbcf65fb55b098b07b7f6132f
Commit Message: Improve migration data consistency checker. (#20407)
Branch: 39850eb7a188b59fbcf65fb55b098b07b7f6132f
Build time: 2022-08-23T11:32:37+0800
```
### Which project did you use? ShardingSphere-JDBC or ShardingSphere-Proxy?
*ShardingSphere-Proxy*
### Expected behavior
```
mysql> select count(*) from t_order;
+----------+
| count(*) |
+----------+
| 5 |
+----------+
1 row in set (0.09 sec)
```
### Actual behavior
```
mysql> show migration status 'j012f8ae37064557dd75569db498cb3ba8c';
+------+-------------+--------------------------+--------+-------------------------------+--------------------------+
| item | data_source | status | active | inventory_finished_percentage | incremental_idle_seconds |
+------+-------------+--------------------------+--------+-------------------------------+--------------------------+
| 0 | source_ds | EXECUTE_INCREMENTAL_TASK | true | 100 | 28 |
+------+-------------+--------------------------+--------+-------------------------------+--------------------------+
1 row in set (0.03 sec)
mysql> show migration status 'j012f8ae37064557dd75569db498cb3ba8';
ERROR 16000 (HY000): Can not find pipeline job `j012f8ae37064557dd75569db498cb3ba8`
mysql> select count(*) from t_order;
ERROR 30000 (42000): Unknown exception: null
mysql> select count(*) from t_order;
ERROR 30000 (42000): Unknown exception: null
mysql> show tables;
Empty set (0.00 sec)
mysql> show migration status 'j012f8ae37064557dd75569db498cb3ba8';
ERROR 16000 (HY000): Can not find pipeline job `j012f8ae37064557dd75569db498cb3ba8`
mysql> show migration status 'j012f8ae37064557dd75569db498cb3ba8c';
+------+-------------+--------------------------+--------+-------------------------------+--------------------------+
| item | data_source | status | active | inventory_finished_percentage | incremental_idle_seconds |
+------+-------------+--------------------------+--------+-------------------------------+--------------------------+
| 0 | source_ds | EXECUTE_INCREMENTAL_TASK | true | 100 | 92 |
+------+-------------+--------------------------+--------+-------------------------------+--------------------------+
1 row in set (0.01 sec)
```
### Reason analyze (If you can)
stdout.log
```
[ERROR] 2022-08-23 16:43:18.822 [ShardingSphere-Command-7] o.a.s.p.f.c.CommandExecutorTask - Exception occur:
java.lang.NullPointerException: null
at org.apache.shardingsphere.sharding.merge.dql.groupby.GroupByMemoryMergedResult.getValueCaseSensitiveFromTables(GroupByMemoryMergedResult.java:135)
at org.apache.shardingsphere.sharding.merge.dql.groupby.GroupByMemoryMergedResult.getValueCaseSensitive(GroupByMemoryMergedResult.java:125)
at org.apache.shardingsphere.sharding.merge.dql.groupby.GroupByMemoryMergedResult.init(GroupByMemoryMergedResult.java:73)
at org.apache.shardingsphere.sharding.merge.dql.groupby.GroupByMemoryMergedResult.init(GroupByMemoryMergedResult.java:53)
at org.apache.shardingsphere.infra.merge.result.impl.memory.MemoryMergedResult.<init>(MemoryMergedResult.java:51)
at org.apache.shardingsphere.sharding.merge.dql.groupby.GroupByMemoryMergedResult.<init>(GroupByMemoryMergedResult.java:56)
at org.apache.shardingsphere.sharding.merge.dql.ShardingDQLResultMerger.getGroupByMergedResult(ShardingDQLResultMerger.java:118)
at org.apache.shardingsphere.sharding.merge.dql.ShardingDQLResultMerger.build(ShardingDQLResultMerger.java:88)
at org.apache.shardingsphere.sharding.merge.dql.ShardingDQLResultMerger.merge(ShardingDQLResultMerger.java:66)
at org.apache.shardingsphere.infra.merge.MergeEngine.executeMerge(MergeEngine.java:82)
at org.apache.shardingsphere.infra.merge.MergeEngine.merge(MergeEngine.java:71)
at org.apache.shardingsphere.proxy.backend.communication.DatabaseCommunicationEngine.mergeQuery(DatabaseCommunicationEngine.java:177)
at org.apache.shardingsphere.proxy.backend.communication.DatabaseCommunicationEngine.processExecuteQuery(DatabaseCommunicationEngine.java:143)
at org.apache.shardingsphere.proxy.backend.communication.jdbc.JDBCDatabaseCommunicationEngine.execute(JDBCDatabaseCommunicationEngine.java:136)
at org.apache.shardingsphere.proxy.frontend.mysql.command.query.text.query.MySQLComQueryPacketExecutor.execute(MySQLComQueryPacketExecutor.java:94)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.executeCommand(CommandExecutorTask.java:110)
at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.run(CommandExecutorTask.java:77)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
### Steps to reproduce the behavior, such as: SQL to execute, sharding rule configuration, when exception occur etc.
1. start proxy with cluster mode
2. create logic db;add resource in logic db.
3. execute `MIGRATE TABLE source_ds.t_order INTO sharding_db.t_order;`
4. execute `select count(*) from table` during MIGRATE;execute `select * from table` during MIGRATE
### Example codes for reproduce this issue (such as a github link).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] azexcy commented on issue #20458: NPE: when execute `MIGRATE TABLE source_ds.t_order INTO sharding_db.t_order;` , `select * from table` occur `ERROR 30000 (42000): Unknown exception: null`
Posted by GitBox <gi...@apache.org>.
azexcy commented on issue #20458:
URL: https://github.com/apache/shardingsphere/issues/20458#issuecomment-1223784798
Please execute `REFRESH TABLE METADATA;` before execute `select count(*) from t_order; `
In subsequent versions we will find ways to optimize this issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] wsm12138 commented on issue #20458: NPE: when execute `MIGRATE TABLE source_ds.t_order INTO sharding_db.t_order;` , `select * from table` occur `ERROR 30000 (42000): Unknown exception: null`
Posted by GitBox <gi...@apache.org>.
wsm12138 commented on issue #20458:
URL: https://github.com/apache/shardingsphere/issues/20458#issuecomment-1225272434
I have another question.
I don't execute `refresh table metadata;`. When execute select count(*) from table during MIGRATE;execute select * from table during MIGRATE, `select * from table ` success,select count(*) from table have error.
<img width="803" alt="image" src="https://user-images.githubusercontent.com/86462784/186349752-2defef7f-cfb4-4ea9-9cdb-d374e78df3af.png">
### so,select * from sharding_table,don't need metadata?
<img width="1310" alt="image" src="https://user-images.githubusercontent.com/86462784/186348945-a66a71f9-c47e-47ba-8ce0-307fff1dc752.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] github-actions[bot] commented on issue #20458: NPE: when execute `MIGRATE TABLE source_ds.t_order INTO sharding_db.t_order;` , `select * from table` occur `ERROR 30000 (42000): Unknown exception: null`
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #20458:
URL: https://github.com/apache/shardingsphere/issues/20458#issuecomment-1272348737
Hello , this issue has not received a reply for several days.
This issue is supposed to be closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] wsm12138 commented on issue #20458: NPE: when execute `MIGRATE TABLE source_ds.t_order INTO sharding_db.t_order;` , `select * from table` occur `ERROR 30000 (42000): Unknown exception: null`
Posted by GitBox <gi...@apache.org>.
wsm12138 commented on issue #20458:
URL: https://github.com/apache/shardingsphere/issues/20458#issuecomment-1225210334
After the `check migration ${job id};` performs data consistency verification, there may be increments during the `stop migration ${job id};`. This leads to inconsistency between source & target data. At present, migrat cannot limit the writing on the source side, so the whole process is controlled by the user , which is easy to cause data inconsistency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] github-actions[bot] closed issue #20458: NPE: when execute `MIGRATE TABLE source_ds.t_order INTO sharding_db.t_order;` , `select * from table` occur `ERROR 30000 (42000): Unknown exception: null`
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #20458: NPE: when execute `MIGRATE TABLE source_ds.t_order INTO sharding_db.t_order;` , `select * from table` occur `ERROR 30000 (42000): Unknown exception: null`
URL: https://github.com/apache/shardingsphere/issues/20458
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org