You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2021/12/29 08:58:54 UTC

[GitHub] [shardingsphere] sandynz opened a new issue #14415: ColumnMetaDataLoader.load throw exception on ShardingSphereDataSource

sandynz opened a new issue #14415:
URL: https://github.com/apache/shardingsphere/issues/14415


   ## Bug Report
   
   ### Which version of ShardingSphere did you use?
   master branch, commit: 3b8dc82bfb8691f52f221ae394587bcfc03bfa9b
   
   ### Which project did you use? ShardingSphere-JDBC or ShardingSphere-Proxy?
   ShardingSphere-Proxy, scaling
   
   ### Expected behavior
   Load column metadata successfully.
   
   It should be a new scernario of `ColumnMetaDataLoader.load` on ShardingSphereDataSource, which is used by scailng data consistency check to get primary key from metadata.
   
   ### Actual behavior
   Exception thrown:
   ```
   [_finished_check_Worker-1] o.a.s.d.p.core.job.FinishedCheckJob - scaling job 5011358192994937 finish check failed!
   org.apache.shardingsphere.data.pipeline.core.exception.DataCheckFailException: get table columns failed
           at org.apache.shardingsphere.data.pipeline.core.check.consistency.DataConsistencyCheckerImpl.getTablesColumnsMap(DataConsistencyCheckerImpl.java:198)
           at org.apache.shardingsphere.data.pipeline.core.check.consistency.DataConsistencyCheckerImpl.checkRecordsContent(DataConsistencyCheckerImpl.java:130)
           at org.apache.shardingsphere.data.pipeline.core.api.impl.PipelineJobAPIImpl.dataConsistencyCheck0(PipelineJobAPIImpl.java:253)
           at org.apache.shardingsphere.data.pipeline.core.api.impl.PipelineJobAPIImpl.dataConsistencyCheck(PipelineJobAPIImpl.java:235)
           at org.apache.shardingsphere.data.pipeline.core.job.FinishedCheckJob.dataConsistencyCheck(FinishedCheckJob.java:100)
           at org.apache.shardingsphere.data.pipeline.core.job.FinishedCheckJob.execute(FinishedCheckJob.java:79)
           at org.apache.shardingsphere.elasticjob.simple.executor.SimpleJobExecutor.process(SimpleJobExecutor.java:33)
           at org.apache.shardingsphere.elasticjob.simple.executor.SimpleJobExecutor.process(SimpleJobExecutor.java:29)
           at org.apache.shardingsphere.elasticjob.executor.ElasticJobExecutor.process(ElasticJobExecutor.java:172)
           at org.apache.shardingsphere.elasticjob.executor.ElasticJobExecutor.process(ElasticJobExecutor.java:141)
           at org.apache.shardingsphere.elasticjob.executor.ElasticJobExecutor.execute(ElasticJobExecutor.java:124)
           at org.apache.shardingsphere.elasticjob.executor.ElasticJobExecutor.execute(ElasticJobExecutor.java:100)
           at org.apache.shardingsphere.elasticjob.lite.internal.schedule.LiteJob.execute(LiteJob.java:35)
           at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
           at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
   Caused by: java.sql.SQLException: Column index out of range.
           at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)
           at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
           at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:89)
           at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:63)
           at com.mysql.cj.jdbc.result.ResultSetMetaData.getField(ResultSetMetaData.java:223)
           at com.mysql.cj.jdbc.result.ResultSetMetaData.isAutoIncrement(ResultSetMetaData.java:277)
           at org.apache.shardingsphere.driver.jdbc.core.resultset.ShardingSphereResultSetMetaData.isAutoIncrement(ShardingSphereResultSetMetaData.java:62)
           at org.apache.shardingsphere.infra.metadata.schema.builder.loader.common.ColumnMetaDataLoader.load(ColumnMetaDataLoader.java:77)
           at org.apache.shardingsphere.infra.metadata.schema.builder.loader.common.TableMetaDataLoader.load(TableMetaDataLoader.java:51)
           at org.apache.shardingsphere.data.pipeline.core.check.consistency.DataConsistencyCheckerImpl.getTablesColumnsMap(DataConsistencyCheckerImpl.java:192)
           ... 14 common frames omitted
   ```
   
   ### Reason analyze (If you can)
   1) `ShardingSphereDatabaseMetaData.getActualTableNamePattern` expand `tableNamePattern` as `% tableName %`, e.g. `t_order` will be changed to `%t_order%`. When it's executed in underlying DB, it'll get sharded tables columns metadata, e.g. t_order_0, t_order_1, etc.
   
   2) `ColumnMetaDataLoader.load` doesn't isolate `columnNames` by table name, they're put together into List<String>.
   
   <img width="1371" alt="图片" src="https://user-images.githubusercontent.com/42492540/147644140-3713ac1e-e877-4df5-bdee-6cd2fd19db2c.png">
   
   
   ### Steps to reproduce the behavior, such as: SQL to execute, sharding rule configuration, when exception occur etc.
   
   ### Example codes for reproduce this issue (such as a github link).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] sandynz commented on issue #14415: ColumnMetaDataLoader.load throw exception on ShardingSphereDataSource

Posted by GitBox <gi...@apache.org>.
sandynz commented on issue #14415:
URL: https://github.com/apache/shardingsphere/issues/14415#issuecomment-1002573547


   Since `ShardingSphereDatabaseMetaData.getActualTableNamePattern` will extend table name pattern, it's not suitable for scaling.
   And `ColumnMetaDataLoader` is the kernel of ShardingSphere, it's better not to change it for new scenario.
   
   Workaround: Pipeline metadata loader is recovered and primary key loading is added for scaling.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] tristaZero closed issue #14415: ColumnMetaDataLoader.load throw exception on ShardingSphereDataSource

Posted by GitBox <gi...@apache.org>.
tristaZero closed issue #14415:
URL: https://github.com/apache/shardingsphere/issues/14415


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org