[GitHub] [shardingsphere] sandynz commented on pull request #16763: Refactor check target data source, protect by global lock

[GitBox <gi...@apache.org>]

sandynz commented on PR #16763:
URL: https://github.com/apache/shardingsphere/pull/16763#issuecomment-1096383835

   Target tables preparation and target dat source check might be executed concurrently, after target tables preparation is protected by global lock and some thread will be blocked for some time, when the later thread check target data source, there's already new data imported, it'll break.
   
   Current solution is put target data source check in the same global lock, it'll be executed once.
   
   Example exception:
   ```
   [ERROR] 2022-04-12 13:32:12.144 [elasticjob-0130317c30317c3054317c7363616c696e675f6462-2] o.a.s.d.p.s.r.RuleAlteredJob - job prepare failed, 0130317c30317c3054317c7363616c696e675f6462-1
   org.apache.shardingsphere.data.pipeline.core.exception.PipelineJobPrepareFailedException: Target table `t_order_item` is not empty, sql: SELECT * FROM t_order_item LIMIT 1.
   	at org.apache.shardingsphere.data.pipeline.core.check.datasource.AbstractDataSourceChecker.checkEmpty(AbstractDataSourceChecker.java:66)
   	at org.apache.shardingsphere.data.pipeline.core.check.datasource.AbstractDataSourceChecker.checkTargetTable(AbstractDataSourceChecker.java:52)
   	at org.apache.shardingsphere.data.pipeline.scenario.rulealtered.RuleAlteredJobPreparer.checkTargetDataSource(RuleAlteredJobPreparer.java:158)
   	at org.apache.shardingsphere.data.pipeline.scenario.rulealtered.RuleAlteredJobPreparer.initAndCheckDataSource(RuleAlteredJobPreparer.java:142)
   	at org.apache.shardingsphere.data.pipeline.scenario.rulealtered.RuleAlteredJobPreparer.prepare(RuleAlteredJobPreparer.java:82)
   	at org.apache.shardingsphere.data.pipeline.scenario.rulealtered.RuleAlteredJob.execute(RuleAlteredJob.java:51)
   ```
   
   Another possible solution doesn't work:
   Every concurrent thread check its own physical target data source, it's impossible, since data will be imported to the same sharding target, there's no isolation.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org