[GitHub] [shardingsphere] sandynz opened a new issue, #22186: data pipeline modules refactoring

[GitBox <gi...@apache.org>]

sandynz opened a new issue, #22186:
URL: https://github.com/apache/shardingsphere/issues/22186

   Hi community,
   
   We've added some new scenarios for data pipeline in recent months, and there will be more scenarios later.
   In order to better support of scenraios development and maintanance, after discussion with @terrymanu and @tristaZero , we'll do some refactoring of data pipeline modules.
   
   data pipeline modules structure:
   ```
   data-pipeline
   	- api
   	- core (in L1 layer)
   	- dialect
   		- mysql
   		- postgresql
   		- opengauss
   	- scenario (new module, in L2 layer)
   	- cdc (new module, in L1 layer)
   	- distsql (new module)
   		- parser
   		- statement
   		- handler
   ```
   
   Major changes:
   
   1, Add `scenario` module, it will be in L2 layer, so it could depend on sharding-core and encrypt-core modules, then we could avoid RequiredSPI and move implementation classes to here. Classes in `core` module `scenario` package will be moved here.
   
   2, Add `cdc` module, it will be in L1 layer, used for CDC feature.
   
   3, Add `distsql` modules, classes in sub-modules of `features/sharding/distsql/` will be moved here.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] sandynz commented on issue #22186: data pipeline modules refactoring

[GitBox <gi...@apache.org>]

sandynz commented on issue #22186:
URL: https://github.com/apache/shardingsphere/issues/22186#issuecomment-1333080774

   From suggestion of @terrymanu , `scenario` module will be splitted by different scenarios, so module dependencies could be more clear.
   `distsql` module won't be splitted for now, since `distsql` module has 3 sub-modules, if there're 5 scenarios, then there will be 3x5=15 modules. And g4 grammar and code of `distsql` sub-modules might be shared by different scenarios.
   
   New modules:
   - scenario
     - scenario-migration
     - scenario-consistencycheck
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] taojintianxia closed issue #22186: data pipeline modules refactoring

[GitBox <gi...@apache.org>]

taojintianxia closed issue #22186: data pipeline modules refactoring
URL: https://github.com/apache/shardingsphere/issues/22186


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org