You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/12/22 06:47:52 UTC
[GitHub] [shardingsphere] azexcy opened a new pull request, #23026: Add CDC job start subscription and add client connection status.
azexcy opened a new pull request, #23026:
URL: https://github.com/apache/shardingsphere/pull/23026
Related #22500 .
Changes proposed in this pull request:
- Add CDC job start subscription
- Add client connection status
---
Before committing this PR, I'm sure that I have checked the following options:
- [ ] My code follows the [code of conduct](https://shardingsphere.apache.org/community/en/involved/conduct/code/) of this project.
- [ ] I have self-reviewed the commit code.
- [ ] I have (or in comment I request) added corresponding labels for the pull request.
- [ ] I have passed maven check locally : `./mvnw clean install -B -T1C -Dmaven.javadoc.skip -Dmaven.jacoco.skip -e`.
- [ ] I have made corresponding changes to the documentation.
- [ ] I have added corresponding unit tests for my changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz merged pull request #23026: Add CDC job start subscription and add client connection status.
Posted by GitBox <gi...@apache.org>.
sandynz merged PR #23026:
URL: https://github.com/apache/shardingsphere/pull/23026
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] codecov-commenter commented on pull request #23026: Add CDC job start subscription and add client connection status.
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #23026:
URL: https://github.com/apache/shardingsphere/pull/23026#issuecomment-1362497370
# [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/23026?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#23026](https://codecov.io/gh/apache/shardingsphere/pull/23026?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (61d7943) into [master](https://codecov.io/gh/apache/shardingsphere/commit/daf568ecad544e311f440b348476bbfd18c9be98?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (daf568e) will **decrease** coverage by `0.03%`.
> The diff coverage is `10.08%`.
```diff
@@ Coverage Diff @@
## master #23026 +/- ##
============================================
- Coverage 50.06% 50.02% -0.04%
+ Complexity 2428 2426 -2
============================================
Files 4111 4111
Lines 57361 57389 +28
Branches 9042 9044 +2
============================================
- Hits 28715 28711 -4
- Misses 26134 26170 +36
+ Partials 2512 2508 -4
```
| [Impacted Files](https://codecov.io/gh/apache/shardingsphere/pull/23026?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../pipeline/spi/importer/ImporterCreatorFactory.java](https://codecov.io/gh/apache/shardingsphere/pull/23026/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a2VybmVsL2RhdGEtcGlwZWxpbmUvYXBpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL3NwaS9pbXBvcnRlci9JbXBvcnRlckNyZWF0b3JGYWN0b3J5LmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...ne/cdc/client/constant/ClientConnectionStatus.java](https://codecov.io/gh/apache/shardingsphere/pull/23026/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a2VybmVsL2RhdGEtcGlwZWxpbmUvY2RjL2NsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvZGF0YS9waXBlbGluZS9jZGMvY2xpZW50L2NvbnN0YW50L0NsaWVudENvbm5lY3Rpb25TdGF0dXMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...ne/cdc/client/context/ClientConnectionContext.java](https://codecov.io/gh/apache/shardingsphere/pull/23026/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a2VybmVsL2RhdGEtcGlwZWxpbmUvY2RjL2NsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvZGF0YS9waXBlbGluZS9jZGMvY2xpZW50L2NvbnRleHQvQ2xpZW50Q29ubmVjdGlvbkNvbnRleHQuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...peline/cdc/client/handler/LoginRequestHandler.java](https://codecov.io/gh/apache/shardingsphere/pull/23026/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a2VybmVsL2RhdGEtcGlwZWxpbmUvY2RjL2NsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvZGF0YS9waXBlbGluZS9jZGMvY2xpZW50L2hhbmRsZXIvTG9naW5SZXF1ZXN0SGFuZGxlci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...cdc/client/handler/SubscriptionRequestHandler.java](https://codecov.io/gh/apache/shardingsphere/pull/23026/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a2VybmVsL2RhdGEtcGlwZWxpbmUvY2RjL2NsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvZGF0YS9waXBlbGluZS9jZGMvY2xpZW50L2hhbmRsZXIvU3Vic2NyaXB0aW9uUmVxdWVzdEhhbmRsZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...here/data/pipeline/cdc/api/impl/CDCJobAPIImpl.java](https://codecov.io/gh/apache/shardingsphere/pull/23026/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a2VybmVsL2RhdGEtcGlwZWxpbmUvY2RjL2NvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL2RhdGEvcGlwZWxpbmUvY2RjL2FwaS9pbXBsL0NEQ0pvYkFQSUltcGwuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...ata/pipeline/cdc/context/CDCConnectionContext.java](https://codecov.io/gh/apache/shardingsphere/pull/23026/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a2VybmVsL2RhdGEtcGlwZWxpbmUvY2RjL2NvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL2RhdGEvcGlwZWxpbmUvY2RjL2NvbnRleHQvQ0RDQ29ubmVjdGlvbkNvbnRleHQuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...ta/pipeline/cdc/context/job/CDCJobItemContext.java](https://codecov.io/gh/apache/shardingsphere/pull/23026/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a2VybmVsL2RhdGEtcGlwZWxpbmUvY2RjL2NvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL2RhdGEvcGlwZWxpbmUvY2RjL2NvbnRleHQvam9iL0NEQ0pvYkl0ZW1Db250ZXh0LmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../core/importer/connector/CDCImporterConnector.java](https://codecov.io/gh/apache/shardingsphere/pull/23026/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a2VybmVsL2RhdGEtcGlwZWxpbmUvY2RjL2NvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL2RhdGEvcGlwZWxpbmUvY2RjL2NvcmUvaW1wb3J0ZXIvY29ubmVjdG9yL0NEQ0ltcG9ydGVyQ29ubmVjdG9yLmphdmE=) | `0.00% <0.00%> (-33.34%)` | :arrow_down: |
| [...rdingsphere/data/pipeline/cdc/core/job/CDCJob.java](https://codecov.io/gh/apache/shardingsphere/pull/23026/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a2VybmVsL2RhdGEtcGlwZWxpbmUvY2RjL2NvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL2RhdGEvcGlwZWxpbmUvY2RjL2NvcmUvam9iL0NEQ0pvYi5qYXZh) | `0.00% <0.00%> (ø)` | |
| ... and [16 more](https://codecov.io/gh/apache/shardingsphere/pull/23026/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz commented on a diff in pull request #23026: Add CDC job start subscription and add client connection status.
Posted by GitBox <gi...@apache.org>.
sandynz commented on code in PR #23026:
URL: https://github.com/apache/shardingsphere/pull/23026#discussion_r1056901933
##########
kernel/data-pipeline/core/src/main/java/org/apache/shardingsphere/data/pipeline/core/metadata/node/event/handler/PipelineJobConfigurationChangedProcessor.java:
##########
@@ -24,8 +24,7 @@
/**
* Pipeline changed job configuration processor.
*/
-// TODO rename to PipelineJobConfigurationChangedProcessor
-public interface PipelineChangedJobConfigurationProcessor extends TypedSPI {
+public interface PipelineJobConfigurationChangedProcessor extends TypedSPI {
Review Comment:
Keep it for now.
1, It's not decided how to rename it.
2, Refactoring could be in another PR.
##########
proxy/backend/src/main/java/org/apache/shardingsphere/proxy/backend/handler/cdc/CDCBackendHandler.java:
##########
@@ -71,13 +88,53 @@ public CDCResponse createSubscription(final CDCRequest request) {
}
CreateSubscriptionJobParameter parameter = new CreateSubscriptionJobParameter(subscriptionRequest.getDatabase(), tableNames, subscriptionRequest.getSubscriptionName(),
subscriptionRequest.getSubscriptionMode().name(), actualDataNodesMap);
- CDCJobAPIFactory.getInstance().createJobAndStart(parameter);
- return CDCResponseGenerator.succeedBuilder(request.getRequestId()).build();
+ if (CDCJobAPIFactory.getInstance().createJob(parameter)) {
+ return CDCResponseGenerator.succeedBuilder(request.getRequestId()).setCreateSubscriptionResult(CreateSubscriptionResult.newBuilder()
+ .setSubscriptionName(subscriptionRequest.getSubscriptionName()).setExisting(false).build()).build();
+ } else {
+ return CDCResponseGenerator.succeedBuilder(request.getRequestId()).setCreateSubscriptionResult(CreateSubscriptionResult.newBuilder()
+ .setSubscriptionName(subscriptionRequest.getSubscriptionName()).setExisting(true).build()).build();
+ }
}
private List<DataNode> getActualDataNodes(final ShardingRule shardingRule, final String logicTableName) {
TableRule tableRule = shardingRule.getTableRule(logicTableName);
// TODO support virtual data source name
return tableRule.getActualDataNodes();
}
+
+ /**
+ * Start subscription.
+ *
+ * @param request request
+ * @param channel channel
+ * @param connectionContext connection context
+ * @return CDC response
+ */
+ public CDCResponse startSubscription(final CDCRequest request, final Channel channel, final CDCConnectionContext connectionContext) {
+ StartSubscriptionRequest startSubscriptionRequest = request.getStartSubscription();
+ CDCJobAPI jobAPI = CDCJobAPIFactory.getInstance();
+ String jobId = jobAPI.marshalJobId(new CDCJobId(startSubscriptionRequest.getDatabase(), startSubscriptionRequest.getSubscriptionName()));
+ String jobConfigText = PipelineContext.getContextManager().getMetaDataContexts().getPersistService().getRepository().getDirectly(PipelineMetaDataNode.getJobConfigPath(jobId));
+ if (null == jobConfigText || jobConfigText.isEmpty()) {
+ return CDCResponseGenerator.failed(request.getRequestId(), CDCResponseErrorCode.SERVER_ERROR, String.format("the %s job config isn't exists",
+ startSubscriptionRequest.getSubscriptionName()));
+ }
+ JobConfigurationPOJO jobConfigPOJO = YamlEngine.unmarshal(jobConfigText, JobConfigurationPOJO.class);
Review Comment:
It's better to get job configuration via jobAPI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org