You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "npofsi (via GitHub)" <gi...@apache.org> on 2025/08/07 08:38:01 UTC
[PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
npofsi opened a new pull request, #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402
<!--Thanks very much for contributing to Apache DolphinScheduler, we are happy that you want to help us improve DolphinScheduler! -->
## Purpose of the pull request
This pull request adds an implementation for gRPC plugin which is introduced in [DSIP-90](github.com/apache/dolphinscheduler/issues/17290). It provides an JSON api to transfer protobuf definitions between webpage and server.
Close #17290
## Brief change log
BrowserSide:
- Add gRPC plugin component in workflow space
- Add gRPC plugin UI to accept Protobuf and messages
- Translate definitions from Protobuf to JSON use protobuf.js
ServerSide:
- Written a JSON Parser to reconstruct the Protobuf from browser side
- Making gRPC call from server and check the response
## Verify this pull request
<!--*(Please pick either of the following options)*-->
This pull request is code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
<!--*(example:)*
- *Added dolphinscheduler-dao tests for end-to-end.*
- *Added CronUtilsTest to verify the change.*
- *Manually verified the change by testing locally.* -->
(or)
## Pull Request Notice
[Pull Request Notice](https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)
If your pull request contains incompatible change, you should also add it to `docs/docs/en/guide/upgrade/incompatible.md`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3393051155
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [221 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3280878153
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343013001
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
All of we need at runtime construction are the things after the protobuf compile. The protobuf workflow is demostrated in following image.
https://i0.wp.com/lab.wallarm.com/wp-content/uploads/2023/12/253-min.jpg?w=770&ssl=1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2340983869
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
To get a structed protobuf descriptor, the only choice is `protoc` or `protobufjs`. The first one needs the dependency of the binary executable protoc in paths. To avoid an additional dependency, we choose protobufjs. We use it to generate a structed protobuf descriptor at frontend, and reconstruct the protobuf descriptor at backend. We can not use a .proto text as a structed definition directly (which is different from json).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3281305741
Please retry analysis of this Pull-Request directly on SonarQube Cloud
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3315114553
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3277740004
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2335296406
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorHelper.java:
##########
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+
+import lombok.val;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorHelper {
+
+ public static Root ProtobufFromJSON(String json) throws JsonProcessingException {
+ ObjectMapper mapper = JsonMapper.builder()
+ .build();
+ return mapper.readValue(json, Root.class);
Review Comment:
Using `JsonUtils` instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3208871890
Please retry analysis of this Pull-Request directly on SonarQube Cloud
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3179680607
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [72 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [70.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3264557416
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [80 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3264558788
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [80 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2329485976
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
+
+ public Descriptors.FileDescriptor buildDescriptor(Root root) throws Descriptors.DescriptorValidationException {
+ return parseRoot(root);
+ }
+
+ private Namespace findInnerNamespace(Namespace ns) {
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return ns;
+ }
+
+ public String readPackageName(Root root) {
+ List<String> packageNameNS = new ArrayList<>();
+ Namespace ns = root;
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ packageNameNS.add((String) ns.nested.keySet().toArray()[0]);
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return String.join(".", packageNameNS);
Review Comment:
Get it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3264399729
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [80 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2341305922
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
We can generate the `.proto` file, then avoid doing the complex parsing here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2341305922
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
We can generate the `.proto` file, then no need to parser this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2340983869
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
To get a structed protobuf descriptor, the only choices are `protoc` or `protobufjs`. The first one needs the dependence of the binary executable protoc in paths. To avoid an additional dependence, we choose protobufjs. We use it to generate a structed protobuf descriptor at frontend, and reconstruct the protobuf descriptor at backend. We can not use a .proto text directly as a structed definition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3305320746
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343540824
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
Not all requests originate from the UI; they may also come from the API, so the frontend validation isn't particularly important, we still need to perform all validation on the backend.
I ask gpt, it tells me that we can use some lib to parse the proto, we don't need to write the parser here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3420729077
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3415518269
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [168 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS merged PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2443637910
##########
docs/docs/en/guide/task/grpc.md:
##########
@@ -0,0 +1,57 @@
+# GRPC Node
+
+## Overview
+
+This node is used to execute gRPC tasks and supports checking gRPC status codes, SSL/TLS, and other features.
+
+## Create Task
+
+- Click Project Management -> Project Name -> Workflow Definition, then click the "Create Workflow" button to enter the DAG editing page:
+
+- Drag the <img src="../../../../img/tasks/icons/grpc.png" width="15"/> task node from the toolbar to the canvas.
+
+## Task Parameters
+
+[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
+[//]: # (- For default parameter descriptions, please refer to [DolphinScheduler Task Parameter Appendix](appendix.md#默认任务参数) under `Default Task Parameters`.)
+
+- For default parameter descriptions, please refer to [DolphinScheduler Task Parameter Appendix](appendix.md) under `Default Task Parameters`.
+
+| **Task Parameter** | **Description** |
+|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| gRPC Url | gRPC request URL, must use the `hostname:port` format |
+| gRPC Credential Type | Supports None (insecure), and client default SSL/TLS credential types |
+| Protobuf Definition | Protobuf code defining the service in the `.proto` file, will be converted to JSON Descriptor when saved |
+| Protobuf Definition JSON | Used for creating tasks via API, generated from the above definition as [JSON Descriptor](https://github.com/protobufjs/protobuf.js/#using-json-descriptors) |
Review Comment:
Is this need to appear at docs? We don't have a task parameter named `Protobuf Definition JSON`in the task form.
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/GrpcParameters.java:
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc;
+
+import org.apache.dolphinscheduler.plugin.task.api.parameters.AbstractParameters;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.GrpcDynamicService;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.JSONDescriptorHelper;
+
+import org.apache.commons.lang3.StringUtils;
+
+import lombok.Data;
+import lombok.EqualsAndHashCode;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.google.protobuf.Descriptors;
+
+@EqualsAndHashCode(callSuper = true)
+@Data
+public class GrpcParameters extends AbstractParameters {
+
+ private String url;
+
+ private GrpcCredentialType channelCredentialType = GrpcCredentialType.INSECURE;
+
+ private String grpcServiceDefinition;
+
+ private String grpcServiceDefinitionJSON;
+
+ private String methodName;
+
+ private String message;
+
+ private GrpcCheckCondition grpcCheckCondition = GrpcCheckCondition.STATUS_CODE_DEFAULT;
+
+ private String condition;
+
+ @JsonProperty("connectTimeout")
Review Comment:
```suggestion
@JsonProperty("connectTimeoutMs")
```
##########
.github/workflows/e2e.yml:
##########
@@ -192,6 +192,7 @@ jobs:
- name: Run Test
run: |
./mvnw -B -f dolphinscheduler-e2e/pom.xml -am \
+ -DtrimStackTrace=false \
Review Comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "github-advanced-security[bot] (via GitHub)" <gi...@apache.org>.
github-advanced-security[bot] commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2434499815
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/Enum.java:
##########
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import java.util.Map;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [ReflectionObject.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5560)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/Field.java:
##########
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import lombok.Data;
+import lombok.Getter;
+import lombok.Setter;
+
+import com.fasterxml.jackson.annotation.JsonSubTypes;
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
+import com.fasterxml.jackson.databind.JsonNode;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [ReflectionObject.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5563)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/MapField.java:
##########
@@ -0,0 +1,26 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [Field.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5562)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/Method.java:
##########
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [ReflectionObject.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5561)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/OneOf.java:
##########
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import java.util.ArrayList;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [ReflectionObject.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5564)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/Type.java:
##########
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import java.util.Map;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [Namespace.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5567)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/Service.java:
##########
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import java.util.Map;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [Namespace.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5568)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/Namespace.java:
##########
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import java.util.Map;
+
+import lombok.Data;
+
+import com.fasterxml.jackson.annotation.JsonSubTypes;
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [ReflectionObject.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5565)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/types/OneofType.java:
##########
@@ -0,0 +1,34 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping.OneOf;
+
+import com.google.protobuf.DescriptorProtos;
+
+public class OneofType {
+
+ public static DescriptorProtos.OneofDescriptorProto.Builder parseOneof(DescriptorProtos.DescriptorProto.Builder descriptorProtoBuilder,
+ String selfName, OneOf oneof) {
Review Comment:
## Useless parameter
The parameter 'oneof' is never used.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5559)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/Root.java:
##########
@@ -0,0 +1,27 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import lombok.Data;
+
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [Namespace.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5566)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3166439246
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [65 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [63.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3395736838
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [219 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3273277222
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "github-advanced-security[bot] (via GitHub)" <gi...@apache.org>.
github-advanced-security[bot] commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2259863708
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,195 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.ReflectionObject;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
+
+ public Descriptors.FileDescriptor buildDescriptor(Root root) throws Descriptors.DescriptorValidationException {
+ return parseRoot(root);
+ }
+
+ private void parse(String name, ReflectionObject pbObject) {
Review Comment:
## Useless parameter
The parameter 'name' is never used.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5501)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,195 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.ReflectionObject;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
+
+ public Descriptors.FileDescriptor buildDescriptor(Root root) throws Descriptors.DescriptorValidationException {
+ return parseRoot(root);
+ }
+
+ private void parse(String name, ReflectionObject pbObject) {
Review Comment:
## Useless parameter
The parameter 'pbObject' is never used.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5502)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/types/Enum.java:
##########
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types;
+
+import java.util.Map;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [ReflectionObject.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5505)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/types/OneOf.java:
##########
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types;
+
+import java.util.ArrayList;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [ReflectionObject.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5510)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/types/Method.java:
##########
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [ReflectionObject.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5508)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/types/MapField.java:
##########
@@ -0,0 +1,26 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [Field.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5506)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,195 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.ReflectionObject;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
+
+ public Descriptors.FileDescriptor buildDescriptor(Root root) throws Descriptors.DescriptorValidationException {
+ return parseRoot(root);
+ }
+
+ private void parse(String name, ReflectionObject pbObject) {
+
+ }
+
+ private Namespace parseNamespace(Namespace ns) {
+ List<String> packageNameNS = new ArrayList<>();
Review Comment:
## Container contents are never accessed
The contents of this container are never accessed.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5503)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/types/Namespace.java:
##########
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types;
+
+import java.util.Map;
+
+import lombok.Data;
+
+import com.fasterxml.jackson.annotation.JsonSubTypes;
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [ReflectionObject.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5509)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/types/Root.java:
##########
@@ -0,0 +1,27 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types;
+
+import lombok.Data;
+
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [Namespace.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5512)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/GrpcTask.java:
##########
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.common.utils.JSONUtils;
+import org.apache.dolphinscheduler.plugin.task.api.AbstractTask;
+import org.apache.dolphinscheduler.plugin.task.api.TaskCallBack;
+import org.apache.dolphinscheduler.plugin.task.api.TaskConstants;
+import org.apache.dolphinscheduler.plugin.task.api.TaskException;
+import org.apache.dolphinscheduler.plugin.task.api.TaskExecutionContext;
+import org.apache.dolphinscheduler.plugin.task.api.enums.DataType;
+import org.apache.dolphinscheduler.plugin.task.api.enums.Direct;
+import org.apache.dolphinscheduler.plugin.task.api.model.Property;
+import org.apache.dolphinscheduler.plugin.task.api.parameters.AbstractParameters;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.GrpcDynamicService;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.JSONDescriptorHelper;
+
+import lombok.Setter;
+import lombok.extern.slf4j.Slf4j;
+
+import com.google.protobuf.Descriptors;
+import com.google.protobuf.DynamicMessage;
+import com.google.protobuf.util.JsonFormat;
+import com.google.protobuf.util.JsonFormat.Printer;
+
+import io.grpc.ManagedChannel;
+import io.grpc.Status;
+import io.grpc.StatusRuntimeException;
+
+@Slf4j
+public class GrpcTask extends AbstractTask {
+
+ private GrpcParameters grpcParameters;
+ private TaskExecutionContext taskExecutionContext;
+
+ /**
+ * constructor
+ *
+ * @param taskExecutionContext taskExecutionContext
+ */
+ protected GrpcTask(TaskExecutionContext taskExecutionContext) {
+ super(taskExecutionContext);
+ this.taskExecutionContext = taskExecutionContext;
+ }
+
+ @Override
+ public void init() {
+ this.grpcParameters = JSONUtils.parseObject(taskExecutionContext.getTaskParams(), GrpcParameters.class);
+ log.info("Initialize gRPC task params: {}", JSONUtils.toPrettyJsonString(grpcParameters));
+
+ if (grpcParameters == null || !grpcParameters.checkParameters()) {
+ throw new RuntimeException("gRPC task params is not valid");
+ }
+ }
+
+ @Override
+ public void handle(TaskCallBack taskCallBack) throws TaskException {
+ try {
+ ManagedChannel channel =
+ isNull(this.channel) ? GrpcDynamicService.ChannelFactory.createChannel(grpcParameters.getUrl())
+ : this.channel;
Review Comment:
## Possible confusion of local and field
Potentially confusing name: method [handle](1) also refers to field [channel](2) (as this.channel).
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5504)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/types/Field.java:
##########
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types;
+
+import java.util.Map;
+
+import lombok.Data;
+
+import com.fasterxml.jackson.annotation.JsonSubTypes;
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [ReflectionObject.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5507)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/types/Service.java:
##########
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types;
+
+import java.util.Map;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [Namespace.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5511)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/types/Type.java:
##########
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types;
+
+import java.util.Map;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [Namespace.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5513)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3166277898
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [70 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [64.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3349730543
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3277420643
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2341293707
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/pom.xml:
##########
@@ -0,0 +1,174 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-task-grpc</artifactId>
+ <packaging>jar</packaging>
+
+ <properties>
+ <plugin.name>task.grpc</plugin.name>
+ <protoc.version>3.17.3</protoc.version>
+ <grpc.version>1.71.0</grpc.version>
+ </properties>
+
+ <dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-bom</artifactId>
+ <version>dev-SNAPSHOT</version>
+ <scope>import</scope>
+ </dependency>
+ </dependencies>
+ </dependencyManagement>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-common</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-services</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty-shaded</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-protobuf</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-stub</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ <scope>provided</scope>
+ </dependency>
Review Comment:
If so, it's better to remove the provided scope.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3263758739
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [80 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3287243544
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [77.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3287244162
Please retry analysis of this Pull-Request directly on SonarQube Cloud
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3409124677
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate failed**
Failed conditions
 [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3262102527
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3262318967
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3263600808
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [80 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "github-advanced-security[bot] (via GitHub)" <gi...@apache.org>.
github-advanced-security[bot] commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2287176606
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
+
+ public Descriptors.FileDescriptor buildDescriptor(Root root) throws Descriptors.DescriptorValidationException {
+ return parseRoot(root);
+ }
+
+ private Namespace findInnerNamespace(Namespace ns) {
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return ns;
+ }
+
+ public String readPackageName(Root root) {
+ List<String> packageNameNS = new ArrayList<>();
+ Namespace ns = root;
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ packageNameNS.add((String) ns.nested.keySet().toArray()[0]);
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return String.join(".", packageNameNS);
+ }
+
+ private Descriptors.FileDescriptor parseRoot(Root root) throws Descriptors.DescriptorValidationException {
+ DescriptorProtos.FileDescriptorProto.Builder fileDescriptorProtoBuilder =
+ DescriptorProtos.FileDescriptorProto.newBuilder()
+ .setPackage(readPackageName(root));
+ Namespace innerNS = findInnerNamespace(root);
+ if (innerNS.nested != null)
+ innerNS.nested.forEach((name, pbObject) -> {
+ if (pbObject instanceof Namespace) {
+ Namespace ns = (Namespace) pbObject;
+ if (ns instanceof Type) {
+ fileDescriptorProtoBuilder.addMessageType(parseType(name, (Type) ns));
+ } else if (ns instanceof Service) {
+ fileDescriptorProtoBuilder.addService(parseService(name, (Service) ns));
+ } else {
+
+ }
+ } else if (pbObject instanceof Enum) {
+ fileDescriptorProtoBuilder.addEnumType(parseEnum(name, (Enum) pbObject));
+ } else if (pbObject instanceof Field) {
+
+ } else if (pbObject instanceof OneOf) {
+
+ } else if (pbObject instanceof Method) {
+
+ }
+ });
+ Descriptors.FileDescriptor fileDescriptor =
+ Descriptors.FileDescriptor.buildFrom(fileDescriptorProtoBuilder.build(),
+ new Descriptors.FileDescriptor[0]);
+ return fileDescriptor;
+ }
+
+ private DescriptorProtos.DescriptorProto.Builder parseType(String selfName, Type type) {
+ DescriptorProtos.DescriptorProto.Builder descriptorProtoBuilder = DescriptorProtos.DescriptorProto.newBuilder()
+ .setName(selfName);
+
+ Map<String, DescriptorProtos.FieldDescriptorProto.Builder> fieldMap = new HashMap<>();
+ if (type.fields != null)
+ type.fields.forEach((name, pbObject) -> {
+ if (pbObject instanceof Field) {
+ if (pbObject instanceof MapField) {
+ DescriptorProtos.FieldDescriptorProto.Builder mapField =
+ parseMapField(descriptorProtoBuilder, name, (MapField) pbObject);
+ fieldMap.put(name, mapField);
+ } else {
+ DescriptorProtos.FieldDescriptorProto.Builder field = parseField(name, (Field) pbObject);
+ fieldMap.put(name, field);
+ }
+
+ }
+ });
+ if (type.nested != null)
+ type.nested.forEach((name, pbObject) -> {
+ if (pbObject instanceof Enum) {
+ descriptorProtoBuilder.addEnumType(parseEnum(name, (Enum) pbObject));
+ } else if (pbObject instanceof Type) {
+ descriptorProtoBuilder.addNestedType(parseType(name, (Type) pbObject));
+ }
+ });
+ if (type.oneofs != null) {
+ type.oneofs.entrySet()
+ .stream()
+ .sorted((entry1, entry2) -> {
+ boolean key1StartsWithUnderscore = entry1.getKey().startsWith("_");
+ boolean key2StartsWithUnderscore = entry2.getKey().startsWith("_");
+ if (key1StartsWithUnderscore && !key2StartsWithUnderscore) {
+ return 1; // Move key1 after key2
+ } else if (!key1StartsWithUnderscore && key2StartsWithUnderscore) {
+ return -1; // Move key1 before key2
+ } else {
+ return 0; // Keep the original order
+ }
+ })
+ .collect(Collectors.toMap(
+ Map.Entry::getKey,
+ Map.Entry::getValue,
+ (oldValue, newValue) -> oldValue,
+ LinkedHashMap::new))
+ .forEach((name, oneof) -> {
+ parseOneof(descriptorProtoBuilder, name, oneof);
+ });
+ int oneofCount = descriptorProtoBuilder.getOneofDeclCount();
+ List<DescriptorProtos.OneofDescriptorProto> oneofDescriptorProtos =
+ descriptorProtoBuilder.getOneofDeclList();
+ oneofDescriptorProtos.forEach((oneof) -> {
+
+ });
+ IntStream.range(0, oneofCount)
+ .forEach(oneofIndex -> {
+ DescriptorProtos.OneofDescriptorProto oneofDescriptorProto =
+ oneofDescriptorProtos.get(oneofIndex);
+ String oneofName = oneofDescriptorProto.getName();
+ type.oneofs.get(oneofName).oneof.forEach((fieldName) -> {
+ fieldMap.get(fieldName).setOneofIndex(oneofIndex);
+ });
+ });
+ }
+ fieldMap.forEach((name, field) -> {
+ descriptorProtoBuilder.addField(field);
+ });
+ return descriptorProtoBuilder;
+ }
+
+ private DescriptorProtos.ServiceDescriptorProto.Builder parseService(String selfName, Service service) {
+ DescriptorProtos.ServiceDescriptorProto.Builder serviceDescriptorProtoBuilder =
+ DescriptorProtos.ServiceDescriptorProto.newBuilder()
+ .setName(selfName);
+ if (service.methods != null)
+ service.methods.forEach((name, pbObject) -> {
+ if (pbObject instanceof Method) {
+ serviceDescriptorProtoBuilder.addMethod(parseMethod(name, (Method) pbObject));
+ }
+ });
+ return serviceDescriptorProtoBuilder;
+ }
+
+ private DescriptorProtos.MethodDescriptorProto.Builder parseMethod(String selfName, Method method) {
+ DescriptorProtos.MethodDescriptorProto.Builder methodDescriptorProtoBuilder =
+ DescriptorProtos.MethodDescriptorProto.newBuilder()
+ .setName(selfName)
+ .setInputType(method.requestType)
+ .setOutputType(method.responseType);
+ return methodDescriptorProtoBuilder;
+ }
+
+ private DescriptorProtos.EnumDescriptorProto.Builder parseEnum(String selfName, Enum enumObj) {
+ DescriptorProtos.EnumDescriptorProto.Builder enumDescriptorProtoBuilder =
+ DescriptorProtos.EnumDescriptorProto.newBuilder()
+ .setName(selfName);
+ enumObj.values.forEach((name, id) -> {
+ DescriptorProtos.EnumValueDescriptorProto.Builder enumValueDescriptorProtoBuilder =
+ DescriptorProtos.EnumValueDescriptorProto.newBuilder()
+ .setName(name)
+ .setNumber(id);
+ enumDescriptorProtoBuilder.addValue(enumValueDescriptorProtoBuilder);
+ });
+ return enumDescriptorProtoBuilder;
+ }
+
+ private DescriptorProtos.FieldDescriptorProto.Builder parseField(String selfName, Field field) {
+ DescriptorProtos.FieldDescriptorProto.Builder fieldDescriptorProtoBuilder =
+ DescriptorProtos.FieldDescriptorProto.newBuilder()
+ .setName(selfName)
+ .setNumber(field.id);
+ if (!isNull(field.options)) {
+ if (!isNull(field.options.get("proto3_optional")) && ((boolean) field.options.get("proto3_optional"))) {
+ fieldDescriptorProtoBuilder.setProto3Optional(true);
+ }
+ }
+ JsonNode rule = field.rule;
+ if (!isNull(rule))
+ if (rule.isTextual()) {
+ String label = rule.asText();
+ try {
+ fieldDescriptorProtoBuilder
+ .setLabel(DescriptorProtos.FieldDescriptorProto.Label
+ .valueOf("LABEL_" + label.toUpperCase()));
+ } catch (IllegalArgumentException e) {
+ throw new RuntimeException("grpc exception: Unrecogenized field label: " + label, e);
+ }
+ }
+ try {
+ fieldDescriptorProtoBuilder
+ .setType(DescriptorProtos.FieldDescriptorProto.Type.valueOf("TYPE_" + field.type.toUpperCase()));
+ } catch (IllegalArgumentException e) {
+ fieldDescriptorProtoBuilder.setTypeName(field.type);
+ }
+ return fieldDescriptorProtoBuilder;
+ }
+
+ private DescriptorProtos.FieldDescriptorProto.Builder parseMapField(DescriptorProtos.DescriptorProto.Builder parentMessage,
+ String selfName, MapField mapField) {
+ DescriptorProtos.FieldDescriptorProto.Builder mapFieldDescriptorProtoBuilder =
+ DescriptorProtos.FieldDescriptorProto.newBuilder()
+ .setName(selfName)
+ .setNumber(mapField.id)
+ .setTypeName("MapEntry_" + selfName)
+ .setLabel(DescriptorProtos.FieldDescriptorProto.Label.LABEL_REPEATED);
+ DescriptorProtos.DescriptorProto.Builder mapEntryDescriptorProtoBuilder =
+ DescriptorProtos.DescriptorProto.newBuilder()
+ .setName("MapEntry_" + selfName)
+ .setOptions(DescriptorProtos.MessageOptions.newBuilder().setMapEntry(true).build());
+ DescriptorProtos.FieldDescriptorProto.Builder keyDescriptorProtoBuilder =
+ DescriptorProtos.FieldDescriptorProto.newBuilder()
+ .setName("key")
+ .setNumber(1);
+ DescriptorProtos.FieldDescriptorProto.Builder valueDescriptorProtoBuilder =
+ DescriptorProtos.FieldDescriptorProto.newBuilder()
+ .setName("value")
+ .setNumber(2);
+ try {
+ keyDescriptorProtoBuilder
+ .setType(DescriptorProtos.FieldDescriptorProto.Type
+ .valueOf("TYPE_" + mapField.keyType.toUpperCase()));
+ } catch (IllegalArgumentException e) {
+ keyDescriptorProtoBuilder.setTypeName(mapField.keyType);
+ }
+ try {
+ valueDescriptorProtoBuilder
+ .setType(DescriptorProtos.FieldDescriptorProto.Type.valueOf("TYPE_" + mapField.type.toUpperCase()));
+ } catch (IllegalArgumentException e) {
+ valueDescriptorProtoBuilder.setTypeName(mapField.type);
+ }
+ mapEntryDescriptorProtoBuilder.addField(keyDescriptorProtoBuilder);
+ mapEntryDescriptorProtoBuilder.addField(valueDescriptorProtoBuilder);
+ parentMessage.addNestedType(mapEntryDescriptorProtoBuilder);
+
+ return mapFieldDescriptorProtoBuilder;
+ }
+
+ private DescriptorProtos.OneofDescriptorProto.Builder parseOneof(DescriptorProtos.DescriptorProto.Builder descriptorProtoBuilder,
+ String selfName, OneOf oneof) {
Review Comment:
## Useless parameter
The parameter 'oneof' is never used.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5521)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3362168718
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [75 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3258422846
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3257629726
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2329028147
##########
dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/java/org/apache/dolphinscheduler/e2e/cases/workflow/BaseWorkflowE2ETest.java:
##########
@@ -167,8 +202,15 @@ protected TaskInstanceTab.Row untilTaskInstanceFailed(String workflowName, Strin
return await()
.until(() -> {
browser.navigate().refresh();
- List<TaskInstanceTab.Row> taskInstances = projectPage
- .goToTab(TaskInstanceTab.class)
+
+ await().ignoreException(NoSuchElementException.class)
+ .untilAsserted(() -> assertThat(browser)
+ .as("task instance is not found")
+ .matches(it -> it.findElement(By.className("n-menu-item-content")).isDisplayed()));
+ TaskInstanceTab taskInstanceTab = projectPage
+ .goToTab(TaskInstanceTab.class);
+
+ List<TaskInstanceTab.Row> taskInstances = taskInstanceTab
Review Comment:
Why change this? The current detection process is normal and does not need to be modified.
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-case/src/test/java/org/apache/dolphinscheduler/api/test/cases/ExecutorAPITest.java:
##########
@@ -145,4 +145,53 @@ public void testStartWorkflowInstance() {
}
}
+ @Test
+ @Order(2)
+ public void testGrpcWorkflowInstance() {
Review Comment:
We should move this test case into `GrpcTaskAPITest` since this is an new task type.
##########
dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/java/org/apache/dolphinscheduler/e2e/pages/LoginPage.java:
##########
@@ -70,7 +75,13 @@ public NavBarPage login(String username, String password) {
inputPassword().sendKeys(password);
buttonLogin().click();
- WebDriverWaitFactory.createWebDriverWait(driver).until(ExpectedConditions.urlContains("/home"));
+ await().ignoreException(NoSuchElementException.class)
+ .untilAsserted(() -> assertThat(driver)
+ .as("can not go to menu")
+ .matches(it -> it
+ .findElement(By.xpath(
+ "//div[contains(@class, 'tab-horizontal')]//div[contains(@role,'menubar')]"))
+ .isDisplayed()));
Review Comment:
Same here. Why change this?
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/pom.xml:
##########
@@ -0,0 +1,297 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-task-grpc</artifactId>
+ <packaging>jar</packaging>
+
+ <properties>
+ <plugin.name>task.grpc</plugin.name>
+ <grpc.version>1.71.0</grpc.version>
+ <protobuf.version>3.17.3</protobuf.version>
+ <protoc.version>3.17.3</protoc.version>
+ </properties>
+
+ <dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-bom</artifactId>
+ <version>${grpc.version}</version>
+ <type>pom</type>
+ <scope>import</scope>
+ </dependency>
+
+ </dependencies>
+ </dependencyManagement>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-common</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>com.squareup.okhttp3</groupId>
+ <artifactId>mockwebserver</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-services</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.code.gson</groupId>
+ <artifactId>gson</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-transport-native-unix-common</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-codec-http2</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-handler-proxy</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-api</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty-shaded</artifactId>
+ <scope>runtime</scope>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-protobuf</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-stub</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ <version>${protobuf.version}</version>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.gson</groupId>
+ <artifactId>gson</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>com.google.j2objc</groupId>
+ <artifactId>j2objc-annotations</artifactId>
+ <version>3.0.0</version>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-testing</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>4.13.2</version>
Review Comment:
Same here.
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
+
+ public Descriptors.FileDescriptor buildDescriptor(Root root) throws Descriptors.DescriptorValidationException {
+ return parseRoot(root);
+ }
+
+ private Namespace findInnerNamespace(Namespace ns) {
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return ns;
+ }
+
+ public String readPackageName(Root root) {
+ List<String> packageNameNS = new ArrayList<>();
+ Namespace ns = root;
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ packageNameNS.add((String) ns.nested.keySet().toArray()[0]);
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return String.join(".", packageNameNS);
Review Comment:
Please using `Constants` instead of hard code it.
##########
dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/java/org/apache/dolphinscheduler/e2e/pages/project/ProjectDetailPage.java:
##########
@@ -53,12 +58,24 @@ public <T extends Tab> T goToTab(Class<T> tab) {
menuWorkflowDefinition().click();
WebDriverWaitFactory.createWebDriverWait(driver)
.until(ExpectedConditions.urlContains("/workflow-definition"));
+
+ await().ignoreException(NoSuchElementException.class)
+ .untilAsserted(() -> assertThat(driver)
+ .as("workflow definition page not found")
+ .matches(it -> it.findElement(By.className("n-data-table-base-table-body")).isDisplayed()));
+
return tab.cast(new WorkflowDefinitionTab(driver));
}
if (tab == WorkflowInstanceTab.class) {
menuWorkflowInstances().click();
WebDriverWaitFactory.createWebDriverWait(driver)
.until(ExpectedConditions.urlContains("/workflow/instances"));
+
+ await().ignoreException(NoSuchElementException.class)
+ .untilAsserted(() -> assertThat(driver)
+ .as("workflow instance page not found")
+ .matches(it -> it.findElement(By.className("n-data-table-base-table-body")).isDisplayed()));
+
Review Comment:
Same here. Why change this?
##########
dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/java/org/apache/dolphinscheduler/e2e/pages/project/ProjectPage.java:
##########
@@ -132,13 +140,21 @@ public ProjectPage verifyAssignedWorkerGroup(String project, String workerGroup)
}
public ProjectDetailPage goTo(String project) {
+ await().untilAsserted(() -> assertThat(driver)
+ .as("not in project detail page")
+ .matches(it -> it.findElement(By.className("n-layout-header")).isDisplayed()));
+
projectList().stream()
.filter(it -> it.getText().contains(project))
.map(it -> it.findElement(By.className("project-name")).findElement(new By.ByTagName("button")))
.findFirst()
.orElseThrow(() -> new RuntimeException("Cannot click the project item"))
.click();
+ await().untilAsserted(() -> assertThat(driver)
+ .as("not in project detail page")
+ .matches(it -> it.findElement(By.className("n-menu-item-content")).isDisplayed()));
+
Review Comment:
Same here. Why change this?
##########
dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/resources/docker/grpc-task/docker-compose.yaml:
##########
@@ -0,0 +1,53 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+version: "3.8"
+
+services:
+ dolphinscheduler:
+ image: apache/dolphinscheduler-standalone-server:ci
+ environment:
+ MASTER_MAX_CPU_LOAD_AVG: 400
+ MASTER_RESERVED_MEMORY: 0.01
+ WORKER_TENANT_AUTO_CREATE: 'true'
+ ports:
+ - "12345:12345"
+ networks:
+ - e2e
+ healthcheck:
+ test: [ "CMD", "curl", "http://localhost:12345/dolphinscheduler/actuator/health" ]
+ interval: 5s
+ timeout: 5s
+ retries: 120
+ greeterServer:
+ image: grpc/java-example-hostname:1.75.0
+# environment:
Review Comment:
```suggestion
```
Please remove this.
##########
dolphinscheduler-e2e/dolphinscheduler-e2e-core/src/main/java/org/apache/dolphinscheduler/e2e/core/DolphinSchedulerExtension.java:
##########
@@ -106,8 +107,9 @@ public void beforeAll(ExtensionContext context) throws IOException {
driver.manage().timeouts()
.implicitlyWait(Duration.ofSeconds(1))
.pageLoadTimeout(Duration.ofSeconds(5));
+ Dimension size = new Dimension(1920, 1080);
Review Comment:
Why change this?
##########
dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/java/org/apache/dolphinscheduler/e2e/pages/project/workflow/WorkflowDefinitionTab.java:
##########
@@ -72,6 +73,11 @@ public WorkflowDefinitionTab(RemoteWebDriver driver) {
public WorkflowForm createWorkflow() {
buttonCreateWorkflow().click();
+ await().ignoreException(NoSuchElementException.class)
+ .untilAsserted(() -> assertThat(driver)
+ .as("created workflow must be displayed")
+ .matches(it -> it.findElement(By.className("n-collapse-item")).isDisplayed()));
+
Review Comment:
Same here. Why change this?
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/test/java/org/apache/dolphinscheduler/plugin/task/grpc/GrpcCleanupExtension.java:
##########
@@ -0,0 +1,176 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+import org.junit.jupiter.api.extension.AfterEachCallback;
+import org.junit.jupiter.api.extension.ExtensionContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.grpc.BindableService;
+import io.grpc.ManagedChannel;
+import io.grpc.Metadata;
+import io.grpc.Server;
+import io.grpc.ServerCall;
+import io.grpc.ServerCallHandler;
+import io.grpc.ServerInterceptor;
+import io.grpc.inprocess.InProcessChannelBuilder;
+import io.grpc.inprocess.InProcessServerBuilder;
+
+public class GrpcCleanupExtension implements AfterEachCallback {
+
+ private final List<CleanupTarget> cleanupTargets = new ArrayList<>();
+
+ private static final Logger logger = LoggerFactory.getLogger(GrpcCleanupExtension.class);
Review Comment:
```suggestion
```
Using `@Slf4j` instead.
##########
dolphinscheduler-ui/pnpm-lock.yaml:
##########
Review Comment:
Please revert unnessnary change in this file.
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/test/java/org/apache/dolphinscheduler/plugin/task/grpc/.gitignore:
##########
Review Comment:
We should remove unnessnary file.
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/pom.xml:
##########
@@ -0,0 +1,297 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-task-grpc</artifactId>
+ <packaging>jar</packaging>
+
+ <properties>
+ <plugin.name>task.grpc</plugin.name>
+ <grpc.version>1.71.0</grpc.version>
+ <protobuf.version>3.17.3</protobuf.version>
+ <protoc.version>3.17.3</protoc.version>
+ </properties>
+
+ <dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-bom</artifactId>
+ <version>${grpc.version}</version>
+ <type>pom</type>
+ <scope>import</scope>
+ </dependency>
+
+ </dependencies>
+ </dependencyManagement>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-common</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>com.squareup.okhttp3</groupId>
+ <artifactId>mockwebserver</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-services</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.code.gson</groupId>
+ <artifactId>gson</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-transport-native-unix-common</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-codec-http2</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-handler-proxy</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-api</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty-shaded</artifactId>
+ <scope>runtime</scope>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-protobuf</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-stub</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ <version>${protobuf.version}</version>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.gson</groupId>
+ <artifactId>gson</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>com.google.j2objc</groupId>
+ <artifactId>j2objc-annotations</artifactId>
+ <version>3.0.0</version>
Review Comment:
Dependeny version should be setting in `bom/pom.xml`.
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/pom.xml:
##########
@@ -0,0 +1,297 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-task-grpc</artifactId>
+ <packaging>jar</packaging>
+
+ <properties>
+ <plugin.name>task.grpc</plugin.name>
+ <grpc.version>1.71.0</grpc.version>
+ <protobuf.version>3.17.3</protobuf.version>
+ <protoc.version>3.17.3</protoc.version>
+ </properties>
+
+ <dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-bom</artifactId>
+ <version>${grpc.version}</version>
+ <type>pom</type>
+ <scope>import</scope>
+ </dependency>
+
+ </dependencies>
+ </dependencyManagement>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-common</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>com.squareup.okhttp3</groupId>
+ <artifactId>mockwebserver</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-services</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.code.gson</groupId>
+ <artifactId>gson</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-transport-native-unix-common</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-codec-http2</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-handler-proxy</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-api</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty-shaded</artifactId>
+ <scope>runtime</scope>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-protobuf</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-stub</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ <version>${protobuf.version}</version>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.gson</groupId>
+ <artifactId>gson</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>com.google.j2objc</groupId>
+ <artifactId>j2objc-annotations</artifactId>
+ <version>3.0.0</version>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-testing</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>4.13.2</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java</artifactId>
+ <version>${protobuf.version}</version>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <executions>
+ <execution>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <phase>package</phase>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>org.xolstice.maven.plugins</groupId>
+ <artifactId>protobuf-maven-plugin</artifactId>
+ <version>0.6.1</version>
+ <configuration>
+ <clearOutputDirectory>false</clearOutputDirectory>
+ <protocArtifact>com.google.protobuf:protoc:${protoc.version}:exe:${os.detected.classifier}</protocArtifact>
+ <pluginId>grpc-java</pluginId>
+ <pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}:exe:${os.detected.classifier}</pluginArtifact>
+ <protoSourceRoot>${project.basedir}/src/test/proto</protoSourceRoot>
+ <outputDirectory>${project.basedir}/src/test/java/</outputDirectory>
+ </configuration>
+ <executions>
+ <execution>
+ <goals>
+ <goal>test-compile</goal>
+ <goal>test-compile-custom</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-enforcer-plugin</artifactId>
+ <version>3.5.0</version>
+ <executions>
+ <execution>
+ <id>enforce</id>
+ <goals>
+ <goal>enforce</goal>
+ </goals>
+ <configuration>
+ <rules>
+ <requireUpperBoundDeps />
+ </rules>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ <extensions>
+ <extension>
+ <groupId>kr.motd.maven</groupId>
+ <artifactId>os-maven-plugin</artifactId>
+ <version>1.7.1</version>
+ </extension>
+ </extensions>
Review Comment:
Why adding this?
I think we dont't need it.
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/test/resources/taskTester.json:
##########
@@ -0,0 +1,61 @@
+{
+ "nested": {
+ "org": {
+ "nested": {
+ "apache": {
+ "nested": {
+ "dolphinscheduler": {
+ "nested": {
+ "task": {
+ "nested": {
+ "grpc": {
+ "nested": {
+ "proto": {
+ "options": {
+ "java_package": "org.apache.dolphinscheduler.plugin.task.grpc.generated",
+ "java_multiple_files": true,
+ "java_outer_classname": "TaskTesterProto"
+ },
+ "nested": {
+ "TaskTester": {
+ "methods": {
+ "TestOK": {
+ "requestType": "StringRequest",
+ "responseType": "StringReply"
+ },
+ "TestUNIMPLEMENTED": {
+ "requestType": "StringRequest",
+ "responseType": "StringReply"
+ }
+ }
+ },
+ "StringRequest": {
+ "fields": {
+ "username": {
+ "type": "string",
+ "id": 1
+ }
+ }
+ },
+ "StringReply": {
+ "fields": {
+ "message": {
+ "type": "string",
+ "id": 1
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+}
Review Comment:
Please leave an blank line at the end of each file.
##########
tools/dependencies/known-dependencies.txt:
##########
@@ -300,6 +300,7 @@ perfmark-api-0.23.0.jar
proto-google-common-protos-2.0.1.jar
protobuf-java-3.17.2.jar
protobuf-java-util-3.17.2.jar
+protobuf-java-util-3.17.3.jar
Review Comment:
Why do we need to introduce a new version? Since we already have `protobuf-java-util-3.17.2`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3201365094
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [73.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3273042326
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3269360743
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3273276269
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3277420779
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343032072
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
Of course we can save the `.proto` into backend and compile it at runtime. For using `protobufjs`, running a js engine to interprete the `protobufjs` in java is not an efficient and clear solution.
For using `protoc`, we always need additional dependencies, even `libprotoc` or `libprotoc-java` needs a binary executable `protoc`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3279499185
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2341073297
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/pom.xml:
##########
@@ -0,0 +1,174 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-task-grpc</artifactId>
+ <packaging>jar</packaging>
+
+ <properties>
+ <plugin.name>task.grpc</plugin.name>
+ <protoc.version>3.17.3</protoc.version>
+ <grpc.version>1.71.0</grpc.version>
+ </properties>
+
+ <dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-bom</artifactId>
+ <version>dev-SNAPSHOT</version>
+ <scope>import</scope>
+ </dependency>
+ </dependencies>
+ </dependencyManagement>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-common</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-services</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty-shaded</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-protobuf</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-stub</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ <scope>provided</scope>
+ </dependency>
Review Comment:
These are used at the construction stage for grpc client.
grpc-service & grpc-netty: create managed channel to an remote address.
grpc-protobuf: convert from primitive protobuf descriptors to protobufs in grpc domain.
protobuf-java-util: convert json descriptors to structed protobuf descriptors.
grpc-stub: create a client accessing to corresponding managed channel.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343057781
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
+
+ public Descriptors.FileDescriptor buildDescriptor(Root root) throws Descriptors.DescriptorValidationException {
+ return parseRoot(root);
+ }
+
+ private Namespace findInnerNamespace(Namespace ns) {
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return ns;
+ }
+
+ public String readPackageName(Root root) {
+ List<String> packageNameNS = new ArrayList<>();
+ Namespace ns = root;
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ packageNameNS.add((String) ns.nested.keySet().toArray()[0]);
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return String.join(GrpcConstants.NAMESPACE_SEPERATOR, packageNameNS);
+ }
+
+ private Descriptors.FileDescriptor parseRoot(Root root) throws Descriptors.DescriptorValidationException {
+ DescriptorProtos.FileDescriptorProto.Builder fileDescriptorProtoBuilder =
+ DescriptorProtos.FileDescriptorProto.newBuilder()
+ .setPackage(readPackageName(root));
+ Namespace innerNS = findInnerNamespace(root);
+ if (innerNS.nested != null)
+ innerNS.nested.forEach((name, pbObject) -> {
+ if (pbObject instanceof Namespace) {
+ Namespace ns = (Namespace) pbObject;
+ if (ns instanceof Type) {
+ fileDescriptorProtoBuilder.addMessageType(parseType(name, (Type) ns));
+ } else if (ns instanceof Service) {
+ fileDescriptorProtoBuilder.addService(parseService(name, (Service) ns));
+ } else {
+
+ }
Review Comment:
Unnecessary branches, will be removed.
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
+
+ public Descriptors.FileDescriptor buildDescriptor(Root root) throws Descriptors.DescriptorValidationException {
+ return parseRoot(root);
+ }
+
+ private Namespace findInnerNamespace(Namespace ns) {
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return ns;
+ }
+
+ public String readPackageName(Root root) {
+ List<String> packageNameNS = new ArrayList<>();
+ Namespace ns = root;
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ packageNameNS.add((String) ns.nested.keySet().toArray()[0]);
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return String.join(GrpcConstants.NAMESPACE_SEPERATOR, packageNameNS);
+ }
+
+ private Descriptors.FileDescriptor parseRoot(Root root) throws Descriptors.DescriptorValidationException {
+ DescriptorProtos.FileDescriptorProto.Builder fileDescriptorProtoBuilder =
+ DescriptorProtos.FileDescriptorProto.newBuilder()
+ .setPackage(readPackageName(root));
+ Namespace innerNS = findInnerNamespace(root);
+ if (innerNS.nested != null)
+ innerNS.nested.forEach((name, pbObject) -> {
+ if (pbObject instanceof Namespace) {
+ Namespace ns = (Namespace) pbObject;
+ if (ns instanceof Type) {
+ fileDescriptorProtoBuilder.addMessageType(parseType(name, (Type) ns));
+ } else if (ns instanceof Service) {
+ fileDescriptorProtoBuilder.addService(parseService(name, (Service) ns));
+ } else {
+
+ }
+ } else if (pbObject instanceof Enum) {
+ fileDescriptorProtoBuilder.addEnumType(parseEnum(name, (Enum) pbObject));
+ } else if (pbObject instanceof Field) {
+
+ } else if (pbObject instanceof OneOf) {
+
+ } else if (pbObject instanceof Method) {
+
+ }
Review Comment:
Also
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3181295880
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [72 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [70.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2422545110
##########
dolphinscheduler-ui/package.json:
##########
@@ -24,6 +24,7 @@
"nprogress": "^0.2.0",
"pinia": "^2.0.22",
"pinia-plugin-persistedstate": "^2.2.0",
+ "protobufjs": "^7.5.4",
Review Comment:
Please add license https://dolphinscheduler.apache.org/en-us/docs/3.3.1/contribute/join/DS-License
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3166439692
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [65 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [63.2% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3172766260
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [65 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [63.2% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3201365733
Please retry analysis of this Pull-Request directly on SonarQube Cloud
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3290774041
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [77.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2340983869
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
To get a structed protobuf descriptor, the only choices is `protoc` or `protobufjs`. The first one needs the dependency of the binary executable protoc in paths. To avoid an additional dependency, we choose protobufjs. We use it to generate a structed protobuf descriptor at frontend, and reconstruct the protobuf descriptor at backend. We can not use a .proto text directly as a structed definition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3270814425
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3249286708
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3241109753
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3393050998
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [221 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343032072
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
Of course we can save the `.proto` into backend and compile it at runtime. But there are some trade off.
For using `protobufjs`, running a js engine to interprete the `protobufjs` in java is not an efficient and clear solution.
For using `protoc`, we always need additional dependencies, even `libprotoc` or `libprotoc-java` needs a binary executable `protoc`.
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
Of course we can save the `.proto` into backend and compile it at runtime. But there are some trade offs.
For using `protobufjs`, running a js engine to interprete the `protobufjs` in java is not an efficient and clear solution.
For using `protoc`, we always need additional dependencies, even `libprotoc` or `libprotoc-java` needs a binary executable `protoc`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2329485270
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/pom.xml:
##########
@@ -0,0 +1,297 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-task-grpc</artifactId>
+ <packaging>jar</packaging>
+
+ <properties>
+ <plugin.name>task.grpc</plugin.name>
+ <grpc.version>1.71.0</grpc.version>
+ <protobuf.version>3.17.3</protobuf.version>
+ <protoc.version>3.17.3</protoc.version>
+ </properties>
+
+ <dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-bom</artifactId>
+ <version>${grpc.version}</version>
+ <type>pom</type>
+ <scope>import</scope>
+ </dependency>
+
+ </dependencies>
+ </dependencyManagement>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-common</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>com.squareup.okhttp3</groupId>
+ <artifactId>mockwebserver</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-services</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.code.gson</groupId>
+ <artifactId>gson</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-transport-native-unix-common</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-codec-http2</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-handler-proxy</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-api</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty-shaded</artifactId>
+ <scope>runtime</scope>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-protobuf</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-stub</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ <version>${protobuf.version}</version>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.gson</groupId>
+ <artifactId>gson</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>com.google.j2objc</groupId>
+ <artifactId>j2objc-annotations</artifactId>
+ <version>3.0.0</version>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-testing</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>4.13.2</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java</artifactId>
+ <version>${protobuf.version}</version>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <executions>
+ <execution>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <phase>package</phase>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>org.xolstice.maven.plugins</groupId>
+ <artifactId>protobuf-maven-plugin</artifactId>
+ <version>0.6.1</version>
+ <configuration>
+ <clearOutputDirectory>false</clearOutputDirectory>
+ <protocArtifact>com.google.protobuf:protoc:${protoc.version}:exe:${os.detected.classifier}</protocArtifact>
+ <pluginId>grpc-java</pluginId>
+ <pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}:exe:${os.detected.classifier}</pluginArtifact>
+ <protoSourceRoot>${project.basedir}/src/test/proto</protoSourceRoot>
+ <outputDirectory>${project.basedir}/src/test/java/</outputDirectory>
+ </configuration>
+ <executions>
+ <execution>
+ <goals>
+ <goal>test-compile</goal>
+ <goal>test-compile-custom</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-enforcer-plugin</artifactId>
+ <version>3.5.0</version>
+ <executions>
+ <execution>
+ <id>enforce</id>
+ <goals>
+ <goal>enforce</goal>
+ </goals>
+ <configuration>
+ <rules>
+ <requireUpperBoundDeps />
+ </rules>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ <extensions>
+ <extension>
+ <groupId>kr.motd.maven</groupId>
+ <artifactId>os-maven-plugin</artifactId>
+ <version>1.7.1</version>
+ </extension>
+ </extensions>
Review Comment:
The test depends on a protobuf compiler, protoc, which is a executable maven plugin using os type. The os-maven-plugin is the plugin to get system os type and pass the type to maven to get the specific version of protoc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2329490770
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/test/resources/taskTester.json:
##########
@@ -0,0 +1,61 @@
+{
+ "nested": {
+ "org": {
+ "nested": {
+ "apache": {
+ "nested": {
+ "dolphinscheduler": {
+ "nested": {
+ "task": {
+ "nested": {
+ "grpc": {
+ "nested": {
+ "proto": {
+ "options": {
+ "java_package": "org.apache.dolphinscheduler.plugin.task.grpc.generated",
+ "java_multiple_files": true,
+ "java_outer_classname": "TaskTesterProto"
+ },
+ "nested": {
+ "TaskTester": {
+ "methods": {
+ "TestOK": {
+ "requestType": "StringRequest",
+ "responseType": "StringReply"
+ },
+ "TestUNIMPLEMENTED": {
+ "requestType": "StringRequest",
+ "responseType": "StringReply"
+ }
+ }
+ },
+ "StringRequest": {
+ "fields": {
+ "username": {
+ "type": "string",
+ "id": 1
+ }
+ }
+ },
+ "StringReply": {
+ "fields": {
+ "message": {
+ "type": "string",
+ "id": 1
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+}
Review Comment:
Get it.
##########
dolphinscheduler-ui/pnpm-lock.yaml:
##########
Review Comment:
Get it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343705864
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
Certainly, we also do parse stage check for input.
If a java lib can compile a .proto at run time, please provide to me. It must be useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3415033492
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [197 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3414373510
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [209 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "github-advanced-security[bot] (via GitHub)" <gi...@apache.org>.
github-advanced-security[bot] commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2261830128
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,196 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
+
+ public Descriptors.FileDescriptor buildDescriptor(Root root) throws Descriptors.DescriptorValidationException {
+ return parseRoot(root);
+ }
+
+ private Namespace findInnerNamespace(Namespace ns) {
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return ns;
+ }
+
+ public String readPackageName(Root root) {
+ List<String> packageNameNS = new ArrayList<>();
+ Namespace ns = root;
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ packageNameNS.add((String) ns.nested.keySet().toArray()[0]);
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return String.join(".", packageNameNS);
+ }
+
+ private Descriptors.FileDescriptor parseRoot(Root root) throws Descriptors.DescriptorValidationException {
+ DescriptorProtos.FileDescriptorProto.Builder fileDescriptorProtoBuilder =
+ DescriptorProtos.FileDescriptorProto.newBuilder()
+ .setPackage(readPackageName(root));
+ Namespace innerNS = findInnerNamespace(root);
+ if (innerNS.nested != null)
+ innerNS.nested.forEach((name, pbObject) -> {
+ if (pbObject instanceof Namespace) {
+ Namespace ns = (Namespace) pbObject;
+ if (ns instanceof Type) {
+ fileDescriptorProtoBuilder.addMessageType(parseType(name, (Type) ns));
+ } else if (ns instanceof Service) {
+ fileDescriptorProtoBuilder.addService(parseService(name, (Service) ns));
+ } else {
+
+ }
+ } else if (pbObject instanceof Enum) {
+ fileDescriptorProtoBuilder.addEnumType(parseEnum(name, (Enum) pbObject));
+ } else if (pbObject instanceof Field) {
+
+ } else if (pbObject instanceof OneOf) {
+
+ } else if (pbObject instanceof Method) {
+
+ }
+ });
+ Descriptors.FileDescriptor fileDescriptor =
+ Descriptors.FileDescriptor.buildFrom(fileDescriptorProtoBuilder.build(),
+ new Descriptors.FileDescriptor[0]);
+ return fileDescriptor;
+ }
+
+ private DescriptorProtos.DescriptorProto.Builder parseType(String selfName, Type type) {
+ DescriptorProtos.DescriptorProto.Builder descriptorProtoBuilder = DescriptorProtos.DescriptorProto.newBuilder()
+ .setName(selfName);
+ if (type.fields != null)
+ type.fields.forEach((name, pbObject) -> {
+ if (pbObject instanceof Field) {
+ if (pbObject instanceof MapField) {
+ descriptorProtoBuilder.addField(parseMapField(name, (MapField) pbObject));
+ } else {
+ descriptorProtoBuilder.addField(parseField(name, (Field) pbObject));
+ }
+ }
+ });
+ if (type.nested != null)
+ type.nested.forEach((name, pbObject) -> {
+ if (pbObject instanceof Enum) {
+ descriptorProtoBuilder.addEnumType(parseEnum(name, (Enum) pbObject));
+ } else if (pbObject instanceof Type) {
+ descriptorProtoBuilder.addNestedType(parseType(name, (Type) pbObject));
+ }
+ });
+ return descriptorProtoBuilder;
+ }
+
+ private DescriptorProtos.ServiceDescriptorProto.Builder parseService(String selfName, Service service) {
+ DescriptorProtos.ServiceDescriptorProto.Builder serviceDescriptorProtoBuilder =
+ DescriptorProtos.ServiceDescriptorProto.newBuilder()
+ .setName(selfName);
+ if (service.methods != null)
+ service.methods.forEach((name, pbObject) -> {
+ if (pbObject instanceof Method) {
+ serviceDescriptorProtoBuilder.addMethod(parseMethod(name, (Method) pbObject));
+ }
+ });
+ return serviceDescriptorProtoBuilder;
+ }
+
+ private DescriptorProtos.MethodDescriptorProto.Builder parseMethod(String selfName, Method method) {
+ DescriptorProtos.MethodDescriptorProto.Builder methodDescriptorProtoBuilder =
+ DescriptorProtos.MethodDescriptorProto.newBuilder()
+ .setName(selfName)
+ .setInputType(method.requestType)
+ .setOutputType(method.responseType);
+ return methodDescriptorProtoBuilder;
+ }
+
+ private DescriptorProtos.EnumDescriptorProto.Builder parseEnum(String selfName, Enum enumObj) {
+ DescriptorProtos.EnumDescriptorProto.Builder enumDescriptorProtoBuilder =
+ DescriptorProtos.EnumDescriptorProto.newBuilder()
+ .setName(selfName);
+ enumObj.values.forEach((name, id) -> {
+ DescriptorProtos.EnumValueDescriptorProto.Builder enumValueDescriptorProtoBuilder =
+ DescriptorProtos.EnumValueDescriptorProto.newBuilder()
+ .setName(name)
+ .setNumber(id);
+ enumDescriptorProtoBuilder.addValue(enumValueDescriptorProtoBuilder);
+ });
+ return enumDescriptorProtoBuilder;
+ }
+
+ private DescriptorProtos.FieldDescriptorProto.Builder parseField(String selfName, Field field) {
+ DescriptorProtos.FieldDescriptorProto.Builder fieldDescriptorProtoBuilder =
+ DescriptorProtos.FieldDescriptorProto.newBuilder()
+ .setName(selfName)
+ .setNumber(field.id);
+ try {
+ fieldDescriptorProtoBuilder
+ .setType(DescriptorProtos.FieldDescriptorProto.Type.valueOf("TYPE_" + field.type.toUpperCase()));
+ } catch (IllegalArgumentException e) {
+ fieldDescriptorProtoBuilder.setTypeName(field.type);
+ }
+ return fieldDescriptorProtoBuilder;
+ }
+
+ private DescriptorProtos.FieldDescriptorProto.Builder parseMapField(String selfName, MapField mapField) {
Review Comment:
## Useless parameter
The parameter 'selfName' is never used.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5514)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3172766934
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [65 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [63.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3257626614
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3266848395
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3267113203
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3261632734
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3257769839
Please retry analysis of this Pull-Request directly on SonarQube Cloud
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343015288
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/pom.xml:
##########
@@ -0,0 +1,174 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-task-grpc</artifactId>
+ <packaging>jar</packaging>
+
+ <properties>
+ <plugin.name>task.grpc</plugin.name>
+ <protoc.version>3.17.3</protoc.version>
+ <grpc.version>1.71.0</grpc.version>
+ </properties>
+
+ <dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-bom</artifactId>
+ <version>dev-SNAPSHOT</version>
+ <scope>import</scope>
+ </dependency>
+ </dependencies>
+ </dependencyManagement>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-common</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-services</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty-shaded</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-protobuf</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-stub</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ <scope>provided</scope>
+ </dependency>
Review Comment:
get
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3270815453
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2340983869
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
To get a structed protobuf descriptor, the only choice is `protoc` or `protobufjs`. The first one needs the dependency of the binary executable protoc in paths. To avoid an additional dependency, we choose protobufjs. We use it to generate a structed protobuf descriptor representated in json at frontend, and reconstruct the protobuf descriptor at backend. We can not use a .proto text as a structed definition directly (which is different from json).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3280874045
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2340983869
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
To get a structed protobuf descriptor, the only choices is `protoc` or `protobufjs`. The first one needs the dependency of the binary executable protoc in paths. To avoid an additional dependency, we choose protobufjs. We use it to generate a structed protobuf descriptor at frontend, and reconstruct the protobuf descriptor at backend. We can not use a .proto text as a structed definition directly (which is different from json).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3287579421
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [77.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3339364594
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [77 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3415502174
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [168 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3420728631
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3195640566
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [73.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3211033639
## PR Stage Summary (Changes or Adds)
### Frontend
- I18n Strings for field hints
- Task Icon
- Add fields in task params for gRPC Task(message, serviceDefinition, serviceDefinitionJSON, methodName, ...)
- Task UI with correction checker(especially serviceDefinition and messageJson)
- Dependency of Protobufjs
### Backend
- gRPC Task and its channel & channelFactory
- CheckCondition implementation based on gRPC Status Code
- gRPC dynamic service for runtime gRPC stub generation
- A full java implemented ProtobufJS Description Parser
- Parser implementation of basic protobuf types(namespace, message, field, nested message, primitive types, service, method(rpc), enum)
- Special field and its labels(map field, optional field, repeated field, oneof field)
### Test
- Backend Task Function Test
- Backend Task Parameter Test
- Workflow Context Binding Test
- JUnit5 gRPC Test Cleanup Extension
- Protobuf Definitions for test(taskTester.proto, parserTester.proto) and corresponding json files in resource
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2340983869
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
To get a structed protobuf descriptor, the only choices is `protoc` or `protobufjs`. The first one needs the dependence of the binary executable protoc in paths. To avoid an additional dependence, we choose protobufjs. We use it to generate a structed protobuf descriptor at frontend, and reconstruct the protobuf descriptor at backend. We can not use a .proto text directly as a structed definition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3281305725
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3263600394
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [80 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3247916195
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3263758472
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [80 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3262462225
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3262458793
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343013001
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
All of we need at runtime construction are the things used after the protobuf compile. The protobuf workflow is demostrated in following image.
https://i0.wp.com/lab.wallarm.com/wp-content/uploads/2023/12/253-min.jpg?w=770&ssl=1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343713042
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
If the DynamicMessage had mentioned by chatgpt, it has already used in current solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3287698801
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [77.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2329494559
##########
tools/dependencies/known-dependencies.txt:
##########
@@ -300,6 +300,7 @@ perfmark-api-0.23.0.jar
proto-google-common-protos-2.0.1.jar
protobuf-java-3.17.2.jar
protobuf-java-util-3.17.2.jar
+protobuf-java-util-3.17.3.jar
Review Comment:
The 3.17.3 is the first protobuf version with a osx-supported protoc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2329476595
##########
dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/java/org/apache/dolphinscheduler/e2e/cases/workflow/BaseWorkflowE2ETest.java:
##########
@@ -167,8 +202,15 @@ protected TaskInstanceTab.Row untilTaskInstanceFailed(String workflowName, Strin
return await()
.until(() -> {
browser.navigate().refresh();
- List<TaskInstanceTab.Row> taskInstances = projectPage
- .goToTab(TaskInstanceTab.class)
+
+ await().ignoreException(NoSuchElementException.class)
+ .untilAsserted(() -> assertThat(browser)
+ .as("task instance is not found")
+ .matches(it -> it.findElement(By.className("n-menu-item-content")).isDisplayed()));
+ TaskInstanceTab taskInstanceTab = projectPage
+ .goToTab(TaskInstanceTab.class);
+
+ List<TaskInstanceTab.Row> taskInstances = taskInstanceTab
Review Comment:
The new awaits added to E2E are ensuring the behaviour stability. Maybe I need some help here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3290531598
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [77.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3290534167
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [77.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3181294967
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [72 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [70.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3417681784
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "github-advanced-security[bot] (via GitHub)" <gi...@apache.org>.
github-advanced-security[bot] commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2439817161
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/MapField.java:
##########
@@ -0,0 +1,26 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [Field.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5570)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/Method.java:
##########
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [ReflectionObject.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5571)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/Service.java:
##########
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import java.util.Map;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [Namespace.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5573)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/Enum.java:
##########
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import java.util.Map;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [ReflectionObject.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5569)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/Type.java:
##########
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import java.util.Map;
+
+import lombok.Data;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [Namespace.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5574)
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/mapping/OneOf.java:
##########
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.mapping;
+
+import java.util.ArrayList;
+
+import lombok.Data;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+@Data
Review Comment:
## Missing Override annotation
This method overrides [ReflectionObject.canEqual](1); it is advisable to add an Override annotation.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5572)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3305318204
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3315113779
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3195634274
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [73.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3204539238
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [82 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2431280409
##########
dolphinscheduler-bom/pom.xml:
##########
@@ -958,6 +1036,15 @@
</dependencies>
</dependencyManagement>
+ <repositories>
+ <repository>
+ <id>confluent-io</id>
+ <name>Confluent Cloud</name>
+ <url>https://packages.confluent.io/maven/</url>
+
+ </repository>
+ </repositories>
+
Review Comment:
Please remove this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3404789787
Please fix the new issue
<img width="1664" height="628" alt="image" src="https://github.com/user-attachments/assets/5936ce03-cf13-429f-b70c-94840426d323" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3349728379
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2419407681
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/test/java/org/apache/dolphinscheduler/plugin/task/grpc/.gitignore:
##########
Review Comment:
Please remove unnessnary file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2329477556
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/pom.xml:
##########
@@ -0,0 +1,297 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-task-grpc</artifactId>
+ <packaging>jar</packaging>
+
+ <properties>
+ <plugin.name>task.grpc</plugin.name>
+ <grpc.version>1.71.0</grpc.version>
+ <protobuf.version>3.17.3</protobuf.version>
+ <protoc.version>3.17.3</protoc.version>
+ </properties>
+
+ <dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-bom</artifactId>
+ <version>${grpc.version}</version>
+ <type>pom</type>
+ <scope>import</scope>
+ </dependency>
+
+ </dependencies>
+ </dependencyManagement>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-common</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>com.squareup.okhttp3</groupId>
+ <artifactId>mockwebserver</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-services</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.code.gson</groupId>
+ <artifactId>gson</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-transport-native-unix-common</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-codec-http2</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-handler-proxy</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-api</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty-shaded</artifactId>
+ <scope>runtime</scope>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-protobuf</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-stub</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ <version>${protobuf.version}</version>
+ <exclusions>
+ <exclusion>
+ <groupId>com.google.code.gson</groupId>
+ <artifactId>gson</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.errorprone</groupId>
+ <artifactId>error_prone_annotations</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>com.google.j2objc</groupId>
+ <artifactId>j2objc-annotations</artifactId>
+ <version>3.0.0</version>
Review Comment:
Get it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2329494559
##########
tools/dependencies/known-dependencies.txt:
##########
@@ -300,6 +300,7 @@ perfmark-api-0.23.0.jar
proto-google-common-protos-2.0.1.jar
protobuf-java-3.17.2.jar
protobuf-java-util-3.17.2.jar
+protobuf-java-util-3.17.3.jar
Review Comment:
The 3.17.3 is the first protobuf version with a osx-supported protoc. If we have no need with test on osx, this can be reverted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3262102404
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3262316304
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3263445707
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3269360344
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3274853904
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3241464587
Please retry analysis of this Pull-Request directly on SonarQube Cloud
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343005536
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
We can't use `.proto` file directly, because the ` .proto` file is designed as a static definition, not a format to transfer data. The inner transmission type of protobuf is a binary format. The file needs to be compiled to a builder of binaries (by `protoc` or `protobufjs`). So that we can use the compiled dist to create a runtime definition (or protobuf descriptor) dynamically. And we choose to compile it at front end.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343705864
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
Certainly, we also do parse stage check for input.
If a java lib can compile a .proto into, please provide to me. It must be useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3241463752
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3249286484
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3208871500
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [82 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3258425327
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3261629078
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3263445849
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3270520470
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3166279159
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [70 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [64.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3204540903
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [82 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343058291
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
+
+ public Descriptors.FileDescriptor buildDescriptor(Root root) throws Descriptors.DescriptorValidationException {
+ return parseRoot(root);
+ }
+
+ private Namespace findInnerNamespace(Namespace ns) {
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return ns;
+ }
+
+ public String readPackageName(Root root) {
+ List<String> packageNameNS = new ArrayList<>();
+ Namespace ns = root;
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ packageNameNS.add((String) ns.nested.keySet().toArray()[0]);
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return String.join(GrpcConstants.NAMESPACE_SEPERATOR, packageNameNS);
+ }
+
+ private Descriptors.FileDescriptor parseRoot(Root root) throws Descriptors.DescriptorValidationException {
+ DescriptorProtos.FileDescriptorProto.Builder fileDescriptorProtoBuilder =
+ DescriptorProtos.FileDescriptorProto.newBuilder()
+ .setPackage(readPackageName(root));
+ Namespace innerNS = findInnerNamespace(root);
+ if (innerNS.nested != null)
+ innerNS.nested.forEach((name, pbObject) -> {
+ if (pbObject instanceof Namespace) {
+ Namespace ns = (Namespace) pbObject;
+ if (ns instanceof Type) {
+ fileDescriptorProtoBuilder.addMessageType(parseType(name, (Type) ns));
+ } else if (ns instanceof Service) {
+ fileDescriptorProtoBuilder.addService(parseService(name, (Service) ns));
+ } else {
+
+ }
+ } else if (pbObject instanceof Enum) {
+ fileDescriptorProtoBuilder.addEnumType(parseEnum(name, (Enum) pbObject));
+ } else if (pbObject instanceof Field) {
+
+ } else if (pbObject instanceof OneOf) {
+
+ } else if (pbObject instanceof Method) {
+
+ }
+ });
+ Descriptors.FileDescriptor fileDescriptor =
+ Descriptors.FileDescriptor.buildFrom(fileDescriptorProtoBuilder.build(),
+ new Descriptors.FileDescriptor[0]);
+ return fileDescriptor;
+ }
+
+ private DescriptorProtos.DescriptorProto.Builder parseType(String selfName, Type type) {
+ DescriptorProtos.DescriptorProto.Builder descriptorProtoBuilder = DescriptorProtos.DescriptorProto.newBuilder()
+ .setName(selfName);
+
+ Map<String, DescriptorProtos.FieldDescriptorProto.Builder> fieldMap = new HashMap<>();
+ if (type.fields != null)
+ type.fields.forEach((name, pbObject) -> {
+ if (pbObject instanceof Field) {
+ if (pbObject instanceof MapField) {
+ DescriptorProtos.FieldDescriptorProto.Builder mapField =
+ parseMapField(descriptorProtoBuilder, name, (MapField) pbObject);
+ fieldMap.put(name, mapField);
+ } else {
+ DescriptorProtos.FieldDescriptorProto.Builder field = parseField(name, (Field) pbObject);
+ fieldMap.put(name, field);
+ }
+
+ }
+ });
+ if (type.nested != null)
+ type.nested.forEach((name, pbObject) -> {
+ if (pbObject instanceof Enum) {
+ descriptorProtoBuilder.addEnumType(parseEnum(name, (Enum) pbObject));
+ } else if (pbObject instanceof Type) {
+ descriptorProtoBuilder.addNestedType(parseType(name, (Type) pbObject));
+ }
+ });
+ if (type.oneofs != null) {
+ type.oneofs.entrySet()
+ .stream()
+ .sorted((entry1, entry2) -> {
+ boolean key1StartsWithUnderscore = entry1.getKey().startsWith("_");
+ boolean key2StartsWithUnderscore = entry2.getKey().startsWith("_");
+ if (key1StartsWithUnderscore && !key2StartsWithUnderscore) {
+ return 1; // Move key1 after key2
+ } else if (!key1StartsWithUnderscore && key2StartsWithUnderscore) {
+ return -1; // Move key1 before key2
+ } else {
+ return 0; // Keep the original order
+ }
+ })
+ .collect(Collectors.toMap(
+ Map.Entry::getKey,
+ Map.Entry::getValue,
+ (oldValue, newValue) -> oldValue,
+ LinkedHashMap::new))
+ .forEach((name, oneof) -> {
+ parseOneof(descriptorProtoBuilder, name, oneof);
+ });
+ int oneofCount = descriptorProtoBuilder.getOneofDeclCount();
+ List<DescriptorProtos.OneofDescriptorProto> oneofDescriptorProtos =
+ descriptorProtoBuilder.getOneofDeclList();
+ oneofDescriptorProtos.forEach((oneof) -> {
+
+ });
Review Comment:
Also
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343049693
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
In the current solution, we compile the .proto at frontend, so we can validate user input at realtime. And the json descriptor of the format between frontend and backend is defined by `protobufjs`, which is more reliable then running a js engine at backend, the only problem is an implementation of reconstructor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3291469736
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [77.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343041115
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
+
+ public Descriptors.FileDescriptor buildDescriptor(Root root) throws Descriptors.DescriptorValidationException {
+ return parseRoot(root);
+ }
+
+ private Namespace findInnerNamespace(Namespace ns) {
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return ns;
+ }
+
+ public String readPackageName(Root root) {
+ List<String> packageNameNS = new ArrayList<>();
+ Namespace ns = root;
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ packageNameNS.add((String) ns.nested.keySet().toArray()[0]);
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return String.join(GrpcConstants.NAMESPACE_SEPERATOR, packageNameNS);
+ }
+
+ private Descriptors.FileDescriptor parseRoot(Root root) throws Descriptors.DescriptorValidationException {
+ DescriptorProtos.FileDescriptorProto.Builder fileDescriptorProtoBuilder =
+ DescriptorProtos.FileDescriptorProto.newBuilder()
+ .setPackage(readPackageName(root));
+ Namespace innerNS = findInnerNamespace(root);
+ if (innerNS.nested != null)
+ innerNS.nested.forEach((name, pbObject) -> {
+ if (pbObject instanceof Namespace) {
+ Namespace ns = (Namespace) pbObject;
+ if (ns instanceof Type) {
+ fileDescriptorProtoBuilder.addMessageType(parseType(name, (Type) ns));
+ } else if (ns instanceof Service) {
+ fileDescriptorProtoBuilder.addService(parseService(name, (Service) ns));
+ } else {
+
+ }
Review Comment:
Why adding empty else here?
##########
dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/java/org/apache/dolphinscheduler/e2e/pages/project/workflow/task/GrpcTaskForm.java:
##########
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.e2e.pages.project.workflow.task;
+
+import org.apache.dolphinscheduler.e2e.pages.common.HttpInput;
+import org.apache.dolphinscheduler.e2e.pages.common.MultipleCodeEditor;
+import org.apache.dolphinscheduler.e2e.pages.project.workflow.WorkflowForm;
+
+import org.openqa.selenium.WebDriver;
+import org.openqa.selenium.WebElement;
+import org.openqa.selenium.support.FindBy;
+import org.openqa.selenium.support.FindBys;
+import org.openqa.selenium.support.PageFactory;
+
+public class GrpcTaskForm extends TaskNodeForm {
+
+ private WebDriver driver;
+
+ private HttpInput url;
+
+ private MultipleCodeEditor editors;
+
+ @FindBys({
+
+ @FindBy(className = "input-method-name"),
+ @FindBy(tagName = "input")
+ })
Review Comment:
```suggestion
@FindBys({
@FindBy(className = "input-method-name"),
@FindBy(tagName = "input")
})
```
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
+
+ public Descriptors.FileDescriptor buildDescriptor(Root root) throws Descriptors.DescriptorValidationException {
+ return parseRoot(root);
+ }
+
+ private Namespace findInnerNamespace(Namespace ns) {
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return ns;
+ }
+
+ public String readPackageName(Root root) {
+ List<String> packageNameNS = new ArrayList<>();
+ Namespace ns = root;
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ packageNameNS.add((String) ns.nested.keySet().toArray()[0]);
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return String.join(GrpcConstants.NAMESPACE_SEPERATOR, packageNameNS);
+ }
+
+ private Descriptors.FileDescriptor parseRoot(Root root) throws Descriptors.DescriptorValidationException {
+ DescriptorProtos.FileDescriptorProto.Builder fileDescriptorProtoBuilder =
+ DescriptorProtos.FileDescriptorProto.newBuilder()
+ .setPackage(readPackageName(root));
+ Namespace innerNS = findInnerNamespace(root);
+ if (innerNS.nested != null)
+ innerNS.nested.forEach((name, pbObject) -> {
+ if (pbObject instanceof Namespace) {
+ Namespace ns = (Namespace) pbObject;
+ if (ns instanceof Type) {
+ fileDescriptorProtoBuilder.addMessageType(parseType(name, (Type) ns));
+ } else if (ns instanceof Service) {
+ fileDescriptorProtoBuilder.addService(parseService(name, (Service) ns));
+ } else {
+
+ }
+ } else if (pbObject instanceof Enum) {
+ fileDescriptorProtoBuilder.addEnumType(parseEnum(name, (Enum) pbObject));
+ } else if (pbObject instanceof Field) {
+
+ } else if (pbObject instanceof OneOf) {
+
+ } else if (pbObject instanceof Method) {
+
+ }
+ });
+ Descriptors.FileDescriptor fileDescriptor =
+ Descriptors.FileDescriptor.buildFrom(fileDescriptorProtoBuilder.build(),
+ new Descriptors.FileDescriptor[0]);
+ return fileDescriptor;
+ }
+
+ private DescriptorProtos.DescriptorProto.Builder parseType(String selfName, Type type) {
+ DescriptorProtos.DescriptorProto.Builder descriptorProtoBuilder = DescriptorProtos.DescriptorProto.newBuilder()
+ .setName(selfName);
+
+ Map<String, DescriptorProtos.FieldDescriptorProto.Builder> fieldMap = new HashMap<>();
+ if (type.fields != null)
+ type.fields.forEach((name, pbObject) -> {
+ if (pbObject instanceof Field) {
+ if (pbObject instanceof MapField) {
+ DescriptorProtos.FieldDescriptorProto.Builder mapField =
+ parseMapField(descriptorProtoBuilder, name, (MapField) pbObject);
+ fieldMap.put(name, mapField);
+ } else {
+ DescriptorProtos.FieldDescriptorProto.Builder field = parseField(name, (Field) pbObject);
+ fieldMap.put(name, field);
+ }
+
+ }
+ });
+ if (type.nested != null)
+ type.nested.forEach((name, pbObject) -> {
+ if (pbObject instanceof Enum) {
+ descriptorProtoBuilder.addEnumType(parseEnum(name, (Enum) pbObject));
+ } else if (pbObject instanceof Type) {
+ descriptorProtoBuilder.addNestedType(parseType(name, (Type) pbObject));
+ }
+ });
+ if (type.oneofs != null) {
+ type.oneofs.entrySet()
+ .stream()
+ .sorted((entry1, entry2) -> {
+ boolean key1StartsWithUnderscore = entry1.getKey().startsWith("_");
+ boolean key2StartsWithUnderscore = entry2.getKey().startsWith("_");
+ if (key1StartsWithUnderscore && !key2StartsWithUnderscore) {
+ return 1; // Move key1 after key2
+ } else if (!key1StartsWithUnderscore && key2StartsWithUnderscore) {
+ return -1; // Move key1 before key2
+ } else {
+ return 0; // Keep the original order
+ }
+ })
+ .collect(Collectors.toMap(
+ Map.Entry::getKey,
+ Map.Entry::getValue,
+ (oldValue, newValue) -> oldValue,
+ LinkedHashMap::new))
+ .forEach((name, oneof) -> {
+ parseOneof(descriptorProtoBuilder, name, oneof);
+ });
+ int oneofCount = descriptorProtoBuilder.getOneofDeclCount();
+ List<DescriptorProtos.OneofDescriptorProto> oneofDescriptorProtos =
+ descriptorProtoBuilder.getOneofDeclList();
+ oneofDescriptorProtos.forEach((oneof) -> {
+
+ });
Review Comment:
Why adding empty logic here?
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
+
+ public Descriptors.FileDescriptor buildDescriptor(Root root) throws Descriptors.DescriptorValidationException {
+ return parseRoot(root);
+ }
+
+ private Namespace findInnerNamespace(Namespace ns) {
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return ns;
+ }
+
+ public String readPackageName(Root root) {
+ List<String> packageNameNS = new ArrayList<>();
+ Namespace ns = root;
+ while (true) {
+ if (ns.nested != null && ns.nested.values().size() == 1
+ && ns.nested.values().toArray()[0] instanceof Namespace && !(ns.nested instanceof Type)
+ && !(ns.nested instanceof Service)) {
+ packageNameNS.add((String) ns.nested.keySet().toArray()[0]);
+ ns = (Namespace) ns.nested.values().toArray()[0];
+ } else {
+ break;
+ }
+ }
+ return String.join(GrpcConstants.NAMESPACE_SEPERATOR, packageNameNS);
+ }
+
+ private Descriptors.FileDescriptor parseRoot(Root root) throws Descriptors.DescriptorValidationException {
+ DescriptorProtos.FileDescriptorProto.Builder fileDescriptorProtoBuilder =
+ DescriptorProtos.FileDescriptorProto.newBuilder()
+ .setPackage(readPackageName(root));
+ Namespace innerNS = findInnerNamespace(root);
+ if (innerNS.nested != null)
+ innerNS.nested.forEach((name, pbObject) -> {
+ if (pbObject instanceof Namespace) {
+ Namespace ns = (Namespace) pbObject;
+ if (ns instanceof Type) {
+ fileDescriptorProtoBuilder.addMessageType(parseType(name, (Type) ns));
+ } else if (ns instanceof Service) {
+ fileDescriptorProtoBuilder.addService(parseService(name, (Service) ns));
+ } else {
+
+ }
+ } else if (pbObject instanceof Enum) {
+ fileDescriptorProtoBuilder.addEnumType(parseEnum(name, (Enum) pbObject));
+ } else if (pbObject instanceof Field) {
+
+ } else if (pbObject instanceof OneOf) {
+
+ } else if (pbObject instanceof Method) {
+
+ }
Review Comment:
Why adding empty else here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3277736720
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2340810918
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/pom.xml:
##########
@@ -0,0 +1,174 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-task-grpc</artifactId>
+ <packaging>jar</packaging>
+
+ <properties>
+ <plugin.name>task.grpc</plugin.name>
+ <protoc.version>3.17.3</protoc.version>
+ <grpc.version>1.71.0</grpc.version>
+ </properties>
+
+ <dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-bom</artifactId>
+ <version>dev-SNAPSHOT</version>
+ <scope>import</scope>
+ </dependency>
+ </dependencies>
+ </dependencyManagement>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-common</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-services</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty-shaded</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-protobuf</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-stub</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>com.google.j2objc</groupId>
+ <artifactId>j2objc-annotations</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-testing</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
Review Comment:
```suggestion
```
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/pom.xml:
##########
@@ -0,0 +1,174 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-task-grpc</artifactId>
+ <packaging>jar</packaging>
+
+ <properties>
+ <plugin.name>task.grpc</plugin.name>
+ <protoc.version>3.17.3</protoc.version>
+ <grpc.version>1.71.0</grpc.version>
+ </properties>
+
+ <dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-bom</artifactId>
+ <version>dev-SNAPSHOT</version>
+ <scope>import</scope>
+ </dependency>
+ </dependencies>
+ </dependencyManagement>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-common</artifactId>
+ <version>${project.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-services</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-netty-shaded</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-protobuf</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.grpc</groupId>
+ <artifactId>grpc-stub</artifactId>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ <scope>provided</scope>
+ </dependency>
Review Comment:
Why add these, these is not needed? if we only need this in test, we should change the scope to test, and the version shouldn't be managed at bom if we only need this in UT
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/GrpcParameters.java:
##########
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc;
+
+import org.apache.dolphinscheduler.plugin.task.api.parameters.AbstractParameters;
+
+import org.apache.commons.lang3.StringUtils;
+
+import lombok.Data;
+import lombok.EqualsAndHashCode;
+
+@EqualsAndHashCode(callSuper = true)
+@Data
+public class GrpcParameters extends AbstractParameters {
+
+ private String url;
+
+ private String grpcServiceDefinition;
+
+ private String grpcServiceDefinitionJSON;
+
+ private String methodName;
+
+ private String message;
+
+ private GrpcCheckCondition grpcCheckCondition = GrpcCheckCondition.STATUS_CODE_DEFAULT;
+
+ private String condition;
+
+ /**
+ * Connect Timeout
+ * Unit: ms
+ */
+ private int connectTimeout = 0;
Review Comment:
```suggestion
private long connectTimeoutMs = 0L;
```
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
Is this needed? If we directly receive the proto content rather than json schema, then we don't need this? This seems meaningless.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3339357603
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [77 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3395738898
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [219 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3416412147
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [2 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3409125973
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate failed**
Failed conditions
 [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3417681623
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3247913944
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3290767387
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [77.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2329494559
##########
tools/dependencies/known-dependencies.txt:
##########
@@ -300,6 +300,7 @@ perfmark-api-0.23.0.jar
proto-google-common-protos-2.0.1.jar
protobuf-java-3.17.2.jar
protobuf-java-util-3.17.2.jar
+protobuf-java-util-3.17.3.jar
Review Comment:
The 3.17.2 is the first protobuf version with a osx-supported protoc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3273044131
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3267111871
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2329490053
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/test/java/org/apache/dolphinscheduler/plugin/task/grpc/.gitignore:
##########
Review Comment:
The tests use protoc to compile the .proto file to java classes, and this ignore file is used to unselect the generated files, because these files will generate at test-compose stage.
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/test/java/org/apache/dolphinscheduler/plugin/task/grpc/GrpcCleanupExtension.java:
##########
@@ -0,0 +1,176 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+import org.junit.jupiter.api.extension.AfterEachCallback;
+import org.junit.jupiter.api.extension.ExtensionContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.grpc.BindableService;
+import io.grpc.ManagedChannel;
+import io.grpc.Metadata;
+import io.grpc.Server;
+import io.grpc.ServerCall;
+import io.grpc.ServerCallHandler;
+import io.grpc.ServerInterceptor;
+import io.grpc.inprocess.InProcessChannelBuilder;
+import io.grpc.inprocess.InProcessServerBuilder;
+
+public class GrpcCleanupExtension implements AfterEachCallback {
+
+ private final List<CleanupTarget> cleanupTargets = new ArrayList<>();
+
+ private static final Logger logger = LoggerFactory.getLogger(GrpcCleanupExtension.class);
Review Comment:
Get it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2329471970
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-case/src/test/java/org/apache/dolphinscheduler/api/test/cases/ExecutorAPITest.java:
##########
@@ -145,4 +145,53 @@ public void testStartWorkflowInstance() {
}
}
+ @Test
+ @Order(2)
+ public void testGrpcWorkflowInstance() {
Review Comment:
This will be moved at following commits.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3274848174
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3270517923
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [78 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3264398821
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [80 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2343005536
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
We can not use `.proto` file directly, because the ` .proto` file is designed as a static definition, not a format to transfer data. The inner transmission type of protobuf is a binary format. The file needs to be compiled to a builder of binaries (by `protoc` or `protobufjs`). So that we can use the compiled dist to create a runtime definition (or protobuf descriptor) dynamically. And we choose to compile it at front end.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3291471561
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [77.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3362169677
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [75 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3179681368
Please retry analysis of this Pull-Request directly on SonarQube Cloud
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3241111802
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3257769477
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3414121897
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [209 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3416412950
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [2 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3414375269
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [209 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [76.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3279515310
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [79 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [75.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "npofsi (via GitHub)" <gi...@apache.org>.
npofsi commented on code in PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#discussion_r2340983869
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-grpc/src/main/java/org/apache/dolphinscheduler/plugin/task/grpc/protobufjs/JSONDescriptorParser.java:
##########
@@ -0,0 +1,297 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.grpc.protobufjs;
+
+import static java.util.Objects.isNull;
+
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Enum;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Field;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.MapField;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Method;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Namespace;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.OneOf;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Root;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Service;
+import org.apache.dolphinscheduler.plugin.task.grpc.protobufjs.types.Type;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.protobuf.DescriptorProtos;
+import com.google.protobuf.Descriptors;
+
+public class JSONDescriptorParser {
Review Comment:
To get a structed protobuf descriptor, the only choices is `protoc` or `protobufjs`. The first one needs the dependency of the binary executable protoc in paths. To avoid an additional dependency, we choose protobufjs. We use it to generate a structed protobuf descriptor at frontend, and reconstruct the protobuf descriptor at backend. We can not use a .proto text as a structed definition directly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3287697784
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [77.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][Task] Add a gRPC task plugin [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17402:
URL: https://github.com/apache/dolphinscheduler/pull/17402#issuecomment-3287575504
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402) **Quality Gate passed**
Issues
 [74 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=17402&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [77.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17402&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17402)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org