You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by "fuweng11 (via GitHub)" <gi...@apache.org> on 2023/11/16 03:11:06 UTC
[PR] [INLONG-9297][Manager] Support configuring multiple sink types of tasks under a single stream [inlong]
fuweng11 opened a new pull request, #9298:
URL: https://github.com/apache/inlong/pull/9298
### Prepare a Pull Request
- Fixes #9297
### Motivation
Support configuring multiple sink types of tasks under a single stream.
### Modifications
Support configuring multiple sink types of tasks under a single stream.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9297][Manager] Support configuring multiple sink types of tasks under a single stream [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #9298:
URL: https://github.com/apache/inlong/pull/9298#discussion_r1405754823
##########
inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/consts/SinkType.java:
##########
@@ -39,4 +47,27 @@ public class SinkType extends StreamType {
* Details: <a href="https://www.tencentcloud.com/products/cls">CLS</a>
*/
public static final String CLS = "CLS";
+
+ public static final Set<String> SORT_FLINK_SINK = new HashSet<>();
+
+ public static final Set<String> SORT_STANDALONE_SINK = new HashSet<>();
+
+ public static boolean containSortFlinkSink(List<String> sinkTypes) {
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9297][Manager] Support configuring multiple sink types of tasks under a single stream [inlong]
Posted by "vernedeng (via GitHub)" <gi...@apache.org>.
vernedeng commented on code in PR #9298:
URL: https://github.com/apache/inlong/pull/9298#discussion_r1405726506
##########
inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/consts/SinkType.java:
##########
@@ -39,4 +47,27 @@ public class SinkType extends StreamType {
* Details: <a href="https://www.tencentcloud.com/products/cls">CLS</a>
*/
public static final String CLS = "CLS";
+
+ public static final Set<String> SORT_FLINK_SINK = new HashSet<>();
+
+ public static final Set<String> SORT_STANDALONE_SINK = new HashSet<>();
+
+ public static boolean containSortFlinkSink(List<String> sinkTypes) {
Review Comment:
use Set::retainAll to get the intersection of two sets
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9297][Manager] Support configuring multiple sink types of tasks under a single stream [inlong]
Posted by "dockerzhang (via GitHub)" <gi...@apache.org>.
dockerzhang merged PR #9298:
URL: https://github.com/apache/inlong/pull/9298
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9297][Manager] Support configuring multiple sink types of tasks under a single stream [inlong]
Posted by "healchow (via GitHub)" <gi...@apache.org>.
healchow commented on code in PR #9298:
URL: https://github.com/apache/inlong/pull/9298#discussion_r1406988936
##########
inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/consts/SinkType.java:
##########
@@ -17,9 +17,17 @@
package org.apache.inlong.manager.common.consts;
+import org.springframework.beans.factory.annotation.Value;
+import org.springframework.stereotype.Component;
+
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
/**
* Constants of sink type.
*/
+@Component
Review Comment:
I mean, you can customize two annotations: one for types supporting `Sort, such as `@SupportSort`, and another for types supporting `SortStandalone`, such as `@SupportSortStandalone`.
On each Sink type, specify the supported annotation types. During the InLong-Manager startup, use reflection to read specific annotations and categorize Sink types into different `Set`.
With this approach, when adding a new Sink type, you only need to annotate it with one or both of the mentioned annotations, eliminating the need to maintain a configuration file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9297][Manager] Support configuring multiple sink types of tasks under a single stream [inlong]
Posted by "healchow (via GitHub)" <gi...@apache.org>.
healchow commented on code in PR #9298:
URL: https://github.com/apache/inlong/pull/9298#discussion_r1407430661
##########
inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/consts/SortType.java:
##########
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.manager.common.consts;
+
+public enum SortType {
Review Comment:
Javadoc, please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9297][Manager] Support configuring multiple sink types of tasks under a single stream [inlong]
Posted by "healchow (via GitHub)" <gi...@apache.org>.
healchow commented on code in PR #9298:
URL: https://github.com/apache/inlong/pull/9298#discussion_r1407430379
##########
inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/consts/SupportSortType.java:
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.manager.common.consts;
+
+import java.lang.annotation.Documented;
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+@Retention(RetentionPolicy.RUNTIME)
+@Documented
+@Target({ElementType.FIELD})
+public @interface SupportSortType {
Review Comment:
Can you please add some javadoc for this annotation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9297][Manager] Support configuring multiple sink types of tasks under a single stream [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #9298:
URL: https://github.com/apache/inlong/pull/9298#discussion_r1407446204
##########
inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/consts/SinkType.java:
##########
@@ -17,9 +17,17 @@
package org.apache.inlong.manager.common.consts;
+import org.springframework.beans.factory.annotation.Value;
+import org.springframework.stereotype.Component;
+
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
/**
* Constants of sink type.
*/
+@Component
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9297][Manager] Support configuring multiple sink types of tasks under a single stream [inlong]
Posted by "healchow (via GitHub)" <gi...@apache.org>.
healchow commented on code in PR #9298:
URL: https://github.com/apache/inlong/pull/9298#discussion_r1398636241
##########
inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/consts/SinkType.java:
##########
@@ -39,4 +45,23 @@ public class SinkType extends StreamType {
* Details: <a href="https://www.tencentcloud.com/products/cls">CLS</a>
*/
public static final String CLS = "CLS";
+
+ public static final Set<String> SORT_FLINK_SINK = new HashSet<>();
+ public static final Set<String> SORT_STANDALONE_SINK = new HashSet<>();
+
+ static {
+ SORT_FLINK_SINK.addAll(
Review Comment:
This method is not very flexible. If a new type is added, the code here needs to be updated. Can we achieve this functionality through annotations?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9297][Manager] Support configuring multiple sink types of tasks under a single stream [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #9298:
URL: https://github.com/apache/inlong/pull/9298#discussion_r1405548618
##########
inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/consts/SinkType.java:
##########
@@ -39,4 +45,23 @@ public class SinkType extends StreamType {
* Details: <a href="https://www.tencentcloud.com/products/cls">CLS</a>
*/
public static final String CLS = "CLS";
+
+ public static final Set<String> SORT_FLINK_SINK = new HashSet<>();
+ public static final Set<String> SORT_STANDALONE_SINK = new HashSet<>();
+
+ static {
+ SORT_FLINK_SINK.addAll(
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9297][Manager] Support configuring multiple sink types of tasks under a single stream [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #9298:
URL: https://github.com/apache/inlong/pull/9298#discussion_r1407437420
##########
inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/consts/SupportSortType.java:
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.manager.common.consts;
+
+import java.lang.annotation.Documented;
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+@Retention(RetentionPolicy.RUNTIME)
+@Documented
+@Target({ElementType.FIELD})
+public @interface SupportSortType {
Review Comment:
Done.
##########
inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/consts/SortType.java:
##########
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.manager.common.consts;
+
+public enum SortType {
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org