You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2022/08/11 13:18:49 UTC

[GitHub] [rocketmq-connect] sunxiaojian opened a new pull request, #248: [ISSUE #238]Add pause and resume to the task, and optimize some rest APIs

sunxiaojian opened a new pull request, #248:
URL: https://github.com/apache/rocketmq-connect/pull/248

   本次优化点：
   - [ 1.]   优化WorkerSourceTask 增加 pause 和 resume的功能，提供source task 状态上报能力
   - [ 2.]  优化WorkerSinkTask 并将 Rocketmq Consumer 替换成DefaultLitePullConsumer, 以使用 pause 、resume 及seek 功能，优化commit 提交流程，提供sink task状态上报能力
   - [ 3.]  优化WorkerConnector, 增加 pause 和 resume 维护功能，提供connector 状态上报能力
   - [ 4.]  增加 StateManagementService 状态管理服务，帮助缓存和向集群传输connector 和 task 任务的状态，并用于rest api状态透出
   - [ 5.]  优化ConfigManagementServiceImpl，将异步每次发送全量配置改为每次只发送变更connector，为了解决配置任务过多而触发的消息body的限制，每次任务的变更以监听的消息为准
   - [ 6.]  优化Worker增加维护connector的线程池，优化graceful的关闭机制
   - [ 7.]  优化AbstractConnectController 添加接口供rest api使用
   - [ 8.]  标准化rest api ，javalin 升级到>4.6 ，规范了api返回的数据格式，标准化了url
   
   以上为本次优化点，有时间可以帮忙review一下  @odbozhou  @duhenglucky   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq-connect] odbozhou commented on a diff in pull request #248: [ISSUE #238]Add pause and resume to the task, and optimize some rest APIs

Posted by GitBox <gi...@apache.org>.
odbozhou commented on code in PR #248:
URL: https://github.com/apache/rocketmq-connect/pull/248#discussion_r953731847


##########
rocketmq-connect-runtime/src/main/java/org/apache/rocketmq/connect/runtime/converter/ConnectKeyValueConverter.java:
##########
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ */
+
+package org.apache.rocketmq.connect.runtime.converter;
+
+import com.alibaba.fastjson.JSON;
+import io.openmessaging.connector.api.data.Converter;
+import org.apache.rocketmq.connect.runtime.common.ConnectKeyValue;
+import org.apache.rocketmq.connect.runtime.common.LoggerName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.UnsupportedEncodingException;
+
+/**
+ * Converter data between ConnAndTaskConfigs and byte[].
+ */
+public class ConnectKeyValueConverter implements Converter<ConnectKeyValue> {

Review Comment:
   JSONConverter？



##########
rocketmq-connect-runtime/src/main/java/org/apache/rocketmq/connect/runtime/converter/ConnAndTasksStatusConverter.java:
##########
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ */
+
+package org.apache.rocketmq.connect.runtime.converter;
+
+import io.openmessaging.connector.api.data.Converter;
+import org.apache.rocketmq.connect.runtime.common.LoggerName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.UnsupportedEncodingException;
+
+/**
+ * Converter data between ConnAndTaskStatus and byte[].
+ */
+public class ConnAndTasksStatusConverter implements Converter<String> {

Review Comment:
   This implementation is like StringConverter, why does it need to be reimplemented?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq-connect] odbozhou merged pull request #248: [ISSUE #238]Add pause and resume to the task, and optimize some rest APIs

Posted by GitBox <gi...@apache.org>.
odbozhou merged PR #248:
URL: https://github.com/apache/rocketmq-connect/pull/248


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq-connect] sunxiaojian closed pull request #248: [ISSUE #238]Add pause and resume to the task, and optimize some rest APIs

Posted by GitBox <gi...@apache.org>.
sunxiaojian closed pull request #248: [ISSUE #238]Add pause and resume to the task, and optimize some rest APIs
URL: https://github.com/apache/rocketmq-connect/pull/248


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org