You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "ityestop (via GitHub)" <gi...@apache.org> on 2023/02/11 16:44:32 UTC
[GitHub] [rocketmq-connect] ityestop opened a new issue, #424: 创建的topic为null的问题
ityestop opened a new issue, #424:
URL: https://github.com/apache/rocketmq-connect/issues/424
**问题描述**
在进行rocketmq-connect-runtime测试时，使用文档中的测试用例，结果运行结果是对的，可以正常生成文件，但是传递的topic是 null。
<img width="1570" alt="image" src="https://user-images.githubusercontent.com/14820088/218269291-3b34b8e8-ec43-46d7-987a-1f8afb7d6614.png">
**配置问题**
使用当前master分支（92c14dac8f5667b64aa532bdab411cb95bc18f2c）打包配置，无变化
**使用rocketMq 版本**
rocketMq版本 4.9.2
rocketMq-client 4.7.1
**触发命令**
```
curl -X POST -H "Content-Type: application/json" http://127.0.0.1:8082/connectors/fileSourceConnector -d '{"connector.class":"org.apache.rocketmq.connect.file.FileSourceConnector","filename":"test-source-file.txt","connect.topicname":"fileTopic"}'
```
**排查问题**
https://github.com/apache/rocketmq-connect/blob/92c14dac8f5667b64aa532bdab411cb95bc18f2c/rocketmq-connect-sample/src/main/java/org/apache/rocketmq/connect/file/FileSourceTask.java#L149
<img width="1188" alt="image" src="https://user-images.githubusercontent.com/14820088/218269388-f3c080a4-7249-41c9-a14d-59c9ef6d92ee.png">
当前代码获取 `fileConfig.topic`，但是在 connector 创建时传入的是 connect.topicname， 不存在赋值。此时一定为null传入，但是根据代码排查，调用的方法为 io.openmessaging.connector.api.data.ConnectRecord#addExtension(java.lang.String, java.lang.String)
最终执行方法为 /Users/cchu/.m2/repository/io/openmessaging/openmessaging-api/0.3.1-alpha/openmessaging-api-0.3.1-alpha.jar!/io/openmessaging/internal/DefaultKeyValue.class:35
<img width="654" alt="image" src="https://user-images.githubusercontent.com/14820088/218269683-67b3d720-08d0-49cd-b77a-08859538c157.png">
根据java.lang.String#valueOf(java.lang.Object) 查看，发现对象传入为null时，返回的是”null“字符串。
https://github.com/apache/rocketmq-connect/blob/92c14dac8f5667b64aa532bdab411cb95bc18f2c/rocketmq-connect-runtime/src/main/java/org/apache/rocketmq/connect/runtime/connectorwrapper/WorkerSourceTask.java#L422
<img width="698" alt="image" src="https://user-images.githubusercontent.com/14820088/218270112-438e2bd8-8b85-4f2a-bc2e-66cccd841825.png">
在org.apache.rocketmq.connect.runtime.connectorwrapper.WorkerSourceTask#maybeCreateAndGetTopic方法中的判断，优先取topic，补偿取 connect.topicname 。由于 topic = "null"，最终导致创建topic为 null
以上是排查的定位，不排除个人用法不当导致的问题，希望有朋友可以一直帮忙double check一下。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-connect] ityestop commented on issue #424: 创建的topic为null的问题
Posted by "ityestop (via GitHub)" <gi...@apache.org>.
ityestop commented on issue #424:
URL: https://github.com/apache/rocketmq-connect/issues/424#issuecomment-1430857563
好的，我再尝试理解一下全流程逻辑后，再进行讨论。感谢
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-connect] sunxiaojian closed issue #424: 创建的topic为null的问题
Posted by "sunxiaojian (via GitHub)" <gi...@apache.org>.
sunxiaojian closed issue #424: 创建的topic为null的问题
URL: https://github.com/apache/rocketmq-connect/issues/424
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-connect] sunxiaojian commented on issue #424: 创建的topic为null的问题
Posted by "sunxiaojian (via GitHub)" <gi...@apache.org>.
sunxiaojian commented on issue #424:
URL: https://github.com/apache/rocketmq-connect/issues/424#issuecomment-1427228601
> 使用文档 https://github.com/apache/rocketmq-connect#source-connector%E7%89%B9%E6%AE%8A%E9%85%8D%E7%BD%AE
>
> <img alt="image" width="903" src="https://user-images.githubusercontent.com/14820088/218295792-da0d6707-77e9-4b84-9a9b-31069356e067.png">
>
> 根据文档的含义与实际代码的追踪，发现在某次的改动后，逻辑出现了变化
>
> <img alt="image" width="1684" src="https://user-images.githubusercontent.com/14820088/218295773-6bc3ba9a-c06c-4365-95b5-3f5d719508b7.png">
提个 issue修复一下文档
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-connect] ityestop commented on issue #424: 创建的topic为null的问题
Posted by "ityestop (via GitHub)" <gi...@apache.org>.
ityestop commented on issue #424:
URL: https://github.com/apache/rocketmq-connect/issues/424#issuecomment-1426951825
使用文档
https://github.com/apache/rocketmq-connect#source-connector%E7%89%B9%E6%AE%8A%E9%85%8D%E7%BD%AE
<img width="903" alt="image" src="https://user-images.githubusercontent.com/14820088/218295792-da0d6707-77e9-4b84-9a9b-31069356e067.png">
根据文档的含义与实际代码的追踪，发现在某次的改动后，逻辑出现了变化
<img width="1684" alt="image" src="https://user-images.githubusercontent.com/14820088/218295773-6bc3ba9a-c06c-4365-95b5-3f5d719508b7.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-connect] ityestop commented on issue #424: 创建的topic为null的问题
Posted by "ityestop (via GitHub)" <gi...@apache.org>.
ityestop commented on issue #424:
URL: https://github.com/apache/rocketmq-connect/issues/424#issuecomment-1427335034
我怀疑是这里的代码逻辑问题，不是文档的问题。逻辑上应该优先取 connect.topicname，不存在时再取 overwriteTopicRecord(record)。connect.topicname是配置指定的，topic是兜底时代码写死的
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-connect] sunxiaojian commented on issue #424: 创建的topic为null的问题
Posted by "sunxiaojian (via GitHub)" <gi...@apache.org>.
sunxiaojian commented on issue #424:
URL: https://github.com/apache/rocketmq-connect/issues/424#issuecomment-1427475635
> 我怀疑是这里的代码逻辑问题，不是文档的问题。逻辑上应该优先取 connect.topicname，不存在时再取 overwriteTopicRecord(record)。connect.topicname是配置指定的，topic是兜底时代码写死的
优先取逻辑里面定义的动态topic ,后取connect.topicname
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org