You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by "castorqin (via GitHub)" <gi...@apache.org> on 2023/10/18 03:13:13 UTC
[PR] [INLONG-9060][Manager] Fix manager return wrong sink configuration to sort standalone [inlong]
castorqin opened a new pull request, #9067:
URL: https://github.com/apache/inlong/pull/9067
### Prepare a Pull Request
*(Change the title refer to the following example)*
- Title Example: [INLONG-XYZ][Component] Title of the pull request
*(The following *XYZ* should be replaced by the actual [GitHub Issue](https://github.com/apache/inlong/issues) number)*
- Fixes #9060
### Motivation
1. update StreamSinkFieldEntityMapper.selectAllFields method, use table inlong_stream_field replace stream_sink_field table
2. if stream use extended fields , add FILED_OFFSET params
### Modifications
The details can be found at [#9060 ](https://github.com/apache/inlong/issues/9060).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9060][Manager] Fix manager return wrong sink configuration to sort standalone [inlong]
Posted by "dockerzhang (via GitHub)" <gi...@apache.org>.
dockerzhang merged PR #9067:
URL: https://github.com/apache/inlong/pull/9067
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9060][Manager] Fix manager return wrong sink configuration to sort standalone [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #9067:
URL: https://github.com/apache/inlong/pull/9067#discussion_r1364831476
##########
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/pulsar/PulsarSinkOperator.java:
##########
@@ -123,7 +123,8 @@ public Map<String, String> parse2IdParams(StreamSinkEntity streamSink, List<Stri
}
private String getFullTopicName(PulsarSinkDTO pulsarSinkDTO) {
- return pulsarSinkDTO.getPulsarTenant() + "/" + pulsarSinkDTO.getNamespace() + "/" + pulsarSinkDTO.getTopic();
+ return "persistent://" + pulsarSinkDTO.getPulsarTenant() + "/" + pulsarSinkDTO.getNamespace() + "/"
Review Comment:
Use `PULSAR_TOPIC_FORMAT`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9060][Manager] Fix manager return wrong sink configuration to sort standalone [inlong]
Posted by "castorqin (via GitHub)" <gi...@apache.org>.
castorqin commented on code in PR #9067:
URL: https://github.com/apache/inlong/pull/9067#discussion_r1368055867
##########
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/pulsar/PulsarSinkOperator.java:
##########
@@ -123,7 +123,8 @@ public Map<String, String> parse2IdParams(StreamSinkEntity streamSink, List<Stri
}
private String getFullTopicName(PulsarSinkDTO pulsarSinkDTO) {
- return pulsarSinkDTO.getPulsarTenant() + "/" + pulsarSinkDTO.getNamespace() + "/" + pulsarSinkDTO.getTopic();
+ return "persistent://" + pulsarSinkDTO.getPulsarTenant() + "/" + pulsarSinkDTO.getNamespace() + "/"
Review Comment:
done
##########
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/core/impl/SortClusterServiceImpl.java:
##########
@@ -278,6 +292,18 @@ private List<Map<String, String>> parseIdParams(List<StreamSinkEntity> streams,
.collect(Collectors.toList());
}
+ private Map<String, String> setFiledOffset(StreamSinkEntity streamSink, Map<String, String> params) {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9060][Manager] Fix manager return wrong sink configuration to sort standalone [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #9067:
URL: https://github.com/apache/inlong/pull/9067#discussion_r1368081385
##########
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/resource/sink/cls/ClsOperator.java:
##########
@@ -55,26 +56,28 @@
@Service
public class ClsOperator {
+ @Value("${cls.manager.endpoint}")
Review Comment:
Please add the configuration to `application-dev.properties` and add relevant comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9060][Manager] Fix manager return wrong sink configuration to sort standalone [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #9067:
URL: https://github.com/apache/inlong/pull/9067#discussion_r1364834627
##########
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/core/impl/SortClusterServiceImpl.java:
##########
@@ -278,6 +292,18 @@ private List<Map<String, String>> parseIdParams(List<StreamSinkEntity> streams,
.collect(Collectors.toList());
}
+ private Map<String, String> setFiledOffset(StreamSinkEntity streamSink, Map<String, String> params) {
Review Comment:
void setFiledOffset(StreamSinkEntity streamSink, Map<String, String> params)
##########
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/core/impl/SortClusterServiceImpl.java:
##########
@@ -278,6 +292,18 @@ private List<Map<String, String>> parseIdParams(List<StreamSinkEntity> streams,
.collect(Collectors.toList());
}
+ private Map<String, String> setFiledOffset(StreamSinkEntity streamSink, Map<String, String> params) {
+
+ SortSourceStreamInfo sortSourceStreamInfo = allStreams.get(streamSink.getInlongGroupId())
+ .get(streamSink.getInlongStreamId());
+ InlongStreamExtParam inlongStreamExtParam = JsonUtils.parseObject(
+ sortSourceStreamInfo.getExtParams(), InlongStreamExtParam.class);
+ if (inlongStreamExtParam != null && !inlongStreamExtParam.getUseExtendedFields()) {
Review Comment:
CollectionUtils.isNotEmpty()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9060][Manager] Fix manager return wrong sink configuration to sort standalone [inlong]
Posted by "castorqin (via GitHub)" <gi...@apache.org>.
castorqin commented on code in PR #9067:
URL: https://github.com/apache/inlong/pull/9067#discussion_r1368083458
##########
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/resource/sink/cls/ClsOperator.java:
##########
@@ -55,26 +56,28 @@
@Service
public class ClsOperator {
+ @Value("${cls.manager.endpoint}")
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-9060][Manager] Fix manager return wrong sink configuration to sort standalone [inlong]
Posted by "castorqin (via GitHub)" <gi...@apache.org>.
castorqin commented on code in PR #9067:
URL: https://github.com/apache/inlong/pull/9067#discussion_r1368055976
##########
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/core/impl/SortClusterServiceImpl.java:
##########
@@ -278,6 +292,18 @@ private List<Map<String, String>> parseIdParams(List<StreamSinkEntity> streams,
.collect(Collectors.toList());
}
+ private Map<String, String> setFiledOffset(StreamSinkEntity streamSink, Map<String, String> params) {
+
+ SortSourceStreamInfo sortSourceStreamInfo = allStreams.get(streamSink.getInlongGroupId())
+ .get(streamSink.getInlongStreamId());
+ InlongStreamExtParam inlongStreamExtParam = JsonUtils.parseObject(
+ sortSourceStreamInfo.getExtParams(), InlongStreamExtParam.class);
+ if (inlongStreamExtParam != null && !inlongStreamExtParam.getUseExtendedFields()) {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org