You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by "fuweng11 (via GitHub)" <gi...@apache.org> on 2023/08/24 03:18:28 UTC
[GitHub] [inlong] fuweng11 opened a new pull request, #8795: [INLONG-8794][Manager] Support add streamField and sinkField
fuweng11 opened a new pull request, #8795:
URL: https://github.com/apache/inlong/pull/8795
### Prepare a Pull Request
- Fixes #8794
### Motivation
Support add streamField and sinkField and remove duplicates when adding fields.
### Modifications
Support add streamField and sinkField and remove duplicates when adding fields.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] vernedeng commented on a diff in pull request #8795: [INLONG-8794][Manager] Support add streamField and sinkField
Posted by "vernedeng (via GitHub)" <gi...@apache.org>.
vernedeng commented on code in PR #8795:
URL: https://github.com/apache/inlong/pull/8795#discussion_r1303741145
##########
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkServiceImpl.java:
##########
@@ -700,6 +703,39 @@ public Boolean updateAfterApprove(List<SinkApproveDTO> approveList, String opera
return true;
}
+ @Override
+ public boolean addFields(StreamSinkEntity sinkEntity, List<SinkField> sinkFieldList) {
+ Set<String> existFields = sinkFieldMapper.selectBySinkId(sinkEntity.getId()).stream()
+ .map(StreamSinkFieldEntity::getFieldName).collect(Collectors.toSet());
+
+ LOGGER.debug("begin to save sink fields={}", sinkFieldList);
+ if (CollectionUtils.isEmpty(sinkFieldList)) {
+ return true;
+ }
+ List<StreamSinkFieldEntity> needAddFieldList = new ArrayList<>();
+ for (SinkField fieldInfo : sinkFieldList) {
+ if (existFields.contains(fieldInfo.getFieldName())) {
+ continue;
+ }
+ StreamSinkFieldEntity fieldEntity = CommonBeanUtils.copyProperties(fieldInfo,
+ StreamSinkFieldEntity::new);
+ if (StringUtils.isEmpty(fieldEntity.getFieldComment())) {
+ fieldEntity.setFieldComment(fieldEntity.getFieldName());
+ }
+ fieldEntity.setInlongGroupId(sinkEntity.getInlongGroupId());
+ fieldEntity.setInlongStreamId(sinkEntity.getInlongStreamId());
+ fieldEntity.setSinkType(sinkEntity.getSinkType());
+ fieldEntity.setSinkId(sinkEntity.getId());
+ fieldEntity.setIsDeleted(InlongConstants.UN_DELETED);
+ needAddFieldList.add(fieldEntity);
+ }
+ if (CollectionUtils.isNotEmpty(needAddFieldList)) {
+ sinkFieldMapper.insertAll(needAddFieldList);
Review Comment:
ditto
##########
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkServiceImpl.java:
##########
@@ -700,6 +703,39 @@ public Boolean updateAfterApprove(List<SinkApproveDTO> approveList, String opera
return true;
}
+ @Override
+ public boolean addFields(StreamSinkEntity sinkEntity, List<SinkField> sinkFieldList) {
+ Set<String> existFields = sinkFieldMapper.selectBySinkId(sinkEntity.getId()).stream()
+ .map(StreamSinkFieldEntity::getFieldName).collect(Collectors.toSet());
+
+ LOGGER.debug("begin to save sink fields={}", sinkFieldList);
+ if (CollectionUtils.isEmpty(sinkFieldList)) {
+ return true;
Review Comment:
Please add more debug log
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] dockerzhang merged pull request #8795: [INLONG-8794][Manager] Support add streamField and sinkField
Posted by "dockerzhang (via GitHub)" <gi...@apache.org>.
dockerzhang merged PR #8795:
URL: https://github.com/apache/inlong/pull/8795
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] fuweng11 commented on pull request #8795: [INLONG-8794][Manager] Support add streamField and sinkField
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on PR #8795:
URL: https://github.com/apache/inlong/pull/8795#issuecomment-1691117552
> Maybe it is better to add some unit tests for it.
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] fuweng11 commented on a diff in pull request #8795: [INLONG-8794][Manager] Support add streamField and sinkField
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #8795:
URL: https://github.com/apache/inlong/pull/8795#discussion_r1303846830
##########
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkServiceImpl.java:
##########
@@ -700,6 +703,39 @@ public Boolean updateAfterApprove(List<SinkApproveDTO> approveList, String opera
return true;
}
+ @Override
+ public boolean addFields(StreamSinkEntity sinkEntity, List<SinkField> sinkFieldList) {
+ Set<String> existFields = sinkFieldMapper.selectBySinkId(sinkEntity.getId()).stream()
+ .map(StreamSinkFieldEntity::getFieldName).collect(Collectors.toSet());
+
+ LOGGER.debug("begin to save sink fields={}", sinkFieldList);
+ if (CollectionUtils.isEmpty(sinkFieldList)) {
+ return true;
Review Comment:
The log above shows whether the field is empty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org