You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by GitBox <gi...@apache.org> on 2022/03/26 03:23:41 UTC
[GitHub] [incubator-inlong] kipshi opened a new pull request #3374: [INLONG-3369] Add streamsource in group/list Api
kipshi opened a new pull request #3374:
URL: https://github.com/apache/incubator-inlong/pull/3374
### Title Name:[INLONG-3369] Add streamsource in group/list Api
where *XYZ* should be replaced by the actual issue number.
Fixes #3369
### Motivation
### Modifications
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
*(Please pick either of the following options)*
This change is a trivial rework / code cleanup without any test coverage.
*(or)*
This change is already covered by existing tests, such as *(please describe tests)*.
*(or)*
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end deployment with large payloads (10MB)*
- *Extended integration test for recovery after broker failure*
### Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
- If a feature is not applicable for documentation, explain why?
- If a feature is not documented yet in this PR, please create a followup issue for adding the documentation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] kipshi commented on a change in pull request #3374: [INLONG-3369] Add streamsource in group/list Api
Posted by GitBox <gi...@apache.org>.
kipshi commented on a change in pull request #3374:
URL: https://github.com/apache/incubator-inlong/pull/3374#discussion_r835735082
##########
File path: inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/core/impl/InlongGroupServiceImpl.java
##########
@@ -206,6 +219,24 @@ public InlongGroupInfo get(String groupId) {
Page<InlongGroupEntity> entityPage = (Page<InlongGroupEntity>) groupMapper.selectByCondition(request);
List<InlongGroupListResponse> groupList = CommonBeanUtils.copyListProperties(entityPage,
InlongGroupListResponse::new);
+ if (request.isListSources() && CollectionUtils.isNotEmpty(groupList)) {
Review comment:
default listsources is false
@ApiModelProperty(value = "If list streamSource for group", hidden = true)
private boolean listSources = false;
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] gosonzhang merged pull request #3374: [INLONG-3369][Manager] Add streamsource in group/list Api
Posted by GitBox <gi...@apache.org>.
gosonzhang merged pull request #3374:
URL: https://github.com/apache/incubator-inlong/pull/3374
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] healchow commented on a change in pull request #3374: [INLONG-3369] Add streamsource in group/list Api
Posted by GitBox <gi...@apache.org>.
healchow commented on a change in pull request #3374:
URL: https://github.com/apache/incubator-inlong/pull/3374#discussion_r835719214
##########
File path: inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/core/impl/InlongGroupServiceImpl.java
##########
@@ -206,6 +219,24 @@ public InlongGroupInfo get(String groupId) {
Page<InlongGroupEntity> entityPage = (Page<InlongGroupEntity>) groupMapper.selectByCondition(request);
List<InlongGroupListResponse> groupList = CommonBeanUtils.copyListProperties(entityPage,
InlongGroupListResponse::new);
+ if (request.isListSources() && CollectionUtils.isNotEmpty(groupList)) {
Review comment:
There may be a lot of streams and sources under these groups, and the result set of the query may be very large.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] kipshi commented on a change in pull request #3374: [INLONG-3369] Add streamsource in group/list Api
Posted by GitBox <gi...@apache.org>.
kipshi commented on a change in pull request #3374:
URL: https://github.com/apache/incubator-inlong/pull/3374#discussion_r835735885
##########
File path: inlong-manager/manager-client/src/main/java/org/apache/inlong/manager/client/api/impl/InlongClientImpl.java
##########
@@ -97,16 +101,41 @@ public InlongGroup forGroup(InlongGroupConf groupConf) {
}
}
+ @Override
+ public Response<PageInfo<InlongGroupListResponse>> listGroup(InlongGroupPageRequest request) throws Exception {
+ InnerInlongManagerClient managerClient = new InnerInlongManagerClient(this);
+ return managerClient.listGroups(request);
+ }
+
/**
- * List group
+ * List group state
*
- * @param request The request
- * @return PageInfo of group
+ * @param groupNames
+ * @return
+ * @throws Exception
*/
@Override
- public Response<PageInfo<InlongGroupListResponse>> listGroup(InlongGroupPageRequest request) throws Exception {
+ public Map<String, InlongGroupState> listGroupState(List<String> groupNames) throws Exception {
InnerInlongManagerClient managerClient = new InnerInlongManagerClient(this);
- return managerClient.listGroups(request);
+ InlongGroupPageRequest request = new InlongGroupPageRequest();
+ request.setNameList(groupNames);
+ request.setPageNum(1);
+ request.setPageSize(groupNames.size());
+ request.setListSources(true);
+ Response<PageInfo<InlongGroupListResponse>> pageInfoResponse = managerClient.listGroups(request);
+ if (!pageInfoResponse.isSuccess() || pageInfoResponse.getErrMsg() != null) {
+ throw new RuntimeException("listGroupStateFailed:" + pageInfoResponse.getErrMsg());
+ }
+ List<InlongGroupListResponse> groupListResponses = pageInfoResponse.getData().getList();
+ Map<String, InlongGroupState> groupStateMap = Maps.newHashMap();
+ groupListResponses.stream().forEach(groupListResponse -> {
+ String groupId = groupListResponse.getInlongGroupId();
+ InlongGroupState groupState = InlongGroupState.parseByBizStatus(groupListResponse.getStatus());
+ List<SourceListResponse> sourceListResponses = groupListResponse.getSourceListResponses();
Review comment:
ok, but groupList is select in mybatis, if no result is return, mybatis return empty list rather than null
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] healchow commented on a change in pull request #3374: [INLONG-3369] Add streamsource in group/list Api
Posted by GitBox <gi...@apache.org>.
healchow commented on a change in pull request #3374:
URL: https://github.com/apache/incubator-inlong/pull/3374#discussion_r835735471
##########
File path: inlong-manager/manager-client/src/main/java/org/apache/inlong/manager/client/api/impl/InlongClientImpl.java
##########
@@ -97,16 +101,41 @@ public InlongGroup forGroup(InlongGroupConf groupConf) {
}
}
+ @Override
+ public Response<PageInfo<InlongGroupListResponse>> listGroup(InlongGroupPageRequest request) throws Exception {
+ InnerInlongManagerClient managerClient = new InnerInlongManagerClient(this);
+ return managerClient.listGroups(request);
+ }
+
/**
- * List group
+ * List group state
*
- * @param request The request
- * @return PageInfo of group
+ * @param groupNames
+ * @return
+ * @throws Exception
*/
@Override
- public Response<PageInfo<InlongGroupListResponse>> listGroup(InlongGroupPageRequest request) throws Exception {
+ public Map<String, InlongGroupState> listGroupState(List<String> groupNames) throws Exception {
InnerInlongManagerClient managerClient = new InnerInlongManagerClient(this);
- return managerClient.listGroups(request);
+ InlongGroupPageRequest request = new InlongGroupPageRequest();
+ request.setNameList(groupNames);
+ request.setPageNum(1);
+ request.setPageSize(groupNames.size());
+ request.setListSources(true);
+ Response<PageInfo<InlongGroupListResponse>> pageInfoResponse = managerClient.listGroups(request);
+ if (!pageInfoResponse.isSuccess() || pageInfoResponse.getErrMsg() != null) {
+ throw new RuntimeException("listGroupStateFailed:" + pageInfoResponse.getErrMsg());
+ }
+ List<InlongGroupListResponse> groupListResponses = pageInfoResponse.getData().getList();
+ Map<String, InlongGroupState> groupStateMap = Maps.newHashMap();
+ groupListResponses.stream().forEach(groupListResponse -> {
+ String groupId = groupListResponse.getInlongGroupId();
+ InlongGroupState groupState = InlongGroupState.parseByBizStatus(groupListResponse.getStatus());
+ List<SourceListResponse> sourceListResponses = groupListResponse.getSourceListResponses();
Review comment:
Here we need to check if the List is empty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org