You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streampipes.apache.org by GitBox <gi...@apache.org> on 2021/08/09 08:24:42 UTC
[GitHub] [incubator-streampipes] yangdh1225 opened a new pull request #48: flink 1.13 rest port config & Keep kafka consumer group for Pipeline action and sepas
yangdh1225 opened a new pull request #48:
URL: https://github.com/apache/incubator-streampipes/pull/48
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
~
-->
<!--
Thanks for contributing! Here are some tips you can follow to help us incorporate your contribution quickly and easily:
1. If this is your first time, please read our contributor guidelines:
- https://streampipes.apache.org/getinvolved.html
- https://cwiki.apache.org/confluence/display/STREAMPIPES/Getting+Started
2. Make sure the PR title is formatted like: `[STREAMPIPES-<Jira issue #>] PR title ...`
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., `[WIP][STREAMPIPES-<Jira issue #>] PR title ...`.
4. Please write your PR title to summarize what this PR proposes/fixes.
5. Be sure to keep the PR description updated to reflect all changes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. Make sure tests pass via `mvn clean install`.
8. (Optional) If the contribution is large, please file an Apache ICLA
- http://apache.org/licenses/icla.pdf
-->
### Purpose
<!--
Please clarify what changes you are proposing and describe how those changes will address the issue.
-->
1. custom flink rest port from configuration for version 1.13
2. Make kafka consumer group id fixed for pipeline action and sepas
### Approach
<!--
Describe how you are implementing the solutions along with the design details.
-->
### Samples
<!--
Provide high-level details about the samples related to this feature.
-->
### Remarks
<!--
List related issues/PRs, link to discussions in the mailing list, todo items, or any other notes related to the PR.
-->
Fixes: <Jira issue link>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampipes] yangdh1225 commented on pull request #48: flink 1.13 rest port config & Keep kafka consumer group for Pipeline action and sepas
Posted by GitBox <gi...@apache.org>.
yangdh1225 commented on pull request #48:
URL: https://github.com/apache/incubator-streampipes/pull/48#issuecomment-895705523
> Hi @yangdh1225
> thanks for the PR!
> Is there a reason why we use the pipelineName instead of the ID for generating the groupId?
>
> Dominik
Thanks for comment.
I have submitted a commit to replace special characters in pipeline name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampipes] dominikriemer merged pull request #48: flink 1.13 rest port config & Keep kafka consumer group for Pipeline action and sepas
Posted by GitBox <gi...@apache.org>.
dominikriemer merged pull request #48:
URL: https://github.com/apache/incubator-streampipes/pull/48
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampipes] yangdh1225 commented on a change in pull request #48: flink 1.13 rest port config & Keep kafka consumer group for Pipeline action and sepas
Posted by GitBox <gi...@apache.org>.
yangdh1225 commented on a change in pull request #48:
URL: https://github.com/apache/incubator-streampipes/pull/48#discussion_r685630742
##########
File path: streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/execution/http/PipelineExecutor.java
##########
@@ -107,7 +108,7 @@ private void updateGroupIds(InvocableStreamPipesEntity entity) {
.filter(is -> is.getEventGrounding().getTransportProtocol() instanceof KafkaTransportProtocol)
.map(is -> is.getEventGrounding().getTransportProtocol())
.map(KafkaTransportProtocol.class::cast)
- .forEach(tp -> tp.setGroupId(UUID.randomUUID().toString()));
+ .forEach(tp -> tp.setGroupId(pipeline.getName() + "-" + MD5.crypt(tp.getElementId())));
Review comment:
The pipeline name is used instead of id, just to make it easier to identify in Kafka monitoring. Certainly if the UI interface does not check special characters, such as spaces, surely we should use id to generate group id.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampipes] dominikriemer commented on pull request #48: flink 1.13 rest port config & Keep kafka consumer group for Pipeline action and sepas
Posted by GitBox <gi...@apache.org>.
dominikriemer commented on pull request #48:
URL: https://github.com/apache/incubator-streampipes/pull/48#issuecomment-896260937
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampipes] dominikriemer commented on a change in pull request #48: flink 1.13 rest port config & Keep kafka consumer group for Pipeline action and sepas
Posted by GitBox <gi...@apache.org>.
dominikriemer commented on a change in pull request #48:
URL: https://github.com/apache/incubator-streampipes/pull/48#discussion_r685519049
##########
File path: streampipes-pipeline-management/src/main/java/org/apache/streampipes/manager/execution/http/PipelineExecutor.java
##########
@@ -107,7 +108,7 @@ private void updateGroupIds(InvocableStreamPipesEntity entity) {
.filter(is -> is.getEventGrounding().getTransportProtocol() instanceof KafkaTransportProtocol)
.map(is -> is.getEventGrounding().getTransportProtocol())
.map(KafkaTransportProtocol.class::cast)
- .forEach(tp -> tp.setGroupId(UUID.randomUUID().toString()));
+ .forEach(tp -> tp.setGroupId(pipeline.getName() + "-" + MD5.crypt(tp.getElementId())));
Review comment:
Can we use pipeline.getId() instead of the name to avoid having blank spaces from the user-defined pipeline name in the group Id?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org