You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@streampipes.apache.org by "Muhammad Faizan (Jira)" <ji...@apache.org> on 2021/03/22 13:36:00 UTC
[jira] [Updated] (STREAMPIPES-321) Fail to stop pipeline if one of
the PE is stopped manually/not running
[ https://issues.apache.org/jira/browse/STREAMPIPES-321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Muhammad Faizan updated STREAMPIPES-321:
----------------------------------------
Description:
I have identified a problem with the following use case.
*Use case:*
1) Run a pipeline with two flink PE's.
2) Go to flink dashboard and cancel one of the flink job (i.e. PE)
3) Go to streampipes UI and try to stop the pipeline.
4) It fails to stop the pipeline completely because of the missing PE.
5) The other PE's are stopped now, but the pipeline is still shown as running.
*Expected behaviour:*
If the flink job is not found, then it means that it is cancelled or failed. So streampipes should continue to stop this PE from its in-memory storage and mark the whole pipeline as Stopped.
See the attached screenshots!
Right now, I see two approaches: # Put the line 141 ([https://github.com/apache/incubator-streampipes/blob/2f10a17b77a93eee48527e4e88baf[…]streampipes/container/api/InvocablePipelineElementResource.java|https://github.com/apache/incubator-streampipes/blob/2f10a17b77a93eee48527e4e88baf7e4b144d6fe/streampipes-container/src/main/java/org/apache/streampipes/container/api/InvocablePipelineElementResource.java#L141]) inside a try/catch block, and execute line 144 in any case.
# Or, make a change in FlinkRuntime.java ([https://github.com/apache/incubator-streampipes/blob/2f10a17b77a93eee48527e4e88baf[…]ain/java/org/apache/streampipes/wrapper/flink/FlinkRuntime.java|https://github.com/apache/incubator-streampipes/blob/2f10a17b77a93eee48527e4e88baf7e4b144d6fe/streampipes-wrapper-flink/src/main/java/org/apache/streampipes/wrapper/flink/FlinkRuntime.java#L264]) in method discardRuntime() to also check if its present and its state is running then cancel the job, else check if its status is CANCELLED or FAILED, then return and don't throw exception. Otherwise throw exception.
was:
I have identified a problem with the following use case.
*Use case:*
1) Run a pipeline with two flink PE's.
2) Go to flink dashboard and cancel one of the flink job (i.e. PE)
3) Go to streampipes UI and try to stop the pipeline.
4) It fails to stop the pipeline completely because of the missing PE.
5) The other PE's are stopped now, but the pipeline is still shown as running.
*Expected behaviour:*
If the flink job is not found, then it means that it is cancelled or failed. So streampipes should continue to stop this PE from its in-memory storage and mark the whole pipeline as Stopped.
See the attached screenshots!
ight now, I see two approaches: # Put the line 141 ([https://github.com/apache/incubator-streampipes/blob/2f10a17b77a93eee48527e4e88baf[…]streampipes/container/api/InvocablePipelineElementResource.java|https://github.com/apache/incubator-streampipes/blob/2f10a17b77a93eee48527e4e88baf7e4b144d6fe/streampipes-container/src/main/java/org/apache/streampipes/container/api/InvocablePipelineElementResource.java#L141]) inside a try/catch block, and execute line 144 in any case.
# Or, make a change in FlinkRuntime.java ([https://github.com/apache/incubator-streampipes/blob/2f10a17b77a93eee48527e4e88baf[…]ain/java/org/apache/streampipes/wrapper/flink/FlinkRuntime.java|https://github.com/apache/incubator-streampipes/blob/2f10a17b77a93eee48527e4e88baf7e4b144d6fe/streampipes-wrapper-flink/src/main/java/org/apache/streampipes/wrapper/flink/FlinkRuntime.java#L264]) in method discardRuntime() to also check if its present and its state is running then cancel the job, else check if its status is CANCELLED or FAILED, then return and don't throw exception. Otherwise throw exception.
> Fail to stop pipeline if one of the PE is stopped manually/not running
> ----------------------------------------------------------------------
>
> Key: STREAMPIPES-321
> URL: https://issues.apache.org/jira/browse/STREAMPIPES-321
> Project: StreamPipes
> Issue Type: Bug
> Components: Backend, Pipeline Elements
> Reporter: Muhammad Faizan
> Priority: Major
> Fix For: 0.68.0
>
> Attachments: zx1.png, zx2.png
>
>
> I have identified a problem with the following use case.
> *Use case:*
> 1) Run a pipeline with two flink PE's.
> 2) Go to flink dashboard and cancel one of the flink job (i.e. PE)
> 3) Go to streampipes UI and try to stop the pipeline.
> 4) It fails to stop the pipeline completely because of the missing PE.
> 5) The other PE's are stopped now, but the pipeline is still shown as running.
>
> *Expected behaviour:*
> If the flink job is not found, then it means that it is cancelled or failed. So streampipes should continue to stop this PE from its in-memory storage and mark the whole pipeline as Stopped.
>
> See the attached screenshots!
>
> Right now, I see two approaches: # Put the line 141 ([https://github.com/apache/incubator-streampipes/blob/2f10a17b77a93eee48527e4e88baf[…]streampipes/container/api/InvocablePipelineElementResource.java|https://github.com/apache/incubator-streampipes/blob/2f10a17b77a93eee48527e4e88baf7e4b144d6fe/streampipes-container/src/main/java/org/apache/streampipes/container/api/InvocablePipelineElementResource.java#L141]) inside a try/catch block, and execute line 144 in any case.
> # Or, make a change in FlinkRuntime.java ([https://github.com/apache/incubator-streampipes/blob/2f10a17b77a93eee48527e4e88baf[…]ain/java/org/apache/streampipes/wrapper/flink/FlinkRuntime.java|https://github.com/apache/incubator-streampipes/blob/2f10a17b77a93eee48527e4e88baf7e4b144d6fe/streampipes-wrapper-flink/src/main/java/org/apache/streampipes/wrapper/flink/FlinkRuntime.java#L264]) in method discardRuntime() to also check if its present and its state is running then cancel the job, else check if its status is CANCELLED or FAILED, then return and don't throw exception. Otherwise throw exception.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)