You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Chirag Dewan <ch...@yahoo.in> on 2018/07/19 06:25:48 UTC
Production readiness of Flink Job Stop Service
Hi,
I am planning to use the Stop Service for stopping/resuming/pausing my Flink Job. My intention is to stop sources before we take the savepoint i.e. stop with savepoint.
I know that since Flink 1.4.2, Stop is not stable/not production ready.
With Flink 1.5 can it be used for stopping jobs?
Regards,
Chirag
Re: Production readiness of Flink Job Stop Service
Posted by vino yang <ya...@gmail.com>.
Hi Chirag,
Actually, if you want to use "stop/resume/pause" pattern, you can trigger a
savepoint before stopping job, then you can resume your job with specify
this savepoint. The savepoint does not bind with cancel command, although
you can only invoke "cancel with savepoint" now.
Now you mentioned "how to implement a sink to make it no duplicate records"
and "stop source before the savepoint is triggered", not only about "stop
with savepoint". Can you describe your use case more clear or provide more
detail information?
Here is a mailing list about discussing "cancel and stop"[1].
[1]:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-flick-cancel-vs-stop-td15534.html
Thanks, vino.
2018-07-23 10:38 GMT+08:00 Chirag Dewan <ch...@yahoo.in>:
> Thanks a lot Fabian and Vino.
>
> Is there anyway I can do that without stop? Although the plan should be to
> use a Two Phase commit sink so that there are no duplicates, I am looking
> for stopping the sources before the savepoint is triggered.
>
> Thanks,
>
> Chirag
>
> Sent from Yahoo Mail on Android
> <https://go.onelink.me/107872968?pid=InProduct&c=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers&af_wl=ym&af_sub1=Internal&af_sub2=Global_YGrowth&af_sub3=EmailSignature>
>
> On Thu, 19 Jul 2018 at 1:35 PM, Fabian Hueske
> <fh...@gmail.com> wrote:
> Hi Chirag,
>
> Stop with savepoint is not mentioned in the 1.5.0 release notes [1].
> Since its a frequently requested feature, I'm pretty sure that it would
> have been mentioned if it was added.
>
> Best, Fabian
>
> [1] http://flink.apache.org/news/2018/05/25/release-1.5.0.html
>
>
> 2018-07-19 8:39 GMT+02:00 vino yang <ya...@gmail.com>:
>
> Hi Chirag,
>
> Did you read the latest stable Flink documentation about Savepoint[1] and
> Cancel with savepoint[2] and Upgrade application[3]?
>
> [1]: https://ci.apache.org/ projects/flink/flink-docs-
> release-1.5/ops/state/ savepoints.html#resuming-from- savepoints
> <https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/state/savepoints.html#resuming-from-savepoints>
> [2]: https://ci.apache.org/ projects/flink/flink-docs-
> release-1.5/monitoring/rest_ api.html#cancel-job-with- savepoint
> <https://ci.apache.org/projects/flink/flink-docs-release-1.5/monitoring/rest_api.html#cancel-job-with-savepoint>
> [3]: https://ci.apache.org/ projects/flink/flink-docs-
> release-1.5/ops/upgrading.html
> <https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/upgrading.html>
>
> Thanks, vino.
>
>
> 2018-07-19 14:25 GMT+08:00 Chirag Dewan <ch...@yahoo.in>:
>
> Hi,
>
> I am planning to use the Stop Service for stopping/resuming/pausing my
> Flink Job. My intention is to stop sources before we take the savepoint
> i.e. stop with savepoint.
>
> I know that since Flink 1.4.2, Stop is not stable/not production ready.
>
> With Flink 1.5 can it be used for stopping jobs?
>
> Regards,
>
> Chirag
>
>
>
>
Re: Production readiness of Flink Job Stop Service
Posted by Chirag Dewan <ch...@yahoo.in>.
Thanks a lot Fabian and Vino.
Is there anyway I can do that without stop? Although the plan should be to use a Two Phase commit sink so that there are no duplicates, I am looking for stopping the sources before the savepoint is triggered.
Thanks,
Chirag
Sent from Yahoo Mail on Android
On Thu, 19 Jul 2018 at 1:35 PM, Fabian Hueske<fh...@gmail.com> wrote: Hi Chirag,
Stop with savepoint is not mentioned in the 1.5.0 release notes [1].Since its a frequently requested feature, I'm pretty sure that it would have been mentioned if it was added.
Best, Fabian
[1] http://flink.apache.org/news/2018/05/25/release-1.5.0.html
2018-07-19 8:39 GMT+02:00 vino yang <ya...@gmail.com>:
Hi Chirag,
Did you read the latest stable Flink documentation about Savepoint[1] and Cancel with savepoint[2] and Upgrade application[3]?
[1]: https://ci.apache.org/ projects/flink/flink-docs- release-1.5/ops/state/ savepoints.html#resuming-from- savepoints[2]: https://ci.apache.org/ projects/flink/flink-docs- release-1.5/monitoring/rest_ api.html#cancel-job-with- savepoint[3]: https://ci.apache.org/ projects/flink/flink-docs- release-1.5/ops/upgrading.html
Thanks, vino.
2018-07-19 14:25 GMT+08:00 Chirag Dewan <ch...@yahoo.in>:
Hi,
I am planning to use the Stop Service for stopping/resuming/pausing my Flink Job. My intention is to stop sources before we take the savepoint i.e. stop with savepoint.
I know that since Flink 1.4.2, Stop is not stable/not production ready.
With Flink 1.5 can it be used for stopping jobs?
Regards,
Chirag
Re: Production readiness of Flink Job Stop Service
Posted by Fabian Hueske <fh...@gmail.com>.
Hi Chirag,
Stop with savepoint is not mentioned in the 1.5.0 release notes [1].
Since its a frequently requested feature, I'm pretty sure that it would
have been mentioned if it was added.
Best, Fabian
[1] http://flink.apache.org/news/2018/05/25/release-1.5.0.html
2018-07-19 8:39 GMT+02:00 vino yang <ya...@gmail.com>:
> Hi Chirag,
>
> Did you read the latest stable Flink documentation about Savepoint[1] and
> Cancel with savepoint[2] and Upgrade application[3]?
>
> [1]: https://ci.apache.org/projects/flink/flink-docs-
> release-1.5/ops/state/savepoints.html#resuming-from-savepoints
> [2]: https://ci.apache.org/projects/flink/flink-docs-
> release-1.5/monitoring/rest_api.html#cancel-job-with-savepoint
> [3]: https://ci.apache.org/projects/flink/flink-docs-
> release-1.5/ops/upgrading.html
>
> Thanks, vino.
>
>
> 2018-07-19 14:25 GMT+08:00 Chirag Dewan <ch...@yahoo.in>:
>
>> Hi,
>>
>> I am planning to use the Stop Service for stopping/resuming/pausing my
>> Flink Job. My intention is to stop sources before we take the savepoint
>> i.e. stop with savepoint.
>>
>> I know that since Flink 1.4.2, Stop is not stable/not production ready.
>>
>> With Flink 1.5 can it be used for stopping jobs?
>>
>> Regards,
>>
>> Chirag
>>
>
>
Re: Production readiness of Flink Job Stop Service
Posted by vino yang <ya...@gmail.com>.
Hi Chirag,
Did you read the latest stable Flink documentation about Savepoint[1] and
Cancel with savepoint[2] and Upgrade application[3]?
[1]:
https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/state/savepoints.html#resuming-from-savepoints
[2]:
https://ci.apache.org/projects/flink/flink-docs-release-1.5/monitoring/rest_api.html#cancel-job-with-savepoint
[3]:
https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/upgrading.html
Thanks, vino.
2018-07-19 14:25 GMT+08:00 Chirag Dewan <ch...@yahoo.in>:
> Hi,
>
> I am planning to use the Stop Service for stopping/resuming/pausing my
> Flink Job. My intention is to stop sources before we take the savepoint
> i.e. stop with savepoint.
>
> I know that since Flink 1.4.2, Stop is not stable/not production ready.
>
> With Flink 1.5 can it be used for stopping jobs?
>
> Regards,
>
> Chirag
>