You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Mihail Vieru <mi...@zalando.de> on 2017/03/24 11:00:29 UTC

Close method call in sinks

Hi,

quick question: is the close method in sinks, e.g. BucketingSink, called
when a job gets cancelled or stopped?

We're trying to have the following: A custom stoppable streaming source
with a modified BucketingSink which moves the files from ephemeral storage
to S3 Frankfurt upon the closing of part files.

Best,
Mihail

Re: Close method call in sinks

Posted by Till Rohrmann <tr...@apache.org>.

Hi Mihail,

the close method of the RichFunction will always be called. Only if the JVM
crashes hard and the task thread does not have a chance to clean up the
user functions, this won't be the case.

Cheers,
Till

On Fri, Mar 24, 2017 at 12:00 PM, Mihail Vieru <mi...@zalando.de>
wrote:

> Hi,
>
> quick question: is the close method in sinks, e.g. BucketingSink, called
> when a job gets cancelled or stopped?
>
> We're trying to have the following: A custom stoppable streaming source
> with a modified BucketingSink which moves the files from ephemeral storage
> to S3 Frankfurt upon the closing of part files.
>
> Best,
> Mihail
>