[jira] [Commented] (FLINK-6958) Async I/O timeout not work

["Aljoscha Krettek (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072391#comment-16072391 ] 

Aljoscha Krettek commented on FLINK-6958:
-----------------------------------------

I just noticed this now but I have a few observations:

 - I seems to have been a conscious decision by [~StephanEwen] do "drop" any pending processing-time timers and shutdown the operator/task as fast as possible. The main motivation for this, I think, is that a task should terminate quickly in case we have a job failure and we need to shutdown tasks and bring them up again.
 - Blocking on the queue of promises when closing in general seems to run counter to this desire of quick shutdown.
 - Instead of manually blocking for the duration of the timeout a fix for this should be to make sure that remaining processing-time timers fire. This would be the more general solution.

In general, I think dropping processing-time timers when a job shuts down is somewhat similar to data loss: a user did schedule a computation in the future that will output some data and we drop that. To reconcile this with the requirement of fast shutdown/restart in case of failures I think we need to have different shutdown paths. In case of a shutdown due to failure we drop everything and bring down as fast as possible. In case of a regular shutdown (because bounded sources finished) we need to make sure to flush out all remaining timers. This requires more discussion, though.

What do you think?

> Async I/O timeout not work
> --------------------------
>
>                 Key: FLINK-6958
>                 URL: https://issues.apache.org/jira/browse/FLINK-6958
>             Project: Flink
>          Issue Type: Bug
>          Components: Streaming
>    Affects Versions: 1.2.1
>            Reporter: feng xiaojie
>            Assignee: Jark Wu
>
> when use Async I/O with UnorderedStreamElementQueue, the queue will always full if you don't  call the AsyncCollector.collect to ack them.
> Timeout shall collect these entries when the timeout trigger,but it isn't work
> I debug find,
> when time out, it will call resultFuture.completeExceptionally(error);
> but not call  UnorderedStreamElementQueue.onCompleteHandler
> it will cause that async i/o hang always



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)