[jira] [Commented] (FLINK-8020) Deadlock found in Flink Streaming job

["Chesnay Schepler (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-8020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346590#comment-16346590 ] 

Chesnay Schepler commented on FLINK-8020:
-----------------------------------------

are you certain that you're actually completing the futures? If the output queue of the {{AsyncWaitOperator}} is filled up with uncompleted futures the job will in fact grind to a halt, with the operator and emitter thread at the exact places they are in the stack traces.

> Deadlock found in Flink Streaming job
> -------------------------------------
>
>                 Key: FLINK-8020
>                 URL: https://issues.apache.org/jira/browse/FLINK-8020
>             Project: Flink
>          Issue Type: Bug
>          Components: Kafka Connector, Streaming, Streaming Connectors
>    Affects Versions: 1.3.2
>         Environment: Kafka 0.8.2 and Flink 1.3.2 on YARN mode
>            Reporter: Weihua Jiang
>            Priority: Blocker
>             Fix For: 1.5.0, 1.4.1
>
>         Attachments: jstack53009(2).out, jstack67976-2.log
>
>
> Our streaming job run into trouble in these days after a long time smooth running. One issue we found is [https://issues.apache.org/jira/browse/FLINK-8019] and another one is this one.
> After analyzing the jstack, we believe  we found a DEAD LOCK in flink:
> 1. The thread "cache-process0 -> async-operator0 -> Sink: hbase-sink0 (8/8)" hold lock 0x00000007b6aa1788 and is waiting for lock 0x00000007b6aa1940.
> 2. The thread "Time Trigger for cache-process0 -> async-operator0 -> Sink: hbase-sink0 (8/8)" hold lock 0x00000007b6aa1940 and is waiting for lock 0x00000007b6aa1788. 
> This DEADLOCK made the job fail to proceed. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)