[jira] [Commented] (AVRO-1391) NettyTransceiver await called in IO thread

["Jan Van Besien (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13813724#comment-13813724 ] 

Jan Van Besien commented on AVRO-1391:
--------------------------------------

FYI. I worked around the problem by starting up new threads for the (avro-ipc) communication with my backend system from within the custom flume sink. Although the whole flow is still synchronous (I start a thread to do the synchronous call and wait for the thread to finish), Netty no longer thinks that the call originates from an IO thread.

The deadlock detection mechanism in Netty is wrong here I think, because the IO thread is actually only an IO thread on a different Netty pipeline.

If the NettyTransceiver implementation in Avro would be truly asynchronous (i.e. do not wait for the channel to be created but add a listener which gets notified when it is ready), the problem would not occur. However, I can imagine that this would be a non trivial change which would ripple up in client code.

> NettyTransceiver await called in IO thread
> ------------------------------------------
>
>                 Key: AVRO-1391
>                 URL: https://issues.apache.org/jira/browse/AVRO-1391
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.7.5
>            Reporter: Jan Van Besien
>
> The NettyTransceiver does a call to channelFuture.await() in the getChannel() method (line 271). This fails with the following exception if avro ipc is used from within an IO thread from another avro-ipc call.
> Caused by: java.lang.IllegalStateException: await*() in I/O thread causes a dead lock or sudden performance drop. Use addListener() instead or call await*() from a different thread.
>         at org.jboss.netty.channel.DefaultChannelFuture.checkDeadLock(DefaultChannelFuture.java:314)
>         at org.jboss.netty.channel.DefaultChannelFuture.await0(DefaultChannelFuture.java:278)
> ...
> The use case where I was able to reproduce this is when apache flume's avro source is coupled with a custom flume sink that also uses avro-ipc to talk to something else. In that case, the thread that ultimately does the above getChannel() call is an "avro io thread".



--
This message was sent by Atlassian JIRA
(v6.1#6144)