You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "László Bodor (Jira)" <ji...@apache.org> on 2022/02/05 16:58:00 UTC
[jira] [Commented] (HIVE-25706) ShuffleHandler: Make sure of properly releasing netty reference counted objects
[ https://issues.apache.org/jira/browse/HIVE-25706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487520#comment-17487520 ]
László Bodor commented on HIVE-25706:
-------------------------------------
merged to master, thanks [~pgaref] for the review!
> ShuffleHandler: Make sure of properly releasing netty reference counted objects
> -------------------------------------------------------------------------------
>
> Key: HIVE-25706
> URL: https://issues.apache.org/jira/browse/HIVE-25706
> Project: Hive
> Issue Type: Bug
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> https://netty.io/wiki/reference-counted-objects.html
> ShuffleHandler log contains issues below when we're on unhappy codepath (sendError). Very easily reproducible e.g. with calling ShuffleHandler's http port without any parameters. Doesn't cause a user-facing issue, because at this point ShuffleHandler already responded to the client, but the exception is annoying and implies invalid .release() calls in netty4.
> {code}
> io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
> at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
> at io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
> at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
> at io.netty.handler.codec.http.DefaultFullHttpResponse.release(DefaultFullHttpResponse.java:116) ~[netty-all-4.1.65.Final.jar:4.1.65.Final]
> at org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.sendError(ShuffleHandler.java:1056) ~[classes/:?]
> at org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.handleRequest(ShuffleHandler.java:827) ~[classes/:?]
> at org.apache.hadoop.hive.llap.shufflehandler.ShuffleHandler$Shuffle.channelRead(ShuffleHandler.java:728) ~[classes/:?]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-all-4.1.65.Final.jar:4.1.65.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-all-4.1.65.Final.jar:4.1.65.Final]
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-all-4.1.65.Final.jar:4.1.65.Final]
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-all-4.1.65.Final.jar:4.1.65.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-all-4.1.65.Final.jar:4.1.65.Final]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)