You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Karthick Thanigaimani <ka...@yahoo.com.INVALID> on 2019/08/02 01:33:52 UTC
Flink issue
Hi Team,
We are facing frequent issues with the Flink job manager in one environment when the processing happens.
CHAIN Join(Remap EDGES id: TO) -> Map (Key Extractor) -> Combine (Deduplicate edges including bi-directional edges) (57/80)Timestamp: 2019-08-02, 4:13:25 Location: flink-taskmanager-c:6121
We have tried changing the EC2 sizes to a bigger one and increased heap size etc but still the same problem. The below is the error message that we see Could someone provide some guidance.
org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException: Lost connection to task manager 'flink-taskmanager-b/<IP>2:6121'. This indicates that the remote task manager was lost. at org.apache.flink.runtime.io.network.netty.PartitionRequestClientHandler.exceptionCaught(PartitionRequestClientHandler.java:146) at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275) at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253) at org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131) at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275) at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253) at org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131) at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275) at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253) at org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:79) at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275) at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253) at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:835) at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:87) at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:162) at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) at java.lang.Thread.run(Thread.java:748)Caused by: java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at sun.nio.ch.IOUtil.read(IOUtil.java:192) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) at org.apache.flink.shaded.netty4.io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:311) at org.apache.flink.shaded.netty4.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:881) at org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:241) at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119) ... 6 more
RegardsKarthtick
Re: Flink issue
Posted by Zhu Zhu <re...@gmail.com>.
For network issue, this answer might help
http://mail-archives.apache.org/mod_mbox/flink-user/201907.mbox/%3Cdb49d6f2-1a6b-490a-8502-edd9562b0163.yungao.gy@aliyun.com%3E
.
Thanks,
Zhu Zhu
Karthick Thanigaimani <ka...@yahoo.com> 于2019年8月2日周五 下午5:34写道：
> Yes Zhu, the server is online and it didn't get died. So it looks like to
> be a network issue but we checked all other things like ports / SG etc
> which looks fine.
>
>
> On Friday, 2 August, 2019, 06:08:37 pm GMT+10, Zhu Zhu <re...@gmail.com>
> wrote:
>
>
> Hi Karthick,
>
> Could you check whether the `lost` TM 'flink-taskmanager-b/<IP>2:6121' is
> still alive when the error is reported?
> If it is still alive then, this seems to be a network issue between
> 'flink-taskmanager-c:6121' and 'flink-taskmanager-b/<IP>2:6121'.
> Otherwise, it is needed to check why the TM exits, for internal
> failure(e.g. cancel timeout) or killed by other services(e.g. K8S). I'm not
> familiar with K8S so I'm not sure in which case the cluster may kill the TM.
>
> Thanks,
> Zhu Zhu
>
> Karthick Thanigaimani <ka...@yahoo.com> 于2019年8月2日周五 下午2:48写道：
>
> Thanks Zhu. That's correct but we don't see any errors in the log of that
> TM or the server logs or the Kube logs. Is there any bug in flink 1.4 that
> causes this or settings in flink that can avoid this.
>
>
> Regards
> Karthick
>
> n Fri, 2 Aug. 2019 at 16:04, Zhu Zhu
>
> <re...@gmail.com> wrote:
> Hi Karthick,
>
> From the log seems the TM "flink-taskmanager-b/<IP>2:6121" is lost
> unexpectedly.
> You may need to check the log of that TM to see why it exits, which should
> be the root cause.
>
> Thanks,
> Zhu Zhu
>
> Karthick Thanigaimani <ka...@yahoo.com.invalid> 于2019年8月2日周五
> 下午1:54写道：
>
> Hi Team,
> We are facing frequent issues with the Flink job manager in one
> environment when the processing happens.
> CHAIN Join(Remap EDGES id: TO) -> Map (Key Extractor) -> Combine
> (Deduplicate edges including bi-directional edges) (57/80)Timestamp:
> 2019-08-02, 4:13:25 Location: flink-taskmanager-c:6121
>
> We have tried changing the EC2 sizes to a bigger one and increased heap
> size etc but still the same problem. The below is the error message that we
> see Could someone provide some guidance.
>
> org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException:
> Lost connection to task manager 'flink-taskmanager-b/<IP>2:6121'. This
> indicates that the remote task manager was lost. at
> org.apache.flink.runtime.io.network.netty.PartitionRequestClientHandler.exceptionCaught(PartitionRequestClientHandler.java:146)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:79)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:835)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:87)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:162)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> at java.lang.Thread.run(Thread.java:748)Caused by: java.io.IOException:
> Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native
> Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at
> sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at
> sun.nio.ch.IOUtil.read(IOUtil.java:192) at
> sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) at
> org.apache.flink.shaded.netty4.io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:311)
> at
> org.apache.flink.shaded.netty4.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:881)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:241)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119)
> ... 6 more
>
>
> RegardsKarthtick
>
>
Re: Flink issue
Posted by Zhu Zhu <re...@gmail.com>.
Hi Karthick,
Could you check whether the `lost` TM 'flink-taskmanager-b/<IP>2:6121' is
still alive when the error is reported?
If it is still alive then, this seems to be a network issue between
'flink-taskmanager-c:6121' and 'flink-taskmanager-b/<IP>2:6121'.
Otherwise, it is needed to check why the TM exits, for internal
failure(e.g. cancel timeout) or killed by other services(e.g. K8S). I'm not
familiar with K8S so I'm not sure in which case the cluster may kill the TM.
Thanks,
Zhu Zhu
Karthick Thanigaimani <ka...@yahoo.com> 于2019年8月2日周五 下午2:48写道：
> Thanks Zhu. That's correct but we don't see any errors in the log of that
> TM or the server logs or the Kube logs. Is there any bug in flink 1.4 that
> causes this or settings in flink that can avoid this.
>
>
> Regards
> Karthick
>
> n Fri, 2 Aug. 2019 at 16:04, Zhu Zhu
>
> <re...@gmail.com> wrote:
> Hi Karthick,
>
> From the log seems the TM "flink-taskmanager-b/<IP>2:6121" is lost
> unexpectedly.
> You may need to check the log of that TM to see why it exits, which should
> be the root cause.
>
> Thanks,
> Zhu Zhu
>
> Karthick Thanigaimani <ka...@yahoo.com.invalid> 于2019年8月2日周五
> 下午1:54写道：
>
> Hi Team,
> We are facing frequent issues with the Flink job manager in one
> environment when the processing happens.
> CHAIN Join(Remap EDGES id: TO) -> Map (Key Extractor) -> Combine
> (Deduplicate edges including bi-directional edges) (57/80)Timestamp:
> 2019-08-02, 4:13:25 Location: flink-taskmanager-c:6121
>
> We have tried changing the EC2 sizes to a bigger one and increased heap
> size etc but still the same problem. The below is the error message that we
> see Could someone provide some guidance.
>
> org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException:
> Lost connection to task manager 'flink-taskmanager-b/<IP>2:6121'. This
> indicates that the remote task manager was lost. at
> org.apache.flink.runtime.io.network.netty.PartitionRequestClientHandler.exceptionCaught(PartitionRequestClientHandler.java:146)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:79)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:835)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:87)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:162)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> at java.lang.Thread.run(Thread.java:748)Caused by: java.io.IOException:
> Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native
> Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at
> sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at
> sun.nio.ch.IOUtil.read(IOUtil.java:192) at
> sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) at
> org.apache.flink.shaded.netty4.io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:311)
> at
> org.apache.flink.shaded.netty4.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:881)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:241)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119)
> ... 6 more
>
>
> RegardsKarthtick
>
>
Re: Flink issue
Posted by Zhu Zhu <re...@gmail.com>.
Hi Karthick,
From the log seems the TM "flink-taskmanager-b/<IP>2:6121" is lost
unexpectedly.
You may need to check the log of that TM to see why it exits, which should
be the root cause.
Thanks,
Zhu Zhu
Karthick Thanigaimani <ka...@yahoo.com.invalid> 于2019年8月2日周五 下午1:54写道：
> Hi Team,
> We are facing frequent issues with the Flink job manager in one
> environment when the processing happens.
> CHAIN Join(Remap EDGES id: TO) -> Map (Key Extractor) -> Combine
> (Deduplicate edges including bi-directional edges) (57/80)Timestamp:
> 2019-08-02, 4:13:25 Location: flink-taskmanager-c:6121
>
> We have tried changing the EC2 sizes to a bigger one and increased heap
> size etc but still the same problem. The below is the error message that we
> see Could someone provide some guidance.
>
> org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException:
> Lost connection to task manager 'flink-taskmanager-b/<IP>2:6121'. This
> indicates that the remote task manager was lost. at
> org.apache.flink.runtime.io.network.netty.PartitionRequestClientHandler.exceptionCaught(PartitionRequestClientHandler.java:146)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:79)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:275)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:835)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:87)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:162)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> at java.lang.Thread.run(Thread.java:748)Caused by: java.io.IOException:
> Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native
> Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at
> sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at
> sun.nio.ch.IOUtil.read(IOUtil.java:192) at
> sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) at
> org.apache.flink.shaded.netty4.io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:311)
> at
> org.apache.flink.shaded.netty4.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:881)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:241)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119)
> ... 6 more
>
>
> RegardsKarthtick