[jira] [Created] (SPARK-35237) In k8s, during running spark job, IllegalArgumentException(too large frame) is raised on spark driver

["Keunhyun Oh (Jira)" <ji...@apache.org>]

Keunhyun Oh created SPARK-35237:
-----------------------------------

             Summary: In k8s, during running spark job, IllegalArgumentException(too large frame) is raised on spark driver
                 Key: SPARK-35237
                 URL: https://issues.apache.org/jira/browse/SPARK-35237
             Project: Spark
          Issue Type: Question
          Components: Kubernetes
    Affects Versions: 3.0.2
            Reporter: Keunhyun Oh


In k8s, during running spark job, IllegalArgumentException(too large frame) is raised on spark driver like that
{code:java}
2021-04-26 13:59:26,961 WARN server.TransportChannelHandler: Exception in connection from /${Prometheus pod Ip addr}:51346
java.lang.IllegalArgumentException: Too large frame: 5135603447297303916
at org.sparkproject.guava.base.Preconditions.checkArgument(Preconditions.java:119)
at org.apache.spark.network.util.TransportFrameDecoder.decodeNext(TransportFrameDecoder.java:148)
at org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:98)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Unknown Source) {code}
 

${Prometheus pod Ip addr} means prometheus pod's cluster ip in k8s.

How to resolve the warning?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org