You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2023/05/22 20:49:00 UTC
[jira] [Updated] (SPARK-43583) When encryption is enabled on the External Shuffle Service, then processing of push meta requests throws NPE
[ https://issues.apache.org/jira/browse/SPARK-43583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-43583:
----------------------------------
Affects Version/s: 3.4.0
3.3.2
3.2.4
(was: 3.2.0)
> When encryption is enabled on the External Shuffle Service, then processing of push meta requests throws NPE
> ------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-43583
> URL: https://issues.apache.org/jira/browse/SPARK-43583
> Project: Spark
> Issue Type: Improvement
> Components: Shuffle
> Affects Versions: 3.2.4, 3.3.2, 3.4.0
> Reporter: Chandni Singh
> Assignee: Chandni Singh
> Priority: Major
> Fix For: 3.5.0
>
>
> After enabling support for over-the-wire encryption for spark shuffle services, the meta requests for push-merged blocks fail with this error:
> {code:java}
> java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.spark.network.server.AbstractAuthRpcHandler.getMergedBlockMetaReqHandler(AbstractAuthRpcHandler.java:110)
> at org.apache.spark.network.crypto.AuthRpcHandler.getMergedBlockMetaReqHandler(AuthRpcHandler.java:144)
> at org.apache.spark.network.server.TransportRequestHandler.processMergedBlockMetaRequest(TransportRequestHandler.java:275)
> at org.apache.spark.network.server.TransportRequestHandler.handle(TransportRequestHandler.java:117)
> at org.apache.spark.network.server.TransportChannelHandler.channelRead0(TransportChannelHandler.java:140)
> at org.apache.spark.network.server.TransportChannelHandler.channelRead0(TransportChannelHandler.java:53)
> at org.sparkproject.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
> at org.sparkproject.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> at org.sparkproject.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
> at org.sparkproject.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
> at org.sparkproject.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org