You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2017/09/20 08:13:01 UTC
[jira] [Commented] (SPARK-22077) RpcEndpointAddress fails to parse
spark URL if it is an ipv6 address.
[ https://issues.apache.org/jira/browse/SPARK-22077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16172889#comment-16172889 ]
Sean Owen commented on SPARK-22077:
-----------------------------------
I'm not sure whether it's that IPv6 doesn't really work here in general, or just the parsing is overly aggressive. You can see the code that parses the URI, which parses successfully, but then it's not happy that it lacks a host or port or name or something.
> RpcEndpointAddress fails to parse spark URL if it is an ipv6 address.
> ---------------------------------------------------------------------
>
> Key: SPARK-22077
> URL: https://issues.apache.org/jira/browse/SPARK-22077
> Project: Spark
> Issue Type: Bug
> Components: Deploy
> Affects Versions: 2.0.0
> Reporter: Eric Vandenberg
> Priority: Minor
>
> RpcEndpointAddress fails to parse spark URL if it is an ipv6 address.
> For example,
> sparkUrl = "spark://HeartbeatReceiver@2401:db00:2111:40a1:face:0:21:0:35243"
> is parsed as:
> host = null
> port = -1
> name = null
> While sparkUrl = spark://HeartbeatReceiver@localhost:55691 is parsed properly.
> This is happening on our production machines and causing spark to not start up.
> org.apache.spark.SparkException: Invalid Spark URL: spark://HeartbeatReceiver@2401:db00:2111:40a1:face:0:21:0:35243
> at org.apache.spark.rpc.RpcEndpointAddress$.apply(RpcEndpointAddress.scala:65)
> at org.apache.spark.rpc.netty.NettyRpcEnv.asyncSetupEndpointRefByURI(NettyRpcEnv.scala:133)
> at org.apache.spark.rpc.RpcEnv.setupEndpointRefByURI(RpcEnv.scala:88)
> at org.apache.spark.rpc.RpcEnv.setupEndpointRef(RpcEnv.scala:96)
> at org.apache.spark.util.RpcUtils$.makeDriverRef(RpcUtils.scala:32)
> at org.apache.spark.executor.Executor.<init>(Executor.scala:121)
> at org.apache.spark.scheduler.local.LocalEndpoint.<init>(LocalSchedulerBackend.scala:59)
> at org.apache.spark.scheduler.local.LocalSchedulerBackend.start(LocalSchedulerBackend.scala:126)
> at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:173)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:507)
> at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2283)
> at org.apache.spark.sql.SparkSession$Builder$$anonfun$8.apply(SparkSession.scala:833)
> at org.apache.spark.sql.SparkSession$Builder$$anonfun$8.apply(SparkSession.scala:825)
> at scala.Option.getOrElse(Option.scala:121)
> at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:825)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org