You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Michael Gummelt (JIRA)" <ji...@apache.org> on 2016/08/25 18:38:20 UTC
[jira] [Commented] (SPARK-17240) SparkConf is Serializable but
contains a non-serializable field
[ https://issues.apache.org/jira/browse/SPARK-17240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437436#comment-15437436 ]
Michael Gummelt commented on SPARK-17240:
-----------------------------------------
cc [~vanzin]
> SparkConf is Serializable but contains a non-serializable field
> ---------------------------------------------------------------
>
> Key: SPARK-17240
> URL: https://issues.apache.org/jira/browse/SPARK-17240
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.0.1
> Reporter: Michael Gummelt
> Fix For: 2.0.1
>
>
> This commit: https://github.com/apache/spark/commit/5da6c4b24f512b63cd4e6ba7dd8968066a9396f5
> Added ConfigReader to SparkConf. SparkConf is Serializable, but ConfigReader is not, which results in the following exception:
> {code}
> java.io.NotSerializableException: org.apache.spark.internal.config.ConfigReader
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
> at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
> at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
> at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
> at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
> at org.apache.spark.util.Utils$.serialize(Utils.scala:134)
> at org.apache.spark.scheduler.cluster.mesos.ZookeeperMesosClusterPersistenceEngine.persist(MesosClusterPersistenceEngine.scala:111)
> at org.apache.spark.scheduler.cluster.mesos.MesosClusterScheduler.submitDriver(MesosClusterScheduler.scala:170)
> at org.apache.spark.deploy.rest.mesos.MesosSubmitRequestServlet.handleSubmit(MesosRestServer.scala:126)
> at org.apache.spark.deploy.rest.SubmitRequestServlet.doPost(RestSubmissionServer.scala:265)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.spark_project.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
> at org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
> at org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.spark_project.jetty.server.Server.handle(Server.java:499)
> at org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:311)
> at org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at org.spark_project.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
> at org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at org.spark_project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org