You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by rxin <gi...@git.apache.org> on 2015/01/18 08:19:54 UTC
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
GitHub user rxin opened a pull request:
https://github.com/apache/spark/pull/4093
[SPARK-5307] SerializationDebugger to help debug NotSerializableException
This patch adds a SerializationDebugger that is used to add more information to a NotSerializableException. When a NotSerializableException is encountered, the debugger tries to serialize the object one more time through a DebugStream that hooks into the internals of ObjectOutputStream to get the serialization stack. This ensures that there is no performance loss to run with SerializationDebugger, unlike setting the sun.io.serialization.extendedDebugInfo flag.
An example output looks like this:
```
org.apache.spark.serializer.NotSerializableClass
Serialization stack (3):
- org.apache.spark.serializer.NotSerializableClass@5e20dc10 (class org.apache.spark.serializer.NotSerializableClass)
- org.apache.spark.serializer.SerializableClass2@521fb14e (class org.apache.spark.serializer.SerializableClass2)
- org.apache.spark.serializer.SerializableClass1@5f54e92c (class org.apache.spark.serializer.SerializableClass1)
Run the JVM with sun.io.serialization.extendedDebugInfo for more information.
```
When sun.io.serialization.extendedDebugInfo is turned on, this debugger no longer adds more information. Note that sun.io.serialization.extendedDebugInfo can show also the field name information, which is harder to get by the SerializationDebugger (technically possible with reflection but fairly convoluted).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rxin/spark serialization
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/4093.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4093
----
commit f7e6320e23c12fb24a2d9b26c772d0f3231f0ec7
Author: Reynold Xin <rx...@databricks.com>
Date: 2015-01-18T07:14:56Z
[SPARK-5307] SerializationDebugger to help debug NotSerializableException.
This patch adds a SerializationDebugger that is used to add more information to
a NotSerializableException. When a NotSerializableException is encountered, the
debugger tries to serialize the object one more time through a DebugStream that
hooks into the internals of ObjectOutputStream to get the serialization stack.
An example output looks like this:
org.apache.spark.serializer.NotSerializableClass
Serialization stack (3):
- org.apache.spark.serializer.NotSerializableClass@5e20dc10 (class org.apache.spark.serializer.NotSerializableClass)
- org.apache.spark.serializer.SerializableClass2@521fb14e (class org.apache.spark.serializer.SerializableClass2)
- org.apache.spark.serializer.SerializableClass1@5f54e92c (class org.apache.spark.serializer.SerializableClass1)
Run the JVM with sun.io.serialization.extendedDebugInfo for more information.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4093#issuecomment-70400537
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25716/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4093#issuecomment-70401521
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25715/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4093#issuecomment-70402071
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25719/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/4093#issuecomment-71757676
Closing this one in favor of https://github.com/apache/spark/pull/4098
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
Posted by rxin <gi...@git.apache.org>.
Github user rxin closed the pull request at:
https://github.com/apache/spark/pull/4093
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4093#issuecomment-70402066
[Test build #25719 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25719/consoleFull) for PR 4093 at commit [`beed86d`](https://github.com/apache/spark/commit/beed86d72ddcd98ac775820c5fc545b7d5b2cfae).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* ` out.stack.map(o => s" - $o (class $`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4093#issuecomment-70400553
[Test build #25719 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25719/consoleFull) for PR 4093 at commit [`beed86d`](https://github.com/apache/spark/commit/beed86d72ddcd98ac775820c5fc545b7d5b2cfae).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4093#issuecomment-70401519
**[Test build #25715 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25715/consoleFull)** for PR 4093 at commit [`f7e6320`](https://github.com/apache/spark/commit/f7e6320e23c12fb24a2d9b26c772d0f3231f0ec7) after a configured wait of `120m`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4093#issuecomment-70399174
[Test build #25716 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25716/consoleFull) for PR 4093 at commit [`bde6512`](https://github.com/apache/spark/commit/bde6512a55765a48ca74f321068f9ab91516edae).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4093#issuecomment-70400534
[Test build #25716 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25716/consoleFull) for PR 4093 at commit [`bde6512`](https://github.com/apache/spark/commit/bde6512a55765a48ca74f321068f9ab91516edae).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* ` out.stack.map(o => s" - $o (class $`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/4093#issuecomment-70401056
This LGTM.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4093#discussion_r23131447
--- Diff: core/src/main/scala/org/apache/spark/serializer/SerializationDebugger.scala ---
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.serializer
+
+import java.io._
+import java.lang.reflect.Field
+import java.security.AccessController
+
+import scala.collection.mutable
+import scala.util.control.NonFatal
+
+
+private[serializer]
+object SerializationDebugger {
+
+ /**
+ * Write an object to the [[ObjectOutputStream]]. If a NotSerializableException is encountered,
+ * use our debug stream to capture the serialization stack leading to the problematic object.
+ */
+ def writeObject(out: ObjectOutputStream, obj: Any): Unit = {
+ try {
+ out.writeObject(obj)
+ } catch {
+ case e: NotSerializableException =>
+ if (enableDebugging) throw improveException(obj, e) else throw e
+ }
+ }
+
+ /**
+ * Improve the given NotSerializableException with the serialization stack leading from the given
+ * object to the problematic object.
+ */
+ private def improveException(obj: Any, e: NotSerializableException): NotSerializableException = {
+ if (depthField != null) {
+ val out = new DebugStream(new ByteArrayOutputStream)
+ try {
+ out.writeObject(obj)
+ e
+ } catch {
+ case nse: NotSerializableException =>
+ new NotSerializableException(
+ nse.getMessage + "\n" +
+ s"\tSerialization stack (${out.stack.size}):\n" +
+ out.stack.map(o => s"\t- $o (class ${o.getClass.getName})").mkString("\n") + "\n" +
+ "\tRun the JVM with sun.io.serialization.extendedDebugInfo for more information.")
--- End diff --
It is actually "-Dsun.io.serialization.extendedDebugInfo=true". Kinda long ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/4093#discussion_r23131429
--- Diff: core/src/main/scala/org/apache/spark/serializer/SerializationDebugger.scala ---
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.serializer
+
+import java.io._
+import java.lang.reflect.Field
+import java.security.AccessController
+
+import scala.collection.mutable
+import scala.util.control.NonFatal
+
+
+private[serializer]
+object SerializationDebugger {
+
+ /**
+ * Write an object to the [[ObjectOutputStream]]. If a NotSerializableException is encountered,
+ * use our debug stream to capture the serialization stack leading to the problematic object.
+ */
+ def writeObject(out: ObjectOutputStream, obj: Any): Unit = {
+ try {
+ out.writeObject(obj)
+ } catch {
+ case e: NotSerializableException =>
+ if (enableDebugging) throw improveException(obj, e) else throw e
+ }
+ }
+
+ /**
+ * Improve the given NotSerializableException with the serialization stack leading from the given
+ * object to the problematic object.
+ */
+ private def improveException(obj: Any, e: NotSerializableException): NotSerializableException = {
+ if (depthField != null) {
+ val out = new DebugStream(new ByteArrayOutputStream)
+ try {
+ out.writeObject(obj)
+ e
+ } catch {
+ case nse: NotSerializableException =>
+ new NotSerializableException(
+ nse.getMessage + "\n" +
+ s"\tSerialization stack (${out.stack.size}):\n" +
+ out.stack.map(o => s"\t- $o (class ${o.getClass.getName})").mkString("\n") + "\n" +
+ "\tRun the JVM with sun.io.serialization.extendedDebugInfo for more information.")
--- End diff --
Maybe `-Dsun.io.serialization.extendedDebugInfo`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4093#issuecomment-70399089
[Test build #25715 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25715/consoleFull) for PR 4093 at commit [`f7e6320`](https://github.com/apache/spark/commit/f7e6320e23c12fb24a2d9b26c772d0f3231f0ec7).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org