You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by rxin <gi...@git.apache.org> on 2014/12/24 20:25:11 UTC
[GitHub] spark pull request: Added Java serialization util functions back i...
GitHub user rxin opened a pull request:
https://github.com/apache/spark/pull/3792
Added Java serialization util functions back in network/common/util/JavaUtils
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rxin/spark java-ser
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3792.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3792
----
commit 2a2ad9d6fcb98bbb7ffca1c0a5273f4ff8cb53a6
Author: Reynold Xin <rx...@databricks.com>
Date: 2014-12-24T19:24:31Z
Added Java serialization util functions back in network/common/util/JavaUtils.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Added Java serialization util functions back i...
Posted by rxin <gi...@git.apache.org>.
Github user rxin closed the pull request at:
https://github.com/apache/spark/pull/3792
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Added Java serialization util functions back i...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3792#issuecomment-68073817
[Test build #24786 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24786/consoleFull) for PR 3792 at commit [`2a2ad9d`](https://github.com/apache/spark/commit/2a2ad9d6fcb98bbb7ffca1c0a5273f4ff8cb53a6).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Added Java serialization util functions back i...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/3792#issuecomment-68070712
cc @aarondav
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Added Java serialization util functions back i...
Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/3792#issuecomment-75497469
@rxin @aarondav Is this PR still relevant?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Added Java serialization util functions back i...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3792#issuecomment-68073819
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24786/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Added Java serialization util functions back i...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3792#discussion_r22305806
--- Diff: network/common/src/main/java/org/apache/spark/network/util/JavaUtils.java ---
@@ -41,6 +41,34 @@
public class JavaUtils {
private static final Logger logger = LoggerFactory.getLogger(JavaUtils.class);
+ /** Deserialize a byte array using Java serialization. */
+ public static <T> T deserialize(byte[] bytes) {
+ try {
+ ObjectInputStream is = new ObjectInputStream(new ByteArrayInputStream(bytes));
+ Object out = is.readObject();
+ is.close();
+ return (T) out;
+ } catch (ClassNotFoundException e) {
+ throw new RuntimeException("Could not deserialize object", e);
--- End diff --
I was thinking that you don't expect to not have the class on hand... But sure, IllegalArgumentException because the bytes describe something invalid? The principle is to avoid RuntimeException since it is the superclass of all unchecked exceptions. If you ever wanted to catch this exception to deal with it you'd have no hope of distinguishing with a catch block. So reach for another standard and slightly more specific exception. Marginal argument here, but I think still common good practice in Java.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Added Java serialization util functions back i...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/3792#issuecomment-87194326
Probably not -- but @zsxwing might want to pull this in the future.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Added Java serialization util functions back i...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3792#issuecomment-68070795
[Test build #24786 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24786/consoleFull) for PR 3792 at commit [`2a2ad9d`](https://github.com/apache/spark/commit/2a2ad9d6fcb98bbb7ffca1c0a5273f4ff8cb53a6).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Added Java serialization util functions back i...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3792#discussion_r22262724
--- Diff: network/common/src/main/java/org/apache/spark/network/util/JavaUtils.java ---
@@ -41,6 +41,34 @@
public class JavaUtils {
private static final Logger logger = LoggerFactory.getLogger(JavaUtils.class);
+ /** Deserialize a byte array using Java serialization. */
+ public static <T> T deserialize(byte[] bytes) {
+ try {
+ ObjectInputStream is = new ObjectInputStream(new ByteArrayInputStream(bytes));
+ Object out = is.readObject();
+ is.close();
+ return (T) out;
+ } catch (ClassNotFoundException e) {
+ throw new RuntimeException("Could not deserialize object", e);
--- End diff --
Yeah pretty standard formulation. Nit suggestion: don't throw general `RuntimeException` but something marginally more specific like `IllegalStateException`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Added Java serialization util functions back i...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/3792#discussion_r22304444
--- Diff: network/common/src/main/java/org/apache/spark/network/util/JavaUtils.java ---
@@ -41,6 +41,34 @@
public class JavaUtils {
private static final Logger logger = LoggerFactory.getLogger(JavaUtils.class);
+ /** Deserialize a byte array using Java serialization. */
+ public static <T> T deserialize(byte[] bytes) {
+ try {
+ ObjectInputStream is = new ObjectInputStream(new ByteArrayInputStream(bytes));
+ Object out = is.readObject();
+ is.close();
+ return (T) out;
+ } catch (ClassNotFoundException e) {
+ throw new RuntimeException("Could not deserialize object", e);
--- End diff --
IllegalStateException doesn't seem to be an accurate description of this, does it? Of course, you can technically label everything as an illegal state ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org