You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by moranmathias <gi...@git.apache.org> on 2015/04/24 17:10:10 UTC
[GitHub] spark pull request: Upgrade the json4s version
GitHub user moranmathias opened a pull request:
https://github.com/apache/spark/pull/5691
Upgrade the json4s version
The new version of json4s adds support to parse Map[String,Any] out the box. This is a very useful feature that i've been using in my custom version of spark.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/moranmathias/spark patch-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5691.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5691
----
commit 1e79f5e4621395d524fab2260478cd14c7bde22c
Author: Matias Morán Losada <mo...@gmail.com>
Date: 2015-04-24T15:06:54Z
Upgrade the jon4s version
The new version of json4s adds support to parse Map[String,Any] out the box. This is a very useful feature that i've been using in my custom version of spark.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5691#issuecomment-96001588
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30939/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5691#issuecomment-95961831
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5691#issuecomment-95965005
Probably OK. Often we want a JIRA too to discuss the change but this is so small. Are you aware of any incompatibilities? I doubt it in a maintenance release, but worth researching in the commit log briefly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5691#issuecomment-96001541
[Test build #30939 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30939/consoleFull) for PR 5691 at commit [`1e79f5e`](https://github.com/apache/spark/commit/1e79f5e4621395d524fab2260478cd14c7bde22c).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
* This patch **adds the following new dependencies:**
* `json4s-ast_2.10-3.2.11.jar`
* `json4s-core_2.10-3.2.11.jar`
* `json4s-jackson_2.10-3.2.11.jar`
* This patch **removes the following dependencies:**
* `json4s-ast_2.10-3.2.10.jar`
* `json4s-core_2.10-3.2.10.jar`
* `json4s-jackson_2.10-3.2.10.jar`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5691#issuecomment-96775263
@moranmathias any updates? I think we just need some confidence that this is compatible to the best of our knowledge.
Do you need this in your code only? you shouldn't be depending on Spark for json4s, note. If so then mind closing this PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by wb14123 <gi...@git.apache.org>.
Github user wb14123 commented on the pull request:
https://github.com/apache/spark/pull/5691#issuecomment-141085138
@srowen Sorry, I don't understand what do you mean. The discussion here seems that the developer don't follow up. But the issue is still here and I think it is important. Do I need to create a JIRA for this?
You said we shouldn't be depending on Spark for json4s. But I'm about to use json4s 3.2.11 in my spark application, I also write it as a dependency in my project, but while I submit the application as a JAR with spark-submit, the spark's json4s just override my own version of json4s.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5691#issuecomment-141053615
@wb14123 I don't think the outstanding issues were resolved? you can see the discussion here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by wb14123 <gi...@git.apache.org>.
Github user wb14123 commented on the pull request:
https://github.com/apache/spark/pull/5691#issuecomment-141092917
"userClassPathFirst" is marked "Experimental" in the doc for now. And it will use other library versions in my project like akka-slf4j which is incompatible with other akka libraries in Spark, and so on.
But maybe I should cleanup my project's dependency to find out which versions do I really need. Thanks for the explain.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5691#issuecomment-96766623
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5691#issuecomment-95965408
[Test build #30939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30939/consoleFull) for PR 5691 at commit [`1e79f5e`](https://github.com/apache/spark/commit/1e79f5e4621395d524fab2260478cd14c7bde22c).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5691#issuecomment-95965013
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/5691
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/5691#issuecomment-95990923
@moranmathias Do you mind creating a JIRA for this? It's good to have an audit trail of any version upgrades, since sometimes they cause maintenance issues downstream
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by wb14123 <gi...@git.apache.org>.
Github user wb14123 commented on the pull request:
https://github.com/apache/spark/pull/5691#issuecomment-141026749
Any update on this? I think this is a very useful feature. I searched on the web and there are many work around for this. It is good to see this to be merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Upgrade the json4s version
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5691#issuecomment-141089285
@wb14123 This issue concerns updating Spark's json4s. If you want to pursue that, you can open a PR, but the question will be the same: does it cause any compatibility problem?
However your purpose is simply to use json4s in your app. It's not shaded in Spark. You need to use the 'userClassPathFirst' options in Spark to make your copy take precedence. This does not involve updating Spark.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org