You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by MaxGekk <gi...@git.apache.org> on 2018/04/19 20:17:14 UTC
[GitHub] spark pull request #21108: [SPARK-24027][SQL] Support MapType with StringTyp...
GitHub user MaxGekk opened a pull request:
https://github.com/apache/spark/pull/21108
[SPARK-24027][SQL] Support MapType with StringType for keys as the root type by from_json
## What changes were proposed in this pull request?
Currently, the from_json function support StructType or ArrayType as the root type. The PR allows to specify MapType(StringType, DataType) as the root type additionally to mentioned types. For example:
```scala
import org.apache.spark.sql.types._
val schema = MapType(StringType, IntegerType)
val in = Seq("""{"a": 1, "b": 2, "c": 3}""").toDS()
in.select(from_json($"value", schema, Map[String, String]())).collect()
```
```
res1: Array[org.apache.spark.sql.Row] = Array([Map(a -> 1, b -> 2, c -> 3)])
```
## How was this patch tested?
It was checked by new tests for the map type with integer type and struct type as value types. Also roundtrip tests like from_json(to_json) and to_json(from_json) for MapType are added.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/MaxGekk/spark-1 from_json-map-type
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21108.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21108
----
commit 48a7f1e40b5942dd9f04d520d07ff8ef7a3c1def
Author: Maxim Gekk <ma...@...>
Date: 2018-04-17T10:31:05Z
Support MapType as root type for from_json
commit 82b79d5fb7a50ec06d160d9263c0ee2778881d98
Author: Maxim Gekk <ma...@...>
Date: 2018-04-18T21:16:54Z
Making simple test work
commit 0f715ef86f3cdfb313a0f3c469b5e669298eead2
Author: Maxim Gekk <ma...@...>
Date: 2018-04-19T14:59:20Z
Additional tests for map type as root type of from_json
commit 51fda7b7180ff851385ac3f3b27f1c28a2d224b0
Author: Maxim Gekk <ma...@...>
Date: 2018-04-19T15:13:56Z
Roundtrip tests are added
commit fcf7ba36d156d337fd28cf1f838c6133cd0fc5df
Author: Maxim Gekk <ma...@...>
Date: 2018-04-19T17:42:42Z
Updating from_json comments
commit b75cca826d124e783216c9d1cd0a79e2bd9700fd
Author: Maxim Gekk <ma...@...>
Date: 2018-04-19T18:15:59Z
Set schema in json format
commit 54ee6ba9b88b7af1915f56f3ec2581b75fe6f811
Author: Maxim Gekk <ma...@...>
Date: 2018-04-19T19:51:16Z
Update docs for PySpark
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89627 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89627/testReport)** for PR 21108 at commit [`619f6b1`](https://github.com/apache/spark/commit/619f6b1dfa809716741e54853c5dddb3e413e22b).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89613/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89594/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89652/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89638 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89638/testReport)** for PR 21108 at commit [`10370cb`](https://github.com/apache/spark/commit/10370cb47c2f11e1b24d4132bfb4aab4def80572).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90013/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #90600 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90600/testReport)** for PR 21108 at commit [`768ef5e`](https://github.com/apache/spark/commit/768ef5ee46973d0f578437e489fc9bc622d77831).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #90476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90476/testReport)** for PR 21108 at commit [`768ef5e`](https://github.com/apache/spark/commit/768ef5ee46973d0f578437e489fc9bc622d77831).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90214/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #90600 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90600/testReport)** for PR 21108 at commit [`768ef5e`](https://github.com/apache/spark/commit/768ef5ee46973d0f578437e489fc9bc622d77831).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #90227 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90227/testReport)** for PR 21108 at commit [`586eb58`](https://github.com/apache/spark/commit/586eb5889fdc293ce9869be94a5182743a2652e1).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89608/testReport)** for PR 21108 at commit [`2e2e6e6`](https://github.com/apache/spark/commit/2e2e6e614e8258d075ee2541c76df915f3658dc3).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89614 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89614/testReport)** for PR 21108 at commit [`2e2e6e6`](https://github.com/apache/spark/commit/2e2e6e614e8258d075ee2541c76df915f3658dc3).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90476/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89594 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89594/testReport)** for PR 21108 at commit [`54ee6ba`](https://github.com/apache/spark/commit/54ee6ba9b88b7af1915f56f3ec2581b75fe6f811).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89608/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89591 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89591/testReport)** for PR 21108 at commit [`54ee6ba`](https://github.com/apache/spark/commit/54ee6ba9b88b7af1915f56f3ec2581b75fe6f811).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89652 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89652/testReport)** for PR 21108 at commit [`7bfe231`](https://github.com/apache/spark/commit/7bfe23180d9dd4c2717d65f862b8a6fc13f7b22a).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90600/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89617/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by MaxGekk <gi...@git.apache.org>.
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21108
jenkins, retest this, please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90227/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90048/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by MaxGekk <gi...@git.apache.org>.
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21108
@gatorsmile Please, review the PR
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89591/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89608 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89608/testReport)** for PR 21108 at commit [`2e2e6e6`](https://github.com/apache/spark/commit/2e2e6e614e8258d075ee2541c76df915f3658dc3).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89614 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89614/testReport)** for PR 21108 at commit [`2e2e6e6`](https://github.com/apache/spark/commit/2e2e6e614e8258d075ee2541c76df915f3658dc3).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #90048 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90048/testReport)** for PR 21108 at commit [`840a3a1`](https://github.com/apache/spark/commit/840a3a14890cb9a03efb806d7f07025618abc604).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89638/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by MaxGekk <gi...@git.apache.org>.
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21108
jenkins, retest this, please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21108
add to whitelist
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89613 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89613/testReport)** for PR 21108 at commit [`2e2e6e6`](https://github.com/apache/spark/commit/2e2e6e614e8258d075ee2541c76df915f3658dc3).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #90214 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90214/testReport)** for PR 21108 at commit [`840a3a1`](https://github.com/apache/spark/commit/840a3a14890cb9a03efb806d7f07025618abc604).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21108: [SPARK-24027][SQL] Support MapType with StringTyp...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21108#discussion_r187383654
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala ---
@@ -326,4 +326,61 @@ class JsonFunctionsSuite extends QueryTest with SharedSQLContext {
assert(errMsg4.getMessage.startsWith(
"A type of keys and values in map() must be string, but got"))
}
+
+ test("SPARK-24027: from_json - map<string, int>") {
+ val in = Seq("""{"a": 1, "b": 2, "c": 3}""").toDS()
+ val schema =
+ """
+ |{
+ | "type" : "map",
+ | "keyType" : "string",
+ | "valueType" : "integer",
+ | "valueContainsNull" : true
+ |}
+ """.stripMargin
+ val out = in.select(from_json($"value", schema, Map[String, String]()))
+
+ assert(out.columns.head == "entries")
+ checkAnswer(out, Row(Map("a" -> 1, "b" -> 2, "c" -> 3)))
+ }
+
+ test("SPARK-24027: from_json - map<string, struct>") {
--- End diff --
Any negative test case? For example, when the key of MapType is not StringType?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21108: [SPARK-24027][SQL] Support MapType with StringTyp...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21108
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by MaxGekk <gi...@git.apache.org>.
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21108
@marmbrus May I ask you to look at the PR.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #90013 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90013/testReport)** for PR 21108 at commit [`840a3a1`](https://github.com/apache/spark/commit/840a3a14890cb9a03efb806d7f07025618abc604).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21108
retest this please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #90048 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90048/testReport)** for PR 21108 at commit [`840a3a1`](https://github.com/apache/spark/commit/840a3a14890cb9a03efb806d7f07025618abc604).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #90214 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90214/testReport)** for PR 21108 at commit [`840a3a1`](https://github.com/apache/spark/commit/840a3a14890cb9a03efb806d7f07025618abc604).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #90227 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90227/testReport)** for PR 21108 at commit [`586eb58`](https://github.com/apache/spark/commit/586eb5889fdc293ce9869be94a5182743a2652e1).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by MaxGekk <gi...@git.apache.org>.
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21108
jenkins, retest this, please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #90013 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90013/testReport)** for PR 21108 at commit [`840a3a1`](https://github.com/apache/spark/commit/840a3a14890cb9a03efb806d7f07025618abc604).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21108: [SPARK-24027][SQL] Support MapType with StringTyp...
Posted by MaxGekk <gi...@git.apache.org>.
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21108#discussion_r187888913
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala ---
@@ -326,4 +326,61 @@ class JsonFunctionsSuite extends QueryTest with SharedSQLContext {
assert(errMsg4.getMessage.startsWith(
"A type of keys and values in map() must be string, but got"))
}
+
+ test("SPARK-24027: from_json - map<string, int>") {
+ val in = Seq("""{"a": 1, "b": 2, "c": 3}""").toDS()
+ val schema =
+ """
+ |{
+ | "type" : "map",
+ | "keyType" : "string",
+ | "valueType" : "integer",
+ | "valueContainsNull" : true
+ |}
+ """.stripMargin
+ val out = in.select(from_json($"value", schema, Map[String, String]()))
+
+ assert(out.columns.head == "entries")
+ checkAnswer(out, Row(Map("a" -> 1, "b" -> 2, "c" -> 3)))
+ }
+
+ test("SPARK-24027: from_json - map<string, struct>") {
--- End diff --
I added a negative test. Please, take a look at the PR again.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89627/testReport)** for PR 21108 at commit [`619f6b1`](https://github.com/apache/spark/commit/619f6b1dfa809716741e54853c5dddb3e413e22b).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by MaxGekk <gi...@git.apache.org>.
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21108
@gatorsmile Kindly ask you to look at it again.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89591 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89591/testReport)** for PR 21108 at commit [`54ee6ba`](https://github.com/apache/spark/commit/54ee6ba9b88b7af1915f56f3ec2581b75fe6f811).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89613 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89613/testReport)** for PR 21108 at commit [`2e2e6e6`](https://github.com/apache/spark/commit/2e2e6e614e8258d075ee2541c76df915f3658dc3).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89627/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #90476 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90476/testReport)** for PR 21108 at commit [`768ef5e`](https://github.com/apache/spark/commit/768ef5ee46973d0f578437e489fc9bc622d77831).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89638 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89638/testReport)** for PR 21108 at commit [`10370cb`](https://github.com/apache/spark/commit/10370cb47c2f11e1b24d4132bfb4aab4def80572).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89594 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89594/testReport)** for PR 21108 at commit [`54ee6ba`](https://github.com/apache/spark/commit/54ee6ba9b88b7af1915f56f3ec2581b75fe6f811).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by MaxGekk <gi...@git.apache.org>.
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21108
jenkins, retest this, please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89617 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89617/testReport)** for PR 21108 at commit [`b26d4ba`](https://github.com/apache/spark/commit/b26d4baa48d816f7c10d998cb3db0c556627637c).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21108
Thanks! Merged to master.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89614/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89652 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89652/testReport)** for PR 21108 at commit [`7bfe231`](https://github.com/apache/spark/commit/7bfe23180d9dd4c2717d65f862b8a6fc13f7b22a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21108
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21108: [SPARK-24027][SQL] Support MapType with StringType for k...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21108
**[Test build #89617 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89617/testReport)** for PR 21108 at commit [`b26d4ba`](https://github.com/apache/spark/commit/b26d4baa48d816f7c10d998cb3db0c556627637c).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org