You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/12/28 09:13:37 UTC
[GitHub] [spark] AngersZhuuuu opened a new pull request #30946: [SPARK-32684][SQL] Script transform hive serde mode null value keep same with hive as '\\N'
AngersZhuuuu opened a new pull request #30946:
URL: https://github.com/apache/spark/pull/30946
### What changes were proposed in this pull request?
In hive script transform serde mode, NULL format default is `\\N`
```
String nullString = tbl.getProperty(
serdeConstants.SERIALIZATION_NULL_FORMAT, "\\N");
nullSequence = new Text(nullString);
```
I make a mistake that in Spark's code we need to fix and keep same with hive too. So add some test case to show this issue.
### Why are the changes needed?
add UT
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Added UT
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-752332712
thanks, merging to master!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-752178525
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38087/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-752206080
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133498/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #30946:
URL: https://github.com/apache/spark/pull/30946
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #30946:
URL: https://github.com/apache/spark/pull/30946#discussion_r549727329
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationSuite.scala
##########
@@ -501,4 +501,28 @@ class HiveScriptTransformationSuite extends BaseScriptTransformationSuite with T
""".stripMargin)
checkAnswer(query4, identity, Row(null) :: Nil)
}
+
+ test("SPARK-32684: Script transform hive serde mode null format is same with hive as '\\N'") {
+ val query1 = sql(
+ """
+ |SELECT TRANSFORM(null, null, null)
+ |USING 'cat'
+ |FROM (SELECT 1 AS a, 2 AS b, 3 AS c) t
+ """.stripMargin)
+ checkAnswer(query1, identity, Row(null, "\\N\t\\N") :: Nil)
+
+ val query2 = sql(
+ """
+ |SELECT TRANSFORM(null, null, null)
+ | ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
+ |USING 'cat'
+ | ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
+ | WITH SERDEPROPERTIES (
+ | 'serialization.last.column.takes.rest' = 'true'
Review comment:
> Why do we need this conf in this test?
Use schemaless mode and use this value to make `value` column take all rest data and show the `\\N`, or it will just show `null`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #30946:
URL: https://github.com/apache/spark/pull/30946#discussion_r549727329
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationSuite.scala
##########
@@ -501,4 +501,28 @@ class HiveScriptTransformationSuite extends BaseScriptTransformationSuite with T
""".stripMargin)
checkAnswer(query4, identity, Row(null) :: Nil)
}
+
+ test("SPARK-32684: Script transform hive serde mode null format is same with hive as '\\N'") {
+ val query1 = sql(
+ """
+ |SELECT TRANSFORM(null, null, null)
+ |USING 'cat'
+ |FROM (SELECT 1 AS a, 2 AS b, 3 AS c) t
+ """.stripMargin)
+ checkAnswer(query1, identity, Row(null, "\\N\t\\N") :: Nil)
+
+ val query2 = sql(
+ """
+ |SELECT TRANSFORM(null, null, null)
+ | ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
+ |USING 'cat'
+ | ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
+ | WITH SERDEPROPERTIES (
+ | 'serialization.last.column.takes.rest' = 'true'
Review comment:
> Why do we need this conf in this test?
Use schemaless mode and use this value to make `value` column take all rest data and show the `\\N`, or it will just show `null`.
Need to change to other way to show `\\N`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-752206080
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133498/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30946: [SPARK-32684][SQL] Script transform hive serde mode null value keep same with hive as '\\N'
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-751687825
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38031/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-752185148
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38087/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #30946:
URL: https://github.com/apache/spark/pull/30946#discussion_r549752294
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationSuite.scala
##########
@@ -501,4 +501,28 @@ class HiveScriptTransformationSuite extends BaseScriptTransformationSuite with T
""".stripMargin)
checkAnswer(query4, identity, Row(null) :: Nil)
}
+
+ test("SPARK-32684: Script transform hive serde mode null format is same with hive as '\\N'") {
+ val query1 = sql(
+ """
+ |SELECT TRANSFORM(null, null, null)
+ |USING 'cat'
+ |FROM (SELECT 1 AS a, 2 AS b, 3 AS c) t
+ """.stripMargin)
+ checkAnswer(query1, identity, Row(null, "\\N\t\\N") :: Nil)
+
+ val query2 = sql(
+ """
+ |SELECT TRANSFORM(null, null, null)
+ | ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
+ |USING 'cat'
+ | ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
+ | WITH SERDEPROPERTIES (
+ | 'serialization.last.column.takes.rest' = 'true'
Review comment:
> Why do we need this conf in this test?
Updated, more clear now?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30946: [SPARK-32684][SQL] Script transform hive serde mode null value keep same with hive as '\\N'
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-751687825
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38031/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30946: [SPARK-32684][SQL] Script transform hive serde mode null value keep same with hive as '\\N'
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-751660016
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133441/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30946: [SPARK-32684][SQL] Script transform hive serde mode null value keep same with hive as '\\N'
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-751660016
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133441/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-752138311
**[Test build #133498 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133498/testReport)** for PR 30946 at commit [`1cd4328`](https://github.com/apache/spark/commit/1cd43286faae0e06c39f9f055eb6a92384abc493).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-752164565
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38087/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30946: [SPARK-32684][SQL] Script transform hive serde mode null value keep same with hive as '\\N'
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-751643600
FYI @maropu @cloud-fan @HyukjinKwon
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-752193257
**[Test build #133498 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133498/testReport)** for PR 30946 at commit [`1cd4328`](https://github.com/apache/spark/commit/1cd43286faae0e06c39f9f055eb6a92384abc493).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-752185148
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38087/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #30946:
URL: https://github.com/apache/spark/pull/30946#discussion_r549725571
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationSuite.scala
##########
@@ -501,4 +501,28 @@ class HiveScriptTransformationSuite extends BaseScriptTransformationSuite with T
""".stripMargin)
checkAnswer(query4, identity, Row(null) :: Nil)
}
+
+ test("SPARK-32684: Script transform hive serde mode null format is same with hive as '\\N'") {
+ val query1 = sql(
+ """
+ |SELECT TRANSFORM(null, null, null)
+ |USING 'cat'
+ |FROM (SELECT 1 AS a, 2 AS b, 3 AS c) t
+ """.stripMargin)
+ checkAnswer(query1, identity, Row(null, "\\N\t\\N") :: Nil)
+
+ val query2 = sql(
+ """
+ |SELECT TRANSFORM(null, null, null)
+ | ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
+ |USING 'cat'
+ | ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
+ | WITH SERDEPROPERTIES (
+ | 'serialization.last.column.takes.rest' = 'true'
Review comment:
Why do we need this conf in this test?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30946: [SPARK-32684][SQL][TESTS] Add a test case to check if null value is same as Hive's '\\N' in script transformation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-752138311
**[Test build #133498 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133498/testReport)** for PR 30946 at commit [`1cd4328`](https://github.com/apache/spark/commit/1cd43286faae0e06c39f9f055eb6a92384abc493).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30946: [SPARK-32684][SQL] Script transform hive serde mode null value keep same with hive as '\\N'
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30946:
URL: https://github.com/apache/spark/pull/30946#issuecomment-751645324
**[Test build #133441 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133441/testReport)** for PR 30946 at commit [`983a3e2`](https://github.com/apache/spark/commit/983a3e2d4dbb358c480fcf86b3a726291a8ce4e1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org