You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by mn-mikke <gi...@git.apache.org> on 2018/04/10 21:00:08 UTC
[GitHub] spark pull request #21034: [SPARK-23926][SQL] Extending reverse function to ...
GitHub user mn-mikke opened a pull request:
https://github.com/apache/spark/pull/21034
[SPARK-23926][SQL] Extending reverse function to support ArrayType arguments
## What changes were proposed in this pull request?
This PR extends reverse functions to be able to operate over array columns and covers:
- Introduction of Reverse expression that represents logic for reversing arrays and also strings
- Removal of StringReverse expression
- A wrapper for PySpark
## How was this patch tested?
New tests added into:
- CollectionExpressionsSuite
- DataFrameFunctionsSuite
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/AbsaOSS/spark feature/array-api-reverse-to-master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21034.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21034
----
commit 3a76d87492a8d03a29aeee877f5564071b0a68a8
Author: mn-mikke <mrkaha12346github>
Date: 2018-04-09T14:12:16Z
[SPARK-23926][SQL] Extending reverse function to support ArrayType arguments.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89209/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89396 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89396/testReport)** for PR 21034 at commit [`460fea6`](https://github.com/apache/spark/commit/460fea632ce23946edebda58cafd7bdf00d80955).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89209 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89209/testReport)** for PR 21034 at commit [`28ed664`](https://github.com/apache/spark/commit/28ed66493fb795228d4c2b2cc445d77f6d95d161).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89219 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89219/testReport)** for PR 21034 at commit [`638a0ff`](https://github.com/apache/spark/commit/638a0ff010764ad9bdc9c85334ffb5f083400e08).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89183 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89183/testReport)** for PR 21034 at commit [`3a76d87`](https://github.com/apache/spark/commit/3a76d87492a8d03a29aeee877f5564071b0a68a8).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class Reverse(child: Expression) extends UnaryExpression with ImplicitCastInputTypes `
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89175/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89301/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89175 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89175/testReport)** for PR 21034 at commit [`3a76d87`](https://github.com/apache/spark/commit/3a76d87492a8d03a29aeee877f5564071b0a68a8).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21034: [SPARK-23926][SQL] Extending reverse function to ...
Posted by mn-mikke <gi...@git.apache.org>.
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21034#discussion_r180686345
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala ---
@@ -212,6 +213,96 @@ case class SortArray(base: Expression, ascendingOrder: Expression)
override def prettyName: String = "sort_array"
}
+/**
+ * Returns a reversed string or an array with reverse order of elements.
+ */
+@ExpressionDescription(
+ usage = "_FUNC_(array) - Returns a reversed string or an array with reverse order of elements.",
+ examples = """
+ Examples:
+ > SELECT _FUNC_('Spark SQL');
+ LQS krapS
+ > SELECT _FUNC_(array(2, 1, 4, 3));
+ [3, 4, 1, 2]
+ """,
+ since = "2.4.0")
+case class Reverse(child: Expression) extends UnaryExpression with ImplicitCastInputTypes {
+
+ // Input types are utilized by type coercion in ImplicitTypeCasts.
+ override def inputTypes: Seq[AbstractDataType] = Seq(StringType)
+
+ val allowedTypes = Seq(StringType, ArrayType)
+
+ override def dataType: DataType = child.dataType
+
+ lazy val elementType: DataType = dataType.asInstanceOf[ArrayType].elementType
+
+ override def checkInputDataTypes(): TypeCheckResult = {
--- End diff --
Great idea!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21034
ok to test
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21034: [SPARK-23926][SQL] Extending reverse function to ...
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21034#discussion_r180850179
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala ---
@@ -212,6 +213,93 @@ case class SortArray(base: Expression, ascendingOrder: Expression)
override def prettyName: String = "sort_array"
}
+/**
+ * Returns a reversed string or an array with reverse order of elements.
+ */
+@ExpressionDescription(
+ usage = "_FUNC_(array) - Returns a reversed string or an array with reverse order of elements.",
+ examples = """
+ Examples:
+ > SELECT _FUNC_('Spark SQL');
+ LQS krapS
+ > SELECT _FUNC_(array(2, 1, 4, 3));
+ [3, 4, 1, 2]
+ """,
+ since = "1.5.0",
+ note = "Reverse logic for arrays is available since 2.4.0."
+)
+case class Reverse(child: Expression) extends UnaryExpression with ImplicitCastInputTypes {
+
+ // Input types are utilized by type coercion in ImplicitTypeCasts.
+ override def inputTypes: Seq[AbstractDataType] = Seq(TypeCollection(StringType, ArrayType))
+
+ override def dataType: DataType = child.dataType
+
+ lazy val elementType: DataType = dataType.asInstanceOf[ArrayType].elementType
+
+ override def nullSafeEval(input: Any): Any = input match {
+ case a: ArrayData => new GenericArrayData(a.toObjectArray(elementType).reverse)
+ case s: UTF8String => s.reverse()
+ }
+
+ override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
+ nullSafeCodeGen(ctx, ev, c => dataType match {
+ case _: StringType => stringCodeGen(ev, c)
+ case _: ArrayType => arrayCodeGen(ctx, ev, c)
+ })
+ }
+
+ private def stringCodeGen(ev: ExprCode, childName: String): String = {
+ s"${ev.value} = ($childName).reverse();"
+ }
+
+ private def arrayCodeGen(ctx: CodegenContext, ev: ExprCode, childName: String): String = {
+ val length = ctx.freshName("length")
+ val javaElementType = CodeGenerator.javaType(elementType)
+ val isPrimitiveType = CodeGenerator.isPrimitiveType(elementType)
+
+ val initialization = if (isPrimitiveType) {
+ s"$childName.copy()"
+ } else {
+ s"new ${classOf[GenericArrayData].getName()}(new Object[$length])"
+ }
+
+ val numberOfIterations = if (isPrimitiveType) s"$length / 2" else length
+
+ val swapAssigments = if (isPrimitiveType) {
+ val setFunc = "set" + CodeGenerator.primitiveTypeName(elementType)
+ val getCall = (index: String) => CodeGenerator.getValue(ev.value, elementType, index)
+ s"""|boolean isNullAtK = ${ev.value}.isNullAt(k);
+ |boolean isNullAtL = ${ev.value}.isNullAt(l);
+ |if(!isNullAtK) {
+ | $javaElementType el = ${getCall("k")};
+ | if(!isNullAtL) {
+ | ${ev.value}.$setFunc(k, ${getCall("l")});
+ | } else {
+ | ${ev.value}.setNullAt(k);
+ | }
+ | ${ev.value}.$setFunc(l, el);
+ |} else if (!isNullAtL) {
+ | ${ev.value}.$setFunc(k, ${getCall("l")});
+ | ${ev.value}.setNullAt(l);
+ |}""".stripMargin
+ } else {
+ s"${ev.value}.update(k, ${CodeGenerator.getValue(childName, elementType, "l")});"
+ }
+
+ s"""
+ |final int $length = $childName.numElements();
--- End diff --
nit: more spaces
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by ueshin <gi...@git.apache.org>.
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/21034
Thanks! merging to master.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by mn-mikke <gi...@git.apache.org>.
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21034
Thanks everybody for code reviews!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89396/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21034: [SPARK-23926][SQL] Extending reverse function to ...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21034#discussion_r180602624
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala ---
@@ -212,6 +213,96 @@ case class SortArray(base: Expression, ascendingOrder: Expression)
override def prettyName: String = "sort_array"
}
+/**
+ * Returns a reversed string or an array with reverse order of elements.
+ */
+@ExpressionDescription(
+ usage = "_FUNC_(array) - Returns a reversed string or an array with reverse order of elements.",
+ examples = """
+ Examples:
+ > SELECT _FUNC_('Spark SQL');
+ LQS krapS
+ > SELECT _FUNC_(array(2, 1, 4, 3));
+ [3, 4, 1, 2]
+ """,
+ since = "2.4.0")
+case class Reverse(child: Expression) extends UnaryExpression with ImplicitCastInputTypes {
+
+ // Input types are utilized by type coercion in ImplicitTypeCasts.
+ override def inputTypes: Seq[AbstractDataType] = Seq(StringType)
+
+ val allowedTypes = Seq(StringType, ArrayType)
+
+ override def dataType: DataType = child.dataType
+
+ lazy val elementType: DataType = dataType.asInstanceOf[ArrayType].elementType
+
+ override def checkInputDataTypes(): TypeCheckResult = {
--- End diff --
I think you just make the `inputTypes` as `Seq(TypeCollection(StringType, ArrayType))`. `ExpectsInputTypes` can check the types.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89219 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89219/testReport)** for PR 21034 at commit [`638a0ff`](https://github.com/apache/spark/commit/638a0ff010764ad9bdc9c85334ffb5f083400e08).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21034: [SPARK-23926][SQL] Extending reverse function to ...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21034#discussion_r180738834
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala ---
@@ -212,6 +213,96 @@ case class SortArray(base: Expression, ascendingOrder: Expression)
override def prettyName: String = "sort_array"
}
+/**
+ * Returns a reversed string or an array with reverse order of elements.
+ */
+@ExpressionDescription(
+ usage = "_FUNC_(array) - Returns a reversed string or an array with reverse order of elements.",
+ examples = """
+ Examples:
+ > SELECT _FUNC_('Spark SQL');
+ LQS krapS
+ > SELECT _FUNC_(array(2, 1, 4, 3));
+ [3, 4, 1, 2]
+ """,
+ since = "2.4.0")
--- End diff --
Hmmm. I think the function `reverse` in SQL is there from 1.5 - SPARK-8258. Can we use 1.5.0 here and add a note via `note` field? there's an example for note:
https://github.com/apache/spark/blob/2ce37b50fc01558f49ad22f89c8659f50544ffec/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala#L101-L103
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89219/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89209/testReport)** for PR 21034 at commit [`28ed664`](https://github.com/apache/spark/commit/28ed66493fb795228d4c2b2cc445d77f6d95d161).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89256/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89345 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89345/testReport)** for PR 21034 at commit [`aca511c`](https://github.com/apache/spark/commit/aca511c23da4cec7120a8876dd54510c77a7d2a6).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89345/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89183/testReport)** for PR 21034 at commit [`3a76d87`](https://github.com/apache/spark/commit/3a76d87492a8d03a29aeee877f5564071b0a68a8).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21034
retest this please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by ueshin <gi...@git.apache.org>.
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/21034
LGTM.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21034: [SPARK-23926][SQL] Extending reverse function to ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21034
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89175/testReport)** for PR 21034 at commit [`3a76d87`](https://github.com/apache/spark/commit/3a76d87492a8d03a29aeee877f5564071b0a68a8).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class Reverse(child: Expression) extends UnaryExpression with ImplicitCastInputTypes `
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89256 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89256/testReport)** for PR 21034 at commit [`30af8cd`](https://github.com/apache/spark/commit/30af8cdbfb57ee7e9b953d4f0431f5d4323343d1).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by mn-mikke <gi...@git.apache.org>.
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21034
Any other comments?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89256 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89256/testReport)** for PR 21034 at commit [`30af8cd`](https://github.com/apache/spark/commit/30af8cdbfb57ee7e9b953d4f0431f5d4323343d1).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* ` logInfo(s\"Failed to load main class $childMainClass.\")`
* ` error(s\"Cannot load main class from JAR $primaryResource\")`
* ` error(\"No main class set in JAR; please specify one with --class\")`
* ` checkArgument(mainClass != null, \"Missing example class name.\");`
* `sealed trait Node extends Serializable `
* `sealed trait ClassificationNode extends Node `
* `sealed trait RegressionNode extends Node `
* `sealed trait LeafNode extends Node `
* `sealed trait InternalNode extends Node `
* `class VectorAssembler(JavaTransformer, HasInputCols, HasOutputCol, HasHandleInvalid, JavaMLReadable,`
* `class KolmogorovSmirnovTest(object):`
* `case class ExprCode(var code: String, var isNull: ExprValue, var value: ExprValue)`
* `case class SubExprEliminationState(isNull: ExprValue, value: ExprValue)`
* ` |class SpecificUnsafeProjection extends $`
* `trait JavaCode `
* `trait ExprValue extends JavaCode `
* `case class SimpleExprValue(expr: String, javaType: Class[_]) extends ExprValue `
* `case class VariableValue(variableName: String, javaType: Class[_]) extends ExprValue `
* `case class GlobalValue(value: String, javaType: Class[_]) extends ExprValue `
* `class LiteralValue(val value: String, val javaType: Class[_]) extends ExprValue with Serializable `
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89183/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89301/testReport)** for PR 21034 at commit [`ad17d49`](https://github.com/apache/spark/commit/ad17d49b79d8d80318d8e9cdb01551975bccadac).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21034
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89396 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89396/testReport)** for PR 21034 at commit [`460fea6`](https://github.com/apache/spark/commit/460fea632ce23946edebda58cafd7bdf00d80955).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89301 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89301/testReport)** for PR 21034 at commit [`ad17d49`](https://github.com/apache/spark/commit/ad17d49b79d8d80318d8e9cdb01551975bccadac).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21034
**[Test build #89345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89345/testReport)** for PR 21034 at commit [`aca511c`](https://github.com/apache/spark/commit/aca511c23da4cec7120a8876dd54510c77a7d2a6).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org