You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by yongtang <gi...@git.apache.org> on 2016/04/02 09:43:05 UTC
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
GitHub user yongtang opened a pull request:
https://github.com/apache/spark/pull/12128
[SPARK-14335][SQL] Describe function command returns wrong output
## What changes were proposed in this pull request?
…because some of built-in functions are not in function registry.
This fix tries to fix issues in `describe function` command where some of the outputs
still shows Hive's function because some built-in functions are not in FunctionRegistry.
The following built-in functions have been added to FunctionRegistry:
```
-
!
*
/
&
%
^
+
<
<=
<=>
=
==
>
>=
|
~
and
in
like
not
or
rlike
when
```
The following listed functions are not added:
```
!=
<>
between
case
```
## How was this patch tested?
Existing tests passed.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yongtang/spark SPARK-14335
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12128.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12128
----
commit 7a84e9bd7dccffce103862a7c8818396d6fecb34
Author: Yong Tang <yo...@outlook.com>
Date: 2016-04-02T07:40:07Z
[SPARK-14335][SQL] Describe function command returns wrong output because some of built-in functions are not in function registry.
This fix tries to fix issues in `describe function` command where some of the outputs
still shows Hive's function because some built-in functions are not in FunctionRegistry.
The following built-in functions have been added to FunctionRegistry:
```
-
!
*
/
&
%
^
+
<
<=
<=>
=
==
>
>=
|
~
and
in
like
not
or
rlike
when
```
The following listed functions are not added:
```
!=
<>
between
case
```
All existing tests passed.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yongtang <gi...@git.apache.org>.
Github user yongtang commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205149209
Hi @hvanhovell the pull request was updated. Let me know if there are any issues.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yongtang <gi...@git.apache.org>.
Github user yongtang commented on a diff in the pull request:
https://github.com/apache/spark/pull/12128#discussion_r58543563
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ---
@@ -213,8 +213,8 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils with TestHiveSingleton {
checkExistence(sql("describe functioN `~`"), true,
"Function: ~",
- "Class: org.apache.hadoop.hive.ql.udf.UDFOPBitNot",
- "Usage: ~ n - Bitwise not")
+ "Class: org.apache.spark.sql.catalyst.expressions.BitwiseNot",
+ "Usage: To be added.")
}
--- End diff --
@hvanhovell Thanks. Additional test cases have been added.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205034161
For composed expressions, if there are only a few of them, maybe we can have special handling in the describe command? For `case`, we do not need to support `describe function case` if it is more of a keyword.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205149968
**[Test build #54829 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54829/consoleFull)** for PR 12128 at commit [`baad78f`](https://github.com/apache/spark/commit/baad78ff31815b8d9e77284713c4bb79d8f2a0f3).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205600527
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54928/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/12128
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205600525
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205024763
@yhuai what do you want to do with the composed expressions, i.e.: `!=`, `<>` & `between`? `case` is more of a keyword.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205182405
**[Test build #54834 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54834/consoleFull)** for PR 12128 at commit [`f318e8c`](https://github.com/apache/spark/commit/f318e8c3208c3bf5aed6d2202fb71052410aab81).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205149524
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-207842013
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55436/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-204668180
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205387681
**[Test build #54855 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54855/consoleFull)** for PR 12128 at commit [`6e1e879`](https://github.com/apache/spark/commit/6e1e87964c83c5b313f7dc7b910fc8f1849c08e1).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205519895
**[Test build #2743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2743/consoleFull)** for PR 12128 at commit [`6e1e879`](https://github.com/apache/spark/commit/6e1e87964c83c5b313f7dc7b910fc8f1849c08e1).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205342671
**[Test build #54855 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54855/consoleFull)** for PR 12128 at commit [`6e1e879`](https://github.com/apache/spark/commit/6e1e87964c83c5b313f7dc7b910fc8f1849c08e1).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/12128#discussion_r58492366
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ---
@@ -213,8 +213,8 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils with TestHiveSingleton {
checkExistence(sql("describe functioN `~`"), true,
"Function: ~",
- "Class: org.apache.hadoop.hive.ql.udf.UDFOPBitNot",
- "Usage: ~ n - Bitwise not")
+ "Class: org.apache.spark.sql.catalyst.expressions.BitwiseNot",
+ "Usage: To be added.")
}
--- End diff --
Could you also add a test case for one of the hard coded functions?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205173960
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yongtang <gi...@git.apache.org>.
Github user yongtang commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205053965
@hvanhovell Thanks. Let me take a look and update the pull request.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-207842009
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/12128#discussion_r58475895
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala ---
@@ -438,20 +438,38 @@ case class DescribeFunction(
}
override def run(sqlContext: SQLContext): Seq[Row] = {
- sqlContext.sessionState.functionRegistry.lookupFunction(functionName) match {
- case Some(info) =>
- val result =
- Row(s"Function: ${info.getName}") ::
- Row(s"Class: ${info.getClassName}") ::
- Row(s"Usage: ${replaceFunctionName(info.getUsage(), info.getName)}") :: Nil
-
- if (isExtended) {
- result :+ Row(s"Extended Usage:\n${replaceFunctionName(info.getExtended, info.getName)}")
- } else {
- result
- }
+ // Hard code "<>", "!=", "between", and "case" for now as there is no corresponding functions.
+ functionName match {
+ case "<>" =>
+ Row(s"Function: $functionName") ::
+ Row(s"Usage: a <> b - Returns TRUE if a is not equal to b") :: Nil
+ case "!=" =>
+ Row(s"Function: $functionName") ::
+ Row(s"Usage: a != b - Returns TRUE if a is not equal to b") :: Nil
+ case "between" =>
+ Row(s"Function: $functionName") ::
+ Row(s"Usage: between a [NOT] BETWEEN b AND c - " +
--- End diff --
`Usage: between a ...` => `Usage: a ...`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205600511
**[Test build #54928 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54928/consoleFull)** for PR 12128 at commit [`353a132`](https://github.com/apache/spark/commit/353a1320a4ca4d9b1ccbc336d8f5613a21e47a59).
* This patch **fails MiMa tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yongtang <gi...@git.apache.org>.
Github user yongtang commented on a diff in the pull request:
https://github.com/apache/spark/pull/12128#discussion_r58475784
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala ---
@@ -438,20 +438,38 @@ case class DescribeFunction(
}
override def run(sqlContext: SQLContext): Seq[Row] = {
- sqlContext.sessionState.functionRegistry.lookupFunction(functionName) match {
- case Some(info) =>
- val result =
- Row(s"Function: ${info.getName}") ::
- Row(s"Class: ${info.getClassName}") ::
- Row(s"Usage: ${replaceFunctionName(info.getUsage(), info.getName)}") :: Nil
-
- if (isExtended) {
- result :+ Row(s"Extended Usage:\n${replaceFunctionName(info.getExtended, info.getName)}")
- } else {
- result
- }
+ // Hard code "<>", "!=", "between", and "case" for now as there is no corresponding functions.
+ functionName match {
+ case "<>" =>
+ Row(s"Function: $functionName") ::
+ Row(s"Usage: a <> b - Returns TRUE if a is not equal to b") :: Nil
+ case "!=" =>
+ Row(s"Function: $functionName") ::
+ Row(s"Usage: a != b - Returns TRUE if a is not equal to b") :: Nil
+ case "between" =>
+ Row(s"Function: $functionName") ::
+ Row(s"Usage: between a [NOT] BETWEEN b AND c - " +
+ s"evaluate if a is [not] in between b and c") :: Nil
+ case "case" =>
--- End diff --
Thanks @yhuai let me update the pull request.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205182551
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54834/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yongtang <gi...@git.apache.org>.
Github user yongtang commented on a diff in the pull request:
https://github.com/apache/spark/pull/12128#discussion_r58476303
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala ---
@@ -438,20 +438,38 @@ case class DescribeFunction(
}
override def run(sqlContext: SQLContext): Seq[Row] = {
- sqlContext.sessionState.functionRegistry.lookupFunction(functionName) match {
- case Some(info) =>
- val result =
- Row(s"Function: ${info.getName}") ::
- Row(s"Class: ${info.getClassName}") ::
- Row(s"Usage: ${replaceFunctionName(info.getUsage(), info.getName)}") :: Nil
-
- if (isExtended) {
- result :+ Row(s"Extended Usage:\n${replaceFunctionName(info.getExtended, info.getName)}")
- } else {
- result
- }
+ // Hard code "<>", "!=", "between", and "case" for now as there is no corresponding functions.
+ functionName match {
+ case "<>" =>
+ Row(s"Function: $functionName") ::
+ Row(s"Usage: a <> b - Returns TRUE if a is not equal to b") :: Nil
+ case "!=" =>
+ Row(s"Function: $functionName") ::
+ Row(s"Usage: a != b - Returns TRUE if a is not equal to b") :: Nil
+ case "between" =>
+ Row(s"Function: $functionName") ::
+ Row(s"Usage: between a [NOT] BETWEEN b AND c - " +
--- End diff --
@yhuai Yes that makes sense. The first `between` appears in the original output (with `org.apache.hadoop.hive.ql.udf.generic.GenericUDFBetween`) which seems to be an bug.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205173603
**[Test build #54829 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54829/consoleFull)** for PR 12128 at commit [`baad78f`](https://github.com/apache/spark/commit/baad78ff31815b8d9e77284713c4bb79d8f2a0f3).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/12128#discussion_r58465327
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala ---
@@ -438,20 +438,38 @@ case class DescribeFunction(
}
override def run(sqlContext: SQLContext): Seq[Row] = {
- sqlContext.sessionState.functionRegistry.lookupFunction(functionName) match {
- case Some(info) =>
- val result =
- Row(s"Function: ${info.getName}") ::
- Row(s"Class: ${info.getClassName}") ::
- Row(s"Usage: ${replaceFunctionName(info.getUsage(), info.getName)}") :: Nil
-
- if (isExtended) {
- result :+ Row(s"Extended Usage:\n${replaceFunctionName(info.getExtended, info.getName)}")
- } else {
- result
- }
+ // Hard code "<>", "!=", "between", and "case" for now as there is no corresponding functions.
+ functionName match {
+ case "<>" =>
+ Row(s"Function: $functionName") ::
+ Row(s"Usage: a <> b - Returns TRUE if a is not equal to b") :: Nil
+ case "!=" =>
+ Row(s"Function: $functionName") ::
+ Row(s"Usage: a != b - Returns TRUE if a is not equal to b") :: Nil
+ case "between" =>
+ Row(s"Function: $functionName") ::
+ Row(s"Usage: between a [NOT] BETWEEN b AND c - " +
+ s"evaluate if a is [not] in between b and c") :: Nil
+ case "case" =>
--- End diff --
Looks like we also need to match upper case letters?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-207823843
**[Test build #55436 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55436/consoleFull)** for PR 12128 at commit [`4c59c81`](https://github.com/apache/spark/commit/4c59c819dabf6cc37248b0c5c1c45a0022e7668c).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205599127
**[Test build #54928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54928/consoleFull)** for PR 12128 at commit [`353a132`](https://github.com/apache/spark/commit/353a1320a4ca4d9b1ccbc336d8f5613a21e47a59).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yongtang <gi...@git.apache.org>.
Github user yongtang commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205159531
Hi @yhuai the pull request has been updated with the usages being added. Let me know if there are any other issues. And thanks for the review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205173973
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54829/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205051607
@yongtang can you see if you can hardcode the descriptions for the composed function into the `DecribeFunction` command? See: https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala#L421-L423
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205387872
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205824874
**[Test build #54986 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54986/consoleFull)** for PR 12128 at commit [`927272c`](https://github.com/apache/spark/commit/927272cdc79b7fd8908faea80e8d842d40c2b468).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yongtang <gi...@git.apache.org>.
Github user yongtang commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205598731
Thanks @yhuai. The pull request has been updated. Please let me know if there are any other issues.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yongtang <gi...@git.apache.org>.
Github user yongtang commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205039361
Hi @yhuai @hvanhovell , as a reference, below is the result of the `describe function` for current master. I haven't been able to identify the corresponding functions:
```
spark-sql> describe function `!=`;
Function: <>
Class: org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotEqual
Usage: a <> b - Returns TRUE if a is not equal to b
```
```
spark-sql> describe function `<>`;
Function: <>
Class: org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotEqual
Usage: a <> b - Returns TRUE if a is not equal to b
```
```
spark-sql> describe function `between`;
Function: between
Class: org.apache.hadoop.hive.ql.udf.generic.GenericUDFBetween
Usage: between a [NOT] BETWEEN b AND c - evaluate if a is [not] in between b and c
```
```
spark-sql> describe function `case`;
Function: case
Class: org.apache.hadoop.hive.ql.udf.generic.GenericUDFCase
Usage: CASE a WHEN b THEN c [WHEN d THEN e]* [ELSE f] END - When a = b, returns c; when a = d, return e; else return f
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-204804258
**[Test build #2735 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2735/consoleFull)** for PR 12128 at commit [`7a84e9b`](https://github.com/apache/spark/commit/7a84e9bd7dccffce103862a7c8818396d6fecb34).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-207841753
**[Test build #55436 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55436/consoleFull)** for PR 12128 at commit [`4c59c81`](https://github.com/apache/spark/commit/4c59c819dabf6cc37248b0c5c1c45a0022e7668c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205868723
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54986/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205482287
**[Test build #2743 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2743/consoleFull)** for PR 12128 at commit [`6e1e879`](https://github.com/apache/spark/commit/6e1e87964c83c5b313f7dc7b910fc8f1849c08e1).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yongtang <gi...@git.apache.org>.
Github user yongtang commented on a diff in the pull request:
https://github.com/apache/spark/pull/12128#discussion_r58331883
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala ---
@@ -438,20 +438,27 @@ case class DescribeFunction(
}
override def run(sqlContext: SQLContext): Seq[Row] = {
- sqlContext.sessionState.functionRegistry.lookupFunction(functionName) match {
- case Some(info) =>
- val result =
- Row(s"Function: ${info.getName}") ::
- Row(s"Class: ${info.getClassName}") ::
- Row(s"Usage: ${replaceFunctionName(info.getUsage(), info.getName)}") :: Nil
-
- if (isExtended) {
- result :+ Row(s"Extended Usage:\n${replaceFunctionName(info.getExtended, info.getName)}")
- } else {
- result
- }
+ // Hard code "<>", "!=", "between", and "case" for now as there is no corresponding functions.
+ functionName match {
+ case "<>" | "!=" | "between" | "case" =>
+ Row(s"Function: $functionName") ::
+ Row(s"Usage: To be added.") :: Nil
--- End diff --
@yhuai Sure let me update the pull request shortly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-207818229
@yongtang This PR looks good to me. Can you rebase?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205387878
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54855/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205868179
**[Test build #54986 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54986/consoleFull)** for PR 12128 at commit [`927272c`](https://github.com/apache/spark/commit/927272cdc79b7fd8908faea80e8d842d40c2b468).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yongtang <gi...@git.apache.org>.
Github user yongtang commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-207847079
Thanks @yhuai. The pull request has been rebased and passed the Jenkins. Please let me know if there are any other issues.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205182548
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205868721
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-204809902
**[Test build #2735 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2735/consoleFull)** for PR 12128 at commit [`7a84e9b`](https://github.com/apache/spark/commit/7a84e9bd7dccffce103862a7c8818396d6fecb34).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12128#issuecomment-205159077
**[Test build #54834 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54834/consoleFull)** for PR 12128 at commit [`f318e8c`](https://github.com/apache/spark/commit/f318e8c3208c3bf5aed6d2202fb71052410aab81).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14335][SQL] Describe function command r...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/12128#discussion_r58331592
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala ---
@@ -438,20 +438,27 @@ case class DescribeFunction(
}
override def run(sqlContext: SQLContext): Seq[Row] = {
- sqlContext.sessionState.functionRegistry.lookupFunction(functionName) match {
- case Some(info) =>
- val result =
- Row(s"Function: ${info.getName}") ::
- Row(s"Class: ${info.getClassName}") ::
- Row(s"Usage: ${replaceFunctionName(info.getUsage(), info.getName)}") :: Nil
-
- if (isExtended) {
- result :+ Row(s"Extended Usage:\n${replaceFunctionName(info.getExtended, info.getName)}")
- } else {
- result
- }
+ // Hard code "<>", "!=", "between", and "case" for now as there is no corresponding functions.
+ functionName match {
+ case "<>" | "!=" | "between" | "case" =>
+ Row(s"Function: $functionName") ::
+ Row(s"Usage: To be added.") :: Nil
--- End diff --
Can we add the usage at here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org