You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by xguo27 <gi...@git.apache.org> on 2015/12/22 00:05:16 UTC
[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...
GitHub user xguo27 opened a pull request:
https://github.com/apache/spark/pull/10423
[SPARK-12456][SQL] Add ExpressionDescription to misc functions
First try, not sure how much information we need to provide in the usage part.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xguo27/spark SPARK-12456
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/10423.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10423
----
commit 8cae22cdaaaa771eea1bc08d1e0903b5e9df6814
Author: Xiu Guo <xg...@gmail.com>
Date: 2015-12-21T22:55:44Z
[SPARK-12456][SQL] Add ExpressionDescription to misc functions
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10423#discussion_r48224048
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/InputFileName.scala ---
@@ -26,6 +26,9 @@ import org.apache.spark.unsafe.types.UTF8String
/**
* Expression that returns the name of the current file being read in using [[SqlNewHadoopRDD]]
*/
+@ExpressionDescription(
+ usage = "_FUNC_() - Returns the name of the current file being read in using [SqlNewHadoopRDD]",
--- End diff --
"in using [SqlNewHadoopRDD]" -> "if available"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10423#issuecomment-166557245
**[Test build #2243 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2243/consoleFull)** for PR 10423 at commit [`b49df75`](https://github.com/apache/spark/commit/b49df756bfc7ef89031dc52c3fbfcac186a5ba9c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...
Posted by xguo27 <gi...@git.apache.org>.
Github user xguo27 commented on the pull request:
https://github.com/apache/spark/pull/10423#issuecomment-166701737
@rxin Great, thanks Reynold! My JIRA id is xguo27.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10423#issuecomment-166536591
**[Test build #2243 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2243/consoleFull)** for PR 10423 at commit [`b49df75`](https://github.com/apache/spark/commit/b49df756bfc7ef89031dc52c3fbfcac186a5ba9c).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/10423
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/10423#issuecomment-166700763
What's your JIRA id?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10423#discussion_r48224112
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -30,6 +30,9 @@ import org.apache.spark.unsafe.types.UTF8String
* A function that calculates an MD5 128-bit checksum and returns it as a hex string
* For input of type [[BinaryType]]
*/
+@ExpressionDescription(
+ usage = "_FUNC_(input) - Returns an MD5 128-bit checksum as a hex string of the input",
+ extended = "> SELECT _FUNC_('SparkSql');\n '49df15ea9f84f6fcf94f2823bf3f861e'")
--- End diff --
I'd prefer to just use "Spark".
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...
Posted by xguo27 <gi...@git.apache.org>.
Github user xguo27 commented on the pull request:
https://github.com/apache/spark/pull/10423#issuecomment-166531147
@rxin Thank you very much for go through the changeset, Reynold! I have updated it per your suggestions.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10423#discussion_r48224082
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -53,6 +56,11 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* asking for an unsupported SHA function, the return value is NULL. If either argument is NULL or
* the hash length is not one of the permitted values, the return value is NULL.
*/
+@ExpressionDescription(
+ usage = "_FUNC_(input) - Returns a checksum of SHA-2 family as a hex string of the input. " +
--- End diff --
this accepts two args?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10423#discussion_r48224065
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/MonotonicallyIncreasingID.scala ---
@@ -32,6 +32,9 @@ import org.apache.spark.sql.types.{LongType, DataType}
*
* Since this expression is stateful, it cannot be a case object.
*/
+@ExpressionDescription(
+ usage = "_FUNC_() - Returns monotonically increasing 64-bit integers",
--- End diff --
You should probably include the entire paragraph up there.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10423#issuecomment-166451269
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org