You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by wangyum <gi...@git.apache.org> on 2017/05/28 04:46:00 UTC
[GitHub] spark pull request #18134: [SPARK-20909][SQL] Add build-int SQL function - D...
GitHub user wangyum opened a pull request:
https://github.com/apache/spark/pull/18134
[SPARK-20909][SQL] Add build-int SQL function - DAYOFWEEK
## What changes were proposed in this pull request?
Add build-int SQL function - DAYOFWEEK
## How was this patch tested?
unit tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wangyum/spark SPARK-20909
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18134.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18134
----
commit 02b62b4bce583a0fd42717b6c930a0d038516807
Author: Yuming Wang <wg...@gmail.com>
Date: 2017-05-28T04:42:31Z
Add build-int SQL function - DAYOFWEEK
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18134
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by ueshin <gi...@git.apache.org>.
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/18134
Thanks! Merging to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18134
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77515/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18134
**[Test build #77514 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77514/testReport)** for PR 18134 at commit [`87defdc`](https://github.com/apache/spark/commit/87defdc91a7be6ba027cc196e76a552bf47a01f1).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18134
**[Test build #77478 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77478/testReport)** for PR 18134 at commit [`02b62b4`](https://github.com/apache/spark/commit/02b62b4bce583a0fd42717b6c930a0d038516807).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by sergiobilello-eb <gi...@git.apache.org>.
Github user sergiobilello-eb commented on the issue:
https://github.com/apache/spark/pull/18134
thanks @gatorsmile :)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18134
Yeah, you always can implement such a UDF.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by sergiobilello-eb <gi...@git.apache.org>.
Github user sergiobilello-eb commented on the issue:
https://github.com/apache/spark/pull/18134
This function is not reported in the API documentation: https://spark.apache.org/docs/2.2.0/api/java/index.html?org/apache/spark/sql/functions.html or https://spark.apache.org/docs/2.1.0/api/java/index.html?org/apache/spark/sql/functions.html ?
How is it possible? I am trying to use it from spark-sql on a spark 2.1.0 cluster
Thanks
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18134
**[Test build #77515 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77515/testReport)** for PR 18134 at commit [`69d4227`](https://github.com/apache/spark/commit/69d42278cf6eeb13415b9627cdb7019c333547fa).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/18134
LGTM except for one comment about the function description.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18134
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77513/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #18134: [SPARK-20909][SQL] Add build-int SQL function - D...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/18134#discussion_r119002317
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ---
@@ -404,6 +404,44 @@ case class DayOfMonth(child: Expression) extends UnaryExpression with ImplicitCa
// scalastyle:off line.size.limit
@ExpressionDescription(
+ usage = "_FUNC_(date) - Returns the weekday index for date/timestamp (1 = Sunday, 2 = Monday, ..., 7 = Saturday).",
--- End diff --
As Sunday, Saturday are included, it is not only weekday. `Returns the day of the week ...`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18134
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18134
**[Test build #77513 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77513/testReport)** for PR 18134 at commit [`dcae776`](https://github.com/apache/spark/commit/dcae77600fc0cca41d2e3e607469232f59a021af).
* This patch passes all tests.
* This patch **does not merge cleanly**.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #18134: [SPARK-20909][SQL] Add build-int SQL function - D...
Posted by ueshin <gi...@git.apache.org>.
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18134#discussion_r118858221
--- Diff: sql/core/src/test/resources/sql-tests/inputs/datetime.sql ---
@@ -6,3 +6,5 @@ select current_date = current_date(), current_timestamp = current_timestamp();
select to_date(null), to_date('2016-12-31'), to_date('2016-12-31', 'yyyy-MM-dd');
select to_timestamp(null), to_timestamp('2016-12-31 00:12:00'), to_timestamp('2016-12-31', 'yyyy-MM-dd');
+
+select dayofweek('2007-02-03'), dayofweek('2009-07-30'), dayofweek('2017-05-27'), dayofweek(null), dayofweek('1582-10-15 13:10:15');
--- End diff --
nit: Can you add a line break at the end of file?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by sergiobilello-eb <gi...@git.apache.org>.
Github user sergiobilello-eb commented on the issue:
https://github.com/apache/spark/pull/18134
```spark-sql> select dayofweek('2007-02-03'), dayofweek('2009-07-30'), dayofweek('2017-05-27'), dayofweek(null), dayofweek('1582-10-15 13:10:15');
17/11/10 16:08:23 INFO SparkSqlParser: Parsing command: select dayofweek('2007-02-03'), dayofweek('2009-07-30'), dayofweek('2017-05-27'), dayofweek(null), dayofweek('1582-10-15 13:10:15')
17/11/10 16:08:23 INFO HiveMetaStore: 0: get_database: default
17/11/10 16:08:23 INFO audit: ugi=sergio.bilello ip=unknown-ip-addr cmd=get_database: default
17/11/10 16:08:23 INFO HiveMetaStore: 0: get_database: default
17/11/10 16:08:23 INFO audit: ugi=sergio.bilello ip=unknown-ip-addr cmd=get_database: default
17/11/10 16:08:23 INFO HiveMetaStore: 0: get_function: default.dayofweek
17/11/10 16:08:23 INFO audit: ugi=sergio.bilello ip=unknown-ip-addr cmd=get_function: default.dayofweek
Error in query: Undefined function: 'dayofweek'. This function is neither a registered temporary function nor a permanent function registered in the database 'default'.; line 1 pos 7```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18134
**[Test build #77513 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77513/testReport)** for PR 18134 at commit [`dcae776`](https://github.com/apache/spark/commit/dcae77600fc0cca41d2e3e607469232f59a021af).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18134
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77514/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by sergiobilello-eb <gi...@git.apache.org>.
Github user sergiobilello-eb commented on the issue:
https://github.com/apache/spark/pull/18134
thanks @gatorsmile! Do you suggest any workaround until then? I mean not rebuilding spark with that patch.... Can I register my UDF that contains that logic?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18134
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #18134: [SPARK-20909][SQL] Add build-int SQL function - D...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/18134#discussion_r118863018
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ---
@@ -402,6 +402,44 @@ case class DayOfMonth(child: Expression) extends UnaryExpression with ImplicitCa
}
}
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = "_FUNC_(date) - Returns the weekday index for date/timestamp (1 = Sunday, 2 = Monday, ..., 7 = Saturday).",
+ extended = """
+ Examples:
+ > SELECT _FUNC_('2009-07-30');
+ 5
+ """)
+// scalastyle:on line.size.limit
+case class DayOfWeek(child: Expression) extends UnaryExpression with ImplicitCastInputTypes {
+
+ override def inputTypes: Seq[AbstractDataType] = Seq(DateType)
+
+ override def dataType: DataType = IntegerType
+
+ @transient private lazy val c = {
+ Calendar.getInstance(DateTimeUtils.getTimeZone("UTC"))
+ }
+
+ override protected def nullSafeEval(date: Any): Any = {
+ c.setTimeInMillis(date.asInstanceOf[Int] * 1000L * 3600L * 24L)
+ c.get(Calendar.DAY_OF_WEEK)
--- End diff --
In `WeekOfYear`, we set `Calendar.MONDAY` as the first day of a week. Here seems we assume it's `Calendar.SUNDAY`. Is there any conflict we will encounter?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18134
Build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18134
https://issues.apache.org/jira/browse/SPARK-20909
This is not part of 2.2. Based on JIRA, it will be included in 2.3
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #18134: [SPARK-20909][SQL] Add build-int SQL function - D...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18134
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18134
**[Test build #77478 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77478/testReport)** for PR 18134 at commit [`02b62b4`](https://github.com/apache/spark/commit/02b62b4bce583a0fd42717b6c930a0d038516807).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class DayOfWeek(child: Expression) extends UnaryExpression with ImplicitCastInputTypes `
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #18134: [SPARK-20909][SQL] Add build-int SQL function - D...
Posted by wangyum <gi...@git.apache.org>.
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/18134#discussion_r119000601
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ---
@@ -402,6 +402,44 @@ case class DayOfMonth(child: Expression) extends UnaryExpression with ImplicitCa
}
}
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = "_FUNC_(date) - Returns the weekday index for date/timestamp (1 = Sunday, 2 = Monday, ..., 7 = Saturday).",
+ extended = """
+ Examples:
+ > SELECT _FUNC_('2009-07-30');
+ 5
+ """)
+// scalastyle:on line.size.limit
+case class DayOfWeek(child: Expression) extends UnaryExpression with ImplicitCastInputTypes {
+
+ override def inputTypes: Seq[AbstractDataType] = Seq(DateType)
+
+ override def dataType: DataType = IntegerType
+
+ @transient private lazy val c = {
+ Calendar.getInstance(DateTimeUtils.getTimeZone("UTC"))
+ }
+
+ override protected def nullSafeEval(date: Any): Any = {
+ c.setTimeInMillis(date.asInstanceOf[Int] * 1000L * 3600L * 24L)
+ c.get(Calendar.DAY_OF_WEEK)
--- End diff --
Keep pace with [Hive's DayOfWeek
](https://github.com/apache/hive/blob/59539885725a96cca4b3f0759a5b26e0d8198dc8/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfWeek.java#L55).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18134
**[Test build #77514 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77514/testReport)** for PR 18134 at commit [`87defdc`](https://github.com/apache/spark/commit/87defdc91a7be6ba027cc196e76a552bf47a01f1).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class UnresolvedRelation(`
* `case class StringReplace(srcExpr: Expression, searchExpr: Expression, replaceExpr: Expression)`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18134
@sergiobilello-eb This is just a SQL function. You can call it in SQL interface or using the `df.select(expr("dayofweek('2009-07-30')"))`. It is not part of the DataFrame functions. You can submit a PR or report it as an issue to add such an API.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18134
**[Test build #77515 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77515/testReport)** for PR 18134 at commit [`69d4227`](https://github.com/apache/spark/commit/69d42278cf6eeb13415b9627cdb7019c333547fa).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18134
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77478/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org