You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sa...@apache.org on 2021/07/27 10:06:14 UTC
[spark] branch master updated: [SPARK-34619][SQL][DOCS] Describe
ANSI interval types at the `Data types` page of the SQL reference
This is an automated email from the ASF dual-hosted git repository.
sarutak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new f483796 [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference
f483796 is described below
commit f4837961a9c4c35eaf71406c22874984b454e8fd
Author: Max Gekk <ma...@gmail.com>
AuthorDate: Tue Jul 27 19:05:39 2021 +0900
[SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference
### What changes were proposed in this pull request?
In the PR, I propose to update the page https://spark.apache.org/docs/latest/sql-ref-datatypes.html and add information about the year-month and day-time interval types introduced by SPARK-27790.
<img width="932" alt="Screenshot 2021-07-27 at 10 38 23" src="https://user-images.githubusercontent.com/1580697/127115289-e633ca3a-2c18-49a0-a7c0-22421ae5c363.png">
### Why are the changes needed?
To inform users about new ANSI interval types, and improve UX with Spark SQL.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Should be tested by a GitHub action.
Closes #33518 from MaxGekk/doc-interval-types.
Authored-by: Max Gekk <ma...@gmail.com>
Signed-off-by: Kousuke Saruta <sa...@oss.nttdata.com>
---
docs/sql-ref-datatypes.md | 38 ++++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)
diff --git a/docs/sql-ref-datatypes.md b/docs/sql-ref-datatypes.md
index ce22d92..89ffa34 100644
--- a/docs/sql-ref-datatypes.md
+++ b/docs/sql-ref-datatypes.md
@@ -49,6 +49,44 @@ Spark SQL and DataFrames support the following data types:
absolute point in time.
- `DateType`: Represents values comprising values of fields year, month and day, without a
time-zone.
+* Interval types
+ - `YearMonthIntervalType(startField, endField)`: Represents a year-month interval which is made up of a contiguous subset of the following fields:
+ - MONTH, months within years `[0..11]`,
+ - YEAR, years in the range `[0..178956970]`.
+
+ Individual interval fields are non-negative, but an interval itself can have a sign, and be negative.
+
+ `startField` is the leftmost field, and `endField` is the rightmost field of the type. Valid values of `startField` and `endField` are 0(MONTH) and 1(YEAR). Supported year-month interval types are:
+
+ |Year-Month Interval Type|SQL type|An instance of the type|
+ |---------|----|-------------------|
+ |`YearMonthIntervalType(YEAR, YEAR)` or `YearMonthIntervalType(YEAR)`|INTERVAL YEAR|`INTERVAL '2021' YEAR`|
+ |`YearMonthIntervalType(YEAR, MONTH)`|INTERVAL YEAR TO MONTH|`INTERVAL '2021-07' YEAR TO MONTH`|
+ |`YearMonthIntervalType(MONTH, MONTH)` or `YearMonthIntervalType(MONTH)`|INTERVAL MONTH|`INTERVAL '10' MONTH`|
+
+ - `DayTimeIntervalType(startField, endField)`: Represents a day-time interval which is made up of a contiguous subset of the following fields:
+ - SECOND, seconds within minutes and possibly fractions of a second `[0..59.999999]`,
+ - MINUTE, minutes within hours `[0..59]`,
+ - HOUR, hours within days `[0..23]`,
+ - DAY, days in the range `[0..106751991]`.
+
+ Individual interval fields are non-negative, but an interval itself can have a sign, and be negative.
+
+ `startField` is the leftmost field, and `endField` is the rightmost field of the type. Valid values of `startField` and `endField` are 0 (DAY), 1 (HOUR), 2 (MINUTE), 3 (SECOND). Supported day-time interval types are:
+
+ |Day-Time Interval Type|SQL type|An instance of the type|
+ |---------|----|-------------------|
+ |`DayTimeIntervalType(DAY, DAY)` or `DayTimeIntervalType(DAY)`|INTERVAL DAY|`INTERVAL '100' DAY`|
+ |`DayTimeIntervalType(DAY, HOUR)`|INTERVAL DAY TO HOUR|`INTERVAL '100 10' DAY TO HOUR`|
+ |`DayTimeIntervalType(DAY, MINUTE)`|INTERVAL DAY TO MINUTE|`INTERVAL '100 10:30' DAY TO MINUTE`|
+ |`DayTimeIntervalType(DAY, SECOND)`|INTERVAL DAY TO SECOND|`INTERVAL '100 10:30:40.999999' DAY TO SECOND`|
+ |`DayTimeIntervalType(HOUR, HOUR)` or `DayTimeIntervalType(HOUR)`|INTERVAL HOUR|`INTERVAL '123' HOUR`|
+ |`DayTimeIntervalType(HOUR, MINUTE)`|INTERVAL HOUR TO MINUTE|`INTERVAL '123:10' HOUR TO MINUTE`|
+ |`DayTimeIntervalType(HOUR, SECOND)`|INTERVAL HOUR TO SECOND|`INTERVAL '123:10:59' HOUR TO SECOND`|
+ |`DayTimeIntervalType(MINUTE, MINUTE)` or `DayTimeIntervalType(MINUTE)`|INTERVAL MINUTE|`INTERVAL '1000' MINUTE`|
+ |`DayTimeIntervalType(MINUTE, SECOND)`|INTERVAL MINUTE TO SECOND|`INTERVAL '1000:01.001' MINUTE TO SECOND`|
+ |`DayTimeIntervalType(SECOND, SECOND)` or `DayTimeIntervalType(SECOND)`|INTERVAL SECOND|`INTERVAL '1000.000001' SECOND`|
+
* Complex types
- `ArrayType(elementType, containsNull)`: Represents values comprising a sequence of
elements with the type of `elementType`. `containsNull` is used to indicate if
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org