You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@linkis.apache.org by GitBox <gi...@apache.org> on 2022/07/08 03:11:09 UTC
[GitHub] [incubator-linkis] peacewong commented on issue #2416: [Feature] support ${yyyy-MM-dd%-2d}
peacewong commented on issue #2416:
URL: https://github.com/apache/incubator-linkis/issues/2416#issuecomment-1178493166
# Linkis内置时间变量介绍
## 1.总述
### 需求背景
用户希望在写代码时，对时间的格式要求千变万化,已有的[Linkis自定义变量](https://mp.weixin.qq.com/s/utjiadAkqF9b2xme_4i35Q)目前还不足以支撑这些需求。另外,已有的时间运算-1有些表示减一个月,有些则是减一天,用户很容易混淆
### 目标
* 其他日期内置变量都是相对run_date计算出来
* 支持Pattern格式时间且用户可以自行随意指定
* 支持±y/±M/±d/±H等等
Pattern格式对照表:
Letter| Date or Time Component| Presentation| Examples
----- | ----- | ----- | -----
G |Era designator |Text |AD
y |Year |Year |1996; 96
Y |Week year |Year |2009; 09
M |Month in year |Month| July; Jul; 07
w |Week in year |Number |27
W |Week in month |Number |2
D |Day in year |Number| 189
d |Day in month |Number |10
F |Day of week in month| Number |2
E |Day name in week |Text |Tuesday; Tue
u |Day number of week (1 = Monday, …, 7 = Sunday) |Number |1
a |Am/pm marker |Text |PM
H |Hour in day (0-23) |Number| 0
k |Hour in day (1-24)| Number |24
K |Hour in am/pm (0-11)| Number |0
h |Hour in am/pm (1-12) |Number |12
m |Minute in hour |Number |30
s |Second in minute |Number| 55
S |Millisecond |Number |978
z |Time zone |General time zone |Pacific Standard Time; PST; GMT-08:00
Z |Time zone |RFC 822 time zone |-0800
X |Time zone |ISO 8601 time zone |-08; -0800; -08:00
## 2. 总体设计
总体设计和技术架构参照[Linkis自定义变量](https://mp.weixin.qq.com/s/utjiadAkqF9b2xme_4i35Q)
## 3. 功能介绍
Linkis支持的变量类型分为自定义变量(不做赘述)和系统内置变量，自定变量日期支持+-。
### 3.1 内置变量举例
可以根据自己的喜好/业务实际情况定义需要动态渲染的参数
variable | result
--- | ---
${yyyy-01-01} | 2021-01-01
${yyyy-01-01%-2y} | 2019-01-01
${yyyy-MM-01%-2M} | 2021-02-01
${yyyy-MM-dd%-2d} | 2021-03-31
${yyyy MM ----- HH%-1H} | 2021 04 ----- 14
${yyyyMMdd%-1d} | 20210401
${yyyyMM01%-1M} | 20210301
${HH%-1H} | 14
### 3.2 自定义变量使用实例
* 例子1: sql
```sql
SELECT * FROM hive.tmp.fund_nav_histories
WHERE dt <= DATE_FORMAT(DATE_ADD('day', -1, DATE(Date_parse('${yyyyMMdd%-1d}', '%Y%m%d'))), '%Y%m%d')
```
渲染后
```sql
SELECT * FROM hive.tmp.fund_nav_histories
WHERE dt <= DATE_FORMAT(DATE_ADD('day', -1, DATE(Date_parse('20220705', '%Y%m%d'))), '%Y%m%d')
```
* 例子2: shell
```shell
aws s3 ls s3://***/ads/tmp/dws_member_active_detail_d_20210601_20211231/pt=${yyyyMMdd%-1d}/
```
渲染后
```shell
aws s3 ls s3://***/ads/tmp/dws_member_active_detail_d_20210601_20211231/pt=20220705/
```
* 例子3: datax json
```json
{
"job": {
"setting": {
"speed": {
"channel": 1
}
},
"content": [
{
"reader": {
"name": "s3reader",
"parameter": {
"bucket": "**************",
"path": [
"ads/tmp/ccass_tm_announcements/${yyyyMMdd%-1d}/"
],
"stored": "parquet",
"compression": "NONE",
"column": [
{
"index": 0,
"type": "int"
},
{
"index": 1,
"type": "string",
"constant": "${yyyyMMdd%-1d}"
}
]
}
},
"writer": {
"name": "streamwriter",
"parameter": {
"print": true
}
}
}
]
}
}
```
渲染后
```json
{
"job": {
"setting": {
"speed": {
"channel": 1
}
},
"content": [
{
"reader": {
"name": "s3reader",
"parameter": {
"bucket": "**************",
"path": [
"ads/tmp/ccass_tm_announcements/20220705/"
],
"stored": "parquet",
"compression": "NONE",
"column": [
{
"index": 0,
"type": "int"
},
{
"index": 1,
"type": "string",
"constant": "20220705"
}
]
}
},
"writer": {
"name": "streamwriter",
"parameter": {
"print": true
}
}
}
]
}
}
```
* 例子4:python
```python
print(${yyyyMMdd%-1d})
```
渲染后
```
20220705
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org