You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by mo...@apache.org on 2021/09/28 03:39:48 UTC
[incubator-doris] branch master updated: [Dynamic Partition]
reserve specific history periods by dynamic partition. (#6554)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new cdf9f9e [Dynamic Partition] reserve specific history periods by dynamic partition. (#6554)
cdf9f9e is described below
commit cdf9f9e980399517fe81380dc8d46827e79801ad
Author: Henry2SS <45...@users.noreply.github.com>
AuthorDate: Tue Sep 28 11:39:35 2021 +0800
[Dynamic Partition] reserve specific history periods by dynamic partition. (#6554)
Add RESERVED_HISTORY_STARTS and RESERVED_HISTORY_ENDS.
Fixes #6514
---
docs/en/administrator-guide/dynamic-partition.md | 46 ++-
.../operation/doris-error-code.md | 5 +-
.../Create/CREATE-TABLE.md | 1 +
.../sql-statements/Data Definition/CREATE TABLE.md | 1 +
.../zh-CN/administrator-guide/dynamic-partition.md | 47 ++-
.../operation/doris-error-code.md | 4 +
.../Create/CREATE-TABLE.md | 1 +
.../sql-statements/Data Definition/CREATE TABLE.md | 1 +
.../doris/analysis/ShowDynamicPartitionStmt.java | 3 +-
.../doris/catalog/DynamicPartitionProperty.java | 17 +-
.../doris/clone/DynamicPartitionScheduler.java | 73 ++++-
.../java/org/apache/doris/common/ErrorCode.java | 8 +
.../doris/common/util/DynamicPartitionUtil.java | 140 ++++++++-
.../org/apache/doris/common/util/RangeUtils.java | 6 +
.../java/org/apache/doris/qe/ShowExecutor.java | 6 +-
.../doris/catalog/DynamicPartitionTableTest.java | 336 +++++++++++++++++++++
16 files changed, 653 insertions(+), 42 deletions(-)
diff --git a/docs/en/administrator-guide/dynamic-partition.md b/docs/en/administrator-guide/dynamic-partition.md
index 9a947dd..402e37d 100644
--- a/docs/en/administrator-guide/dynamic-partition.md
+++ b/docs/en/administrator-guide/dynamic-partition.md
@@ -151,6 +151,32 @@ The rules of dynamic partition are prefixed with `dynamic_partition.`:
p20210523: ["2021-05-23", "2021-05-24") storage_medium=SSD storage_cooldown_time=2021-05-25 00:00:00
```
+* `dynamic_partition.reserved_history_periods`
+
+ The range of reserved history periods. It should be in the form of `[yyyy-MM-dd,yyyy-MM-dd],[...,...]` while the `dynamic_partition.time_unit` is "DAY, WEEK, and MONTH". And it should be in the form of `[yyyy-MM-dd HH:mm:ss,yyyy-MM-dd HH:mm:ss],[...,...]` while the dynamic_partition.time_unit` is "HOUR". And no more spaces expected. The default value is `"NULL"`, which means it is not set.
+
+ Let us give an example. Suppose today is 2021-09-06，partitioned by day, and the properties of dynamic partition are set to:
+
+ ```time_unit="DAY/WEEK/MONTH", end=3, start=-3, reserved_history_periods="[2020-06-01,2020-06-20],[2020-10-31,2020-11-15]"```.
+
+ The the system will automatically reserve following partitions in following period :
+
+ ```
+ ["2020-06-01","2020-06-20"],
+ ["2020-10-31","2020-11-15"]
+ ```
+ or
+
+ ```time_unit="HOUR", end=3, start=-3, reserved_history_periods="[2020-06-01 00:00:00,2020-06-01 03:00:00]"```.
+
+ The the system will automatically reserve following partitions in following period :
+
+ ```
+ ["2020-06-01 00:00:00","2020-06-01 03:00:00"]
+ ```
+
+ Otherwise, every `[...,...]` in `reserved_history_periods` is a couple of properties, and they should be set at the same time. And the first date can't be larger than the second one.
+
#### Create History Partition Rules
When `create_history_partition` is `true`, i.e. history partition creation is enabled, Doris determines the number of history partitions to be created based on `dynamic_partition.start` and `dynamic_partition.history_partition_num`.
@@ -357,16 +383,16 @@ You can further view the scheduling of dynamic partitioned tables by using the f
```
mysql> SHOW DYNAMIC PARTITION TABLES;
-+-----------+--------+----------+-------------+------+--------+---------+-----------+----------------+---------------------+--------+------------------------+----------------------+
-| TableName | Enable | TimeUnit | Start | End | Prefix | Buckets | StartOf | LastUpdateTime | LastSchedulerTime | State | LastCreatePartitionMsg | LastDropPartitionMsg |
-+-----------+--------+----------+-------------+------+--------+---------+-----------+----------------+---------------------+--------+------------------------+----------------------+
-| d3 | true | WEEK | -3 | 3 | p | 1 | MONDAY | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A |
-| d5 | true | DAY | -7 | 3 | p | 32 | N/A | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A |
-| d4 | true | WEEK | -3 | 3 | p | 1 | WEDNESDAY | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A |
-| d6 | true | MONTH | -2147483648 | 2 | p | 8 | 3rd | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A |
-| d2 | true | DAY | -3 | 3 | p | 32 | N/A | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A |
-| d7 | true | MONTH | -2147483648 | 5 | p | 8 | 24th | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A |
-+-----------+--------+----------+-------------+------+--------+---------+-----------+----------------+---------------------+--------+------------------------+----------------------+
++-----------+--------+----------+-------------+------+--------+---------+-----------+----------------+---------------------+--------+------------------------+----------------------+-------------------------+
+| TableName | Enable | TimeUnit | Start | End | Prefix | Buckets | StartOf | LastUpdateTime | LastSchedulerTime | State | LastCreatePartitionMsg | LastDropPartitionMsg | ReservedHistoryPeriods |
++-----------+--------+----------+-------------+------+--------+---------+-----------+----------------+---------------------+--------+------------------------+----------------------+-------------------------+
+| d3 | true | WEEK | -3 | 3 | p | 1 | MONDAY | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A | [2021-12-01,2021-12-31] |
+| d5 | true | DAY | -7 | 3 | p | 32 | N/A | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A | NULL |
+| d4 | true | WEEK | -3 | 3 | p | 1 | WEDNESDAY | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A | NULL |
+| d6 | true | MONTH | -2147483648 | 2 | p | 8 | 3rd | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A | NULL |
+| d2 | true | DAY | -3 | 3 | p | 32 | N/A | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A | NULL |
+| d7 | true | MONTH | -2147483648 | 5 | p | 8 | 24th | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A | NULL |
++-----------+--------+----------+-------------+------+--------+---------+-----------+----------------+---------------------+--------+------------------------+----------------------+-------------------------+
7 rows in set (0.02 sec)
```
diff --git a/docs/en/administrator-guide/operation/doris-error-code.md b/docs/en/administrator-guide/operation/doris-error-code.md
index 1ee7bed..9e24322 100644
--- a/docs/en/administrator-guide/operation/doris-error-code.md
+++ b/docs/en/administrator-guide/operation/doris-error-code.md
@@ -172,4 +172,7 @@ under the License.
| 5072 | The dynamic partition copy value is not a valid number |
| 5073 | The original created table stmt is empty |
| 5074 | Create historical dynamic partition parameters: create_history_partition is invalid, what is expected is: true or false |
-
+| 5076 | The specified dynamic partition reserved_history_periods is null or empty |
+| 5077 | The specified dynamic partition reserved_history_periods is invalid |
+| 5078 | The length of specified dynamic partition reserved_history_periods must have pairs of date value |
+| 5079 | The specified dynamic partition reserved_history_periods' first date is larger than the second one |
diff --git a/docs/en/sql-reference-v2/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE.md b/docs/en/sql-reference-v2/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE.md
index ee34196..fd44cf0 100644
--- a/docs/en/sql-reference-v2/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE.md
+++ b/docs/en/sql-reference-v2/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE.md
@@ -295,6 +295,7 @@ distribution_info
* `dynamic_partition.buckets`: Used to specify the number of partition buckets that are automatically created.
* `dynamic_partition.create_history_partition`: Whether to create a history partition.
* `dynamic_partition.history_partition_num`: Specify the number of historical partitions to be created.
+ * `dynamic_partition.reserved_history_periods`: Used to specify the range of reserved history periods.
### Example
diff --git a/docs/en/sql-reference/sql-statements/Data Definition/CREATE TABLE.md b/docs/en/sql-reference/sql-statements/Data Definition/CREATE TABLE.md
index 20c3360..b8275a2 100644
--- a/docs/en/sql-reference/sql-statements/Data Definition/CREATE TABLE.md
+++ b/docs/en/sql-reference/sql-statements/Data Definition/CREATE TABLE.md
@@ -314,6 +314,7 @@ Syntax:
dynamic_partition.buckets: specifies the number of partition buckets that are automatically created
dynamic_partition.create_history_partition: specifies whether create history partitions, default value is false
dynamic_partition.history_partition_num: used to specify the number of history partitions when enable create_history_partition
+ dynamic_partition.reserved_history_periods: Used to specify the range of reserved history periods
```
5) You can create multiple Rollups in bulk when building a table
grammar:
diff --git a/docs/zh-CN/administrator-guide/dynamic-partition.md b/docs/zh-CN/administrator-guide/dynamic-partition.md
index efe84fa..246559e 100644
--- a/docs/zh-CN/administrator-guide/dynamic-partition.md
+++ b/docs/zh-CN/administrator-guide/dynamic-partition.md
@@ -149,6 +149,33 @@ under the License.
p20210523：["2021-05-23", "2021-05-24") storage_medium=SSD storage_cooldown_time=2021-05-25 00:00:00
```
+* `dynamic_partition.reserved_history_periods`
+
+ 需要保留的历史分区的时间范围。当`dynamic_partition.time_unit` 设置为 "DAY/WEEK/MONTH" 时，需要以 `[yyyy-MM-dd,yyyy-MM-dd],[...,...]` 格式进行设置。当`dynamic_partition.time_unit` 设置为 "HOUR" 时，需要以 `[yyyy-MM-dd HH:mm:ss,yyyy-MM-dd HH:mm:ss],[...,...]` 的格式来进行设置。如果不设置，默认为 `"NULL"`。
+
+ 我们举例说明。假设今天是 2021-09-06，按天分类，动态分区的属性设置为：
+
+ ```time_unit="DAY/WEEK/MONTH", end=3, start=-3, reserved_history_periods="[2020-06-01,2020-06-20],[2020-10-31,2020-11-15]"```。
+
+ 则系统会自动保留：
+
+ ```
+ ["2020-06-01","2020-06-20"],
+ ["2020-10-31","2020-11-15"]
+ ```
+
+ 或者
+
+ ```time_unit="HOUR", end=3, start=-3, reserved_history_periods="[2020-06-01 00:00:00,2020-06-01 03:00:00]"```.
+
+ 则系统会自动保留：
+
+ ```
+ ["2020-06-01 00:00:00","2020-06-01 03:00:00"]
+ ```
+
+ 这两个时间段的分区。其中，`reserved_history_periods` 的每一个 `[...,...]` 是一对设置项，两者需要同时被设置，且第一个时间不能大于第二个时间``。
+
#### 创建历史分区规则
当 `create_history_partition` 为 `true`，即开启创建历史分区功能时，Doris 会根据 `dynamic_partition.start` 和 `dynamic_partition.history_partition_num` 来决定创建历史分区的个数。
@@ -355,16 +382,16 @@ p20200521: ["2020-05-21", "2020-05-22")
```
mysql> SHOW DYNAMIC PARTITION TABLES;
-+-----------+--------+----------+-------------+------+--------+---------+-----------+----------------+---------------------+--------+------------------------+----------------------+
-| TableName | Enable | TimeUnit | Start | End | Prefix | Buckets | StartOf | LastUpdateTime | LastSchedulerTime | State | LastCreatePartitionMsg | LastDropPartitionMsg |
-+-----------+--------+----------+-------------+------+--------+---------+-----------+----------------+---------------------+--------+------------------------+----------------------+
-| d3 | true | WEEK | -3 | 3 | p | 1 | MONDAY | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A |
-| d5 | true | DAY | -7 | 3 | p | 32 | N/A | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A |
-| d4 | true | WEEK | -3 | 3 | p | 1 | WEDNESDAY | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A |
-| d6 | true | MONTH | -2147483648 | 2 | p | 8 | 3rd | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A |
-| d2 | true | DAY | -3 | 3 | p | 32 | N/A | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A |
-| d7 | true | MONTH | -2147483648 | 5 | p | 8 | 24th | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A |
-+-----------+--------+----------+-------------+------+--------+---------+-----------+----------------+---------------------+--------+------------------------+----------------------+
++-----------+--------+----------+-------------+------+--------+---------+-----------+----------------+---------------------+--------+------------------------+----------------------+-------------------------+
+| TableName | Enable | TimeUnit | Start | End | Prefix | Buckets | StartOf | LastUpdateTime | LastSchedulerTime | State | LastCreatePartitionMsg | LastDropPartitionMsg | ReservedHistoryPeriods |
++-----------+--------+----------+-------------+------+--------+---------+-----------+----------------+---------------------+--------+------------------------+----------------------+-------------------------+
+| d3 | true | WEEK | -3 | 3 | p | 1 | MONDAY | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A | [2021-12-01,2021-12-31] |
+| d5 | true | DAY | -7 | 3 | p | 32 | N/A | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A | NULL |
+| d4 | true | WEEK | -3 | 3 | p | 1 | WEDNESDAY | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A | NULL |
+| d6 | true | MONTH | -2147483648 | 2 | p | 8 | 3rd | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A | NULL |
+| d2 | true | DAY | -3 | 3 | p | 32 | N/A | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A | NULL |
+| d7 | true | MONTH | -2147483648 | 5 | p | 8 | 24th | N/A | 2020-05-25 14:29:24 | NORMAL | N/A | N/A | NULL |
++-----------+--------+----------+-------------+------+--------+---------+-----------+----------------+---------------------+--------+------------------------+----------------------+-------------------------+
7 rows in set (0.02 sec)
```
diff --git a/docs/zh-CN/administrator-guide/operation/doris-error-code.md b/docs/zh-CN/administrator-guide/operation/doris-error-code.md
index 07123ed..122c0d1 100644
--- a/docs/zh-CN/administrator-guide/operation/doris-error-code.md
+++ b/docs/zh-CN/administrator-guide/operation/doris-error-code.md
@@ -172,4 +172,8 @@ under the License.
| 5072 | 动态分区副本值不是有效的数字 |
| 5073 | 原始创建表stmt为空 |
| 5074 | 创建历史动态分区参数：create_history_partition无效，期望的是：true或者false |
+| 5076 | 指定的保留历史分区时间段为空 |
+| 5077 | 指定的保留历史分区时间段无效 |
+| 5078 | 指定的保留历史分区时间段必须是成对的时间 |
+| 5079 | 指定的保留历史分区时间段对应位置的第一个时间比第二个时间大（起始时间大于结束时间） |
diff --git a/docs/zh-CN/sql-reference-v2/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE.md b/docs/zh-CN/sql-reference-v2/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE.md
index d766a62..7e393ed 100644
--- a/docs/zh-CN/sql-reference-v2/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE.md
+++ b/docs/zh-CN/sql-reference-v2/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE.md
@@ -295,6 +295,7 @@ distribution_info
* `dynamic_partition.buckets`: 用于指定自动创建的分区分桶数量。
* `dynamic_partition.create_history_partition`: 是否创建历史分区。
* `dynamic_partition.history_partition_num`: 指定创建历史分区的数量。
+ * `dynamic_partition.reserved_history_periods`: 用于指定保留的历史分区的时间段。
### Example
diff --git a/docs/zh-CN/sql-reference/sql-statements/Data Definition/CREATE TABLE.md b/docs/zh-CN/sql-reference/sql-statements/Data Definition/CREATE TABLE.md
index 2b01e35..9623a5d 100644
--- a/docs/zh-CN/sql-reference/sql-statements/Data Definition/CREATE TABLE.md
+++ b/docs/zh-CN/sql-reference/sql-statements/Data Definition/CREATE TABLE.md
@@ -336,6 +336,7 @@ under the License.
dynamic_partition.buckets: 用于指定自动创建的分区分桶数量
dynamic_partition.create_history_partition: 用于创建历史分区功能是否开启。默认为 false。
dynamic_partition.history_partition_num: 当开启创建历史分区功能时，用于指定创建历史分区数量。
+ dynamic_partition.reserved_history_periods: 用于指定保留的历史分区的时间段。
5) 建表时可以批量创建多个 Rollup
语法：
diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowDynamicPartitionStmt.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowDynamicPartitionStmt.java
index ab8994a..a80e8e5 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowDynamicPartitionStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowDynamicPartitionStmt.java
@@ -46,6 +46,7 @@ public class ShowDynamicPartitionStmt extends ShowStmt {
.addColumn(new Column("State", ScalarType.createVarchar(20)))
.addColumn(new Column("LastCreatePartitionMsg", ScalarType.createVarchar(20)))
.addColumn(new Column("LastDropPartitionMsg", ScalarType.createVarchar(20)))
+ .addColumn(new Column("ReservedHistoryPeriods", ScalarType.createVarchar(20)))
.build();
ShowDynamicPartitionStmt(String db) {
@@ -95,4 +96,4 @@ public class ShowDynamicPartitionStmt extends ShowStmt {
public RedirectStatus getRedirectStatus() {
return RedirectStatus.FORWARD_NO_SYNC;
}
-}
\ No newline at end of file
+}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/DynamicPartitionProperty.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/DynamicPartitionProperty.java
index ecbe7f2..ca8bf90 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/DynamicPartitionProperty.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/DynamicPartitionProperty.java
@@ -19,7 +19,9 @@ package org.apache.doris.catalog;
import org.apache.doris.analysis.TimestampArithmeticExpr.TimeUnit;
import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.DdlException;
import org.apache.doris.common.FeConstants;
+import org.apache.doris.common.util.DynamicPartitionUtil;
import org.apache.doris.common.util.DynamicPartitionUtil.StartOfDate;
import org.apache.doris.common.util.PropertyAnalyzer;
import org.apache.doris.common.util.TimeUtils;
@@ -43,11 +45,13 @@ public class DynamicPartitionProperty {
public static final String CREATE_HISTORY_PARTITION = "dynamic_partition.create_history_partition";
public static final String HISTORY_PARTITION_NUM = "dynamic_partition.history_partition_num";
public static final String HOT_PARTITION_NUM = "dynamic_partition.hot_partition_num";
+ public static final String RESERVED_HISTORY_PERIODS = "dynamic_partition.reserved_history_periods";
public static final int MIN_START_OFFSET = Integer.MIN_VALUE;
public static final int MAX_END_OFFSET = Integer.MAX_VALUE;
public static final int NOT_SET_REPLICATION_NUM = -1;
public static final int NOT_SET_HISTORY_PARTITION_NUM = -1;
+ public static final String NOT_SET_RESERVED_HISTORY_PERIODS = "NULL";
private boolean exist;
@@ -67,6 +71,7 @@ public class DynamicPartitionProperty {
// This property are used to describe the number of partitions that need to be reserved on the high-speed storage.
// If not set, default is 0
private int hotPartitionNum;
+ private String reservedHistoryPeriods;
public DynamicPartitionProperty(Map<String, String> properties) {
if (properties != null && !properties.isEmpty()) {
@@ -83,6 +88,7 @@ public class DynamicPartitionProperty {
this.createHistoryPartition = Boolean.parseBoolean(properties.get(CREATE_HISTORY_PARTITION));
this.historyPartitionNum = Integer.parseInt(properties.getOrDefault(HISTORY_PARTITION_NUM, String.valueOf(NOT_SET_HISTORY_PARTITION_NUM)));
this.hotPartitionNum = Integer.parseInt(properties.getOrDefault(HOT_PARTITION_NUM, "0"));
+ this.reservedHistoryPeriods = properties.getOrDefault(RESERVED_HISTORY_PERIODS, NOT_SET_RESERVED_HISTORY_PERIODS);
createStartOfs(properties);
} else {
this.exist = false;
@@ -180,6 +186,14 @@ public class DynamicPartitionProperty {
return replicaAlloc;
}
+ public String getReservedHistoryPeriods() {
+ return reservedHistoryPeriods;
+ }
+
+ public String getSortedReservedHistoryPeriods(String reservedHistoryPeriods, String timeUnit) throws DdlException {
+ return DynamicPartitionUtil.sortedListedToString(reservedHistoryPeriods, timeUnit);
+ }
+
/**
* use table replication_num as dynamic_partition.replication_num default value
*/
@@ -195,7 +209,8 @@ public class DynamicPartitionProperty {
",\n\"" + BUCKETS + "\" = \"" + buckets + "\"" +
",\n\"" + CREATE_HISTORY_PARTITION + "\" = \"" + createHistoryPartition + "\"" +
",\n\"" + HISTORY_PARTITION_NUM + "\" = \"" + historyPartitionNum + "\"" +
- ",\n\"" + HOT_PARTITION_NUM + "\" = \"" + hotPartitionNum + "\"";
+ ",\n\"" + HOT_PARTITION_NUM + "\" = \"" + hotPartitionNum + "\"" +
+ ",\n\"" + RESERVED_HISTORY_PERIODS + "\" = \"" + reservedHistoryPeriods + "\"";
if (getTimeUnit().equalsIgnoreCase(TimeUnit.WEEK.toString())) {
res += ",\n\"" + START_DAY_OF_WEEK + "\" = \"" + startOfWeek.dayOfWeek + "\"";
} else if (getTimeUnit().equalsIgnoreCase(TimeUnit.MONTH.toString())) {
diff --git a/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java b/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java
index 62905a7..4a6fc1d 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java
@@ -245,11 +245,29 @@ public class DynamicPartitionScheduler extends MasterDaemon {
partitionProperties.put(PropertyAnalyzer.PROPERTIES_STORAGE_COLDOWN_TIME, cooldownTime);
}
+ private Range<PartitionKey> getClosedRange(Database db, OlapTable olapTable, Column partitionColumn, String partitionFormat,
+ String lowerBorderOfReservedHistory, String upperBorderOfReservedHistory) {
+ Range<PartitionKey> reservedHistoryPartitionKeyRange = null;
+ PartitionValue lowerBorderPartitionValue = new PartitionValue(lowerBorderOfReservedHistory);
+ PartitionValue upperBorderPartitionValue = new PartitionValue(upperBorderOfReservedHistory);
+ try {
+ PartitionKey lowerBorderBound = PartitionKey.createPartitionKey(Collections.singletonList(lowerBorderPartitionValue), Collections.singletonList(partitionColumn));
+ PartitionKey upperBorderBound = PartitionKey.createPartitionKey(Collections.singletonList(upperBorderPartitionValue), Collections.singletonList(partitionColumn));
+ reservedHistoryPartitionKeyRange = Range.closed(lowerBorderBound, upperBorderBound);
+ } catch (AnalysisException e) {
+ // AnalysisException: keys.size is always equal to column.size, cannot reach this exception
+ // IllegalArgumentException: lb is greater than ub
+ LOG.warn("Error in gen reservePartitionKeyRange. Error={}, db: {}, table: {}", e.getMessage(),
+ db.getFullName(), olapTable.getName());
+ }
+ return reservedHistoryPartitionKeyRange;
+ }
+
/**
* 1. get the range of [start, 0) as a reserved range.
* 2. get DropPartitionClause of partitions which range are before this reserved range.
*/
- private ArrayList<DropPartitionClause> getDropPartitionClause(Database db, OlapTable olapTable, Column partitionColumn, String partitionFormat) {
+ private ArrayList<DropPartitionClause> getDropPartitionClause(Database db, OlapTable olapTable, Column partitionColumn, String partitionFormat) throws DdlException {
ArrayList<DropPartitionClause> dropPartitionClauses = new ArrayList<>();
DynamicPartitionProperty dynamicPartitionProperty = olapTable.getTableProperty().getDynamicPartitionProperty();
if (dynamicPartitionProperty.getStart() == DynamicPartitionProperty.MIN_START_OFFSET) {
@@ -261,14 +279,16 @@ public class DynamicPartitionScheduler extends MasterDaemon {
String lowerBorder = DynamicPartitionUtil.getPartitionRangeString(dynamicPartitionProperty,
now, dynamicPartitionProperty.getStart(), partitionFormat);
String upperBorder = DynamicPartitionUtil.getPartitionRangeString(dynamicPartitionProperty,
- now, 0, partitionFormat);
+ now, dynamicPartitionProperty.getEnd() + 1, partitionFormat);
PartitionValue lowerPartitionValue = new PartitionValue(lowerBorder);
PartitionValue upperPartitionValue = new PartitionValue(upperBorder);
+ List<Range<PartitionKey>> reservedHistoryPartitionKeyRangeList = new ArrayList<Range<PartitionKey>>();
Range<PartitionKey> reservePartitionKeyRange;
try {
PartitionKey lowerBound = PartitionKey.createPartitionKey(Collections.singletonList(lowerPartitionValue), Collections.singletonList(partitionColumn));
PartitionKey upperBound = PartitionKey.createPartitionKey(Collections.singletonList(upperPartitionValue), Collections.singletonList(partitionColumn));
reservePartitionKeyRange = Range.closedOpen(lowerBound, upperBound);
+ reservedHistoryPartitionKeyRangeList.add(reservePartitionKeyRange);
} catch (AnalysisException | IllegalArgumentException e) {
// AnalysisException: keys.size is always equal to column.size, cannot reach this exception
// IllegalArgumentException: lb is greater than ub
@@ -276,23 +296,44 @@ public class DynamicPartitionScheduler extends MasterDaemon {
db.getFullName(), olapTable.getName());
return dropPartitionClauses;
}
+
+ String reservedHistoryPeriods = dynamicPartitionProperty.getReservedHistoryPeriods();
+ List<Range> ranges = DynamicPartitionUtil.convertStringToPeriodsList(reservedHistoryPeriods, dynamicPartitionProperty.getTimeUnit());
+
+ if (ranges.size() != 0) {
+ for (Range range : ranges) {
+ try {
+ String lowerBorderOfReservedHistory = DynamicPartitionUtil.getHistoryPartitionRangeString(dynamicPartitionProperty, range.lowerEndpoint().toString(), partitionFormat);
+ String upperBorderOfReservedHistory = DynamicPartitionUtil.getHistoryPartitionRangeString(dynamicPartitionProperty, range.upperEndpoint().toString(), partitionFormat);
+ Range<PartitionKey> reservedHistoryPartitionKeyRange = getClosedRange(db, olapTable, partitionColumn, partitionFormat, lowerBorderOfReservedHistory, upperBorderOfReservedHistory);
+ reservedHistoryPartitionKeyRangeList.add(reservedHistoryPartitionKeyRange);
+ } catch (IllegalArgumentException e) {
+ return dropPartitionClauses;
+ }
+ }
+ }
RangePartitionInfo info = (RangePartitionInfo) (olapTable.getPartitionInfo());
List<Map.Entry<Long, PartitionItem>> idToItems = new ArrayList<>(info.getIdToItem(false).entrySet());
idToItems.sort(Comparator.comparing(o -> ((RangePartitionItem) o.getValue()).getItems().upperEndpoint()));
+ Map<Long, Boolean> isContaineds = new HashMap<>();
for (Map.Entry<Long, PartitionItem> idToItem : idToItems) {
- try {
- Long checkDropPartitionId = idToItem.getKey();
- Range<PartitionKey> checkDropPartitionKey = idToItem.getValue().getItems();
- RangeUtils.checkRangeIntersect(reservePartitionKeyRange, checkDropPartitionKey);
- if (checkDropPartitionKey.upperEndpoint().compareTo(reservePartitionKeyRange.lowerEndpoint()) <= 0) {
- String dropPartitionName = olapTable.getPartition(checkDropPartitionId).getName();
- // Do not drop the partition "by force", or the partition will be dropped directly instread of being in
- // catalog recycle bin. This is for safe reason.
- dropPartitionClauses.add(new DropPartitionClause(false, dropPartitionName, false, false));
+ isContaineds.put(idToItem.getKey(), false);
+ Long checkDropPartitionId = idToItem.getKey();
+ Range<PartitionKey> checkDropPartitionKey = idToItem.getValue().getItems();
+ for (Range<PartitionKey> reserveHistoryPartitionKeyRange : reservedHistoryPartitionKeyRangeList) {
+ if (RangeUtils.checkIsTwoRangesIntersect(reserveHistoryPartitionKeyRange, checkDropPartitionKey)) {
+ isContaineds.put(checkDropPartitionId, true);
}
- } catch (DdlException e) {
- break;
+ }
+ }
+
+ for (Long dropPartitionId : isContaineds.keySet()) {
+ // Do not drop the partition "by force", or the partition will be dropped directly instread of being in
+ // catalog recycle bin. This is for safe reason.
+ if(!isContaineds.get(dropPartitionId)) {
+ String dropPartitionName = olapTable.getPartition(dropPartitionId).getName();
+ dropPartitionClauses.add(new DropPartitionClause(false, dropPartitionName, false, false));
}
}
return dropPartitionClauses;
@@ -311,8 +352,8 @@ public class DynamicPartitionScheduler extends MasterDaemon {
}
ArrayList<AddPartitionClause> addPartitionClauses = new ArrayList<>();
- ArrayList<DropPartitionClause> dropPartitionClauses;
- String tableName;
+ ArrayList<DropPartitionClause> dropPartitionClauses = null;
+ String tableName = null;
boolean skipAddPartition = false;
OlapTable olapTable;
olapTable = (OlapTable) db.getTableNullable(tableId);
@@ -358,6 +399,8 @@ public class DynamicPartitionScheduler extends MasterDaemon {
}
dropPartitionClauses = getDropPartitionClause(db, olapTable, partitionColumn, partitionFormat);
tableName = olapTable.getName();
+ } catch (DdlException e) {
+ e.printStackTrace();
} finally {
olapTable.readUnlock();
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/ErrorCode.java b/fe/fe-core/src/main/java/org/apache/doris/common/ErrorCode.java
index d6497dd..69b9bae 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/ErrorCode.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/ErrorCode.java
@@ -244,6 +244,14 @@ public enum ErrorCode {
"Invalid dynamic partition create_history_partition: %s. Expected true or false"),
ERROR_DYNAMIC_PARTITION_HISTORY_PARTITION_NUM_ZERO(5075, new byte[] {'4', '2', '0', '0', '0'},
"Dynamic history partition num must greater than 0"),
+ ERROR_DYNAMIC_PARTITION_RESERVED_HISTORY_PERIODS_EMPTY(5076, new byte[] {'4', '2', '0', '0', '0'},
+ "Dynamic reserved history periods is empty."),
+ ERROR_DYNAMIC_PARTITION_RESERVED_HISTORY_PERIODS_INVALID(5077, new byte[] {'4', '2', '0', '0', '0'},
+ "Invalid \" %s \" value %s. It must be like \"[yyyy-MM-dd,yyyy-MM-dd],[...,...]\" while time_unit is DAY/WEEK/MONTH or \"[yyyy-MM-dd HH:mm:ss,yyyy-MM-dd HH:mm:ss],[...,...]\" while time_unit is HOUR."),
+ ERROR_DYNAMIC_PARTITION_RESERVED_HISTORY_PERIODS_START_ENDS_LENGTH_NOT_EQUAL(5078, new byte[] {'4', '2', '0', '0', '0'},
+ "RESERVED_HISTORY_PERIODS must have pairs of date value. The input %s is not valid."),
+ ERROR_DYNAMIC_PARTITION_RESERVED_HISTORY_PERIODS_START_LARGER_THAN_ENDS(5079, new byte[] {'4', '2', '0', '0', '0'},
+ "The first date is larger than the second date, [%s,%s] is invalid."),
ERROR_LDAP_CONFIGURATION_ERR(5080, new byte[] {'4', '2', '0', '0', '0'},
"LDAP configuration is incorrect or LDAP admin password is not set."),
ERROR_LDAP_USER_NOT_UNIQUE_ERR(5081, new byte[] {'4', '2', '0', '0', '0'},
diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/util/DynamicPartitionUtil.java b/fe/fe-core/src/main/java/org/apache/doris/common/util/DynamicPartitionUtil.java
index 174673b..1725ff4 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/util/DynamicPartitionUtil.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/util/DynamicPartitionUtil.java
@@ -41,20 +41,30 @@ import org.apache.doris.common.UserException;
import com.google.common.base.Preconditions;
import com.google.common.base.Strings;
+import com.google.common.collect.Range;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
+import java.sql.Timestamp;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.time.DayOfWeek;
import java.time.Month;
+import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;
-import java.util.Calendar;
+import java.util.ArrayList;
+import java.util.Comparator;
import java.util.HashMap;
+import java.util.List;
import java.util.Map;
import java.util.TimeZone;
+import java.util.Date;
+import java.util.Calendar;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
public class DynamicPartitionUtil {
private static final Logger LOG = LogManager.getLogger(DynamicPartitionUtil.class);
@@ -230,6 +240,101 @@ public class DynamicPartitionUtil {
}
}
+ public static List<Range> convertStringToPeriodsList(String reservedHistoryPeriods, String timeUnit) throws DdlException {
+ List<Range> reservedHistoryPeriodsToRangeList = new ArrayList<Range>();
+ if (DynamicPartitionProperty.NOT_SET_RESERVED_HISTORY_PERIODS.equals(reservedHistoryPeriods)) {
+ return reservedHistoryPeriodsToRangeList;
+ }
+
+ Pattern pattern = getPattern(timeUnit);
+ Matcher matcher = pattern.matcher(reservedHistoryPeriods);
+ while (matcher.find()) {
+ String lowerBorderOfReservedHistory = matcher.group(1);
+ String upperBorderOfReservedHistory = matcher.group(2);
+ if (lowerBorderOfReservedHistory.compareTo(upperBorderOfReservedHistory) > 0) {
+ ErrorReport.reportDdlException(ErrorCode.ERROR_DYNAMIC_PARTITION_RESERVED_HISTORY_PERIODS_START_LARGER_THAN_ENDS, lowerBorderOfReservedHistory, upperBorderOfReservedHistory);
+ } else {
+ reservedHistoryPeriodsToRangeList.add(Range.closed(lowerBorderOfReservedHistory, upperBorderOfReservedHistory));
+ }
+ }
+ return reservedHistoryPeriodsToRangeList;
+ }
+
+ private static Pattern getPattern(String timeUnit) {
+ if (timeUnit.equalsIgnoreCase(TimeUnit.HOUR.toString())) {
+ return Pattern.compile("\\[([0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}),([0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2})\\]");
+ } else {
+ return Pattern.compile("\\[([0-9]{4}-[0-9]{2}-[0-9]{2}),([0-9]{4}-[0-9]{2}-[0-9]{2})\\]");
+ }
+ }
+
+ public static String sortedListedToString(String reservedHistoryPeriods, String timeUnit) throws DdlException {
+ if (DynamicPartitionProperty.NOT_SET_RESERVED_HISTORY_PERIODS.equals(reservedHistoryPeriods)) {
+ return reservedHistoryPeriods;
+ }
+ List<Range> reservedHistoryPeriodsToRangeList = convertStringToPeriodsList(reservedHistoryPeriods, timeUnit);
+ reservedHistoryPeriodsToRangeList.sort(new Comparator<Range>() {
+ @Override
+ public int compare(Range o1, Range o2) {
+ return o1.lowerEndpoint().compareTo(o2.lowerEndpoint());
+ }
+ });
+ List<String> sortedReservedHistoryPeriods = reservedHistoryPeriodsToRangeList.stream().
+ map(e -> "[" + e.lowerEndpoint() + "," + e.upperEndpoint() + "]").collect(Collectors.toList());
+
+ return String.join(",", sortedReservedHistoryPeriods);
+ }
+
+ private static void checkReservedHistoryPeriodValidate(String reservedHistoryPeriods, String timeUnit) throws DdlException {
+ if (Strings.isNullOrEmpty(reservedHistoryPeriods)) {
+ ErrorReport.reportDdlException(ErrorCode.ERROR_DYNAMIC_PARTITION_RESERVED_HISTORY_PERIODS_EMPTY);
+ }
+ if (DynamicPartitionProperty.NOT_SET_RESERVED_HISTORY_PERIODS.equals(reservedHistoryPeriods)) {
+ return;
+ }
+ // it has 5 kinds of situation
+ // 1. "dynamic_partition.reserved_history_periods" = "[2021-07-01,]," invalid one
+ // 2. "dynamic_partition.reserved_history_periods" = "2021-07-01", invalid. It must be surrounded by []
+ // 1. "dynamic_partition.reserved_history_periods" = "[2021-07-01,]" invalid one, needs pairs of values
+ // 2. "dynamic_partition.reserved_history_periods" = "[,2021-08-01]" invalid one, needs pairs of values
+ // 3. "dynamic_partition.reserved_history_periods" = "[2021-07-01,2020-08-01,]" invalid format
+ if (!reservedHistoryPeriods.startsWith("[") || !reservedHistoryPeriods.endsWith("]")) {
+ ErrorReport.reportDdlException(ErrorCode.ERROR_DYNAMIC_PARTITION_RESERVED_HISTORY_PERIODS_INVALID, DynamicPartitionProperty.RESERVED_HISTORY_PERIODS, reservedHistoryPeriods);
+ }
+
+ List<Range> reservedHistoryPeriodsToRangeList = convertStringToPeriodsList(reservedHistoryPeriods, timeUnit);
+ Integer sizeOfPeriods = reservedHistoryPeriods.split("],\\[").length;
+ SimpleDateFormat sdf = getSimpleDateFormat(timeUnit);
+
+ if (reservedHistoryPeriodsToRangeList.size() != sizeOfPeriods) {
+ ErrorReport.reportDdlException(ErrorCode.ERROR_DYNAMIC_PARTITION_RESERVED_HISTORY_PERIODS_INVALID, DynamicPartitionProperty.RESERVED_HISTORY_PERIODS, reservedHistoryPeriods);
+ } else {
+ try {
+ for (Range range : reservedHistoryPeriodsToRangeList) {
+ String formattedLowerBound = sdf.format(sdf.parse(range.lowerEndpoint().toString()));
+ String formattedUpperBound = sdf.format(sdf.parse(range.upperEndpoint().toString()));
+ if (!range.lowerEndpoint().toString().equals(formattedLowerBound) || !range.upperEndpoint().toString().equals(formattedUpperBound)) {
+ throw new DdlException("Invalid " + DynamicPartitionProperty.RESERVED_HISTORY_PERIODS +
+ " value. It must be correct DATE value \"[yyyy-MM-dd,yyyy-MM-dd],[...,...]\" while time_unit is DAY/WEEK/MONTH " +
+ "or \"[yyyy-MM-dd HH:mm:ss,yyyy-MM-dd HH:mm:ss],[...,...]\" while time_unit is HOUR.");
+ }
+ }
+ } catch (ParseException e) {
+ throw new DdlException("Invalid " + DynamicPartitionProperty.RESERVED_HISTORY_PERIODS +
+ " value. It must be like \"[yyyy-MM-dd,yyyy-MM-dd],[...,...]\" while time_unit is DAY/WEEK/MONTH " +
+ "or \"[yyyy-MM-dd HH:mm:ss,yyyy-MM-dd HH:mm:ss],[...,...]\" while time_unit is HOUR.");
+ }
+ }
+ }
+
+ private static SimpleDateFormat getSimpleDateFormat(String timeUnit) {
+ if (timeUnit.equalsIgnoreCase(TimeUnit.HOUR.toString())) {
+ return new SimpleDateFormat(DATETIME_FORMAT);
+ } else {
+ return new SimpleDateFormat(DATE_FORMAT);
+ }
+ }
+
public static boolean checkDynamicPartitionPropertiesExist(Map<String, String> properties) {
if (properties == null) {
return false;
@@ -261,6 +366,8 @@ public class DynamicPartitionUtil {
String enable = properties.get(DynamicPartitionProperty.ENABLE);
String createHistoryPartition = properties.get(DynamicPartitionProperty.CREATE_HISTORY_PARTITION);
String historyPartitionNum = properties.get(DynamicPartitionProperty.HISTORY_PARTITION_NUM);
+ String reservedHistoryPeriods = properties.get(DynamicPartitionProperty.RESERVED_HISTORY_PERIODS);
+
if (!(Strings.isNullOrEmpty(enable) &&
Strings.isNullOrEmpty(timeUnit) &&
Strings.isNullOrEmpty(timeZone) &&
@@ -269,7 +376,8 @@ public class DynamicPartitionUtil {
Strings.isNullOrEmpty(end) &&
Strings.isNullOrEmpty(buckets) &&
Strings.isNullOrEmpty(createHistoryPartition) &&
- Strings.isNullOrEmpty(historyPartitionNum))) {
+ Strings.isNullOrEmpty(historyPartitionNum) &&
+ Strings.isNullOrEmpty(reservedHistoryPeriods))) {
if (Strings.isNullOrEmpty(enable)) {
properties.put(DynamicPartitionProperty.ENABLE, "true");
}
@@ -298,6 +406,10 @@ public class DynamicPartitionUtil {
properties.put(DynamicPartitionProperty.HISTORY_PARTITION_NUM,
String.valueOf(DynamicPartitionProperty.NOT_SET_HISTORY_PARTITION_NUM));
}
+ if (Strings.isNullOrEmpty(reservedHistoryPeriods)) {
+ properties.put(DynamicPartitionProperty.RESERVED_HISTORY_PERIODS,
+ String.valueOf(DynamicPartitionProperty.NOT_SET_RESERVED_HISTORY_PERIODS));
+ }
}
return true;
}
@@ -450,7 +562,12 @@ public class DynamicPartitionUtil {
properties.remove(DynamicPartitionProperty.HOT_PARTITION_NUM);
analyzedProperties.put(DynamicPartitionProperty.HOT_PARTITION_NUM, val);
}
-
+ if (properties.containsKey(DynamicPartitionProperty.RESERVED_HISTORY_PERIODS)) {
+ String reservedHistoryPeriods = properties.get(DynamicPartitionProperty.RESERVED_HISTORY_PERIODS);
+ checkReservedHistoryPeriodValidate(reservedHistoryPeriods, analyzedProperties.get(DynamicPartitionProperty.TIME_UNIT));
+ properties.remove(DynamicPartitionProperty.RESERVED_HISTORY_PERIODS);
+ analyzedProperties.put(DynamicPartitionProperty.RESERVED_HISTORY_PERIODS, reservedHistoryPeriods);
+ }
return analyzedProperties;
}
@@ -553,6 +670,23 @@ public class DynamicPartitionUtil {
}
}
+ public static String getHistoryPartitionRangeString(DynamicPartitionProperty dynamicPartitionProperty, String time, String format) {
+ ZoneId zoneId = dynamicPartitionProperty.getTimeZone().toZoneId();
+ Date date = null;
+ Timestamp timestamp = null;
+ String timeUnit = dynamicPartitionProperty.getTimeUnit();
+ DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.s").withZone(zoneId);
+ SimpleDateFormat simpleDateFormat = getSimpleDateFormat(timeUnit);
+ try {
+ date = simpleDateFormat.parse(time);
+ } catch (ParseException e) {
+ LOG.warn("Parse dynamic partition periods error. Error={}", e.getMessage());
+ return getFormattedTimeWithoutMinuteSecond(ZonedDateTime.parse(timestamp.toString(), dateTimeFormatter), format);
+ }
+ timestamp = new Timestamp(date.getTime());
+ return getFormattedTimeWithoutMinuteSecond(ZonedDateTime.parse(timestamp.toString(), dateTimeFormatter), format);
+ }
+
/**
* return formatted string of partition range in HOUR granularity.
* offset: The offset from the current hour. 0 means current hour, 1 means next hour, -1 last hour.
diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/util/RangeUtils.java b/fe/fe-core/src/main/java/org/apache/doris/common/util/RangeUtils.java
index eeef22e..13e0f5b 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/util/RangeUtils.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/util/RangeUtils.java
@@ -51,6 +51,12 @@ public class RangeUtils {
}
}
+ public static boolean checkIsTwoRangesIntersect(Range<PartitionKey> range1, Range<PartitionKey> range2) {
+ if (range2.isConnected(range1) && !range2.intersection(range1).isEmpty()) {
+ return true;
+ }
+ return false;
+ }
/*
* Pass only if the 2 range lists are exactly same
* What is "exactly same"?
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java b/fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java
index e7f7003..195c293 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java
@@ -1820,6 +1820,7 @@ public class ShowExecutor {
if (replicaAlloc.isNotSet()) {
replicaAlloc = olapTable.getDefaultReplicaAllocation();
}
+ String unsortedReservedHistoryPeriods = dynamicPartitionProperty.getReservedHistoryPeriods();
rows.add(Lists.newArrayList(
tableName,
String.valueOf(dynamicPartitionProperty.getEnable()),
@@ -1835,7 +1836,10 @@ public class ShowExecutor {
dynamicPartitionScheduler.getRuntimeInfo(olapTable.getId(), DynamicPartitionScheduler.LAST_SCHEDULER_TIME),
dynamicPartitionScheduler.getRuntimeInfo(olapTable.getId(), DynamicPartitionScheduler.DYNAMIC_PARTITION_STATE),
dynamicPartitionScheduler.getRuntimeInfo(olapTable.getId(), DynamicPartitionScheduler.CREATE_PARTITION_MSG),
- dynamicPartitionScheduler.getRuntimeInfo(olapTable.getId(), DynamicPartitionScheduler.DROP_PARTITION_MSG)));
+ dynamicPartitionScheduler.getRuntimeInfo(olapTable.getId(), DynamicPartitionScheduler.DROP_PARTITION_MSG),
+ dynamicPartitionProperty.getSortedReservedHistoryPeriods(unsortedReservedHistoryPeriods, dynamicPartitionProperty.getTimeUnit().toUpperCase())));
+ } catch (DdlException e) {
+ e.printStackTrace();
} finally {
olapTable.readUnlock();
}
diff --git a/fe/fe-core/src/test/java/org/apache/doris/catalog/DynamicPartitionTableTest.java b/fe/fe-core/src/test/java/org/apache/doris/catalog/DynamicPartitionTableTest.java
index efc2685..6c36ba8 100644
--- a/fe/fe-core/src/test/java/org/apache/doris/catalog/DynamicPartitionTableTest.java
+++ b/fe/fe-core/src/test/java/org/apache/doris/catalog/DynamicPartitionTableTest.java
@@ -1186,4 +1186,340 @@ public class DynamicPartitionTableTest {
scheduler.removeRuntimeInfo(tableId);
Assert.assertTrue(scheduler.getRuntimeInfo(tableId, key1) == FeConstants.null_string);
}
+
+ @Test
+ public void testMissReservedHistoryPeriods() throws Exception {
+ String createOlapTblStmt = "CREATE TABLE test.`dynamic_partition_miss_reserved_history_periods` (\n" +
+ " `k1` date NULL COMMENT \"\",\n" +
+ " `k2` int NULL COMMENT \"\",\n" +
+ " `k3` smallint NULL COMMENT \"\",\n" +
+ " `v1` varchar(2048) NULL COMMENT \"\",\n" +
+ " `v2` datetime NULL COMMENT \"\"\n" +
+ ") ENGINE=OLAP\n" +
+ "DUPLICATE KEY(`k1`, `k2`, `k3`)\n" +
+ "COMMENT \"OLAP\"\n" +
+ "PARTITION BY RANGE (k1)\n" +
+ "(\n" +
+ "PARTITION p1 VALUES LESS THAN (\"2014-01-01\"),\n" +
+ "PARTITION p2 VALUES LESS THAN (\"2014-06-01\"),\n" +
+ "PARTITION p3 VALUES LESS THAN (\"2014-12-01\")\n" +
+ ")\n" +
+ "DISTRIBUTED BY HASH(`k1`) BUCKETS 32\n" +
+ "PROPERTIES (\n" +
+ "\"replication_num\" = \"1\",\n" +
+ "\"dynamic_partition.enable\" = \"true\",\n" +
+ "\"dynamic_partition.start\" = \"-3\",\n" +
+ "\"dynamic_partition.end\" = \"3\",\n" +
+ "\"dynamic_partition.buckets\" = \"3\",\n" +
+ "\"dynamic_partition.time_unit\" = \"day\",\n" +
+ "\"dynamic_partition.prefix\" = \"p\"\n" +
+ ");";
+ createTable(createOlapTblStmt);
+ OlapTable table = (OlapTable) Catalog.getCurrentCatalog().getDbOrAnalysisException("default_cluster:test").getTableOrAnalysisException("dynamic_partition_miss_reserved_history_periods");
+ Assert.assertEquals("NULL", table.getTableProperty().getDynamicPartitionProperty().getReservedHistoryPeriods());
+ }
+
+ @Test
+ public void testNormalReservedHisrotyPeriods() throws Exception {
+ String createOlapTblStmt = "CREATE TABLE test.`dynamic_partition_normal_reserved_history_periods` (\n" +
+ " `k1` date NULL COMMENT \"\",\n" +
+ " `k2` int NULL COMMENT \"\",\n" +
+ " `k3` smallint NULL COMMENT \"\",\n" +
+ " `v1` varchar(2048) NULL COMMENT \"\",\n" +
+ " `v2` datetime NULL COMMENT \"\"\n" +
+ ") ENGINE=OLAP\n" +
+ "DUPLICATE KEY(`k1`, `k2`, `k3`)\n" +
+ "COMMENT \"OLAP\"\n" +
+ "PARTITION BY RANGE (k1)\n" +
+ "(\n" +
+ "PARTITION p1 VALUES LESS THAN (\"2014-01-01\"),\n" +
+ "PARTITION p2 VALUES LESS THAN (\"2014-06-01\"),\n" +
+ "PARTITION p3 VALUES LESS THAN (\"2014-12-01\"),\n" +
+ "PARTITION p4 VALUES LESS THAN (\"2020-06-01\"),\n" +
+ "PARTITION p5 VALUES LESS THAN (\"2020-06-20\"),\n" +
+ "PARTITION p6 VALUES LESS THAN (\"2020-10-25\"),\n" +
+ "PARTITION p7 VALUES LESS THAN (\"2020-11-01\"),\n" +
+ "PARTITION p8 VALUES LESS THAN (\"2020-11-11\"),\n" +
+ "PARTITION p9 VALUES LESS THAN (\"2020-11-21\"),\n" +
+ "PARTITION p10 VALUES LESS THAN (\"2021-04-20\"),\n" +
+ "PARTITION p11 VALUES LESS THAN (\"2021-05-20\")\n" +
+ ")\n" +
+ "DISTRIBUTED BY HASH(`k1`) BUCKETS 32\n" +
+ "PROPERTIES (\n" +
+ "\"replication_num\" = \"1\",\n" +
+ "\"dynamic_partition.enable\" = \"true\",\n" +
+ "\"dynamic_partition.start\" = \"-3\",\n" +
+ "\"dynamic_partition.end\" = \"3\",\n" +
+ "\"dynamic_partition.buckets\" = \"3\",\n" +
+ "\"dynamic_partition.time_unit\" = \"day\",\n" +
+ "\"dynamic_partition.prefix\" = \"p\",\n" +
+ "\"dynamic_partition.reserved_history_periods\" = \"[2020-06-01,2020-06-20],[2020-10-25,2020-11-15],[2021-06-01,2021-06-20]\"\n" +
+ ");";
+ createTable(createOlapTblStmt);
+ OlapTable table = (OlapTable) Catalog.getCurrentCatalog().getDbOrAnalysisException("default_cluster:test").getTableOrAnalysisException("dynamic_partition_normal_reserved_history_periods");
+ Assert.assertEquals("[2020-06-01,2020-06-20],[2020-10-25,2020-11-15],[2021-06-01,2021-06-20]", table.getTableProperty().getDynamicPartitionProperty().getReservedHistoryPeriods());
+ Assert.assertEquals(table.getAllPartitions().size(), 9);
+
+ String createOlapTblStmt2 = "CREATE TABLE test.`dynamic_partition_normal_reserved_history_periods2` (\n" +
+ " `k1` datetime NULL COMMENT \"\",\n" +
+ " `k2` int NULL COMMENT \"\",\n" +
+ " `k3` smallint NULL COMMENT \"\",\n" +
+ " `v1` varchar(2048) NULL COMMENT \"\",\n" +
+ " `v2` datetime NULL COMMENT \"\"\n" +
+ ") ENGINE=OLAP\n" +
+ "DUPLICATE KEY(`k1`, `k2`, `k3`)\n" +
+ "COMMENT \"OLAP\"\n" +
+ "PARTITION BY RANGE (k1)\n" +
+ "(\n" +
+ "PARTITION p1 VALUES LESS THAN (\"2014-01-01 00:00:00\"),\n" +
+ "PARTITION p2 VALUES LESS THAN (\"2014-01-01 03:00:00\"),\n" +
+ "PARTITION p3 VALUES LESS THAN (\"2014-01-01 04:00:00\"),\n" +
+ "PARTITION p4 VALUES LESS THAN (\"2020-01-01 08:00:00\"),\n" +
+ "PARTITION p5 VALUES LESS THAN (\"2020-06-20 00:00:00\")\n" +
+ ")\n" +
+ "DISTRIBUTED BY HASH(`k1`) BUCKETS 32\n" +
+ "PROPERTIES (\n" +
+ "\"replication_num\" = \"1\",\n" +
+ "\"dynamic_partition.enable\" = \"true\",\n" +
+ "\"dynamic_partition.start\" = \"-3\",\n" +
+ "\"dynamic_partition.end\" = \"3\",\n" +
+ "\"dynamic_partition.buckets\" = \"3\",\n" +
+ "\"dynamic_partition.time_unit\" = \"hour\",\n" +
+ "\"dynamic_partition.prefix\" = \"p\",\n" +
+ "\"dynamic_partition.reserved_history_periods\" = \"[2014-01-01 00:00:00,2014-01-01 03:00:00]\"\n" +
+ ");";
+ createTable(createOlapTblStmt2);
+ OlapTable table2 = (OlapTable) Catalog.getCurrentCatalog().getDbOrAnalysisException("default_cluster:test").getTableOrAnalysisException("dynamic_partition_normal_reserved_history_periods2");
+ Assert.assertEquals("[2014-01-01 00:00:00,2014-01-01 03:00:00]", table2.getTableProperty().getDynamicPartitionProperty().getReservedHistoryPeriods());
+ Assert.assertEquals(table2.getAllPartitions().size(), 6);
+
+ String createOlapTblStmt3 = "CREATE TABLE test.`dynamic_partition_normal_reserved_history_periods3` (\n" +
+ " `k1` int NULL COMMENT \"\",\n" +
+ " `k2` int NULL COMMENT \"\",\n" +
+ " `k3` smallint NULL COMMENT \"\",\n" +
+ " `v1` varchar(2048) NULL COMMENT \"\",\n" +
+ " `v2` datetime NULL COMMENT \"\"\n" +
+ ") ENGINE=OLAP\n" +
+ "DUPLICATE KEY(`k1`, `k2`, `k3`)\n" +
+ "COMMENT \"OLAP\"\n" +
+ "PARTITION BY RANGE (k1)\n" +
+ "(\n" +
+ "PARTITION p202127 VALUES [(\"20200527\"), (\"20200628\"))\n" +
+ ")\n" +
+ "DISTRIBUTED BY HASH(`k2`) BUCKETS 32\n" +
+ "PROPERTIES (\n" +
+ "\"replication_num\" = \"1\",\n" +
+ "\"dynamic_partition.enable\" = \"true\",\n" +
+ "\"dynamic_partition.start\" = \"-3\",\n" +
+ "\"dynamic_partition.end\" = \"3\",\n" +
+ "\"dynamic_partition.buckets\" = \"1\",\n" +
+ "\"dynamic_partition.time_unit\" = \"day\",\n" +
+ "\"dynamic_partition.prefix\" = \"p\",\n" +
+ "\"dynamic_partition.reserved_history_periods\" = \"[2020-06-01,2020-06-30]\"\n" +
+ ");";
+ createTable(createOlapTblStmt3);
+ OlapTable table3 = (OlapTable) Catalog.getCurrentCatalog().getDbOrAnalysisException("default_cluster:test").getTableOrAnalysisException("dynamic_partition_normal_reserved_history_periods3");
+ Assert.assertEquals("[2020-06-01,2020-06-30]", table3.getTableProperty().getDynamicPartitionProperty().getReservedHistoryPeriods());
+ Assert.assertEquals(table3.getAllPartitions().size(), 5);
+ }
+
+ @Test
+ public void testInvalidReservedHistoryPeriods() throws Exception {
+ String createOlapTblStmt1 = "CREATE TABLE test.`dynamic_partition_invalid_reserved_history_periods1` (\n" +
+ " `k1` date NULL COMMENT \"\",\n" +
+ " `k2` int NULL COMMENT \"\",\n" +
+ " `k3` smallint NULL COMMENT \"\",\n" +
+ " `v1` varchar(2048) NULL COMMENT \"\",\n" +
+ " `v2` datetime NULL COMMENT \"\"\n" +
+ ") ENGINE=OLAP\n" +
+ "DUPLICATE KEY(`k1`, `k2`, `k3`)\n" +
+ "COMMENT \"OLAP\"\n" +
+ "PARTITION BY RANGE (k1)\n" +
+ "(\n" +
+ "PARTITION p1 VALUES LESS THAN (\"2014-01-01\"),\n" +
+ "PARTITION p2 VALUES LESS THAN (\"2014-06-01\"),\n" +
+ "PARTITION p3 VALUES LESS THAN (\"2014-12-01\")\n" +
+ ")\n" +
+ "DISTRIBUTED BY HASH(`k1`) BUCKETS 32\n" +
+ "PROPERTIES (\n" +
+ "\"replication_num\" = \"1\",\n" +
+ "\"dynamic_partition.enable\" = \"true\",\n" +
+ "\"dynamic_partition.start\" = \"-3\",\n" +
+ "\"dynamic_partition.end\" = \"3\",\n" +
+ "\"dynamic_partition.buckets\" = \"3\",\n" +
+ "\"dynamic_partition.time_unit\" = \"day\",\n" +
+ "\"dynamic_partition.prefix\" = \"p\",\n" +
+ "\"dynamic_partition.reserved_history_periods\" = \"[20210101,2021-10-10]\"\n" +
+ ");";
+ ExceptionChecker.expectThrowsWithMsg(DdlException.class,
+ "errCode = 2, detailMessage = Invalid \" dynamic_partition.reserved_history_periods \" value [20210101,2021-10-10]. " +
+ "It must be like \"[yyyy-MM-dd,yyyy-MM-dd],[...,...]\" while time_unit is DAY/WEEK/MONTH or " +
+ "\"[yyyy-MM-dd HH:mm:ss,yyyy-MM-dd HH:mm:ss],[...,...]\" while time_unit is HOUR.",
+ () -> createTable(createOlapTblStmt1));
+
+ String createOlapTblStmt2 = "CREATE TABLE test.`dynamic_partition_invalid_reserved_history_periods2` (\n" +
+ " `k1` date NULL COMMENT \"\",\n" +
+ " `k2` int NULL COMMENT \"\",\n" +
+ " `k3` smallint NULL COMMENT \"\",\n" +
+ " `v1` varchar(2048) NULL COMMENT \"\",\n" +
+ " `v2` datetime NULL COMMENT \"\"\n" +
+ ") ENGINE=OLAP\n" +
+ "DUPLICATE KEY(`k1`, `k2`, `k3`)\n" +
+ "COMMENT \"OLAP\"\n" +
+ "PARTITION BY RANGE (k1)\n" +
+ "(\n" +
+ "PARTITION p1 VALUES LESS THAN (\"2014-01-01\"),\n" +
+ "PARTITION p2 VALUES LESS THAN (\"2014-06-01\"),\n" +
+ "PARTITION p3 VALUES LESS THAN (\"2014-12-01\")\n" +
+ ")\n" +
+ "DISTRIBUTED BY HASH(`k1`) BUCKETS 32\n" +
+ "PROPERTIES (\n" +
+ "\"replication_num\" = \"1\",\n" +
+ "\"dynamic_partition.enable\" = \"true\",\n" +
+ "\"dynamic_partition.start\" = \"-3\",\n" +
+ "\"dynamic_partition.end\" = \"3\",\n" +
+ "\"dynamic_partition.buckets\" = \"3\",\n" +
+ "\"dynamic_partition.time_unit\" = \"day\",\n" +
+ "\"dynamic_partition.prefix\" = \"p\",\n" +
+ "\"dynamic_partition.reserved_history_periods\" = \"[0000-00-00,2021-10-10]\"\n" +
+ ");";
+ ExceptionChecker.expectThrowsWithMsg(DdlException.class,
+ "errCode = 2, detailMessage = Invalid dynamic_partition.reserved_history_periods value. " +
+ "It must be correct DATE value " +
+ "\"[yyyy-MM-dd,yyyy-MM-dd],[...,...]\" while time_unit is DAY/WEEK/MONTH or " +
+ "\"[yyyy-MM-dd HH:mm:ss,yyyy-MM-dd HH:mm:ss],[...,...]\" while time_unit is HOUR.",
+ () -> createTable(createOlapTblStmt2));
+ }
+
+ @Test
+ public void testReservedHistoryPeriodsValidate() throws Exception {
+ String createOlapTblStmt1 = "CREATE TABLE test.`dynamic_partition_reserved_history_periods_validate1` (\n" +
+ " `k1` date NULL COMMENT \"\",\n" +
+ " `k2` int NULL COMMENT \"\",\n" +
+ " `k3` smallint NULL COMMENT \"\",\n" +
+ " `v1` varchar(2048) NULL COMMENT \"\",\n" +
+ " `v2` datetime NULL COMMENT \"\"\n" +
+ ") ENGINE=OLAP\n" +
+ "DUPLICATE KEY(`k1`, `k2`, `k3`)\n" +
+ "COMMENT \"OLAP\"\n" +
+ "PARTITION BY RANGE (k1)\n" +
+ "(\n" +
+ "PARTITION p1 VALUES LESS THAN (\"2014-01-01\"),\n" +
+ "PARTITION p2 VALUES LESS THAN (\"2014-06-01\"),\n" +
+ "PARTITION p3 VALUES LESS THAN (\"2014-12-01\")\n" +
+ ")\n" +
+ "DISTRIBUTED BY HASH(`k1`) BUCKETS 32\n" +
+ "PROPERTIES (\n" +
+ "\"replication_num\" = \"1\",\n" +
+ "\"dynamic_partition.enable\" = \"true\",\n" +
+ "\"dynamic_partition.start\" = \"-3\",\n" +
+ "\"dynamic_partition.end\" = \"3\",\n" +
+ "\"dynamic_partition.buckets\" = \"3\",\n" +
+ "\"dynamic_partition.time_unit\" = \"day\",\n" +
+ "\"dynamic_partition.prefix\" = \"p\",\n" +
+ "\"dynamic_partition.reserved_history_periods\" = \"[2021-01-01,]\"\n" +
+ ");";
+ ExceptionChecker.expectThrowsWithMsg(DdlException.class,
+ "errCode = 2, detailMessage = Invalid \" dynamic_partition.reserved_history_periods \" value [2021-01-01,]. " +
+ "It must be like " +
+ "\"[yyyy-MM-dd,yyyy-MM-dd],[...,...]\" while time_unit is DAY/WEEK/MONTH " +
+ "or \"[yyyy-MM-dd HH:mm:ss,yyyy-MM-dd HH:mm:ss],[...,...]\" while time_unit is HOUR.",
+ () -> createTable(createOlapTblStmt1));
+
+ String createOlapTblStmt2 = "CREATE TABLE test.`dynamic_partition_reserved_history_periods_validate2` (\n" +
+ " `k1` date NULL COMMENT \"\",\n" +
+ " `k2` int NULL COMMENT \"\",\n" +
+ " `k3` smallint NULL COMMENT \"\",\n" +
+ " `v1` varchar(2048) NULL COMMENT \"\",\n" +
+ " `v2` datetime NULL COMMENT \"\"\n" +
+ ") ENGINE=OLAP\n" +
+ "DUPLICATE KEY(`k1`, `k2`, `k3`)\n" +
+ "COMMENT \"OLAP\"\n" +
+ "PARTITION BY RANGE (k1)\n" +
+ "(\n" +
+ "PARTITION p1 VALUES LESS THAN (\"2014-01-01\"),\n" +
+ "PARTITION p2 VALUES LESS THAN (\"2014-06-01\"),\n" +
+ "PARTITION p3 VALUES LESS THAN (\"2014-12-01\")\n" +
+ ")\n" +
+ "DISTRIBUTED BY HASH(`k1`) BUCKETS 32\n" +
+ "PROPERTIES (\n" +
+ "\"replication_num\" = \"1\",\n" +
+ "\"dynamic_partition.enable\" = \"true\",\n" +
+ "\"dynamic_partition.start\" = \"-3\",\n" +
+ "\"dynamic_partition.end\" = \"3\",\n" +
+ "\"dynamic_partition.buckets\" = \"3\",\n" +
+ "\"dynamic_partition.time_unit\" = \"day\",\n" +
+ "\"dynamic_partition.prefix\" = \"p\",\n" +
+ "\"dynamic_partition.reserved_history_periods\" = \"[,2021-01-01]\"\n" +
+ ");";
+ ExceptionChecker.expectThrowsWithMsg(DdlException.class,
+ "errCode = 2, detailMessage = Invalid \" dynamic_partition.reserved_history_periods \" value [,2021-01-01]. " +
+ "It must be like " +
+ "\"[yyyy-MM-dd,yyyy-MM-dd],[...,...]\" while time_unit is DAY/WEEK/MONTH or " +
+ "\"[yyyy-MM-dd HH:mm:ss,yyyy-MM-dd HH:mm:ss],[...,...]\" while time_unit is HOUR.",
+ () -> createTable(createOlapTblStmt2));
+
+ String createOlapTblStmt3 = "CREATE TABLE test.`dynamic_partition_reserved_history_periods_validate3` (\n" +
+ " `k1` date NULL COMMENT \"\",\n" +
+ " `k2` int NULL COMMENT \"\",\n" +
+ " `k3` smallint NULL COMMENT \"\",\n" +
+ " `v1` varchar(2048) NULL COMMENT \"\",\n" +
+ " `v2` datetime NULL COMMENT \"\"\n" +
+ ") ENGINE=OLAP\n" +
+ "DUPLICATE KEY(`k1`, `k2`, `k3`)\n" +
+ "COMMENT \"OLAP\"\n" +
+ "PARTITION BY RANGE (k1)\n" +
+ "(\n" +
+ "PARTITION p1 VALUES LESS THAN (\"2014-01-01\"),\n" +
+ "PARTITION p2 VALUES LESS THAN (\"2014-06-01\"),\n" +
+ "PARTITION p3 VALUES LESS THAN (\"2014-12-01\")\n" +
+ ")\n" +
+ "DISTRIBUTED BY HASH(`k1`) BUCKETS 32\n" +
+ "PROPERTIES (\n" +
+ "\"replication_num\" = \"1\",\n" +
+ "\"dynamic_partition.enable\" = \"true\",\n" +
+ "\"dynamic_partition.start\" = \"-3\",\n" +
+ "\"dynamic_partition.end\" = \"3\",\n" +
+ "\"dynamic_partition.buckets\" = \"3\",\n" +
+ "\"dynamic_partition.time_unit\" = \"day\",\n" +
+ "\"dynamic_partition.prefix\" = \"p\",\n" +
+ "\"dynamic_partition.reserved_history_periods\" = \"[2020-01-01,2020-03-01],[2021-10-01,2021-09-01]\"\n" +
+ ");";
+ ExceptionChecker.expectThrowsWithMsg(DdlException.class,
+ "errCode = 2, detailMessage = The first date is larger than the second date, [2021-10-01,2021-09-01] is invalid.",
+ () -> createTable(createOlapTblStmt3));
+
+ String createOlapTblStmt4 = "CREATE TABLE test.`dynamic_partition_reserved_history_periods_validate4` (\n" +
+ " `k1` datetime NULL COMMENT \"\",\n" +
+ " `k2` int NULL COMMENT \"\",\n" +
+ " `k3` smallint NULL COMMENT \"\",\n" +
+ " `v1` varchar(2048) NULL COMMENT \"\",\n" +
+ " `v2` datetime NULL COMMENT \"\"\n" +
+ ") ENGINE=OLAP\n" +
+ "DUPLICATE KEY(`k1`, `k2`, `k3`)\n" +
+ "COMMENT \"OLAP\"\n" +
+ "PARTITION BY RANGE (k1)\n" +
+ "(\n" +
+ "PARTITION p1 VALUES LESS THAN (\"2014-01-01 00:00:00\"),\n" +
+ "PARTITION p2 VALUES LESS THAN (\"2014-06-01 00:00:00\"),\n" +
+ "PARTITION p3 VALUES LESS THAN (\"2014-12-01 00:00:00\")\n" +
+ ")\n" +
+ "DISTRIBUTED BY HASH(`k1`) BUCKETS 32\n" +
+ "PROPERTIES (\n" +
+ "\"replication_num\" = \"1\",\n" +
+ "\"dynamic_partition.enable\" = \"true\",\n" +
+ "\"dynamic_partition.start\" = \"-3\",\n" +
+ "\"dynamic_partition.end\" = \"3\",\n" +
+ "\"dynamic_partition.buckets\" = \"3\",\n" +
+ "\"dynamic_partition.time_unit\" = \"hour\",\n" +
+ "\"dynamic_partition.prefix\" = \"p\",\n" +
+ "\"dynamic_partition.reserved_history_periods\" = \"[2020-01-01,2020-03-01]\"\n" +
+ ");";
+ ExceptionChecker.expectThrowsWithMsg(DdlException.class,
+ "errCode = 2, detailMessage = Invalid \" dynamic_partition.reserved_history_periods \" value [2020-01-01,2020-03-01]. " +
+ "It must be like " +
+ "\"[yyyy-MM-dd,yyyy-MM-dd],[...,...]\" while time_unit is DAY/WEEK/MONTH " +
+ "or \"[yyyy-MM-dd HH:mm:ss,yyyy-MM-dd HH:mm:ss],[...,...]\" while time_unit is HOUR.",
+ () -> createTable(createOlapTblStmt4));
+ }
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org