You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/02/04 01:56:12 UTC
[GitHub] [incubator-doris] morningman opened a new pull request #2828: [Temp
Partition] Support add/drop/replace temp partitions
morningman opened a new pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828
This CL implements 3 new operations:
```
ALTER TABLE tbl ADD TEMP PARTITION (p1, p2, ...);
ALTER TABLE tbl DROP TEMP PARTITION (p1, p2, ...);
ALTER TABLE tbl REPLACE TEMP PARTITION (p1, p2, ...);
```
Details can be found in document:
`docs/documentation/cn/administrator-guide/alter-table/alter-table-temp-partition.md`
I did not update the grammar manual of `alter-table.md`.
This manual is too confusing and too big, I will reorganize this manual after.
This is the first part to implement the "overwrite load" feature mentioned in issue #2663.
I will implement the "load to temp partition" feature in next PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386069047
##########
File path: fe/src/main/java/org/apache/doris/catalog/OlapTable.java
##########
@@ -1241,4 +1264,141 @@ public void setIsInMemory(boolean isInMemory) {
tableProperty.modifyTableProperties(PropertyAnalyzer.PROPERTIES_INMEMORY, Boolean.valueOf(isInMemory).toString());
tableProperty.buildInMemory();
}
+
+ // return true if partition with given name already exist, both in partitions and temp partitions.
+ // return false otherwise
+ public boolean checkPartitionNameExist(String partitionName) {
+ if (nameToPartition.containsKey(partitionName)) {
+ return true;
+ }
+ return tempPartitions.hasPartition(partitionName);
+ }
+
+ // if includeTempPartition is true, check if temp partition with given name exist,
+ // if includeTempPartition is false, check if temp partition with given name exist.
+ // return true if exist, otherwise, return false;
+ public boolean checkPartitionNameExist(String partitionName, boolean isTempPartition) {
Review comment:
Same reason as `getPartition`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] imay commented on a change in pull request #2828:
[Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
imay commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r375619199
##########
File path: docs/documentation/cn/administrator-guide/alter-table/alter-table-temp-partition.md
##########
@@ -0,0 +1,117 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 临时分区
+
+在 0.12 版本中，Doris 支持了临时分区功能。临时分区是对 Doris 中**已存在的数据表分区**的一个复制。这个复制操作仅复制分区的元数据，如分区值范围，分桶数，副本数等等，而**不复制数据**。
Review comment:
@kangkaisen
I think you may have misunderstood what I meant. What I want to emphasize is not how to deal with temporary partitions during split and merge. I want to emphasize that we can also complete the functions of partition split and merge through temporary partitions. So what I mean is that we should not assume that it is related to an existing partition when doing temporary partitioning.
Currently we can just generate from other partitions, but this premise should not be made.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on issue #2828: [Temp
Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on issue #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#issuecomment-582389201
> @kangkaisen Would you help to review this PR?
OK. I will review this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386029522
##########
File path: fe/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
##########
@@ -407,4 +410,22 @@ public static boolean analyzeInMemory(Map<String, String> properties, boolean de
}
return defaultInMemory;
}
+
+ public static boolean analyzeStrictRange(Map<String, String> properties, boolean defaultVal) {
+ if (properties != null && properties.containsKey(PROPERTIES_STRICT_RANGE)) {
+ String inMemory = properties.get(PROPERTIES_STRICT_RANGE);
+ properties.remove(PROPERTIES_STRICT_RANGE);
+ return Boolean.parseBoolean(inMemory);
+ }
+ return defaultVal;
+ }
+
+ public static boolean analyzeUseTempPartitionName(Map<String, String> properties, boolean defaultVal) {
+ if (properties != null && properties.containsKey(PROPERTIES_USE_TEMP_PARTITION_NAME)) {
+ String inMemory = properties.get(PROPERTIES_USE_TEMP_PARTITION_NAME);
Review comment:
idem
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386027864
##########
File path: docs/documentation/cn/administrator-guide/alter-table/alter-table-temp-partition.md
##########
@@ -0,0 +1,200 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 临时分区
+
+在 0.12 版本中，Doris 支持了临时分区功能。
+
+临时分区是归属于某一分区表的。只有分区表可以创建临时分区。
+
+## 规则
+
+* 临时分区的分区列和正式分区相同，且不可修改。
+* 一张表所有临时分区之间的分区范围不可重叠，但临时分区的范围和正式分区范围可以重叠。
+* 临时分区的分区名称不能和正式分区以及其他临时分区重复。
+
+## 支持的操作
+
+临时分区支持添加、删除、替换操作。
+
+### 添加临时分区
+
+可以通过 `ALTER TABLE ADD TEMPORARY PARTITION` 语句对一个表添加临时分区：
+
+```
+ALTER TABLE tbl1 ADD TEMPORARY PARTITION tp1 VALUES LESS THAN("2020-02-01");
+
+ALTER TABLE tbl2 ADD TEMPORARY PARTITION tp1 VALUES [("2020-01-01"), ("2020-02-01"));
+
+ALTER TABLE tbl1 ADD TEMPORARY PARTITION tp1 VALUES LESS THAN("2020-02-01")
+("in_memory" = "true", "replication_num" = "1")
+DISTRIBUTED BY HASH(k1) BUCKETS 5;
+```
+
+通过 `HELP ALTER TABLE;` 查看更多帮助和示例。
+
+添加操作的一些说明：
+
+* 临时分区的添加和正式分区的添加操作相似。临时分区的分区范围独立于正式分区。
+* 临时分区可以独立指定一些属性。包括分桶数、副本数、是否是内存表、存储介质等信息。
+
+### 删除临时分区
+
+可以通过 `ALTER TABLE DROP TEMPORARY PARTITION` 语句删除一个表的临时分区：
+
+```
+ALTER TABLE tbl1 DROP TEMPORARY PARTITION tp1;
+```
+
+通过 `HELP ALTER TABLE;` 查看更多帮助和示例。
+
+删除操作的一些说明：
+
+* 删除临时分区，不影响正式分区的数据。
+
+### 替换分区
+
+可以通过 `ALTER TABLE REPLACE PARTITION` 语句将一个表的正式分区替换为临时分区。
+
+```
+ALTER TABLE tbl1 REPLACE PARTITION (p1) WITH TEMPORARY PARTITION (tp1);
+
+ALTER TABLE tbl1 REPLACE PARTITION (p1, p2) WITH TEMPORARY PARTITION (tp1, tp2, tp3);
+
+ALTER TABLE tbl1 REPLACE PARTITION (p1, p2) WITH TEMPORARY PARTITION (tp1, tp2)
+PROPERTIES (
+ "strict_range" = "false",
+ "use_temp_partition_name" = "true"
+);
+```
+
+通过 `HELP ALTER TABLE;` 查看更多帮助和示例。
+
+替换操作有两个特殊的可选参数：
+
+1. `strict_range`
+
+ 默认为 true。当该参数为 true 时，表示要被替换的所有正式分区的范围并集需要和替换的临时分区的范围并集完全相同。当置为 false 时，只需要保证替换后，新的正式分区间的范围不重叠即可。下面举例说明：
+
+ * 示例1
+
+ 待替换的分区 p1, p2, p3 的范围 (=> 并集)：
+
+ ```
+ [10, 20), [20, 30), [40, 50) => [10, 30), [40, 50)
+ ```
+
+ 替换分区 tp1, tp2 的范围(=> 并集)：
+
+ ```
+ [10, 30), [40, 45), [45, 50) => [10, 30), [40, 50)
+ ```
+
+ 范围并集相同，则可以使用 tp1 和 tp2 替换 p1, p2, p3。
+
+ * 示例2
+
+ 待替换的分区 p1 的范围 (=> 并集)：
+
+ ```
+ [10, 50) => [10, 50)
+ ```
+
+ 替换分区 tp1, tp2 的范围(=> 并集)：
+
+ ```
+ [10, 30), [40, 50) => [10, 30), [40, 50)
+ ```
+
+ 范围并集不相同，如果 `strict_range` 为 true，则不可以使用 tp1 和 tp2 替换 p1。如果为 false，且替换后的两个分区范围 `[10, 30), [40, 50)` 和其他正式分区不重叠，则可以替换。
+
+2. `use_temp_partition_name`
+
+ 默认为 false。当该参数为 false，并且待替换的分区和替换分区的个数相同时，则替换后的正式分区名称维持不变。如果为 true，则替换后，正式分区的名称为替换分区的名称。下面举例说明：
+
+ * 示例1
+
+ ```
+ ALTER TABLE tbl1 REPLACE PARTITION (p1) WITH TEMPORARY PARTITION (tp1);
+ ```
+
+ `use_temp_partition_name` 默认为 false，则在替换后，分区的名称依然为 p1，但是相关的数据和属性都替换为 tp1 的。
+
+ 如果 `use_temp_partition_name` 默认为 true，则在替换后，分区的名称为 tp1。p1 分区不再存在。
+
+ * 示例2
+
+ ```
+ ALTER TABLE tbl1 REPLACE PARTITION (p1, p2) WITH TEMPORARY PARTITION (tp1);
+ ```
+
+ `use_temp_partition_name` 默认为 false，但因为待替换分区的个数和替换分区的个数不同，则该参数无效。替换后，分区名称为 tp1，p1 和 p2 不再存在。
+
+替换操作的一些说明：
+
+* 分区替换成功后，被替换的分区将被删除且不可恢复。
+
+## 临时分区的导入
+
+（TODO）
+
+## 和其他操作的关系
+
+### DROP
+
+* 使用 Drop 操作直接删除数据库或表后，可以通过 Recover 命令恢复数据库或表（限定时间内），同时，对应的临时分区也会被一同恢复。
+* 使用 Alter 命令删除正式分区后，可以通过 Recover 命令恢复分区（限定时间内）。操作正式分区和临时分区无关。
+* 使用 Alter 命令删除临时分区后，无法通过 Recover 命令恢复临时分区。
+
+### TRUNCATE
+
+* 使用 Truncate 命令清空表，表的临时分区会被删除，且不可恢复。
+* 使用 Truncate 命令清空正是分区时，不影响临时分区。
+* 不可使用 Truncate 命令清空临时分区。
+
+### ALTER
+
+* 当表存在临时分区时，无法使用 Alter 命令对表进行 Schema Change、Rollup 等变更操作。
+* 当表在进行变更操作时，无法对表添加临时分区。
+
+
+## 最佳实践
+
+1. 原子的覆盖写操作
+
+ 某些情况下，用户希望能够重写某一分区的数据，但如果采用先删除在导入的方式进行，在中间会有一段时间无法查看数据。这是，用户可以先创建一个对应的临时分区，将新的数据导入到临时分区后，通过替换操作，原子的替换原有分区，以达到目的。
Review comment:
```suggestion
某些情况下，用户希望能够重写某一分区的数据，但如果采用先删除再导入的方式进行，在中间会有一段时间无法查看数据。这时，用户可以先创建一个对应的临时分区，将新的数据导入到临时分区后，通过替换操作，原子的替换原有分区，以达到目的。
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386074597
##########
File path: fe/src/main/java/org/apache/doris/catalog/OlapTable.java
##########
@@ -624,6 +632,14 @@ public Partition getPartition(String partitionName) {
return nameToPartition.get(partitionName);
}
+ public Partition getPartition(String partitionName, boolean isTempPartition) {
Review comment:
>If split into 2 methods, we have to write if .. else .. every time we call getPartition(), which is not convenient.
split to two methods is in order to avoid ` if .. else .. `.
>But maybe I will change it when implementing loading into temp partition. It depends on how convenient it is to use in the loading process.
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386027797
##########
File path: docs/documentation/cn/administrator-guide/alter-table/alter-table-temp-partition.md
##########
@@ -0,0 +1,200 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 临时分区
+
+在 0.12 版本中，Doris 支持了临时分区功能。
+
+临时分区是归属于某一分区表的。只有分区表可以创建临时分区。
+
+## 规则
+
+* 临时分区的分区列和正式分区相同，且不可修改。
+* 一张表所有临时分区之间的分区范围不可重叠，但临时分区的范围和正式分区范围可以重叠。
+* 临时分区的分区名称不能和正式分区以及其他临时分区重复。
+
+## 支持的操作
+
+临时分区支持添加、删除、替换操作。
+
+### 添加临时分区
+
+可以通过 `ALTER TABLE ADD TEMPORARY PARTITION` 语句对一个表添加临时分区：
+
+```
+ALTER TABLE tbl1 ADD TEMPORARY PARTITION tp1 VALUES LESS THAN("2020-02-01");
+
+ALTER TABLE tbl2 ADD TEMPORARY PARTITION tp1 VALUES [("2020-01-01"), ("2020-02-01"));
+
+ALTER TABLE tbl1 ADD TEMPORARY PARTITION tp1 VALUES LESS THAN("2020-02-01")
+("in_memory" = "true", "replication_num" = "1")
+DISTRIBUTED BY HASH(k1) BUCKETS 5;
+```
+
+通过 `HELP ALTER TABLE;` 查看更多帮助和示例。
+
+添加操作的一些说明：
+
+* 临时分区的添加和正式分区的添加操作相似。临时分区的分区范围独立于正式分区。
+* 临时分区可以独立指定一些属性。包括分桶数、副本数、是否是内存表、存储介质等信息。
+
+### 删除临时分区
+
+可以通过 `ALTER TABLE DROP TEMPORARY PARTITION` 语句删除一个表的临时分区：
+
+```
+ALTER TABLE tbl1 DROP TEMPORARY PARTITION tp1;
+```
+
+通过 `HELP ALTER TABLE;` 查看更多帮助和示例。
+
+删除操作的一些说明：
+
+* 删除临时分区，不影响正式分区的数据。
+
+### 替换分区
+
+可以通过 `ALTER TABLE REPLACE PARTITION` 语句将一个表的正式分区替换为临时分区。
+
+```
+ALTER TABLE tbl1 REPLACE PARTITION (p1) WITH TEMPORARY PARTITION (tp1);
+
+ALTER TABLE tbl1 REPLACE PARTITION (p1, p2) WITH TEMPORARY PARTITION (tp1, tp2, tp3);
+
+ALTER TABLE tbl1 REPLACE PARTITION (p1, p2) WITH TEMPORARY PARTITION (tp1, tp2)
+PROPERTIES (
+ "strict_range" = "false",
+ "use_temp_partition_name" = "true"
+);
+```
+
+通过 `HELP ALTER TABLE;` 查看更多帮助和示例。
+
+替换操作有两个特殊的可选参数：
+
+1. `strict_range`
+
+ 默认为 true。当该参数为 true 时，表示要被替换的所有正式分区的范围并集需要和替换的临时分区的范围并集完全相同。当置为 false 时，只需要保证替换后，新的正式分区间的范围不重叠即可。下面举例说明：
+
+ * 示例1
+
+ 待替换的分区 p1, p2, p3 的范围 (=> 并集)：
+
+ ```
+ [10, 20), [20, 30), [40, 50) => [10, 30), [40, 50)
+ ```
+
+ 替换分区 tp1, tp2 的范围(=> 并集)：
+
+ ```
+ [10, 30), [40, 45), [45, 50) => [10, 30), [40, 50)
+ ```
+
+ 范围并集相同，则可以使用 tp1 和 tp2 替换 p1, p2, p3。
+
+ * 示例2
+
+ 待替换的分区 p1 的范围 (=> 并集)：
+
+ ```
+ [10, 50) => [10, 50)
+ ```
+
+ 替换分区 tp1, tp2 的范围(=> 并集)：
+
+ ```
+ [10, 30), [40, 50) => [10, 30), [40, 50)
+ ```
+
+ 范围并集不相同，如果 `strict_range` 为 true，则不可以使用 tp1 和 tp2 替换 p1。如果为 false，且替换后的两个分区范围 `[10, 30), [40, 50)` 和其他正式分区不重叠，则可以替换。
+
+2. `use_temp_partition_name`
+
+ 默认为 false。当该参数为 false，并且待替换的分区和替换分区的个数相同时，则替换后的正式分区名称维持不变。如果为 true，则替换后，正式分区的名称为替换分区的名称。下面举例说明：
+
+ * 示例1
+
+ ```
+ ALTER TABLE tbl1 REPLACE PARTITION (p1) WITH TEMPORARY PARTITION (tp1);
+ ```
+
+ `use_temp_partition_name` 默认为 false，则在替换后，分区的名称依然为 p1，但是相关的数据和属性都替换为 tp1 的。
+
+ 如果 `use_temp_partition_name` 默认为 true，则在替换后，分区的名称为 tp1。p1 分区不再存在。
+
+ * 示例2
+
+ ```
+ ALTER TABLE tbl1 REPLACE PARTITION (p1, p2) WITH TEMPORARY PARTITION (tp1);
+ ```
+
+ `use_temp_partition_name` 默认为 false，但因为待替换分区的个数和替换分区的个数不同，则该参数无效。替换后，分区名称为 tp1，p1 和 p2 不再存在。
+
+替换操作的一些说明：
+
+* 分区替换成功后，被替换的分区将被删除且不可恢复。
+
+## 临时分区的导入
+
+（TODO）
+
+## 和其他操作的关系
+
+### DROP
+
+* 使用 Drop 操作直接删除数据库或表后，可以通过 Recover 命令恢复数据库或表（限定时间内），同时，对应的临时分区也会被一同恢复。
+* 使用 Alter 命令删除正式分区后，可以通过 Recover 命令恢复分区（限定时间内）。操作正式分区和临时分区无关。
+* 使用 Alter 命令删除临时分区后，无法通过 Recover 命令恢复临时分区。
+
+### TRUNCATE
+
+* 使用 Truncate 命令清空表，表的临时分区会被删除，且不可恢复。
+* 使用 Truncate 命令清空正是分区时，不影响临时分区。
Review comment:
```suggestion
* 使用 Truncate 命令清空正式分区时，不影响临时分区。
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r375643507
##########
File path: docs/documentation/cn/administrator-guide/alter-table/alter-table-temp-partition.md
##########
@@ -0,0 +1,117 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 临时分区
+
+在 0.12 版本中，Doris 支持了临时分区功能。临时分区是对 Doris 中**已存在的数据表分区**的一个复制。这个复制操作仅复制分区的元数据，如分区值范围，分桶数，副本数等等，而**不复制数据**。
Review comment:
> we can also complete the functions of partition split and merge through temporary partitions.
What you mean is temp partitions need to support split and merge ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r376194260
##########
File path: docs/documentation/cn/administrator-guide/alter-table/alter-table-temp-partition.md
##########
@@ -0,0 +1,117 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 临时分区
+
+在 0.12 版本中，Doris 支持了临时分区功能。临时分区是对 Doris 中**已存在的数据表分区**的一个复制。这个复制操作仅复制分区的元数据，如分区值范围，分桶数，副本数等等，而**不复制数据**。
Review comment:
Let me think.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386034144
##########
File path: docs/documentation/cn/administrator-guide/alter-table/alter-table-temp-partition.md
##########
@@ -0,0 +1,200 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 临时分区
+
+在 0.12 版本中，Doris 支持了临时分区功能。
+
+临时分区是归属于某一分区表的。只有分区表可以创建临时分区。
+
+## 规则
+
+* 临时分区的分区列和正式分区相同，且不可修改。
+* 一张表所有临时分区之间的分区范围不可重叠，但临时分区的范围和正式分区范围可以重叠。
+* 临时分区的分区名称不能和正式分区以及其他临时分区重复。
+
+## 支持的操作
+
+临时分区支持添加、删除、替换操作。
+
+### 添加临时分区
+
+可以通过 `ALTER TABLE ADD TEMPORARY PARTITION` 语句对一个表添加临时分区：
+
+```
+ALTER TABLE tbl1 ADD TEMPORARY PARTITION tp1 VALUES LESS THAN("2020-02-01");
+
+ALTER TABLE tbl2 ADD TEMPORARY PARTITION tp1 VALUES [("2020-01-01"), ("2020-02-01"));
+
+ALTER TABLE tbl1 ADD TEMPORARY PARTITION tp1 VALUES LESS THAN("2020-02-01")
+("in_memory" = "true", "replication_num" = "1")
+DISTRIBUTED BY HASH(k1) BUCKETS 5;
+```
+
+通过 `HELP ALTER TABLE;` 查看更多帮助和示例。
+
+添加操作的一些说明：
+
+* 临时分区的添加和正式分区的添加操作相似。临时分区的分区范围独立于正式分区。
+* 临时分区可以独立指定一些属性。包括分桶数、副本数、是否是内存表、存储介质等信息。
+
+### 删除临时分区
+
+可以通过 `ALTER TABLE DROP TEMPORARY PARTITION` 语句删除一个表的临时分区：
+
+```
+ALTER TABLE tbl1 DROP TEMPORARY PARTITION tp1;
+```
+
+通过 `HELP ALTER TABLE;` 查看更多帮助和示例。
+
+删除操作的一些说明：
+
+* 删除临时分区，不影响正式分区的数据。
+
+### 替换分区
+
+可以通过 `ALTER TABLE REPLACE PARTITION` 语句将一个表的正式分区替换为临时分区。
+
+```
+ALTER TABLE tbl1 REPLACE PARTITION (p1) WITH TEMPORARY PARTITION (tp1);
+
+ALTER TABLE tbl1 REPLACE PARTITION (p1, p2) WITH TEMPORARY PARTITION (tp1, tp2, tp3);
+
+ALTER TABLE tbl1 REPLACE PARTITION (p1, p2) WITH TEMPORARY PARTITION (tp1, tp2)
+PROPERTIES (
+ "strict_range" = "false",
+ "use_temp_partition_name" = "true"
+);
+```
+
+通过 `HELP ALTER TABLE;` 查看更多帮助和示例。
+
+替换操作有两个特殊的可选参数：
+
+1. `strict_range`
+
+ 默认为 true。当该参数为 true 时，表示要被替换的所有正式分区的范围并集需要和替换的临时分区的范围并集完全相同。当置为 false 时，只需要保证替换后，新的正式分区间的范围不重叠即可。下面举例说明：
+
+ * 示例1
+
+ 待替换的分区 p1, p2, p3 的范围 (=> 并集)：
+
+ ```
+ [10, 20), [20, 30), [40, 50) => [10, 30), [40, 50)
+ ```
+
+ 替换分区 tp1, tp2 的范围(=> 并集)：
+
+ ```
+ [10, 30), [40, 45), [45, 50) => [10, 30), [40, 50)
+ ```
+
+ 范围并集相同，则可以使用 tp1 和 tp2 替换 p1, p2, p3。
+
+ * 示例2
+
+ 待替换的分区 p1 的范围 (=> 并集)：
+
+ ```
+ [10, 50) => [10, 50)
+ ```
+
+ 替换分区 tp1, tp2 的范围(=> 并集)：
+
+ ```
+ [10, 30), [40, 50) => [10, 30), [40, 50)
+ ```
+
+ 范围并集不相同，如果 `strict_range` 为 true，则不可以使用 tp1 和 tp2 替换 p1。如果为 false，且替换后的两个分区范围 `[10, 30), [40, 50)` 和其他正式分区不重叠，则可以替换。
+
+2. `use_temp_partition_name`
+
+ 默认为 false。当该参数为 false，并且待替换的分区和替换分区的个数相同时，则替换后的正式分区名称维持不变。如果为 true，则替换后，正式分区的名称为替换分区的名称。下面举例说明：
+
+ * 示例1
+
+ ```
+ ALTER TABLE tbl1 REPLACE PARTITION (p1) WITH TEMPORARY PARTITION (tp1);
+ ```
+
+ `use_temp_partition_name` 默认为 false，则在替换后，分区的名称依然为 p1，但是相关的数据和属性都替换为 tp1 的。
+
+ 如果 `use_temp_partition_name` 默认为 true，则在替换后，分区的名称为 tp1。p1 分区不再存在。
+
+ * 示例2
+
+ ```
+ ALTER TABLE tbl1 REPLACE PARTITION (p1, p2) WITH TEMPORARY PARTITION (tp1);
+ ```
+
+ `use_temp_partition_name` 默认为 false，但因为待替换分区的个数和替换分区的个数不同，则该参数无效。替换后，分区名称为 tp1，p1 和 p2 不再存在。
+
+替换操作的一些说明：
+
+* 分区替换成功后，被替换的分区将被删除且不可恢复。
+
+## 临时分区的导入
+
+（TODO）
+
+## 和其他操作的关系
+
+### DROP
+
+* 使用 Drop 操作直接删除数据库或表后，可以通过 Recover 命令恢复数据库或表（限定时间内），同时，对应的临时分区也会被一同恢复。
+* 使用 Alter 命令删除正式分区后，可以通过 Recover 命令恢复分区（限定时间内）。操作正式分区和临时分区无关。
+* 使用 Alter 命令删除临时分区后，无法通过 Recover 命令恢复临时分区。
+
+### TRUNCATE
+
+* 使用 Truncate 命令清空表，表的临时分区会被删除，且不可恢复。
+* 使用 Truncate 命令清空正是分区时，不影响临时分区。
+* 不可使用 Truncate 命令清空临时分区。
+
+### ALTER
+
+* 当表存在临时分区时，无法使用 Alter 命令对表进行 Schema Change、Rollup 等变更操作。
+* 当表在进行变更操作时，无法对表添加临时分区。
+
+
+## 最佳实践
+
+1. 原子的覆盖写操作
+
+ 某些情况下，用户希望能够重写某一分区的数据，但如果采用先删除在导入的方式进行，在中间会有一段时间无法查看数据。这是，用户可以先创建一个对应的临时分区，将新的数据导入到临时分区后，通过替换操作，原子的替换原有分区，以达到目的。
+
+2. 修改分桶数
+
+ 某些情况下，用户在创建分区时使用了不合适的分桶数。则用户可以先创建一个对应分区范围的临时分区，并指定新的分桶数。然后通过 `INSERT INTO` 命令将正式分区的数据导入到临时分区中，通过替换操作，原子的替换原有分区，以达到目的。
+
+3. 合并或分割分区
+
+ 某些情况下，用户希望对分区的范围进行修改，比如合并两个分区，或将一个大分区分割成多个小分区。则用户可以先建立对应合并或分割后范围的临时分区，然后通过 `INSERT INTO` 命令将正式分区的数据导入到临时分区中，通过替换操作，原子的替换原有分区，以达到目的。
Review comment:
可以，这个不冲突。后面可以考虑下，insert into这种只是个临时的方案。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #2828: [Temp
Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
morningman commented on issue #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#issuecomment-582297628
@kangkaisen Would you help to review this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386069171
##########
File path: fe/src/main/java/org/apache/doris/catalog/TempPartitions.java
##########
@@ -0,0 +1,149 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog;
+
+import org.apache.doris.catalog.MaterializedIndex.IndexExtState;
+import org.apache.doris.common.io.Writable;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+
+// This class saved all temp partitions of a table.
+// temp partition is used to implement the overwrite load.
+// user can load data into some of the temp partitions,
+// and then replace the formal partitions with these temp partitions
+// to make a overwrite load.
+public class TempPartitions implements Writable {
+ private Map<Long, Partition> idToPartition = Maps.newHashMap();
+ private Map<String, Partition> nameToPartition = Maps.newHashMap();
+ private RangePartitionInfo partitionInfo;
+
+ public TempPartitions() {
+ }
+
+ public TempPartitions(List<Column> partCols) {
+ partitionInfo = new RangePartitionInfo(partCols);
+ }
+
+ public RangePartitionInfo getPartitionInfo() {
+ return partitionInfo;
+ }
+
+ public void addPartition(Partition partition) {
+ idToPartition.put(partition.getId(), partition);
+ nameToPartition.put(partition.getName(), partition);
+ }
+
+ /*
+ * Drop temp partitions.
+ * If needDropTablet is true, also drop the tablet from tablet inverted index.
+ */
+ public void dropPartition(String partitionName, boolean needDropTablet) {
+ Partition partition = nameToPartition.get(partitionName);
+ if (partition != null) {
+ idToPartition.remove(partition.getId());
+ nameToPartition.remove(partitionName);
+
+ Preconditions.checkState(partitionInfo.getType() == PartitionType.RANGE);
+ RangePartitionInfo rangePartitionInfo = (RangePartitionInfo) partitionInfo;
+ // drop partition info
+ rangePartitionInfo.dropPartition(partition.getId());
+
+ if (!Catalog.isCheckpointThread() && needDropTablet) {
+ TabletInvertedIndex invertedIndex = Catalog.getCurrentInvertedIndex();
+ for (MaterializedIndex index : partition.getMaterializedIndices(IndexExtState.ALL)) {
+ for (Tablet tablet : index.getTablets()) {
+ invertedIndex.deleteTablet(tablet.getId());
+ }
+ }
+ }
+ }
+ }
+
+ public Partition getPartition(long partitionId) {
+ return idToPartition.get(partitionId);
+ }
+
+ public Partition getPartition(String partitionName) {
+ return nameToPartition.get(partitionName);
+ }
+
+ public List<Partition> getAllPartitions() {
+ return Lists.newArrayList(idToPartition.values());
+ }
+
+ public boolean hasPartition(String partName) {
+ if (partName == null) {
+ return !idToPartition.isEmpty();
Review comment:
`if (partName == null)` is removed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386068923
##########
File path: fe/src/main/java/org/apache/doris/catalog/OlapTable.java
##########
@@ -624,6 +632,14 @@ public Partition getPartition(String partitionName) {
return nameToPartition.get(partitionName);
}
+ public Partition getPartition(String partitionName, boolean isTempPartition) {
Review comment:
If split into 2 methods, we have to write `if .. else ..` every time we call `getPartition()`, which is not convenient.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r375336823
##########
File path: fe/src/main/java/org/apache/doris/alter/MaterializedViewHandler.java
##########
@@ -99,6 +99,11 @@ public MaterializedViewHandler() {
*/
public void processCreateMaterializedView(CreateMaterializedViewStmt addMVClause, Database db, OlapTable olapTable)
throws DdlException, AnalysisException {
+
+ if (olapTable.hasTempPartition(null)) {
Review comment:
This check is strange. we could add a `existTempPartition` method to olapTable, and add a `isEmpty` method to
TempPartitions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386029689
##########
File path: fe/src/main/java/org/apache/doris/common/util/RangeUtils.java
##########
@@ -0,0 +1,210 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common.util;
+
+import org.apache.doris.catalog.PartitionKey;
+import org.apache.doris.common.DdlException;
+
+import com.google.common.collect.BoundType;
+import com.google.common.collect.Range;
+import com.google.common.collect.RangeMap;
+import com.google.common.collect.TreeRangeMap;
+
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+
+public class RangeUtils {
+
+ public static final Comparator<Map.Entry<Long, Range<PartitionKey>>> RANGE_MAP_ENTRY_COMPARATOR =
+ Comparator.comparing(o -> o.getValue().lowerEndpoint());
+
+ public static final Comparator<Range<PartitionKey>> RANGE_COMPARATOR =
+ Comparator.comparing(o -> o.lowerEndpoint());
+
+ public static void checkRangeIntersect(Range<PartitionKey> range1, Range<PartitionKey> range2) throws DdlException {
+ if (range2.isConnected(range1)) {
+ if (!range2.intersection(range1).isEmpty()) {
+ throw new DdlException("Range " + range1 + " is intersected with range: " + range2);
+ }
+ }
+ }
+
+ /*
+ * Pass only if the 2 range lists are exactly same
+ * What is "exactly same"?
+ * 1. {[0, 10), [10, 20)} exactly same as {[0, 20)}
+ * 2. {[0, 10), [15, 20)} exactly same as {[0, 10), [15, 18), [18, 20)}
+ * 3. {[0, 10), [15, 20)} exactly same as {[0, 10), [15, 20)}
+ * 4. {[0, 10), [15, 20)} NOT exactly same as {[0, 20)}
+ *
+ * Here I will use an example to explain the algorithm:
+ * list1: {[0, 10), [15, 20)}
+ * list2: {[0, 10), [15, 18), [18, 20)}
+ *
+ * 1. sort 2 lists first (the above 2 lists are already sorted)
+ * 2. Begin to compare ranges from index 0: [0, 10) and [0, 10)
+ * 2.1 lower bounds (0 and 0) are equal.
+ * 2.2 upper bounds (10 and 10) are equal.
+ * 3. Begin to compare next 2 ranges [15, 20) and [15, 18)
+ * 3.1 lower bounds (15 and 15) are equal.
+ * 3.2 upper bounds (20 and 18) are not equal. and 20 > 18
+ * 3.3 Split range [15, 20) to [15, 18) and [18, 20)
+ * 4. Begin to compare next 2 ranges [18, 20) and [18, 20), the first [18, 20) is the splitted range
+ * 4.1 lower bounds (18 and 18) are equal.
+ * 4.2 upper bounds (20 and 20) are equal.
+ * 5. Not more next ranges, so 2 lists are equal.
+ */
+ public static void checkRangeListsMatch(List<Range<PartitionKey>> list1, List<Range<PartitionKey>> list2) throws DdlException {
Review comment:
Return bool instead of void. I think we shouldn't throw DdlException for util class.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #2828: [Temp
Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
morningman merged pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386069625
##########
File path: fe/src/main/java/org/apache/doris/qe/ShowExecutor.java
##########
@@ -1215,12 +1215,17 @@ private void handleShowTablet() throws AnalysisException {
}
boolean stop = false;
Collection<Partition> partitions = new ArrayList<Partition>();
- List<String> partitionNames = showStmt.getPartitionNames();
if (showStmt.hasPartition()) {
- for (Partition partition : olapTable.getPartitions()) {
- if (partitionNames.contains(partition.getName())) {
- partitions.add(partition);
+ List<String> partitionNames = showStmt.getPartitionNames();
+ for (String partName : partitionNames) {
+ Partition partition = olapTable.getPartition(partName);
+ if (partition == null) {
Review comment:
I will reconsider this when implementing loading process of temp partition.
If I change the origin `getPartition` method, it may change lots of codes in this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386029497
##########
File path: fe/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
##########
@@ -407,4 +410,22 @@ public static boolean analyzeInMemory(Map<String, String> properties, boolean de
}
return defaultInMemory;
}
+
+ public static boolean analyzeStrictRange(Map<String, String> properties, boolean defaultVal) {
+ if (properties != null && properties.containsKey(PROPERTIES_STRICT_RANGE)) {
+ String inMemory = properties.get(PROPERTIES_STRICT_RANGE);
Review comment:
```suggestion
String isStrictRange = properties.get(PROPERTIES_STRICT_RANGE);
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #2828: [Temp
Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
morningman commented on issue #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#issuecomment-583703732
@imay @kangkaisen
I redesign the implementation, see here: #2663
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386030052
##########
File path: fe/src/main/java/org/apache/doris/common/util/RangeUtils.java
##########
@@ -0,0 +1,210 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common.util;
+
+import org.apache.doris.catalog.PartitionKey;
+import org.apache.doris.common.DdlException;
+
+import com.google.common.collect.BoundType;
+import com.google.common.collect.Range;
+import com.google.common.collect.RangeMap;
+import com.google.common.collect.TreeRangeMap;
+
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+
+public class RangeUtils {
Review comment:
Add a UT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386029952
##########
File path: fe/src/main/java/org/apache/doris/qe/ShowExecutor.java
##########
@@ -1215,12 +1215,17 @@ private void handleShowTablet() throws AnalysisException {
}
boolean stop = false;
Collection<Partition> partitions = new ArrayList<Partition>();
- List<String> partitionNames = showStmt.getPartitionNames();
if (showStmt.hasPartition()) {
- for (Partition partition : olapTable.getPartitions()) {
- if (partitionNames.contains(partition.getName())) {
- partitions.add(partition);
+ List<String> partitionNames = showStmt.getPartitionNames();
+ for (String partName : partitionNames) {
+ Partition partition = olapTable.getPartition(partName);
+ if (partition == null) {
Review comment:
As comment above, I think we should split `getPartition` method, otherwise，this logic is strange.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386029354
##########
File path: fe/src/main/java/org/apache/doris/catalog/TempPartitions.java
##########
@@ -0,0 +1,149 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog;
+
+import org.apache.doris.catalog.MaterializedIndex.IndexExtState;
+import org.apache.doris.common.io.Writable;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+
+// This class saved all temp partitions of a table.
+// temp partition is used to implement the overwrite load.
+// user can load data into some of the temp partitions,
+// and then replace the formal partitions with these temp partitions
+// to make a overwrite load.
+public class TempPartitions implements Writable {
+ private Map<Long, Partition> idToPartition = Maps.newHashMap();
+ private Map<String, Partition> nameToPartition = Maps.newHashMap();
+ private RangePartitionInfo partitionInfo;
+
+ public TempPartitions() {
+ }
+
+ public TempPartitions(List<Column> partCols) {
+ partitionInfo = new RangePartitionInfo(partCols);
+ }
+
+ public RangePartitionInfo getPartitionInfo() {
+ return partitionInfo;
+ }
+
+ public void addPartition(Partition partition) {
+ idToPartition.put(partition.getId(), partition);
+ nameToPartition.put(partition.getName(), partition);
+ }
+
+ /*
+ * Drop temp partitions.
+ * If needDropTablet is true, also drop the tablet from tablet inverted index.
+ */
+ public void dropPartition(String partitionName, boolean needDropTablet) {
+ Partition partition = nameToPartition.get(partitionName);
+ if (partition != null) {
+ idToPartition.remove(partition.getId());
+ nameToPartition.remove(partitionName);
+
+ Preconditions.checkState(partitionInfo.getType() == PartitionType.RANGE);
+ RangePartitionInfo rangePartitionInfo = (RangePartitionInfo) partitionInfo;
+ // drop partition info
+ rangePartitionInfo.dropPartition(partition.getId());
+
+ if (!Catalog.isCheckpointThread() && needDropTablet) {
+ TabletInvertedIndex invertedIndex = Catalog.getCurrentInvertedIndex();
+ for (MaterializedIndex index : partition.getMaterializedIndices(IndexExtState.ALL)) {
+ for (Tablet tablet : index.getTablets()) {
+ invertedIndex.deleteTablet(tablet.getId());
+ }
+ }
+ }
+ }
+ }
+
+ public Partition getPartition(long partitionId) {
+ return idToPartition.get(partitionId);
+ }
+
+ public Partition getPartition(String partitionName) {
+ return nameToPartition.get(partitionName);
+ }
+
+ public List<Partition> getAllPartitions() {
+ return Lists.newArrayList(idToPartition.values());
+ }
+
+ public boolean hasPartition(String partName) {
+ if (partName == null) {
+ return !idToPartition.isEmpty();
Review comment:
return idToPartition.isEmpty(); ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386028086
##########
File path: docs/documentation/cn/administrator-guide/alter-table/alter-table-temp-partition.md
##########
@@ -0,0 +1,200 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 临时分区
+
+在 0.12 版本中，Doris 支持了临时分区功能。
+
+临时分区是归属于某一分区表的。只有分区表可以创建临时分区。
+
+## 规则
+
+* 临时分区的分区列和正式分区相同，且不可修改。
+* 一张表所有临时分区之间的分区范围不可重叠，但临时分区的范围和正式分区范围可以重叠。
+* 临时分区的分区名称不能和正式分区以及其他临时分区重复。
+
+## 支持的操作
+
+临时分区支持添加、删除、替换操作。
+
+### 添加临时分区
+
+可以通过 `ALTER TABLE ADD TEMPORARY PARTITION` 语句对一个表添加临时分区：
+
+```
+ALTER TABLE tbl1 ADD TEMPORARY PARTITION tp1 VALUES LESS THAN("2020-02-01");
+
+ALTER TABLE tbl2 ADD TEMPORARY PARTITION tp1 VALUES [("2020-01-01"), ("2020-02-01"));
+
+ALTER TABLE tbl1 ADD TEMPORARY PARTITION tp1 VALUES LESS THAN("2020-02-01")
+("in_memory" = "true", "replication_num" = "1")
+DISTRIBUTED BY HASH(k1) BUCKETS 5;
+```
+
+通过 `HELP ALTER TABLE;` 查看更多帮助和示例。
+
+添加操作的一些说明：
+
+* 临时分区的添加和正式分区的添加操作相似。临时分区的分区范围独立于正式分区。
+* 临时分区可以独立指定一些属性。包括分桶数、副本数、是否是内存表、存储介质等信息。
+
+### 删除临时分区
+
+可以通过 `ALTER TABLE DROP TEMPORARY PARTITION` 语句删除一个表的临时分区：
+
+```
+ALTER TABLE tbl1 DROP TEMPORARY PARTITION tp1;
+```
+
+通过 `HELP ALTER TABLE;` 查看更多帮助和示例。
+
+删除操作的一些说明：
+
+* 删除临时分区，不影响正式分区的数据。
+
+### 替换分区
+
+可以通过 `ALTER TABLE REPLACE PARTITION` 语句将一个表的正式分区替换为临时分区。
+
+```
+ALTER TABLE tbl1 REPLACE PARTITION (p1) WITH TEMPORARY PARTITION (tp1);
+
+ALTER TABLE tbl1 REPLACE PARTITION (p1, p2) WITH TEMPORARY PARTITION (tp1, tp2, tp3);
+
+ALTER TABLE tbl1 REPLACE PARTITION (p1, p2) WITH TEMPORARY PARTITION (tp1, tp2)
+PROPERTIES (
+ "strict_range" = "false",
+ "use_temp_partition_name" = "true"
+);
+```
+
+通过 `HELP ALTER TABLE;` 查看更多帮助和示例。
+
+替换操作有两个特殊的可选参数：
+
+1. `strict_range`
+
+ 默认为 true。当该参数为 true 时，表示要被替换的所有正式分区的范围并集需要和替换的临时分区的范围并集完全相同。当置为 false 时，只需要保证替换后，新的正式分区间的范围不重叠即可。下面举例说明：
+
+ * 示例1
+
+ 待替换的分区 p1, p2, p3 的范围 (=> 并集)：
+
+ ```
+ [10, 20), [20, 30), [40, 50) => [10, 30), [40, 50)
+ ```
+
+ 替换分区 tp1, tp2 的范围(=> 并集)：
+
+ ```
+ [10, 30), [40, 45), [45, 50) => [10, 30), [40, 50)
+ ```
+
+ 范围并集相同，则可以使用 tp1 和 tp2 替换 p1, p2, p3。
+
+ * 示例2
+
+ 待替换的分区 p1 的范围 (=> 并集)：
+
+ ```
+ [10, 50) => [10, 50)
+ ```
+
+ 替换分区 tp1, tp2 的范围(=> 并集)：
+
+ ```
+ [10, 30), [40, 50) => [10, 30), [40, 50)
+ ```
+
+ 范围并集不相同，如果 `strict_range` 为 true，则不可以使用 tp1 和 tp2 替换 p1。如果为 false，且替换后的两个分区范围 `[10, 30), [40, 50)` 和其他正式分区不重叠，则可以替换。
+
+2. `use_temp_partition_name`
+
+ 默认为 false。当该参数为 false，并且待替换的分区和替换分区的个数相同时，则替换后的正式分区名称维持不变。如果为 true，则替换后，正式分区的名称为替换分区的名称。下面举例说明：
+
+ * 示例1
+
+ ```
+ ALTER TABLE tbl1 REPLACE PARTITION (p1) WITH TEMPORARY PARTITION (tp1);
+ ```
+
+ `use_temp_partition_name` 默认为 false，则在替换后，分区的名称依然为 p1，但是相关的数据和属性都替换为 tp1 的。
+
+ 如果 `use_temp_partition_name` 默认为 true，则在替换后，分区的名称为 tp1。p1 分区不再存在。
+
+ * 示例2
+
+ ```
+ ALTER TABLE tbl1 REPLACE PARTITION (p1, p2) WITH TEMPORARY PARTITION (tp1);
+ ```
+
+ `use_temp_partition_name` 默认为 false，但因为待替换分区的个数和替换分区的个数不同，则该参数无效。替换后，分区名称为 tp1，p1 和 p2 不再存在。
+
+替换操作的一些说明：
+
+* 分区替换成功后，被替换的分区将被删除且不可恢复。
+
+## 临时分区的导入
+
+（TODO）
+
+## 和其他操作的关系
+
+### DROP
+
+* 使用 Drop 操作直接删除数据库或表后，可以通过 Recover 命令恢复数据库或表（限定时间内），同时，对应的临时分区也会被一同恢复。
+* 使用 Alter 命令删除正式分区后，可以通过 Recover 命令恢复分区（限定时间内）。操作正式分区和临时分区无关。
+* 使用 Alter 命令删除临时分区后，无法通过 Recover 命令恢复临时分区。
+
+### TRUNCATE
+
+* 使用 Truncate 命令清空表，表的临时分区会被删除，且不可恢复。
+* 使用 Truncate 命令清空正是分区时，不影响临时分区。
+* 不可使用 Truncate 命令清空临时分区。
+
+### ALTER
+
+* 当表存在临时分区时，无法使用 Alter 命令对表进行 Schema Change、Rollup 等变更操作。
+* 当表在进行变更操作时，无法对表添加临时分区。
+
+
+## 最佳实践
+
+1. 原子的覆盖写操作
+
+ 某些情况下，用户希望能够重写某一分区的数据，但如果采用先删除在导入的方式进行，在中间会有一段时间无法查看数据。这是，用户可以先创建一个对应的临时分区，将新的数据导入到临时分区后，通过替换操作，原子的替换原有分区，以达到目的。
+
+2. 修改分桶数
+
+ 某些情况下，用户在创建分区时使用了不合适的分桶数。则用户可以先创建一个对应分区范围的临时分区，并指定新的分桶数。然后通过 `INSERT INTO` 命令将正式分区的数据导入到临时分区中，通过替换操作，原子的替换原有分区，以达到目的。
+
+3. 合并或分割分区
+
+ 某些情况下，用户希望对分区的范围进行修改，比如合并两个分区，或将一个大分区分割成多个小分区。则用户可以先建立对应合并或分割后范围的临时分区，然后通过 `INSERT INTO` 命令将正式分区的数据导入到临时分区中，通过替换操作，原子的替换原有分区，以达到目的。
Review comment:
`然后通过 `INSERT INTO` 命令将正式分区的数据导入到临时分区中`
我理解我们之后应该是要支持auto merge 和split 的。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386069252
##########
File path: fe/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
##########
@@ -407,4 +410,22 @@ public static boolean analyzeInMemory(Map<String, String> properties, boolean de
}
return defaultInMemory;
}
+
+ public static boolean analyzeStrictRange(Map<String, String> properties, boolean defaultVal) {
+ if (properties != null && properties.containsKey(PROPERTIES_STRICT_RANGE)) {
+ String inMemory = properties.get(PROPERTIES_STRICT_RANGE);
Review comment:
I will merge analyze of `inMemory, strictRange, useTempPartitionName` into one common method `analyzeBooleanProp`. Because they looks just same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386028217
##########
File path: fe/src/main/java/org/apache/doris/alter/MaterializedViewHandler.java
##########
@@ -1019,6 +1024,11 @@ private void getOldAlterJobInfos(Database db, List<List<Comparable>> rollupJobIn
@Override
public void process(List<AlterClause> alterClauses, String clusterName, Database db, OlapTable olapTable)
throws DdlException, AnalysisException {
+
+ if (olapTable.existTempPartitions()) {
+ throw new DdlException("Can not alter table when there is temp partitions in table");
Review comment:
```suggestion
throw new DdlException("Can not alter table when there are temp partitions in table");
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386069567
##########
File path: fe/src/main/java/org/apache/doris/common/util/RangeUtils.java
##########
@@ -0,0 +1,210 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common.util;
+
+import org.apache.doris.catalog.PartitionKey;
+import org.apache.doris.common.DdlException;
+
+import com.google.common.collect.BoundType;
+import com.google.common.collect.Range;
+import com.google.common.collect.RangeMap;
+import com.google.common.collect.TreeRangeMap;
+
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+
+public class RangeUtils {
+
+ public static final Comparator<Map.Entry<Long, Range<PartitionKey>>> RANGE_MAP_ENTRY_COMPARATOR =
+ Comparator.comparing(o -> o.getValue().lowerEndpoint());
+
+ public static final Comparator<Range<PartitionKey>> RANGE_COMPARATOR =
+ Comparator.comparing(o -> o.lowerEndpoint());
+
+ public static void checkRangeIntersect(Range<PartitionKey> range1, Range<PartitionKey> range2) throws DdlException {
+ if (range2.isConnected(range1)) {
+ if (!range2.intersection(range1).isEmpty()) {
+ throw new DdlException("Range " + range1 + " is intersected with range: " + range2);
+ }
+ }
+ }
+
+ /*
+ * Pass only if the 2 range lists are exactly same
+ * What is "exactly same"?
+ * 1. {[0, 10), [10, 20)} exactly same as {[0, 20)}
+ * 2. {[0, 10), [15, 20)} exactly same as {[0, 10), [15, 18), [18, 20)}
+ * 3. {[0, 10), [15, 20)} exactly same as {[0, 10), [15, 20)}
+ * 4. {[0, 10), [15, 20)} NOT exactly same as {[0, 20)}
+ *
+ * Here I will use an example to explain the algorithm:
+ * list1: {[0, 10), [15, 20)}
+ * list2: {[0, 10), [15, 18), [18, 20)}
+ *
+ * 1. sort 2 lists first (the above 2 lists are already sorted)
+ * 2. Begin to compare ranges from index 0: [0, 10) and [0, 10)
+ * 2.1 lower bounds (0 and 0) are equal.
+ * 2.2 upper bounds (10 and 10) are equal.
+ * 3. Begin to compare next 2 ranges [15, 20) and [15, 18)
+ * 3.1 lower bounds (15 and 15) are equal.
+ * 3.2 upper bounds (20 and 18) are not equal. and 20 > 18
+ * 3.3 Split range [15, 20) to [15, 18) and [18, 20)
+ * 4. Begin to compare next 2 ranges [18, 20) and [18, 20), the first [18, 20) is the splitted range
+ * 4.1 lower bounds (18 and 18) are equal.
+ * 4.2 upper bounds (20 and 20) are equal.
+ * 5. Not more next ranges, so 2 lists are equal.
+ */
+ public static void checkRangeListsMatch(List<Range<PartitionKey>> list1, List<Range<PartitionKey>> list2) throws DdlException {
Review comment:
An exception can bring some detail error msgs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386069660
##########
File path: fe/src/main/java/org/apache/doris/common/util/RangeUtils.java
##########
@@ -0,0 +1,210 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common.util;
+
+import org.apache.doris.catalog.PartitionKey;
+import org.apache.doris.common.DdlException;
+
+import com.google.common.collect.BoundType;
+import com.google.common.collect.Range;
+import com.google.common.collect.RangeMap;
+import com.google.common.collect.TreeRangeMap;
+
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Map;
+
+public class RangeUtils {
Review comment:
Actually, this utils has been test in TempPartitionsTest, both in `testForStrictRangeCheck()` and `testForMultiPartitionTable()`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386068923
##########
File path: fe/src/main/java/org/apache/doris/catalog/OlapTable.java
##########
@@ -624,6 +632,14 @@ public Partition getPartition(String partitionName) {
return nameToPartition.get(partitionName);
}
+ public Partition getPartition(String partitionName, boolean isTempPartition) {
Review comment:
If split into 2 methods, we have to write `if .. else ..` every time we call `getPartition()`, which is not convenient.
But maybe I will change it when implementing `loading into temp partition`. It depends on how convenient it is to use in the loading process.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386069252
##########
File path: fe/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
##########
@@ -407,4 +410,22 @@ public static boolean analyzeInMemory(Map<String, String> properties, boolean de
}
return defaultInMemory;
}
+
+ public static boolean analyzeStrictRange(Map<String, String> properties, boolean defaultVal) {
+ if (properties != null && properties.containsKey(PROPERTIES_STRICT_RANGE)) {
+ String inMemory = properties.get(PROPERTIES_STRICT_RANGE);
Review comment:
I will merge analyze of `inMemory, strictRange, useTempPartitionName` into one common method. Because they looks just same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] imay commented on a change in pull request #2828:
[Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
imay commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r375674869
##########
File path: docs/documentation/cn/administrator-guide/alter-table/alter-table-temp-partition.md
##########
@@ -0,0 +1,117 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 临时分区
+
+在 0.12 版本中，Doris 支持了临时分区功能。临时分区是对 Doris 中**已存在的数据表分区**的一个复制。这个复制操作仅复制分区的元数据，如分区值范围，分桶数，副本数等等，而**不复制数据**。
Review comment:
> What you mean is temp partitions need to support split and merge ?
No.
I think we can complete the partition split and merge functions through temporary partitions. For example, we create a temporary partition and import the partition data that needs to be merged into this new temporary partition. When the data is imported, replace the previous partitions with this partition. In the same way, partition split can also be achieved through this mechanism.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386074466
##########
File path: fe/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
##########
@@ -407,4 +410,22 @@ public static boolean analyzeInMemory(Map<String, String> properties, boolean de
}
return defaultInMemory;
}
+
+ public static boolean analyzeStrictRange(Map<String, String> properties, boolean defaultVal) {
+ if (properties != null && properties.containsKey(PROPERTIES_STRICT_RANGE)) {
+ String inMemory = properties.get(PROPERTIES_STRICT_RANGE);
Review comment:
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386074617
##########
File path: fe/src/main/java/org/apache/doris/qe/ShowExecutor.java
##########
@@ -1215,12 +1215,17 @@ private void handleShowTablet() throws AnalysisException {
}
boolean stop = false;
Collection<Partition> partitions = new ArrayList<Partition>();
- List<String> partitionNames = showStmt.getPartitionNames();
if (showStmt.hasPartition()) {
- for (Partition partition : olapTable.getPartitions()) {
- if (partitionNames.contains(partition.getName())) {
- partitions.add(partition);
+ List<String> partitionNames = showStmt.getPartitionNames();
+ for (String partName : partitionNames) {
+ Partition partition = olapTable.getPartition(partName);
+ if (partition == null) {
Review comment:
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r375676702
##########
File path: docs/documentation/cn/administrator-guide/alter-table/alter-table-temp-partition.md
##########
@@ -0,0 +1,117 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 临时分区
+
+在 0.12 版本中，Doris 支持了临时分区功能。临时分区是对 Doris 中**已存在的数据表分区**的一个复制。这个复制操作仅复制分区的元数据，如分区值范围，分桶数，副本数等等，而**不复制数据**。
Review comment:
Oh. My misunderstood...
I see and I agree with you ！
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386029002
##########
File path: fe/src/main/java/org/apache/doris/catalog/OlapTable.java
##########
@@ -1241,4 +1264,141 @@ public void setIsInMemory(boolean isInMemory) {
tableProperty.modifyTableProperties(PropertyAnalyzer.PROPERTIES_INMEMORY, Boolean.valueOf(isInMemory).toString());
tableProperty.buildInMemory();
}
+
+ // return true if partition with given name already exist, both in partitions and temp partitions.
+ // return false otherwise
+ public boolean checkPartitionNameExist(String partitionName) {
+ if (nameToPartition.containsKey(partitionName)) {
+ return true;
+ }
+ return tempPartitions.hasPartition(partitionName);
+ }
+
+ // if includeTempPartition is true, check if temp partition with given name exist,
+ // if includeTempPartition is false, check if temp partition with given name exist.
+ // return true if exist, otherwise, return false;
+ public boolean checkPartitionNameExist(String partitionName, boolean isTempPartition) {
Review comment:
Split this method to `checkNormalPartitionExist` and `checkTempPartitionExist`, then, we don't need the long comment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] imay commented on a change in pull request #2828:
[Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
imay commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r375328471
##########
File path: docs/documentation/cn/administrator-guide/alter-table/alter-table-temp-partition.md
##########
@@ -0,0 +1,117 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 临时分区
+
+在 0.12 版本中，Doris 支持了临时分区功能。临时分区是对 Doris 中**已存在的数据表分区**的一个复制。这个复制操作仅复制分区的元数据，如分区值范围，分桶数，副本数等等，而**不复制数据**。
Review comment:
I don't think the temporary partition should be a metadata copy of an existing partition. How do I accomplish the following things? How to split a particularly large partition into multiple small partitions? Or how to merge multiple partitions into one large partition?
So my point is that temporary partitions should not be bound to existing partitions, they should exist independently. Including the temporary partition should have a separate partition name, and this name has nothing to do with the existing visible partition.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r386028821
##########
File path: fe/src/main/java/org/apache/doris/catalog/OlapTable.java
##########
@@ -624,6 +632,14 @@ public Partition getPartition(String partitionName) {
return nameToPartition.get(partitionName);
}
+ public Partition getPartition(String partitionName, boolean isTempPartition) {
Review comment:
Split `getPartition ` method to `getPartition` and `getTempPartition`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#2828: [Temp Partition] Support add/drop/replace temp partitions
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #2828: [Temp Partition] Support add/drop/replace temp partitions
URL: https://github.com/apache/incubator-doris/pull/2828#discussion_r375344047
##########
File path: docs/documentation/cn/administrator-guide/alter-table/alter-table-temp-partition.md
##########
@@ -0,0 +1,117 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 临时分区
+
+在 0.12 版本中，Doris 支持了临时分区功能。临时分区是对 Doris 中**已存在的数据表分区**的一个复制。这个复制操作仅复制分区的元数据，如分区值范围，分桶数，副本数等等，而**不复制数据**。
Review comment:
**Tmp partition is mainly for overwrite load, and it should alive very short time.**
For the partition merge and split, we could disable partition merge and split if there is tmp partition, like Alter operation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org