You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/10/23 15:27:18 UTC
[GitHub] [doris] morningman commented on a diff in pull request #13533: [improvement](planner) support delete from partitioned table without partition specified
morningman commented on code in PR #13533:
URL: https://github.com/apache/doris/pull/13533#discussion_r1002727749
##########
fe/fe-core/src/main/java/org/apache/doris/load/DeleteHandler.java:
##########
@@ -162,14 +173,46 @@ public void process(DeleteStmt stmt) throws DdlException, QueryStateException {
}
if (noPartitionSpecified) {
+ // Try to get selected partitions if no partition specified in delete statement
+ // Use PartitionPruner to generate the select partitions
if (olapTable.getPartitionInfo().getType() == PartitionType.RANGE
|| olapTable.getPartitionInfo().getType() == PartitionType.LIST) {
- if (!ConnectContext.get().getSessionVariable().isDeleteWithoutPartition()) {
- throw new DdlException("This is a range or list partitioned table."
- + " You should specify partition in delete stmt,"
- + " or set delete_without_partition to true");
+ Set<String> partitionColumnNameSet = Utils.execWithReturnVal(
+ olapTable::getPartitionColumnNames);
+ Map<String, ColumnRange> columnNameToRange = Maps.newHashMap();
+ for (String colName : partitionColumnNameSet) {
+ ColumnRange columnRange = createColumnRange(colName, conditions);
+ // Not all partition columns are involved in predicate conditions
+ if (columnRange != null) {
+ columnNameToRange.put(colName, columnRange);
+ }
+ }
+
+ Collection<Long> selectedPartitionId = null;
+ if (!columnNameToRange.isEmpty()) {
+ PartitionInfo partitionInfo = olapTable.getPartitionInfo();
+ Map<Long, PartitionItem> keyItemMap = partitionInfo.getIdToItem(false);
+ PartitionPruner pruner = olapTable.getPartitionInfo().getType() == PartitionType.RANGE
+ ? new RangePartitionPrunerV2(keyItemMap, partitionInfo.getPartitionColumns(),
+ columnNameToRange)
+ : new ListPartitionPrunerV2(keyItemMap, partitionInfo.getPartitionColumns(),
+ columnNameToRange);
+ selectedPartitionId = Utils.execWithReturnVal(pruner::prune);
Review Comment:
Call `prune` directly?
##########
fe/fe-core/src/main/java/org/apache/doris/load/DeleteHandler.java:
##########
@@ -162,14 +173,46 @@ public void process(DeleteStmt stmt) throws DdlException, QueryStateException {
}
if (noPartitionSpecified) {
+ // Try to get selected partitions if no partition specified in delete statement
+ // Use PartitionPruner to generate the select partitions
if (olapTable.getPartitionInfo().getType() == PartitionType.RANGE
|| olapTable.getPartitionInfo().getType() == PartitionType.LIST) {
- if (!ConnectContext.get().getSessionVariable().isDeleteWithoutPartition()) {
- throw new DdlException("This is a range or list partitioned table."
- + " You should specify partition in delete stmt,"
- + " or set delete_without_partition to true");
+ Set<String> partitionColumnNameSet = Utils.execWithReturnVal(
Review Comment:
Why not just call `olapTable.getPartitionColumnNames()`?
I think we should not eliminate those exceptions.
##########
fe/fe-core/src/main/java/org/apache/doris/load/DeleteHandler.java:
##########
@@ -369,6 +412,77 @@ public void process(DeleteStmt stmt) throws DdlException, QueryStateException {
}
}
+ // Return null if there is no filter for the partition column
+ private ColumnRange createColumnRange(String colName, List<Predicate> conditions) {
+ ColumnRange result = ColumnRange.create();
+ boolean hasRange = false;
+ for (Predicate predicate : conditions) {
+ List<Range<ColumnBound>> bounds = createColumnRange(colName, predicate);
+ if (bounds != null) {
+ hasRange = true;
+ result.intersect(bounds);
+ }
+ }
+ if (hasRange) {
+ return result;
+ } else {
+ return null;
+ }
+ }
+
+ // Return null if the condition is not related to the partition column,
+ // or the operator is not supported.
+ private List<Range<ColumnBound>> createColumnRange(String colName, Predicate condition) {
+ List<Range<ColumnBound>> result = Lists.newLinkedList();
+ if (condition instanceof BinaryPredicate) {
+ BinaryPredicate binaryPredicate = (BinaryPredicate) condition;
+ String columnName = ((SlotRef) binaryPredicate.getChild(0)).getColumnName();
+ if (!colName.equals(columnName)) {
Review Comment:
```suggestion
if (!colName.equalsIgnoreCase(columnName)) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org