You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/10/12 01:29:39 UTC
[GitHub] [doris] weizhengte opened a new pull request, #13303: [enhancement](statistics) Support to drop table or partition statistics
weizhengte opened a new pull request, #13303:
URL: https://github.com/apache/doris/pull/13303
# Proposed changes
Manually drop statistics for tables or partitions. Table or partition can be specified, if neither is specified, all statistics under the current database will be deleted.
syntax:
```SQL
DROP STATS [TableName [PARTITIONS(partitionNames)]];
```
Issue Number: close #xxx
## Problem summary
Describe your changes.
## Checklist(Required)
1. Does it affect the original behavior:
- [ ] Yes
- [ ] No
- [ ] I don't know
2. Has unit tests been added:
- [ ] Yes
- [ ] No
- [ ] No Need
3. Has document been added or modified:
- [ ] Yes
- [ ] No
- [ ] No Need
4. Does it need to update dependencies:
- [ ] Yes
- [ ] No
5. Are there any changes that cannot be rolled back:
- [ ] Yes (If Yes, please explain WHY)
- [ ] No
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #13303: [enhancement](statistics) Support to drop table or partition statistics
Posted by GitBox <gi...@apache.org>.
weizhengte commented on code in PR #13303:
URL: https://github.com/apache/doris/pull/13303#discussion_r993143111
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/Statistics.java:
##########
@@ -159,6 +164,26 @@ public void updateColumnStats(long tableId, String partitionName, String columnN
}
}
+ public void dropTableStats(long tableId) {
+ dropPartitionStats(tableId, null);
+ }
+
+ public void dropPartitionStats(long tableId, String partitionName) {
Review Comment:
> you can simply add synchronized modifier to this method, since whole code in it is in the synced block
It is true that all delete operations are performed here, but idToTableStats may be updated by the method of updating statistics.
> Besides, I think the ` if (idToTableStats.containsKey(tableId))` is a bit redundant, since you will use the value anyway in the inner block
It's okay to remove it here, but it needs to be judged again in `idToTableStats.get(tableId)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on a diff in pull request #13303: [enhancement](statistics) Support to drop table or partition statistics
Posted by GitBox <gi...@apache.org>.
Kikyou1997 commented on code in PR #13303:
URL: https://github.com/apache/doris/pull/13303#discussion_r993050520
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/DropTableStatsStmt.java:
##########
@@ -0,0 +1,140 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.Table;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.Util;
+import org.apache.doris.mysql.privilege.PaloAuth;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.Maps;
+import com.google.common.collect.Sets;
+
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Manually drop statistics for tables or partitions.
+ * Table or partition can be specified, if neither is specified,
+ * all statistics under the current database will be deleted.
+ *
+ * syntax:
+ * DROP STATS [TableName [PARTITIONS(partitionNames)]];
+ */
+public class DropTableStatsStmt extends DdlStmt {
+ private final TableName tableName;
+ private final PartitionNames optPartitionNames;
+
+ // after analyzed
+ private final Map<Long, Set<String>> tblIdToPartition = Maps.newHashMap();
+
+ public DropTableStatsStmt(TableName tableName, PartitionNames optPartitionNames) {
+ this.tableName = tableName;
+ this.optPartitionNames = optPartitionNames;
+ }
+
+ public Map<Long, Set<String>> getTblIdToPartition() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The partition name must be obtained after the parsing is complete");
+ return tblIdToPartition;
+ }
+
+ @Override
+ public void analyze(Analyzer analyzer) throws UserException {
+ super.analyze(analyzer);
+
+ if (tableName != null) {
+ if (Strings.isNullOrEmpty(tableName.getDb())) {
+ tableName.setDb(analyzer.getDefaultDb());
+ }
+
+ tableName.analyze(analyzer);
+
+ // check whether the deletion permission is granted
+ checkAnalyzePriv(tableName.getDb(), tableName.getTbl());
+
+ // disallow external catalog
+ Util.prohibitExternalCatalog(tableName.getCtl(), this.getClass().getSimpleName());
+
+ Database db = analyzer.getEnv().getInternalCatalog()
+ .getDbOrAnalysisException(tableName.getDb());
+ long tableId = db.getTableOrAnalysisException(tableName.getTbl()).getId();
+
+ if (optPartitionNames == null) {
+ tblIdToPartition.put(tableId, null);
+ } else {
+ optPartitionNames.analyze(analyzer);
+ List<String> pNames = optPartitionNames.getPartitionNames();
+ HashSet<String> partitionNames = Sets.newHashSet(pNames);
+ tblIdToPartition.put(tableId, partitionNames);
+ }
+ } else {
+ Database db = analyzer.getEnv().getInternalCatalog()
+ .getDbOrAnalysisException(analyzer.getDefaultDb());
+ for (Table table : db.getTables()) {
Review Comment:
Why put partitions of every tables in db into the map
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/Statistics.java:
##########
@@ -159,6 +164,26 @@ public void updateColumnStats(long tableId, String partitionName, String columnN
}
}
+ public void dropTableStats(long tableId) {
+ dropPartitionStats(tableId, null);
+ }
+
+ public void dropPartitionStats(long tableId, String partitionName) {
Review Comment:
you can simply add synchronized modifier to this method, since whole code in it is in the synced block
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/Statistics.java:
##########
@@ -159,6 +164,26 @@ public void updateColumnStats(long tableId, String partitionName, String columnN
}
}
+ public void dropTableStats(long tableId) {
+ dropPartitionStats(tableId, null);
+ }
+
+ public void dropPartitionStats(long tableId, String partitionName) {
Review Comment:
Besides, I think the ` if (idToTableStats.containsKey(tableId))` is a bit redundant, since you will use the value anyway in the inner block
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/TableStats.java:
##########
@@ -78,7 +78,7 @@ public TableStats(double rowCount, long dataSize) {
}
public double getRowCount() {
- if (rowCount == -1) {
+ if (rowCount == -1 && !nameToPartitionStats.isEmpty()) {
Review Comment:
Is latter condition necessary?
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsManager.java:
##########
@@ -58,6 +60,27 @@ public Statistics getStatistics() {
return statistics;
}
+ /**
+ * Support for deleting table or partition statistics.
+ *
+ * @param stmt get table name and partition name from it.
+ */
+ public void dropStats(DropTableStatsStmt stmt) {
Review Comment:
seems that the grammer doesn't allow to drop multiple table in single statement though
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #13303: [enhancement](statistics) Support to drop table or partition statistics
Posted by GitBox <gi...@apache.org>.
morrySnow commented on code in PR #13303:
URL: https://github.com/apache/doris/pull/13303#discussion_r994316905
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/DropTableStatsStmt.java:
##########
@@ -0,0 +1,140 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.Table;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.Util;
+import org.apache.doris.mysql.privilege.PaloAuth;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.Maps;
+import com.google.common.collect.Sets;
+
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Manually drop statistics for tables or partitions.
+ * Table or partition can be specified, if neither is specified,
+ * all statistics under the current database will be deleted.
+ *
+ * syntax:
+ * DROP STATS [TableName [PARTITIONS(partitionNames)]];
+ */
+public class DropTableStatsStmt extends DdlStmt {
+ private final TableName tableName;
+ private final PartitionNames optPartitionNames;
+
+ // after analyzed
+ private final Map<Long, Set<String>> tblIdToPartition = Maps.newHashMap();
+
+ public DropTableStatsStmt(TableName tableName, PartitionNames optPartitionNames) {
+ this.tableName = tableName;
+ this.optPartitionNames = optPartitionNames;
+ }
+
+ public Map<Long, Set<String>> getTblIdToPartition() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The partition name must be obtained after the parsing is complete");
+ return tblIdToPartition;
+ }
+
+ @Override
+ public void analyze(Analyzer analyzer) throws UserException {
+ super.analyze(analyzer);
+
+ if (tableName != null) {
+ if (Strings.isNullOrEmpty(tableName.getDb())) {
+ tableName.setDb(analyzer.getDefaultDb());
+ }
+
+ tableName.analyze(analyzer);
+
+ // check whether the deletion permission is granted
+ checkAnalyzePriv(tableName.getDb(), tableName.getTbl());
+
+ // disallow external catalog
+ Util.prohibitExternalCatalog(tableName.getCtl(), this.getClass().getSimpleName());
+
+ Database db = analyzer.getEnv().getInternalCatalog()
+ .getDbOrAnalysisException(tableName.getDb());
+ long tableId = db.getTableOrAnalysisException(tableName.getTbl()).getId();
+
+ if (optPartitionNames == null) {
+ tblIdToPartition.put(tableId, null);
+ } else {
+ optPartitionNames.analyze(analyzer);
+ List<String> pNames = optPartitionNames.getPartitionNames();
+ HashSet<String> partitionNames = Sets.newHashSet(pNames);
+ tblIdToPartition.put(tableId, partitionNames);
+ }
+ } else {
+ Database db = analyzer.getEnv().getInternalCatalog()
+ .getDbOrAnalysisException(analyzer.getDefaultDb());
+ for (Table table : db.getTables()) {
+ checkAnalyzePriv(db.getFullName(), table.getName());
+ tblIdToPartition.put(table.getId(), null);
+ }
+ }
+ }
+
+ private void checkAnalyzePriv(String dbName, String tblName) throws AnalysisException {
+ PaloAuth auth = Env.getCurrentEnv().getAuth();
+ if (!auth.checkTblPriv(ConnectContext.get(), dbName, tblName, PrivPredicate.DROP)) {
+ ErrorReport.reportAnalysisException(
+ ErrorCode.ERR_TABLEACCESS_DENIED_ERROR,
+ "DROP",
+ ConnectContext.get().getQualifiedUser(),
+ ConnectContext.get().getRemoteIP(),
+ dbName + ": " + tblName);
Review Comment:
dbName + "." + tblName is better?
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/TableStats.java:
##########
@@ -78,7 +78,7 @@ public TableStats(double rowCount, long dataSize) {
}
public double getRowCount() {
- if (rowCount == -1) {
+ if (rowCount == -1 && !nameToPartitionStats.isEmpty()) {
Review Comment:
it is better add comment to record the reason
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #13303: [enhancement](statistics) Support to drop table or partition statistics
Posted by GitBox <gi...@apache.org>.
weizhengte commented on code in PR #13303:
URL: https://github.com/apache/doris/pull/13303#discussion_r994625349
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/TableStats.java:
##########
@@ -78,7 +78,7 @@ public TableStats(double rowCount, long dataSize) {
}
public double getRowCount() {
- if (rowCount == -1) {
+ if (rowCount == -1 && !nameToPartitionStats.isEmpty()) {
Review Comment:
Sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow merged pull request #13303: [enhancement](statistics) Support to drop table or partition statistics
Posted by GitBox <gi...@apache.org>.
morrySnow merged PR #13303:
URL: https://github.com/apache/doris/pull/13303
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #13303: [enhancement](statistics) Support to drop table or partition statistics
Posted by GitBox <gi...@apache.org>.
weizhengte commented on code in PR #13303:
URL: https://github.com/apache/doris/pull/13303#discussion_r993136661
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/TableStats.java:
##########
@@ -78,7 +78,7 @@ public TableStats(double rowCount, long dataSize) {
}
public double getRowCount() {
- if (rowCount == -1) {
+ if (rowCount == -1 && !nameToPartitionStats.isEmpty()) {
Review Comment:
This condition is added mainly because the result returns `0` instead of the expected `-1` when `nameToPartitionStats` is empty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on pull request #13303: [enhancement](statistics) Support to drop table or partition statistics
Posted by GitBox <gi...@apache.org>.
Kikyou1997 commented on PR #13303:
URL: https://github.com/apache/doris/pull/13303#issuecomment-1275656637
I have a little question is that when will a user want to delete stats of some partition rather than whole table
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #13303: [enhancement](statistics) Support to drop table or partition statistics
Posted by GitBox <gi...@apache.org>.
weizhengte commented on code in PR #13303:
URL: https://github.com/apache/doris/pull/13303#discussion_r994624256
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/DropTableStatsStmt.java:
##########
@@ -0,0 +1,140 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.Table;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.Util;
+import org.apache.doris.mysql.privilege.PaloAuth;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.Maps;
+import com.google.common.collect.Sets;
+
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Manually drop statistics for tables or partitions.
+ * Table or partition can be specified, if neither is specified,
+ * all statistics under the current database will be deleted.
+ *
+ * syntax:
+ * DROP STATS [TableName [PARTITIONS(partitionNames)]];
+ */
+public class DropTableStatsStmt extends DdlStmt {
+ private final TableName tableName;
+ private final PartitionNames optPartitionNames;
+
+ // after analyzed
+ private final Map<Long, Set<String>> tblIdToPartition = Maps.newHashMap();
+
+ public DropTableStatsStmt(TableName tableName, PartitionNames optPartitionNames) {
+ this.tableName = tableName;
+ this.optPartitionNames = optPartitionNames;
+ }
+
+ public Map<Long, Set<String>> getTblIdToPartition() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The partition name must be obtained after the parsing is complete");
+ return tblIdToPartition;
+ }
+
+ @Override
+ public void analyze(Analyzer analyzer) throws UserException {
+ super.analyze(analyzer);
+
+ if (tableName != null) {
+ if (Strings.isNullOrEmpty(tableName.getDb())) {
+ tableName.setDb(analyzer.getDefaultDb());
+ }
+
+ tableName.analyze(analyzer);
+
+ // check whether the deletion permission is granted
+ checkAnalyzePriv(tableName.getDb(), tableName.getTbl());
+
+ // disallow external catalog
+ Util.prohibitExternalCatalog(tableName.getCtl(), this.getClass().getSimpleName());
+
+ Database db = analyzer.getEnv().getInternalCatalog()
+ .getDbOrAnalysisException(tableName.getDb());
+ long tableId = db.getTableOrAnalysisException(tableName.getTbl()).getId();
+
+ if (optPartitionNames == null) {
+ tblIdToPartition.put(tableId, null);
+ } else {
+ optPartitionNames.analyze(analyzer);
+ List<String> pNames = optPartitionNames.getPartitionNames();
+ HashSet<String> partitionNames = Sets.newHashSet(pNames);
+ tblIdToPartition.put(tableId, partitionNames);
+ }
+ } else {
+ Database db = analyzer.getEnv().getInternalCatalog()
+ .getDbOrAnalysisException(analyzer.getDefaultDb());
+ for (Table table : db.getTables()) {
+ checkAnalyzePriv(db.getFullName(), table.getName());
+ tblIdToPartition.put(table.getId(), null);
+ }
+ }
+ }
+
+ private void checkAnalyzePriv(String dbName, String tblName) throws AnalysisException {
+ PaloAuth auth = Env.getCurrentEnv().getAuth();
+ if (!auth.checkTblPriv(ConnectContext.get(), dbName, tblName, PrivPredicate.DROP)) {
+ ErrorReport.reportAnalysisException(
+ ErrorCode.ERR_TABLEACCESS_DENIED_ERROR,
+ "DROP",
+ ConnectContext.get().getQualifiedUser(),
+ ConnectContext.get().getRemoteIP(),
+ dbName + ": " + tblName);
Review Comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #13303: [enhancement](statistics) Support to drop table or partition statistics
Posted by GitBox <gi...@apache.org>.
weizhengte commented on PR #13303:
URL: https://github.com/apache/doris/pull/13303#issuecomment-1275765669
> I have a little question is that when will a user want to delete stats of some partitions rather than whole table, just curious
This is mainly related to the statistics of partition granularity. partition statistics is mainly used for queries that contain partitions. This statistics is more useful if the partition statistics involved can be used instead of all other partition statistics. In addition, we can also collect statistics incrementally by partition, etc.
Coming back to your question, I feel that partition statistics may need to be removed in this scenario. For example, a table has been collected statistics, but the partition time is a long time ago, then you can delete this "expired" partition statistics separately. Also, if the partition data is changed, then it's statistics is invalid and can be deleted. Of course, there are other ways to solve similar problems, if not necessary, it can be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #13303: [enhancement](statistics) Support to drop table or partition statistics
Posted by GitBox <gi...@apache.org>.
weizhengte commented on PR #13303:
URL: https://github.com/apache/doris/pull/13303#issuecomment-1275557017
@morrySnow @Kikyou1997 review plz
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #13303: [enhancement](statistics) Support to drop table or partition statistics
Posted by GitBox <gi...@apache.org>.
weizhengte commented on code in PR #13303:
URL: https://github.com/apache/doris/pull/13303#discussion_r993136661
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/TableStats.java:
##########
@@ -78,7 +78,7 @@ public TableStats(double rowCount, long dataSize) {
}
public double getRowCount() {
- if (rowCount == -1) {
+ if (rowCount == -1 && !nameToPartitionStats.isEmpty()) {
Review Comment:
This condition is added mainly because the result returns 0 instead of the expected -1 when `nameToPartitionStats` is empty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org