You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/04/06 08:53:12 UTC
[GitHub] [incubator-doris] weizhengte opened a new pull request, #8862: [feature-wip](statistics) step5: show statistics job information
weizhengte opened a new pull request, #8862:
URL: https://github.com/apache/incubator-doris/pull/8862
# Proposed changes
This pull request includes some implementations of the statistics(https://github.com/apache/incubator-doris/issues/6370), it will not affect any existing code and users will not be able to create statistics job.
## Problem Summary:
Describe the overview of changes.
## Checklist(Required)
1. Does it affect the original behavior: (Yes/No/I Don't know)
2. Has unit tests been added: (Yes/No/No Need)
3. Has document been added or modified: (Yes/No/No Need)
4. Does it need to update dependencies: (Yes/No)
5. Are there any changes that cannot be rolled back: (Yes/No)
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #8862: [feature-wip](statistics) step5: show statistics job information
Posted by GitBox <gi...@apache.org>.
weizhengte commented on code in PR #8862:
URL: https://github.com/apache/doris/pull/8862#discussion_r935850716
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsJobManager.java:
##########
@@ -140,4 +150,84 @@ private void checkRestrict(long dbId, Set<Long> tableIds) throws AnalysisExcepti
+ Config.cbo_max_statistics_job_num);
}
}
+
+ public List<List<String>> getAnalyzeJobInfos(ShowAnalyzeStmt showStmt) throws AnalysisException {
+ List<List<Comparable>> results = Lists.newArrayList();
+
+ String stateValue = showStmt.getStateValue();
+ StatisticsJob.JobState jobState = null;
+ if (!Strings.isNullOrEmpty(stateValue)) {
+ jobState = StatisticsJob.JobState.valueOf(stateValue);
+ }
+
+ // step 1: get job infos
+ List<Long> jobIds = showStmt.getJobIds();
+ if (jobIds != null && !jobIds.isEmpty()) {
+ for (Long jobId : jobIds) {
+ StatisticsJob statisticsJob = idToStatisticsJob.get(jobId);
+ if (statisticsJob == null) {
+ throw new AnalysisException("No such job id: " + jobId);
+ }
+ if (jobState == null || jobState == statisticsJob.getJobState()) {
+ List<Comparable> showInfo = statisticsJob.getShowInfo(null);
+ results.add(showInfo);
+ }
+ }
+ } else {
+ long dbId = showStmt.getDbId();
+ Set<Long> tblIds = showStmt.getTblIds();
+ for (StatisticsJob statisticsJob : idToStatisticsJob.values()) {
+ long jobDbId = statisticsJob.getDbId();
+ if (jobDbId == dbId) {
+ // check the state
+ if (jobState == null || jobState == statisticsJob.getJobState()) {
+ Set<Long> jobTblIds = statisticsJob.getTblIds();
+ // get the intersection of two sets
+ Set<Long> set = Sets.newHashSet();
+ set.addAll(jobTblIds);
+ set.retainAll(tblIds);
+ for (long tblId : set) {
+ List<Comparable> showInfo = statisticsJob.getShowInfo(tblId);
+ results.add(showInfo);
+ }
+ }
+ }
+ }
+ }
+
+ // step2: order the result
Review Comment:
the where condition is mainly to filter the job status(see jobState), which has been processed above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #8862: [feature-wip](statistics) step5: show statistics job information
Posted by GitBox <gi...@apache.org>.
morrySnow commented on code in PR #8862:
URL: https://github.com/apache/doris/pull/8862#discussion_r935308090
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsJob.java:
##########
@@ -265,4 +273,86 @@ public static StatisticsJob fromAnalyzeStmt(AnalyzeStmt stmt) throws AnalysisExc
Map<String, String> properties = stmt.getProperties();
return new StatisticsJob(dbId, tblIds, tableIdToPartitionName, tableIdToColumnName, properties);
}
+
+ public List<Comparable> getShowInfo(@Nullable Long tableId) throws AnalysisException {
+ List<Comparable> result = Lists.newArrayList();
+
+ result.add(Long.toString(id));
+
+ SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS");
+ result.add(TimeUtils.longToTimeString(createTime, dateFormat));
+ result.add(startTime != -1L ? TimeUtils.longToTimeString(startTime, dateFormat) : "N/A");
+ result.add(finishTime != -1L ? TimeUtils.longToTimeString(finishTime, dateFormat) : "N/A");
+
+ StringBuilder sb = new StringBuilder();
+ for (String errorMsg : errorMsgs) {
+ sb.append(errorMsg).append("\n");
+ }
+ result.add(sb.toString());
+
+ int totalTaskNum = 0;
+ int finishedTaskNum = 0;
+ Map<Long, Set<String>> tblIdToCols = Maps.newHashMap();
+
+ for (StatisticsTask task : tasks) {
+ List<StatisticsDesc> statsDescs = task.getStatsDescs();
+
+ if (!statsDescs.isEmpty()) {
+ // The same task has the same stats properties
+ StatsCategory statsCategory = statsDescs.get(0).getStatsCategory();
+ long tblId = statsCategory.getTableId();
+
+ if (tableId == null || tableId == tblId) {
+ totalTaskNum++;
+ if (task.getTaskState() == StatisticsTask.TaskState.FINISHED) {
+ finishedTaskNum++;
+ }
+
+ String col = statsCategory.getColumnName();
+ if (Strings.isNullOrEmpty(col)) {
+ continue;
+ }
+ if (tblIdToCols.containsKey(tblId)) {
+ tblIdToCols.get(tblId).add(col);
+ } else {
+ Set<String> cols = Sets.newHashSet();
+ cols.add(col);
+ tblIdToCols.put(tblId, cols);
+ }
+ }
+ }
+ }
+
+ List<String> scope = Lists.newArrayList();
+ Database db = Env.getCurrentEnv()
+ .getInternalDataSource().getDbOrAnalysisException(dbId);
+ for (Long tblId : tblIds) {
+ try {
+ Table table = db.getTableOrAnalysisException(tblId);
+ List<Column> baseSchema = table.getBaseSchema();
+ Set<String> cols = tblIdToCols.get(tblId);
+ if (cols != null) {
+ if (baseSchema.size() == cols.size()) {
+ scope.add(table.getName() + "(*)");
+ } else {
+ scope.add(table.getName() + "(" + StringUtils.join(cols.toArray(), ",") + ")");
Review Comment:
```suggestion
scope.add(table.getName() + "(" + StringUtils.join(cols.toArray(), ", ") + ")");
```
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsJob.java:
##########
@@ -265,4 +273,86 @@ public static StatisticsJob fromAnalyzeStmt(AnalyzeStmt stmt) throws AnalysisExc
Map<String, String> properties = stmt.getProperties();
return new StatisticsJob(dbId, tblIds, tableIdToPartitionName, tableIdToColumnName, properties);
}
+
+ public List<Comparable> getShowInfo(@Nullable Long tableId) throws AnalysisException {
+ List<Comparable> result = Lists.newArrayList();
+
+ result.add(Long.toString(id));
+
+ SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS");
+ result.add(TimeUtils.longToTimeString(createTime, dateFormat));
+ result.add(startTime != -1L ? TimeUtils.longToTimeString(startTime, dateFormat) : "N/A");
+ result.add(finishTime != -1L ? TimeUtils.longToTimeString(finishTime, dateFormat) : "N/A");
+
+ StringBuilder sb = new StringBuilder();
+ for (String errorMsg : errorMsgs) {
+ sb.append(errorMsg).append("\n");
+ }
+ result.add(sb.toString());
+
+ int totalTaskNum = 0;
+ int finishedTaskNum = 0;
+ Map<Long, Set<String>> tblIdToCols = Maps.newHashMap();
+
+ for (StatisticsTask task : tasks) {
+ List<StatisticsDesc> statsDescs = task.getStatsDescs();
+
+ if (!statsDescs.isEmpty()) {
+ // The same task has the same stats properties
+ StatsCategory statsCategory = statsDescs.get(0).getStatsCategory();
+ long tblId = statsCategory.getTableId();
+
+ if (tableId == null || tableId == tblId) {
+ totalTaskNum++;
+ if (task.getTaskState() == StatisticsTask.TaskState.FINISHED) {
+ finishedTaskNum++;
+ }
+
+ String col = statsCategory.getColumnName();
+ if (Strings.isNullOrEmpty(col)) {
+ continue;
+ }
+ if (tblIdToCols.containsKey(tblId)) {
+ tblIdToCols.get(tblId).add(col);
+ } else {
+ Set<String> cols = Sets.newHashSet();
+ cols.add(col);
+ tblIdToCols.put(tblId, cols);
+ }
+ }
+ }
+ }
+
+ List<String> scope = Lists.newArrayList();
+ Database db = Env.getCurrentEnv()
+ .getInternalDataSource().getDbOrAnalysisException(dbId);
+ for (Long tblId : tblIds) {
+ try {
+ Table table = db.getTableOrAnalysisException(tblId);
+ List<Column> baseSchema = table.getBaseSchema();
+ Set<String> cols = tblIdToCols.get(tblId);
+ if (cols != null) {
+ if (baseSchema.size() == cols.size()) {
+ scope.add(table.getName() + "(*)");
+ } else {
+ scope.add(table.getName() + "(" + StringUtils.join(cols.toArray(), ",") + ")");
+ }
+ }
+ } catch (AnalysisException e) {
+ // catch this exception when table is dropped
+ LOG.info("get table failed, tableId: " + tblId, e);
+ }
+ }
+
+ result.add(StringUtils.join(scope.toArray(), ","));
+ result.add(finishedTaskNum + "/" + totalTaskNum);
+
+ if (totalTaskNum == finishedTaskNum) {
Review Comment:
why need to process FINISHED specially
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsJob.java:
##########
@@ -265,4 +273,86 @@ public static StatisticsJob fromAnalyzeStmt(AnalyzeStmt stmt) throws AnalysisExc
Map<String, String> properties = stmt.getProperties();
return new StatisticsJob(dbId, tblIds, tableIdToPartitionName, tableIdToColumnName, properties);
}
+
+ public List<Comparable> getShowInfo(@Nullable Long tableId) throws AnalysisException {
+ List<Comparable> result = Lists.newArrayList();
+
+ result.add(Long.toString(id));
+
+ SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS");
+ result.add(TimeUtils.longToTimeString(createTime, dateFormat));
+ result.add(startTime != -1L ? TimeUtils.longToTimeString(startTime, dateFormat) : "N/A");
+ result.add(finishTime != -1L ? TimeUtils.longToTimeString(finishTime, dateFormat) : "N/A");
+
+ StringBuilder sb = new StringBuilder();
+ for (String errorMsg : errorMsgs) {
+ sb.append(errorMsg).append("\n");
+ }
+ result.add(sb.toString());
+
+ int totalTaskNum = 0;
+ int finishedTaskNum = 0;
+ Map<Long, Set<String>> tblIdToCols = Maps.newHashMap();
+
+ for (StatisticsTask task : tasks) {
+ List<StatisticsDesc> statsDescs = task.getStatsDescs();
+
+ if (!statsDescs.isEmpty()) {
+ // The same task has the same stats properties
+ StatsCategory statsCategory = statsDescs.get(0).getStatsCategory();
+ long tblId = statsCategory.getTableId();
+
+ if (tableId == null || tableId == tblId) {
+ totalTaskNum++;
+ if (task.getTaskState() == StatisticsTask.TaskState.FINISHED) {
+ finishedTaskNum++;
+ }
+
+ String col = statsCategory.getColumnName();
+ if (Strings.isNullOrEmpty(col)) {
+ continue;
+ }
+ if (tblIdToCols.containsKey(tblId)) {
+ tblIdToCols.get(tblId).add(col);
+ } else {
+ Set<String> cols = Sets.newHashSet();
+ cols.add(col);
+ tblIdToCols.put(tblId, cols);
Review Comment:
could use computeIfAbsent
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowAnalyzeStmt.java:
##########
@@ -0,0 +1,366 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.catalog.Table;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.OrderByPair;
+import org.apache.doris.mysql.privilege.PaloAuth;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSetMetaData;
+import org.apache.doris.statistics.StatisticsJob;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Sets;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.IntStream;
+
+/**
+ * ShowAnalyzeStmt is used to show statistics job info.
+ * syntax:
+ * SHOW ANALYZE
+ * [TABLE | ID]
+ * [
+ * WHERE
+ * [STATE = ["PENDING"|"SCHEDULING"|"RUNNING"|"FINISHED"|"FAILED"|"CANCELLED"]]
+ * ]
+ * [ORDER BY ...]
+ * [LIMIT limit][OFFSET offset];
+ */
+public class ShowAnalyzeStmt extends ShowStmt {
+ private static final String STATE_NAME = "state";
+ private static final ImmutableList<String> TITLE_NAMES = new ImmutableList.Builder<String>()
+ .add("id")
+ .add("create_time")
+ .add("start_time")
+ .add("finish_time")
+ .add("error_msg")
+ .add("scope")
+ .add("progress")
+ .add("state")
+ .build();
+
+ private List<Long> jobIds;
+ private TableName dbTableName;
+ private Expr whereClause;
+ private LimitElement limitElement;
+ private List<OrderByElement> orderByElements;
+
+ // after analyzed
+ private long dbId;
+ private final Set<Long> tblIds = Sets.newHashSet();
+
+ private String stateValue;
+ private ArrayList<OrderByPair> orderByPairs;
+
+ public ShowAnalyzeStmt() {
+ }
+
+ public ShowAnalyzeStmt(List<Long> jobIds) {
+ this.jobIds = jobIds;
+ }
+
+ public ShowAnalyzeStmt(TableName dbTableName,
+ Expr whereClause,
+ List<OrderByElement> orderByElements,
+ LimitElement limitElement) {
+ this.dbTableName = dbTableName;
+ this.whereClause = whereClause;
+ this.orderByElements = orderByElements;
+ this.limitElement = limitElement;
+ }
+
+ public List<Long> getJobIds() {
+ return jobIds;
+ }
+
+ public long getDbId() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The dbId must be obtained after the parsing is complete");
+ return dbId;
+ }
+
+ public Set<Long> getTblIds() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The dbId must be obtained after the parsing is complete");
+ return tblIds;
+ }
+
+ public String getStateValue() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return stateValue;
+ }
+
+ public ArrayList<OrderByPair> getOrderByPairs() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return orderByPairs;
+ }
+
+ public long getLimit() {
+ if (limitElement != null && limitElement.hasLimit()) {
+ return limitElement.getLimit();
+ }
+ return -1L;
+ }
+
+ public long getOffset() {
+ if (limitElement != null && limitElement.hasOffset()) {
+ return limitElement.getOffset();
+ }
+ return -1L;
+ }
+
+ @Override
+ public void analyze(Analyzer analyzer) throws AnalysisException, UserException {
+ super.analyze(analyzer);
+
+ if (dbTableName != null) {
+ dbTableName.analyze(analyzer);
+ String dbName = dbTableName.getDb();
+ String tblName = dbTableName.getTbl();
+ checkShowAnalyzePriv(dbName, tblName);
+
+ Database db = analyzer.getEnv()
+ .getInternalDataSource().getDbOrAnalysisException(dbName);
+ Table table = db.getTableOrAnalysisException(tblName);
+ dbId = db.getId();
+ tblIds.add(table.getId());
+ } else {
+ // analyze the current default db
+ String dbName = analyzer.getDefaultDb();
+ if (Strings.isNullOrEmpty(dbName)) {
+ ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR);
+ }
+
+ Database db = analyzer.getEnv()
+ .getInternalDataSource().getDbOrAnalysisException(dbName);
+
+ db.readLock();
+ try {
+ List<Table> tables = db.getTables();
+ for (Table table : tables) {
+ checkShowAnalyzePriv(dbName, table.getName());
+ }
+
+ dbId = db.getId();
+ for (Table table : tables) {
+ long tblId = table.getId();
+ tblIds.add(tblId);
+ }
+ } finally {
+ db.readUnlock();
+ }
+ }
+
+ // analyze where clause if not null
+ if (whereClause != null) {
+ if (whereClause instanceof CompoundPredicate) {
+ CompoundPredicate cp = (CompoundPredicate) whereClause;
+ if (cp.getOp() != CompoundPredicate.Operator.AND) {
+ throw new AnalysisException("Only allow compound predicate with operator AND");
+ }
+ // check whether left.columnName equals to right.columnName
+ checkPredicateName(cp.getChild(0), cp.getChild(1));
+ analyzeSubPredicate(cp.getChild(0));
+ analyzeSubPredicate(cp.getChild(1));
+ } else {
+ analyzeSubPredicate(whereClause);
+ }
+ }
+
+ // analyze order by
+ if (orderByElements != null && !orderByElements.isEmpty()) {
+ orderByPairs = new ArrayList<>();
+ for (OrderByElement orderByElement : orderByElements) {
+ if (orderByElement.getExpr() instanceof SlotRef) {
+ SlotRef slotRef = (SlotRef) orderByElement.getExpr();
+ int index = analyzeColumn(slotRef.getColumnName());
+ OrderByPair orderByPair = new OrderByPair(index, !orderByElement.getIsAsc());
+ orderByPairs.add(orderByPair);
+ } else {
+ throw new AnalysisException("Should order by column");
+ }
+ }
+ }
+ }
+
+ @Override
+ public ShowResultSetMetaData getMetaData() {
+ ShowResultSetMetaData.Builder builder = ShowResultSetMetaData.builder();
+ for (String title : TITLE_NAMES) {
+ builder.addColumn(new Column(title, ScalarType.createVarchar(128)));
+ }
+ return builder.build();
+ }
+
+ @Override
+ public RedirectStatus getRedirectStatus() {
+ return RedirectStatus.FORWARD_NO_SYNC;
+ }
+
+ private void checkShowAnalyzePriv(String dbName, String tblName) throws AnalysisException {
+ PaloAuth auth = Env.getCurrentEnv().getAuth();
+ if (!auth.checkTblPriv(ConnectContext.get(), dbName, tblName, PrivPredicate.SHOW)) {
+ ErrorReport.reportAnalysisException(
+ ErrorCode.ERR_TABLEACCESS_DENIED_ERROR,
+ "SHOW ANALYZE",
+ ConnectContext.get().getQualifiedUser(),
+ ConnectContext.get().getRemoteIP(),
+ dbName + ": " + tblName);
+ }
+ }
+
+ private void checkPredicateName(Expr leftChild, Expr rightChild) throws AnalysisException {
+ String leftChildColumnName = ((SlotRef) leftChild.getChild(0)).getColumnName();
+ String rightChildColumnName = ((SlotRef) rightChild.getChild(0)).getColumnName();
Review Comment:
type cast without type check is OK?
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsJobManager.java:
##########
@@ -140,4 +150,84 @@ private void checkRestrict(long dbId, Set<Long> tableIds) throws AnalysisExcepti
+ Config.cbo_max_statistics_job_num);
}
}
+
+ public List<List<String>> getAnalyzeJobInfos(ShowAnalyzeStmt showStmt) throws AnalysisException {
+ List<List<Comparable>> results = Lists.newArrayList();
+
+ String stateValue = showStmt.getStateValue();
+ StatisticsJob.JobState jobState = null;
+ if (!Strings.isNullOrEmpty(stateValue)) {
+ jobState = StatisticsJob.JobState.valueOf(stateValue);
+ }
+
+ // step 1: get job infos
+ List<Long> jobIds = showStmt.getJobIds();
+ if (jobIds != null && !jobIds.isEmpty()) {
+ for (Long jobId : jobIds) {
+ StatisticsJob statisticsJob = idToStatisticsJob.get(jobId);
+ if (statisticsJob == null) {
+ throw new AnalysisException("No such job id: " + jobId);
+ }
+ if (jobState == null || jobState == statisticsJob.getJobState()) {
+ List<Comparable> showInfo = statisticsJob.getShowInfo(null);
+ results.add(showInfo);
+ }
+ }
+ } else {
+ long dbId = showStmt.getDbId();
+ Set<Long> tblIds = showStmt.getTblIds();
+ for (StatisticsJob statisticsJob : idToStatisticsJob.values()) {
+ long jobDbId = statisticsJob.getDbId();
+ if (jobDbId == dbId) {
+ // check the state
+ if (jobState == null || jobState == statisticsJob.getJobState()) {
+ Set<Long> jobTblIds = statisticsJob.getTblIds();
+ // get the intersection of two sets
+ Set<Long> set = Sets.newHashSet();
+ set.addAll(jobTblIds);
+ set.retainAll(tblIds);
+ for (long tblId : set) {
+ List<Comparable> showInfo = statisticsJob.getShowInfo(tblId);
+ results.add(showInfo);
+ }
+ }
+ }
+ }
+ }
+
+ // step2: order the result
Review Comment:
not process where clause?
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowAnalyzeStmt.java:
##########
@@ -0,0 +1,366 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.catalog.Table;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.OrderByPair;
+import org.apache.doris.mysql.privilege.PaloAuth;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSetMetaData;
+import org.apache.doris.statistics.StatisticsJob;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Sets;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.IntStream;
+
+/**
+ * ShowAnalyzeStmt is used to show statistics job info.
+ * syntax:
+ * SHOW ANALYZE
+ * [TABLE | ID]
+ * [
+ * WHERE
+ * [STATE = ["PENDING"|"SCHEDULING"|"RUNNING"|"FINISHED"|"FAILED"|"CANCELLED"]]
+ * ]
+ * [ORDER BY ...]
+ * [LIMIT limit][OFFSET offset];
+ */
+public class ShowAnalyzeStmt extends ShowStmt {
+ private static final String STATE_NAME = "state";
+ private static final ImmutableList<String> TITLE_NAMES = new ImmutableList.Builder<String>()
+ .add("id")
+ .add("create_time")
+ .add("start_time")
+ .add("finish_time")
+ .add("error_msg")
+ .add("scope")
+ .add("progress")
+ .add("state")
+ .build();
+
+ private List<Long> jobIds;
+ private TableName dbTableName;
+ private Expr whereClause;
+ private LimitElement limitElement;
+ private List<OrderByElement> orderByElements;
+
+ // after analyzed
+ private long dbId;
+ private final Set<Long> tblIds = Sets.newHashSet();
+
+ private String stateValue;
+ private ArrayList<OrderByPair> orderByPairs;
+
+ public ShowAnalyzeStmt() {
+ }
+
+ public ShowAnalyzeStmt(List<Long> jobIds) {
+ this.jobIds = jobIds;
+ }
+
+ public ShowAnalyzeStmt(TableName dbTableName,
+ Expr whereClause,
+ List<OrderByElement> orderByElements,
+ LimitElement limitElement) {
+ this.dbTableName = dbTableName;
+ this.whereClause = whereClause;
+ this.orderByElements = orderByElements;
+ this.limitElement = limitElement;
+ }
+
+ public List<Long> getJobIds() {
+ return jobIds;
+ }
+
+ public long getDbId() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The dbId must be obtained after the parsing is complete");
+ return dbId;
+ }
+
+ public Set<Long> getTblIds() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The dbId must be obtained after the parsing is complete");
+ return tblIds;
+ }
+
+ public String getStateValue() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return stateValue;
+ }
+
+ public ArrayList<OrderByPair> getOrderByPairs() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return orderByPairs;
+ }
+
+ public long getLimit() {
+ if (limitElement != null && limitElement.hasLimit()) {
+ return limitElement.getLimit();
+ }
+ return -1L;
+ }
+
+ public long getOffset() {
+ if (limitElement != null && limitElement.hasOffset()) {
+ return limitElement.getOffset();
+ }
+ return -1L;
+ }
+
+ @Override
+ public void analyze(Analyzer analyzer) throws AnalysisException, UserException {
+ super.analyze(analyzer);
+
+ if (dbTableName != null) {
+ dbTableName.analyze(analyzer);
+ String dbName = dbTableName.getDb();
+ String tblName = dbTableName.getTbl();
+ checkShowAnalyzePriv(dbName, tblName);
+
+ Database db = analyzer.getEnv()
+ .getInternalDataSource().getDbOrAnalysisException(dbName);
+ Table table = db.getTableOrAnalysisException(tblName);
+ dbId = db.getId();
+ tblIds.add(table.getId());
+ } else {
+ // analyze the current default db
+ String dbName = analyzer.getDefaultDb();
+ if (Strings.isNullOrEmpty(dbName)) {
+ ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR);
+ }
+
+ Database db = analyzer.getEnv()
+ .getInternalDataSource().getDbOrAnalysisException(dbName);
+
+ db.readLock();
+ try {
+ List<Table> tables = db.getTables();
+ for (Table table : tables) {
+ checkShowAnalyzePriv(dbName, table.getName());
+ }
+
+ dbId = db.getId();
+ for (Table table : tables) {
+ long tblId = table.getId();
+ tblIds.add(tblId);
+ }
+ } finally {
+ db.readUnlock();
+ }
+ }
+
+ // analyze where clause if not null
+ if (whereClause != null) {
+ if (whereClause instanceof CompoundPredicate) {
+ CompoundPredicate cp = (CompoundPredicate) whereClause;
+ if (cp.getOp() != CompoundPredicate.Operator.AND) {
+ throw new AnalysisException("Only allow compound predicate with operator AND");
+ }
+ // check whether left.columnName equals to right.columnName
+ checkPredicateName(cp.getChild(0), cp.getChild(1));
+ analyzeSubPredicate(cp.getChild(0));
Review Comment:
only support one level CompoundPredicate?
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsJobManager.java:
##########
@@ -140,4 +150,84 @@ private void checkRestrict(long dbId, Set<Long> tableIds) throws AnalysisExcepti
+ Config.cbo_max_statistics_job_num);
}
}
+
+ public List<List<String>> getAnalyzeJobInfos(ShowAnalyzeStmt showStmt) throws AnalysisException {
+ List<List<Comparable>> results = Lists.newArrayList();
+
+ String stateValue = showStmt.getStateValue();
+ StatisticsJob.JobState jobState = null;
+ if (!Strings.isNullOrEmpty(stateValue)) {
+ jobState = StatisticsJob.JobState.valueOf(stateValue);
+ }
+
+ // step 1: get job infos
+ List<Long> jobIds = showStmt.getJobIds();
+ if (jobIds != null && !jobIds.isEmpty()) {
+ for (Long jobId : jobIds) {
+ StatisticsJob statisticsJob = idToStatisticsJob.get(jobId);
+ if (statisticsJob == null) {
+ throw new AnalysisException("No such job id: " + jobId);
+ }
+ if (jobState == null || jobState == statisticsJob.getJobState()) {
+ List<Comparable> showInfo = statisticsJob.getShowInfo(null);
+ results.add(showInfo);
+ }
+ }
+ } else {
+ long dbId = showStmt.getDbId();
+ Set<Long> tblIds = showStmt.getTblIds();
+ for (StatisticsJob statisticsJob : idToStatisticsJob.values()) {
+ long jobDbId = statisticsJob.getDbId();
+ if (jobDbId == dbId) {
+ // check the state
+ if (jobState == null || jobState == statisticsJob.getJobState()) {
+ Set<Long> jobTblIds = statisticsJob.getTblIds();
+ // get the intersection of two sets
+ Set<Long> set = Sets.newHashSet();
+ set.addAll(jobTblIds);
+ set.retainAll(tblIds);
+ for (long tblId : set) {
+ List<Comparable> showInfo = statisticsJob.getShowInfo(tblId);
+ results.add(showInfo);
+ }
+ }
+ }
+ }
+ }
+
+ // step2: order the result
+ ListComparator<List<Comparable>> comparator;
+ List<OrderByPair> orderByPairs = showStmt.getOrderByPairs();
+ if (orderByPairs == null) {
+ // sort by id asc
+ comparator = new ListComparator<>(0);
+ } else {
+ OrderByPair[] orderByPairArr = new OrderByPair[orderByPairs.size()];
+ comparator = new ListComparator<>(orderByPairs.toArray(orderByPairArr));
+ }
+ results.sort(comparator);
+
+ // step3: filter by limit
+ long limit = showStmt.getLimit();
+ long offset = showStmt.getOffset() == -1L ? 0 : showStmt.getOffset();
+ if (offset >= results.size()) {
+ results = Collections.emptyList();
+ } else if (limit != -1L) {
+ if ((limit + offset) >= results.size()) {
+ results = results.subList((int) offset, results.size());
+ } else {
+ results = results.subList((int) offset, (int) (limit + offset));
+ }
+ }
+
+ // step4: convert to result and return it
+ List<List<String>> rows = Lists.newArrayList();
+ for (List<Comparable> result : results) {
+ List<String> row = result.stream().map(Object::toString)
+ .collect(Collectors.toCollection(() -> new ArrayList<>(result.size())));
Review Comment:
why not use toList directly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #8862: [feature-wip](statistics) step5: show statistics job information
Posted by GitBox <gi...@apache.org>.
weizhengte commented on code in PR #8862:
URL: https://github.com/apache/doris/pull/8862#discussion_r935847810
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowAnalyzeStmt.java:
##########
@@ -0,0 +1,366 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.catalog.Table;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.OrderByPair;
+import org.apache.doris.mysql.privilege.PaloAuth;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSetMetaData;
+import org.apache.doris.statistics.StatisticsJob;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Sets;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.IntStream;
+
+/**
+ * ShowAnalyzeStmt is used to show statistics job info.
+ * syntax:
+ * SHOW ANALYZE
+ * [TABLE | ID]
+ * [
+ * WHERE
+ * [STATE = ["PENDING"|"SCHEDULING"|"RUNNING"|"FINISHED"|"FAILED"|"CANCELLED"]]
+ * ]
+ * [ORDER BY ...]
+ * [LIMIT limit][OFFSET offset];
+ */
+public class ShowAnalyzeStmt extends ShowStmt {
+ private static final String STATE_NAME = "state";
+ private static final ImmutableList<String> TITLE_NAMES = new ImmutableList.Builder<String>()
+ .add("id")
+ .add("create_time")
+ .add("start_time")
+ .add("finish_time")
+ .add("error_msg")
+ .add("scope")
+ .add("progress")
+ .add("state")
+ .build();
+
+ private List<Long> jobIds;
+ private TableName dbTableName;
+ private Expr whereClause;
+ private LimitElement limitElement;
+ private List<OrderByElement> orderByElements;
+
+ // after analyzed
+ private long dbId;
+ private final Set<Long> tblIds = Sets.newHashSet();
+
+ private String stateValue;
+ private ArrayList<OrderByPair> orderByPairs;
+
+ public ShowAnalyzeStmt() {
+ }
+
+ public ShowAnalyzeStmt(List<Long> jobIds) {
+ this.jobIds = jobIds;
+ }
+
+ public ShowAnalyzeStmt(TableName dbTableName,
+ Expr whereClause,
+ List<OrderByElement> orderByElements,
+ LimitElement limitElement) {
+ this.dbTableName = dbTableName;
+ this.whereClause = whereClause;
+ this.orderByElements = orderByElements;
+ this.limitElement = limitElement;
+ }
+
+ public List<Long> getJobIds() {
+ return jobIds;
+ }
+
+ public long getDbId() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The dbId must be obtained after the parsing is complete");
+ return dbId;
+ }
+
+ public Set<Long> getTblIds() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The dbId must be obtained after the parsing is complete");
+ return tblIds;
+ }
+
+ public String getStateValue() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return stateValue;
+ }
+
+ public ArrayList<OrderByPair> getOrderByPairs() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return orderByPairs;
+ }
+
+ public long getLimit() {
+ if (limitElement != null && limitElement.hasLimit()) {
+ return limitElement.getLimit();
+ }
+ return -1L;
+ }
+
+ public long getOffset() {
+ if (limitElement != null && limitElement.hasOffset()) {
+ return limitElement.getOffset();
+ }
+ return -1L;
+ }
+
+ @Override
+ public void analyze(Analyzer analyzer) throws AnalysisException, UserException {
+ super.analyze(analyzer);
+
+ if (dbTableName != null) {
+ dbTableName.analyze(analyzer);
+ String dbName = dbTableName.getDb();
+ String tblName = dbTableName.getTbl();
+ checkShowAnalyzePriv(dbName, tblName);
+
+ Database db = analyzer.getEnv()
+ .getInternalDataSource().getDbOrAnalysisException(dbName);
+ Table table = db.getTableOrAnalysisException(tblName);
+ dbId = db.getId();
+ tblIds.add(table.getId());
+ } else {
+ // analyze the current default db
+ String dbName = analyzer.getDefaultDb();
+ if (Strings.isNullOrEmpty(dbName)) {
+ ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR);
+ }
+
+ Database db = analyzer.getEnv()
+ .getInternalDataSource().getDbOrAnalysisException(dbName);
+
+ db.readLock();
+ try {
+ List<Table> tables = db.getTables();
+ for (Table table : tables) {
+ checkShowAnalyzePriv(dbName, table.getName());
+ }
+
+ dbId = db.getId();
+ for (Table table : tables) {
+ long tblId = table.getId();
+ tblIds.add(tblId);
+ }
+ } finally {
+ db.readUnlock();
+ }
+ }
+
+ // analyze where clause if not null
+ if (whereClause != null) {
+ if (whereClause instanceof CompoundPredicate) {
+ CompoundPredicate cp = (CompoundPredicate) whereClause;
+ if (cp.getOp() != CompoundPredicate.Operator.AND) {
+ throw new AnalysisException("Only allow compound predicate with operator AND");
+ }
+ // check whether left.columnName equals to right.columnName
+ checkPredicateName(cp.getChild(0), cp.getChild(1));
+ analyzeSubPredicate(cp.getChild(0));
Review Comment:
for statistical job, I think the user mainly cares which state it belongs to, and I've simplified it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #8862: [feature-wip](statistics) step5: show statistics job information
Posted by GitBox <gi...@apache.org>.
weizhengte commented on code in PR #8862:
URL: https://github.com/apache/doris/pull/8862#discussion_r935838686
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsJob.java:
##########
@@ -265,4 +273,86 @@ public static StatisticsJob fromAnalyzeStmt(AnalyzeStmt stmt) throws AnalysisExc
Map<String, String> properties = stmt.getProperties();
return new StatisticsJob(dbId, tblIds, tableIdToPartitionName, tableIdToColumnName, properties);
}
+
+ public List<Comparable> getShowInfo(@Nullable Long tableId) throws AnalysisException {
+ List<Comparable> result = Lists.newArrayList();
+
+ result.add(Long.toString(id));
+
+ SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS");
+ result.add(TimeUtils.longToTimeString(createTime, dateFormat));
+ result.add(startTime != -1L ? TimeUtils.longToTimeString(startTime, dateFormat) : "N/A");
+ result.add(finishTime != -1L ? TimeUtils.longToTimeString(finishTime, dateFormat) : "N/A");
+
+ StringBuilder sb = new StringBuilder();
+ for (String errorMsg : errorMsgs) {
+ sb.append(errorMsg).append("\n");
+ }
+ result.add(sb.toString());
+
+ int totalTaskNum = 0;
+ int finishedTaskNum = 0;
+ Map<Long, Set<String>> tblIdToCols = Maps.newHashMap();
+
+ for (StatisticsTask task : tasks) {
+ List<StatisticsDesc> statsDescs = task.getStatsDescs();
+
+ if (!statsDescs.isEmpty()) {
+ // The same task has the same stats properties
+ StatsCategory statsCategory = statsDescs.get(0).getStatsCategory();
+ long tblId = statsCategory.getTableId();
+
+ if (tableId == null || tableId == tblId) {
+ totalTaskNum++;
+ if (task.getTaskState() == StatisticsTask.TaskState.FINISHED) {
+ finishedTaskNum++;
+ }
+
+ String col = statsCategory.getColumnName();
+ if (Strings.isNullOrEmpty(col)) {
+ continue;
+ }
+ if (tblIdToCols.containsKey(tblId)) {
+ tblIdToCols.get(tblId).add(col);
+ } else {
+ Set<String> cols = Sets.newHashSet();
+ cols.add(col);
+ tblIdToCols.put(tblId, cols);
+ }
+ }
+ }
+ }
+
+ List<String> scope = Lists.newArrayList();
+ Database db = Env.getCurrentEnv()
+ .getInternalDataSource().getDbOrAnalysisException(dbId);
+ for (Long tblId : tblIds) {
+ try {
+ Table table = db.getTableOrAnalysisException(tblId);
+ List<Column> baseSchema = table.getBaseSchema();
+ Set<String> cols = tblIdToCols.get(tblId);
+ if (cols != null) {
+ if (baseSchema.size() == cols.size()) {
+ scope.add(table.getName() + "(*)");
+ } else {
+ scope.add(table.getName() + "(" + StringUtils.join(cols.toArray(), ",") + ")");
+ }
+ }
+ } catch (AnalysisException e) {
+ // catch this exception when table is dropped
+ LOG.info("get table failed, tableId: " + tblId, e);
+ }
+ }
+
+ result.add(StringUtils.join(scope.toArray(), ","));
+ result.add(finishedTaskNum + "/" + totalTaskNum);
+
+ if (totalTaskNum == finishedTaskNum) {
Review Comment:
a statistical job will be divided into multiple tasks to execute, so the job status is finished means that all tasks have been finished.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morrySnow commented on a diff in pull request #8862: [feature-wip](statistics) step5: show statistics job information
Posted by GitBox <gi...@apache.org>.
morrySnow commented on code in PR #8862:
URL: https://github.com/apache/incubator-doris/pull/8862#discussion_r846797887
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowAnalyzeStmt.java:
##########
@@ -0,0 +1,340 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Catalog;
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.catalog.Table;
+import org.apache.doris.cluster.ClusterNamespace;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.OrderByPair;
+import org.apache.doris.mysql.privilege.PaloAuth;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSetMetaData;
+import org.apache.doris.statistics.StatisticsJob;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableList;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * ShowAnalyzeStmt is used to show statistics job info.
+ * <p>
+ * Grammar:
+ * SHOW ANALYZE
+ * [TABLE | ID]
+ * [
+ * WHERE
+ * [STATE = ["PENDING"|"SCHEDULING"|"RUNNING"|"FINISHED"|"FAILED"|"CANCELLED"]]
+ * ]
+ * [ORDER BY ...]
+ * [LIMIT limit][OFFSET offset];
+ */
+public class ShowAnalyzeStmt extends ShowStmt {
+ private static final String STATE_NAME = "state";
+ private static final ImmutableList<String> TITLE_NAMES = new ImmutableList.Builder<String>()
+ .add("id")
+ .add("create_time")
+ .add("start_time")
+ .add("finish_time")
+ .add("error_msg")
+ .add("scope")
+ .add("progress")
+ .add("state")
+ .build();
+
+ private List<Long> jobIds;
+ private TableName dbTableName;
+ private Expr whereClause;
+ private LimitElement limitElement;
+ private List<OrderByElement> orderByElements;
+
+ // after analyzed
+ private Database db;
+ private Table table;
+ private String stateValue;
+ private ArrayList<OrderByPair> orderByPairs;
+
+ public ShowAnalyzeStmt() {
+ }
+
+ public ShowAnalyzeStmt(List<Long> jobIds) {
+ this.jobIds = jobIds;
+ }
+
+ public ShowAnalyzeStmt(TableName dbTableName, Expr whereClause, List<OrderByElement> orderByElements, LimitElement limitElement) {
+ this.dbTableName = dbTableName;
+ this.whereClause = whereClause;
+ this.orderByElements = orderByElements;
+ this.limitElement = limitElement;
+ }
+
+ public List<Long> getJobIds() {
+ return this.jobIds;
+ }
+
+ public Database getDb() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The db name must be obtained after the parsing is complete");
+ return this.db;
+ }
+
+ public Table getTable() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.table;
+ }
+
+ public String getStateValue() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.stateValue;
+ }
+
+ public ArrayList<OrderByPair> getOrderByPairs() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.orderByPairs;
+ }
+
+ public long getLimit() {
+ if (this.limitElement != null && this.limitElement.hasLimit()) {
+ return this.limitElement.getLimit();
+ }
+ return -1L;
+ }
+
+ public long getOffset() {
+ if (this.limitElement != null && this.limitElement.hasOffset()) {
+ return this.limitElement.getOffset();
+ }
+ return -1L;
+ }
+
+ @Override
+ public void analyze(Analyzer analyzer) throws AnalysisException, UserException {
+ super.analyze(analyzer);
+
+ String dbName = null;
+ String tblName = null;
+ if (this.dbTableName != null) {
+ dbName = this.dbTableName.getDb();
+ tblName = this.dbTableName.getTbl();
+ }
+
+ if (Strings.isNullOrEmpty(dbName)) {
+ dbName = analyzer.getDefaultDb();
+ } else {
+ dbName = ClusterNamespace.getFullName(analyzer.getClusterName(), dbName);
+ }
+
+ if (Strings.isNullOrEmpty(dbName)) {
+ ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR);
+ }
+ this.db = analyzer.getCatalog().getDbOrAnalysisException(dbName);
+
+ if (Strings.isNullOrEmpty(tblName)) {
+ this.table = null;
+ List<Table> tables = this.db.getTables();
+ for (Table table : tables) {
+ checkShowAnalyzePriv(dbName, table.getName());
+ }
+ } else {
+ this.table = this.db.getTableOrAnalysisException(tblName);
+ checkShowAnalyzePriv(dbName, this.table.getName());
+ }
+
+ // analyze where clause if not null
+ if (this.whereClause != null) {
+ if (this.whereClause instanceof CompoundPredicate) {
+ CompoundPredicate cp = (CompoundPredicate) this.whereClause;
+ if (cp.getOp() != CompoundPredicate.Operator.AND) {
+ throw new AnalysisException("Only allow compound predicate with operator AND");
+ }
+ // check whether left.columnName equals to right.columnName
+ checkPredicateName(cp.getChild(0), cp.getChild(1));
+ analyzeSubPredicate(cp.getChild(0));
+ analyzeSubPredicate(cp.getChild(1));
+ } else {
+ analyzeSubPredicate(this.whereClause);
+ }
+ }
+
+ // analyze order by
+ if (this.orderByElements != null && !this.orderByElements.isEmpty()) {
+ this.orderByPairs = new ArrayList<>();
+ for (OrderByElement orderByElement : this.orderByElements) {
+ if (!(orderByElement.getExpr() instanceof SlotRef)) {
+ throw new AnalysisException("Should order by column");
+ }
+ SlotRef slotRef = (SlotRef) orderByElement.getExpr();
+ int index = analyzeColumn(slotRef.getColumnName());
+ OrderByPair orderByPair = new OrderByPair(index, !orderByElement.getIsAsc());
+ this.orderByPairs.add(orderByPair);
+ }
+ }
+ }
+
+ @Override
+ public ShowResultSetMetaData getMetaData() {
+ ShowResultSetMetaData.Builder builder = ShowResultSetMetaData.builder();
+ for (String title : TITLE_NAMES) {
+ builder.addColumn(new Column(title, ScalarType.createVarchar(128)));
+ }
+ return builder.build();
+ }
+
+ @Override
+ public RedirectStatus getRedirectStatus() {
+ return RedirectStatus.FORWARD_NO_SYNC;
+ }
+
+ private void checkShowAnalyzePriv(String dbName, String tblName) throws AnalysisException {
+ PaloAuth auth = Catalog.getCurrentCatalog().getAuth();
+ if (!auth.checkTblPriv(ConnectContext.get(), dbName, tblName, PrivPredicate.SHOW)) {
+ ErrorReport.reportAnalysisException(
+ ErrorCode.ERR_TABLEACCESS_DENIED_ERROR,
+ "SHOW ANALYZE",
+ ConnectContext.get().getQualifiedUser(),
+ ConnectContext.get().getRemoteIP(),
+ dbName + ": " + tblName);
+ }
+ }
+
+ private void checkPredicateName(Expr leftChild, Expr rightChild) throws AnalysisException {
+ String leftChildColumnName = ((SlotRef) leftChild.getChild(0)).getColumnName();
+ String rightChildColumnName = ((SlotRef) rightChild.getChild(0)).getColumnName();
+ if (leftChildColumnName.equals(rightChildColumnName)) {
+ throw new AnalysisException("column names on both sides of operator AND should be different");
+ }
+ }
+
+ private void analyzeSubPredicate(Expr subExpr) throws AnalysisException {
+ if (subExpr == null) {
+ return;
+ }
+
+ boolean valid = true;
+
+ CHECK:
+ {
+ if (subExpr instanceof BinaryPredicate) {
+ BinaryPredicate binaryPredicate = (BinaryPredicate) subExpr;
+ if (binaryPredicate.getOp() != BinaryPredicate.Operator.EQ) {
+ valid = false;
+ break CHECK;
+ }
+ } else {
+ valid = false;
+ break CHECK;
+ }
+
+ // left child
+ if (!(subExpr.getChild(0) instanceof SlotRef)) {
+ valid = false;
+ break CHECK;
+ }
+ String leftKey = ((SlotRef) subExpr.getChild(0)).getColumnName();
+ if (!STATE_NAME.equalsIgnoreCase(leftKey)) {
+ valid = false;
+ break CHECK;
+ }
+
+ // right child
+ if (!(subExpr.getChild(1) instanceof StringLiteral)) {
+ valid = false;
+ break CHECK;
+ }
+
+ String value = subExpr.getChild(1).getStringValue();
+ if (Strings.isNullOrEmpty(value)) {
+ valid = false;
+ break CHECK;
+ }
+
+ this.stateValue = value.toUpperCase();
+ try {
+ StatisticsJob.JobState.valueOf(this.stateValue);
+ } catch (Exception e) {
+ valid = false;
+ }
+ }
+
+ if (!valid) {
+ throw new AnalysisException("Where clause should looks like: " +
+ "STATE = \"PENDING|SCHEDULING|RUNNING|FINISHED|FAILED|CANCELLED\", " +
+ "or compound predicate with operator AND");
+ }
+ }
+
+ private int analyzeColumn(String columnName) throws AnalysisException {
+ for (String title : TITLE_NAMES) {
+ if (title.equalsIgnoreCase(columnName)) {
+ return TITLE_NAMES.indexOf(title);
+ }
+ }
+ throw new AnalysisException("Title name[" + columnName + "] does not exist");
+ }
+
+ @Override
+ public String toSql() {
+ StringBuilder sb = new StringBuilder();
+ sb.append("SHOW ANALYZE ");
+ if (this.table != null) {
+ sb.append("`").append(this.table.getName()).append("`");
Review Comment:
should we need to add db name here?
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowAnalyzeStmt.java:
##########
@@ -0,0 +1,340 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Catalog;
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.catalog.Table;
+import org.apache.doris.cluster.ClusterNamespace;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.OrderByPair;
+import org.apache.doris.mysql.privilege.PaloAuth;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSetMetaData;
+import org.apache.doris.statistics.StatisticsJob;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableList;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * ShowAnalyzeStmt is used to show statistics job info.
+ * <p>
+ * Grammar:
+ * SHOW ANALYZE
+ * [TABLE | ID]
+ * [
+ * WHERE
+ * [STATE = ["PENDING"|"SCHEDULING"|"RUNNING"|"FINISHED"|"FAILED"|"CANCELLED"]]
+ * ]
+ * [ORDER BY ...]
+ * [LIMIT limit][OFFSET offset];
+ */
+public class ShowAnalyzeStmt extends ShowStmt {
+ private static final String STATE_NAME = "state";
+ private static final ImmutableList<String> TITLE_NAMES = new ImmutableList.Builder<String>()
+ .add("id")
+ .add("create_time")
+ .add("start_time")
+ .add("finish_time")
+ .add("error_msg")
+ .add("scope")
+ .add("progress")
+ .add("state")
+ .build();
+
+ private List<Long> jobIds;
+ private TableName dbTableName;
+ private Expr whereClause;
+ private LimitElement limitElement;
+ private List<OrderByElement> orderByElements;
+
+ // after analyzed
+ private Database db;
+ private Table table;
+ private String stateValue;
+ private ArrayList<OrderByPair> orderByPairs;
+
+ public ShowAnalyzeStmt() {
+ }
+
+ public ShowAnalyzeStmt(List<Long> jobIds) {
+ this.jobIds = jobIds;
+ }
+
+ public ShowAnalyzeStmt(TableName dbTableName, Expr whereClause, List<OrderByElement> orderByElements, LimitElement limitElement) {
+ this.dbTableName = dbTableName;
+ this.whereClause = whereClause;
+ this.orderByElements = orderByElements;
+ this.limitElement = limitElement;
+ }
+
+ public List<Long> getJobIds() {
+ return this.jobIds;
+ }
+
+ public Database getDb() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The db name must be obtained after the parsing is complete");
+ return this.db;
+ }
+
+ public Table getTable() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.table;
+ }
+
+ public String getStateValue() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.stateValue;
+ }
+
+ public ArrayList<OrderByPair> getOrderByPairs() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.orderByPairs;
+ }
+
+ public long getLimit() {
+ if (this.limitElement != null && this.limitElement.hasLimit()) {
+ return this.limitElement.getLimit();
+ }
+ return -1L;
+ }
+
+ public long getOffset() {
+ if (this.limitElement != null && this.limitElement.hasOffset()) {
+ return this.limitElement.getOffset();
+ }
+ return -1L;
+ }
+
+ @Override
+ public void analyze(Analyzer analyzer) throws AnalysisException, UserException {
+ super.analyze(analyzer);
+
+ String dbName = null;
+ String tblName = null;
+ if (this.dbTableName != null) {
+ dbName = this.dbTableName.getDb();
+ tblName = this.dbTableName.getTbl();
+ }
+
+ if (Strings.isNullOrEmpty(dbName)) {
+ dbName = analyzer.getDefaultDb();
+ } else {
+ dbName = ClusterNamespace.getFullName(analyzer.getClusterName(), dbName);
+ }
+
+ if (Strings.isNullOrEmpty(dbName)) {
+ ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR);
+ }
+ this.db = analyzer.getCatalog().getDbOrAnalysisException(dbName);
+
+ if (Strings.isNullOrEmpty(tblName)) {
+ this.table = null;
+ List<Table> tables = this.db.getTables();
+ for (Table table : tables) {
+ checkShowAnalyzePriv(dbName, table.getName());
+ }
+ } else {
+ this.table = this.db.getTableOrAnalysisException(tblName);
+ checkShowAnalyzePriv(dbName, this.table.getName());
+ }
+
+ // analyze where clause if not null
+ if (this.whereClause != null) {
+ if (this.whereClause instanceof CompoundPredicate) {
+ CompoundPredicate cp = (CompoundPredicate) this.whereClause;
+ if (cp.getOp() != CompoundPredicate.Operator.AND) {
+ throw new AnalysisException("Only allow compound predicate with operator AND");
+ }
+ // check whether left.columnName equals to right.columnName
+ checkPredicateName(cp.getChild(0), cp.getChild(1));
+ analyzeSubPredicate(cp.getChild(0));
+ analyzeSubPredicate(cp.getChild(1));
+ } else {
+ analyzeSubPredicate(this.whereClause);
+ }
+ }
+
+ // analyze order by
+ if (this.orderByElements != null && !this.orderByElements.isEmpty()) {
+ this.orderByPairs = new ArrayList<>();
+ for (OrderByElement orderByElement : this.orderByElements) {
+ if (!(orderByElement.getExpr() instanceof SlotRef)) {
+ throw new AnalysisException("Should order by column");
+ }
+ SlotRef slotRef = (SlotRef) orderByElement.getExpr();
+ int index = analyzeColumn(slotRef.getColumnName());
+ OrderByPair orderByPair = new OrderByPair(index, !orderByElement.getIsAsc());
+ this.orderByPairs.add(orderByPair);
+ }
+ }
+ }
+
+ @Override
+ public ShowResultSetMetaData getMetaData() {
+ ShowResultSetMetaData.Builder builder = ShowResultSetMetaData.builder();
+ for (String title : TITLE_NAMES) {
+ builder.addColumn(new Column(title, ScalarType.createVarchar(128)));
+ }
+ return builder.build();
+ }
+
+ @Override
+ public RedirectStatus getRedirectStatus() {
+ return RedirectStatus.FORWARD_NO_SYNC;
+ }
+
+ private void checkShowAnalyzePriv(String dbName, String tblName) throws AnalysisException {
+ PaloAuth auth = Catalog.getCurrentCatalog().getAuth();
+ if (!auth.checkTblPriv(ConnectContext.get(), dbName, tblName, PrivPredicate.SHOW)) {
+ ErrorReport.reportAnalysisException(
+ ErrorCode.ERR_TABLEACCESS_DENIED_ERROR,
+ "SHOW ANALYZE",
+ ConnectContext.get().getQualifiedUser(),
+ ConnectContext.get().getRemoteIP(),
+ dbName + ": " + tblName);
+ }
+ }
+
+ private void checkPredicateName(Expr leftChild, Expr rightChild) throws AnalysisException {
+ String leftChildColumnName = ((SlotRef) leftChild.getChild(0)).getColumnName();
+ String rightChildColumnName = ((SlotRef) rightChild.getChild(0)).getColumnName();
+ if (leftChildColumnName.equals(rightChildColumnName)) {
+ throw new AnalysisException("column names on both sides of operator AND should be different");
+ }
+ }
+
+ private void analyzeSubPredicate(Expr subExpr) throws AnalysisException {
+ if (subExpr == null) {
+ return;
+ }
+
+ boolean valid = true;
+
+ CHECK:
+ {
+ if (subExpr instanceof BinaryPredicate) {
+ BinaryPredicate binaryPredicate = (BinaryPredicate) subExpr;
+ if (binaryPredicate.getOp() != BinaryPredicate.Operator.EQ) {
+ valid = false;
+ break CHECK;
+ }
+ } else {
+ valid = false;
+ break CHECK;
+ }
+
+ // left child
+ if (!(subExpr.getChild(0) instanceof SlotRef)) {
+ valid = false;
+ break CHECK;
+ }
+ String leftKey = ((SlotRef) subExpr.getChild(0)).getColumnName();
+ if (!STATE_NAME.equalsIgnoreCase(leftKey)) {
+ valid = false;
+ break CHECK;
+ }
+
+ // right child
+ if (!(subExpr.getChild(1) instanceof StringLiteral)) {
+ valid = false;
+ break CHECK;
+ }
+
+ String value = subExpr.getChild(1).getStringValue();
+ if (Strings.isNullOrEmpty(value)) {
+ valid = false;
+ break CHECK;
+ }
+
+ this.stateValue = value.toUpperCase();
+ try {
+ StatisticsJob.JobState.valueOf(this.stateValue);
+ } catch (Exception e) {
+ valid = false;
+ }
+ }
+
+ if (!valid) {
+ throw new AnalysisException("Where clause should looks like: " +
+ "STATE = \"PENDING|SCHEDULING|RUNNING|FINISHED|FAILED|CANCELLED\", " +
+ "or compound predicate with operator AND");
+ }
+ }
+
+ private int analyzeColumn(String columnName) throws AnalysisException {
+ for (String title : TITLE_NAMES) {
+ if (title.equalsIgnoreCase(columnName)) {
+ return TITLE_NAMES.indexOf(title);
+ }
+ }
+ throw new AnalysisException("Title name[" + columnName + "] does not exist");
+ }
+
+ @Override
+ public String toSql() {
+ StringBuilder sb = new StringBuilder();
+ sb.append("SHOW ANALYZE ");
+ if (this.table != null) {
+ sb.append("`").append(this.table.getName()).append("`");
+ }
+
+ if (this.whereClause != null) {
+ sb.append(" WHERE ").append(this.whereClause.toSql());
+ }
+
+ // Order By clause
+ if (this.orderByElements != null) {
+ sb.append(" ORDER BY ");
+ for (int i = 0; i < this.orderByElements.size(); ++i) {
+ sb.append(this.orderByElements.get(i).getExpr().toSql());
+ sb.append((this.orderByElements.get(i).getIsAsc()) ? " ASC" : " DESC");
+ sb.append((i + 1 != this.orderByElements.size()) ? ", " : "");
Review Comment:
nit: suggest code
```java
String.join(", ", orderByElements.stream().map(o -> {o.getExpr().toSql() + o.getIsAsc() ? " ASC" : " DESC" }).collect(Collectors.toList()))
```
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowAnalyzeStmt.java:
##########
@@ -0,0 +1,340 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Catalog;
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.catalog.Table;
+import org.apache.doris.cluster.ClusterNamespace;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.OrderByPair;
+import org.apache.doris.mysql.privilege.PaloAuth;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSetMetaData;
+import org.apache.doris.statistics.StatisticsJob;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableList;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * ShowAnalyzeStmt is used to show statistics job info.
+ * <p>
+ * Grammar:
+ * SHOW ANALYZE
+ * [TABLE | ID]
+ * [
+ * WHERE
+ * [STATE = ["PENDING"|"SCHEDULING"|"RUNNING"|"FINISHED"|"FAILED"|"CANCELLED"]]
+ * ]
+ * [ORDER BY ...]
+ * [LIMIT limit][OFFSET offset];
+ */
+public class ShowAnalyzeStmt extends ShowStmt {
+ private static final String STATE_NAME = "state";
+ private static final ImmutableList<String> TITLE_NAMES = new ImmutableList.Builder<String>()
+ .add("id")
+ .add("create_time")
+ .add("start_time")
+ .add("finish_time")
+ .add("error_msg")
+ .add("scope")
+ .add("progress")
+ .add("state")
+ .build();
+
+ private List<Long> jobIds;
+ private TableName dbTableName;
+ private Expr whereClause;
+ private LimitElement limitElement;
+ private List<OrderByElement> orderByElements;
+
+ // after analyzed
+ private Database db;
+ private Table table;
+ private String stateValue;
+ private ArrayList<OrderByPair> orderByPairs;
+
+ public ShowAnalyzeStmt() {
+ }
+
+ public ShowAnalyzeStmt(List<Long> jobIds) {
+ this.jobIds = jobIds;
+ }
+
+ public ShowAnalyzeStmt(TableName dbTableName, Expr whereClause, List<OrderByElement> orderByElements, LimitElement limitElement) {
+ this.dbTableName = dbTableName;
+ this.whereClause = whereClause;
+ this.orderByElements = orderByElements;
+ this.limitElement = limitElement;
+ }
+
+ public List<Long> getJobIds() {
+ return this.jobIds;
+ }
+
+ public Database getDb() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The db name must be obtained after the parsing is complete");
+ return this.db;
+ }
+
+ public Table getTable() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.table;
+ }
+
+ public String getStateValue() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.stateValue;
+ }
+
+ public ArrayList<OrderByPair> getOrderByPairs() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.orderByPairs;
+ }
+
+ public long getLimit() {
+ if (this.limitElement != null && this.limitElement.hasLimit()) {
+ return this.limitElement.getLimit();
+ }
+ return -1L;
+ }
+
+ public long getOffset() {
+ if (this.limitElement != null && this.limitElement.hasOffset()) {
+ return this.limitElement.getOffset();
+ }
+ return -1L;
+ }
+
+ @Override
+ public void analyze(Analyzer analyzer) throws AnalysisException, UserException {
+ super.analyze(analyzer);
+
+ String dbName = null;
+ String tblName = null;
+ if (this.dbTableName != null) {
+ dbName = this.dbTableName.getDb();
+ tblName = this.dbTableName.getTbl();
+ }
+
+ if (Strings.isNullOrEmpty(dbName)) {
+ dbName = analyzer.getDefaultDb();
+ } else {
+ dbName = ClusterNamespace.getFullName(analyzer.getClusterName(), dbName);
+ }
+
+ if (Strings.isNullOrEmpty(dbName)) {
+ ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR);
+ }
+ this.db = analyzer.getCatalog().getDbOrAnalysisException(dbName);
+
+ if (Strings.isNullOrEmpty(tblName)) {
+ this.table = null;
+ List<Table> tables = this.db.getTables();
+ for (Table table : tables) {
+ checkShowAnalyzePriv(dbName, table.getName());
+ }
+ } else {
+ this.table = this.db.getTableOrAnalysisException(tblName);
+ checkShowAnalyzePriv(dbName, this.table.getName());
+ }
+
+ // analyze where clause if not null
+ if (this.whereClause != null) {
+ if (this.whereClause instanceof CompoundPredicate) {
+ CompoundPredicate cp = (CompoundPredicate) this.whereClause;
+ if (cp.getOp() != CompoundPredicate.Operator.AND) {
+ throw new AnalysisException("Only allow compound predicate with operator AND");
+ }
+ // check whether left.columnName equals to right.columnName
+ checkPredicateName(cp.getChild(0), cp.getChild(1));
+ analyzeSubPredicate(cp.getChild(0));
+ analyzeSubPredicate(cp.getChild(1));
+ } else {
+ analyzeSubPredicate(this.whereClause);
+ }
+ }
+
+ // analyze order by
+ if (this.orderByElements != null && !this.orderByElements.isEmpty()) {
+ this.orderByPairs = new ArrayList<>();
+ for (OrderByElement orderByElement : this.orderByElements) {
+ if (!(orderByElement.getExpr() instanceof SlotRef)) {
+ throw new AnalysisException("Should order by column");
+ }
+ SlotRef slotRef = (SlotRef) orderByElement.getExpr();
+ int index = analyzeColumn(slotRef.getColumnName());
+ OrderByPair orderByPair = new OrderByPair(index, !orderByElement.getIsAsc());
+ this.orderByPairs.add(orderByPair);
+ }
+ }
+ }
+
+ @Override
+ public ShowResultSetMetaData getMetaData() {
Review Comment:
since the `ShowResultSetMetaData` is always same, we could use static variable to avoid to generate `ShowResultSetMetaData` every time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #8862: [feature-wip](statistics) step5: show statistics job information
Posted by GitBox <gi...@apache.org>.
weizhengte commented on code in PR #8862:
URL: https://github.com/apache/doris/pull/8862#discussion_r935848320
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowAnalyzeStmt.java:
##########
@@ -0,0 +1,366 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.catalog.Table;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.OrderByPair;
+import org.apache.doris.mysql.privilege.PaloAuth;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSetMetaData;
+import org.apache.doris.statistics.StatisticsJob;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Sets;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.IntStream;
+
+/**
+ * ShowAnalyzeStmt is used to show statistics job info.
+ * syntax:
+ * SHOW ANALYZE
+ * [TABLE | ID]
+ * [
+ * WHERE
+ * [STATE = ["PENDING"|"SCHEDULING"|"RUNNING"|"FINISHED"|"FAILED"|"CANCELLED"]]
+ * ]
+ * [ORDER BY ...]
+ * [LIMIT limit][OFFSET offset];
+ */
+public class ShowAnalyzeStmt extends ShowStmt {
+ private static final String STATE_NAME = "state";
+ private static final ImmutableList<String> TITLE_NAMES = new ImmutableList.Builder<String>()
+ .add("id")
+ .add("create_time")
+ .add("start_time")
+ .add("finish_time")
+ .add("error_msg")
+ .add("scope")
+ .add("progress")
+ .add("state")
+ .build();
+
+ private List<Long> jobIds;
+ private TableName dbTableName;
+ private Expr whereClause;
+ private LimitElement limitElement;
+ private List<OrderByElement> orderByElements;
+
+ // after analyzed
+ private long dbId;
+ private final Set<Long> tblIds = Sets.newHashSet();
+
+ private String stateValue;
+ private ArrayList<OrderByPair> orderByPairs;
+
+ public ShowAnalyzeStmt() {
+ }
+
+ public ShowAnalyzeStmt(List<Long> jobIds) {
+ this.jobIds = jobIds;
+ }
+
+ public ShowAnalyzeStmt(TableName dbTableName,
+ Expr whereClause,
+ List<OrderByElement> orderByElements,
+ LimitElement limitElement) {
+ this.dbTableName = dbTableName;
+ this.whereClause = whereClause;
+ this.orderByElements = orderByElements;
+ this.limitElement = limitElement;
+ }
+
+ public List<Long> getJobIds() {
+ return jobIds;
+ }
+
+ public long getDbId() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The dbId must be obtained after the parsing is complete");
+ return dbId;
+ }
+
+ public Set<Long> getTblIds() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The dbId must be obtained after the parsing is complete");
+ return tblIds;
+ }
+
+ public String getStateValue() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return stateValue;
+ }
+
+ public ArrayList<OrderByPair> getOrderByPairs() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return orderByPairs;
+ }
+
+ public long getLimit() {
+ if (limitElement != null && limitElement.hasLimit()) {
+ return limitElement.getLimit();
+ }
+ return -1L;
+ }
+
+ public long getOffset() {
+ if (limitElement != null && limitElement.hasOffset()) {
+ return limitElement.getOffset();
+ }
+ return -1L;
+ }
+
+ @Override
+ public void analyze(Analyzer analyzer) throws AnalysisException, UserException {
+ super.analyze(analyzer);
+
+ if (dbTableName != null) {
+ dbTableName.analyze(analyzer);
+ String dbName = dbTableName.getDb();
+ String tblName = dbTableName.getTbl();
+ checkShowAnalyzePriv(dbName, tblName);
+
+ Database db = analyzer.getEnv()
+ .getInternalDataSource().getDbOrAnalysisException(dbName);
+ Table table = db.getTableOrAnalysisException(tblName);
+ dbId = db.getId();
+ tblIds.add(table.getId());
+ } else {
+ // analyze the current default db
+ String dbName = analyzer.getDefaultDb();
+ if (Strings.isNullOrEmpty(dbName)) {
+ ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR);
+ }
+
+ Database db = analyzer.getEnv()
+ .getInternalDataSource().getDbOrAnalysisException(dbName);
+
+ db.readLock();
+ try {
+ List<Table> tables = db.getTables();
+ for (Table table : tables) {
+ checkShowAnalyzePriv(dbName, table.getName());
+ }
+
+ dbId = db.getId();
+ for (Table table : tables) {
+ long tblId = table.getId();
+ tblIds.add(tblId);
+ }
+ } finally {
+ db.readUnlock();
+ }
+ }
+
+ // analyze where clause if not null
+ if (whereClause != null) {
+ if (whereClause instanceof CompoundPredicate) {
+ CompoundPredicate cp = (CompoundPredicate) whereClause;
+ if (cp.getOp() != CompoundPredicate.Operator.AND) {
+ throw new AnalysisException("Only allow compound predicate with operator AND");
+ }
+ // check whether left.columnName equals to right.columnName
+ checkPredicateName(cp.getChild(0), cp.getChild(1));
+ analyzeSubPredicate(cp.getChild(0));
+ analyzeSubPredicate(cp.getChild(1));
+ } else {
+ analyzeSubPredicate(whereClause);
+ }
+ }
+
+ // analyze order by
+ if (orderByElements != null && !orderByElements.isEmpty()) {
+ orderByPairs = new ArrayList<>();
+ for (OrderByElement orderByElement : orderByElements) {
+ if (orderByElement.getExpr() instanceof SlotRef) {
+ SlotRef slotRef = (SlotRef) orderByElement.getExpr();
+ int index = analyzeColumn(slotRef.getColumnName());
+ OrderByPair orderByPair = new OrderByPair(index, !orderByElement.getIsAsc());
+ orderByPairs.add(orderByPair);
+ } else {
+ throw new AnalysisException("Should order by column");
+ }
+ }
+ }
+ }
+
+ @Override
+ public ShowResultSetMetaData getMetaData() {
+ ShowResultSetMetaData.Builder builder = ShowResultSetMetaData.builder();
+ for (String title : TITLE_NAMES) {
+ builder.addColumn(new Column(title, ScalarType.createVarchar(128)));
+ }
+ return builder.build();
+ }
+
+ @Override
+ public RedirectStatus getRedirectStatus() {
+ return RedirectStatus.FORWARD_NO_SYNC;
+ }
+
+ private void checkShowAnalyzePriv(String dbName, String tblName) throws AnalysisException {
+ PaloAuth auth = Env.getCurrentEnv().getAuth();
+ if (!auth.checkTblPriv(ConnectContext.get(), dbName, tblName, PrivPredicate.SHOW)) {
+ ErrorReport.reportAnalysisException(
+ ErrorCode.ERR_TABLEACCESS_DENIED_ERROR,
+ "SHOW ANALYZE",
+ ConnectContext.get().getQualifiedUser(),
+ ConnectContext.get().getRemoteIP(),
+ dbName + ": " + tblName);
+ }
+ }
+
+ private void checkPredicateName(Expr leftChild, Expr rightChild) throws AnalysisException {
+ String leftChildColumnName = ((SlotRef) leftChild.getChild(0)).getColumnName();
+ String rightChildColumnName = ((SlotRef) rightChild.getChild(0)).getColumnName();
Review Comment:
same below
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] weizhengte commented on a diff in pull request #8862: [feature-wip](statistics) step5: show statistics job information
Posted by GitBox <gi...@apache.org>.
weizhengte commented on code in PR #8862:
URL: https://github.com/apache/incubator-doris/pull/8862#discussion_r846894404
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowAnalyzeStmt.java:
##########
@@ -0,0 +1,340 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Catalog;
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.catalog.Table;
+import org.apache.doris.cluster.ClusterNamespace;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.OrderByPair;
+import org.apache.doris.mysql.privilege.PaloAuth;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSetMetaData;
+import org.apache.doris.statistics.StatisticsJob;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableList;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * ShowAnalyzeStmt is used to show statistics job info.
+ * <p>
+ * Grammar:
+ * SHOW ANALYZE
+ * [TABLE | ID]
+ * [
+ * WHERE
+ * [STATE = ["PENDING"|"SCHEDULING"|"RUNNING"|"FINISHED"|"FAILED"|"CANCELLED"]]
+ * ]
+ * [ORDER BY ...]
+ * [LIMIT limit][OFFSET offset];
+ */
+public class ShowAnalyzeStmt extends ShowStmt {
+ private static final String STATE_NAME = "state";
+ private static final ImmutableList<String> TITLE_NAMES = new ImmutableList.Builder<String>()
+ .add("id")
+ .add("create_time")
+ .add("start_time")
+ .add("finish_time")
+ .add("error_msg")
+ .add("scope")
+ .add("progress")
+ .add("state")
+ .build();
+
+ private List<Long> jobIds;
+ private TableName dbTableName;
+ private Expr whereClause;
+ private LimitElement limitElement;
+ private List<OrderByElement> orderByElements;
+
+ // after analyzed
+ private Database db;
+ private Table table;
+ private String stateValue;
+ private ArrayList<OrderByPair> orderByPairs;
+
+ public ShowAnalyzeStmt() {
+ }
+
+ public ShowAnalyzeStmt(List<Long> jobIds) {
+ this.jobIds = jobIds;
+ }
+
+ public ShowAnalyzeStmt(TableName dbTableName, Expr whereClause, List<OrderByElement> orderByElements, LimitElement limitElement) {
+ this.dbTableName = dbTableName;
+ this.whereClause = whereClause;
+ this.orderByElements = orderByElements;
+ this.limitElement = limitElement;
+ }
+
+ public List<Long> getJobIds() {
+ return this.jobIds;
+ }
+
+ public Database getDb() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The db name must be obtained after the parsing is complete");
+ return this.db;
+ }
+
+ public Table getTable() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.table;
+ }
+
+ public String getStateValue() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.stateValue;
+ }
+
+ public ArrayList<OrderByPair> getOrderByPairs() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.orderByPairs;
+ }
+
+ public long getLimit() {
+ if (this.limitElement != null && this.limitElement.hasLimit()) {
+ return this.limitElement.getLimit();
+ }
+ return -1L;
+ }
+
+ public long getOffset() {
+ if (this.limitElement != null && this.limitElement.hasOffset()) {
+ return this.limitElement.getOffset();
+ }
+ return -1L;
+ }
+
+ @Override
+ public void analyze(Analyzer analyzer) throws AnalysisException, UserException {
+ super.analyze(analyzer);
+
+ String dbName = null;
+ String tblName = null;
+ if (this.dbTableName != null) {
+ dbName = this.dbTableName.getDb();
+ tblName = this.dbTableName.getTbl();
+ }
+
+ if (Strings.isNullOrEmpty(dbName)) {
+ dbName = analyzer.getDefaultDb();
+ } else {
+ dbName = ClusterNamespace.getFullName(analyzer.getClusterName(), dbName);
+ }
+
+ if (Strings.isNullOrEmpty(dbName)) {
+ ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR);
+ }
+ this.db = analyzer.getCatalog().getDbOrAnalysisException(dbName);
+
+ if (Strings.isNullOrEmpty(tblName)) {
+ this.table = null;
+ List<Table> tables = this.db.getTables();
+ for (Table table : tables) {
+ checkShowAnalyzePriv(dbName, table.getName());
+ }
+ } else {
+ this.table = this.db.getTableOrAnalysisException(tblName);
+ checkShowAnalyzePriv(dbName, this.table.getName());
+ }
+
+ // analyze where clause if not null
+ if (this.whereClause != null) {
+ if (this.whereClause instanceof CompoundPredicate) {
+ CompoundPredicate cp = (CompoundPredicate) this.whereClause;
+ if (cp.getOp() != CompoundPredicate.Operator.AND) {
+ throw new AnalysisException("Only allow compound predicate with operator AND");
+ }
+ // check whether left.columnName equals to right.columnName
+ checkPredicateName(cp.getChild(0), cp.getChild(1));
+ analyzeSubPredicate(cp.getChild(0));
+ analyzeSubPredicate(cp.getChild(1));
+ } else {
+ analyzeSubPredicate(this.whereClause);
+ }
+ }
+
+ // analyze order by
+ if (this.orderByElements != null && !this.orderByElements.isEmpty()) {
+ this.orderByPairs = new ArrayList<>();
+ for (OrderByElement orderByElement : this.orderByElements) {
+ if (!(orderByElement.getExpr() instanceof SlotRef)) {
+ throw new AnalysisException("Should order by column");
+ }
+ SlotRef slotRef = (SlotRef) orderByElement.getExpr();
+ int index = analyzeColumn(slotRef.getColumnName());
+ OrderByPair orderByPair = new OrderByPair(index, !orderByElement.getIsAsc());
+ this.orderByPairs.add(orderByPair);
+ }
+ }
+ }
+
+ @Override
+ public ShowResultSetMetaData getMetaData() {
+ ShowResultSetMetaData.Builder builder = ShowResultSetMetaData.builder();
+ for (String title : TITLE_NAMES) {
+ builder.addColumn(new Column(title, ScalarType.createVarchar(128)));
+ }
+ return builder.build();
+ }
+
+ @Override
+ public RedirectStatus getRedirectStatus() {
+ return RedirectStatus.FORWARD_NO_SYNC;
+ }
+
+ private void checkShowAnalyzePriv(String dbName, String tblName) throws AnalysisException {
+ PaloAuth auth = Catalog.getCurrentCatalog().getAuth();
+ if (!auth.checkTblPriv(ConnectContext.get(), dbName, tblName, PrivPredicate.SHOW)) {
+ ErrorReport.reportAnalysisException(
+ ErrorCode.ERR_TABLEACCESS_DENIED_ERROR,
+ "SHOW ANALYZE",
+ ConnectContext.get().getQualifiedUser(),
+ ConnectContext.get().getRemoteIP(),
+ dbName + ": " + tblName);
+ }
+ }
+
+ private void checkPredicateName(Expr leftChild, Expr rightChild) throws AnalysisException {
+ String leftChildColumnName = ((SlotRef) leftChild.getChild(0)).getColumnName();
+ String rightChildColumnName = ((SlotRef) rightChild.getChild(0)).getColumnName();
+ if (leftChildColumnName.equals(rightChildColumnName)) {
+ throw new AnalysisException("column names on both sides of operator AND should be different");
+ }
+ }
+
+ private void analyzeSubPredicate(Expr subExpr) throws AnalysisException {
+ if (subExpr == null) {
+ return;
+ }
+
+ boolean valid = true;
+
+ CHECK:
+ {
+ if (subExpr instanceof BinaryPredicate) {
+ BinaryPredicate binaryPredicate = (BinaryPredicate) subExpr;
+ if (binaryPredicate.getOp() != BinaryPredicate.Operator.EQ) {
+ valid = false;
+ break CHECK;
+ }
+ } else {
+ valid = false;
+ break CHECK;
+ }
+
+ // left child
+ if (!(subExpr.getChild(0) instanceof SlotRef)) {
+ valid = false;
+ break CHECK;
+ }
+ String leftKey = ((SlotRef) subExpr.getChild(0)).getColumnName();
+ if (!STATE_NAME.equalsIgnoreCase(leftKey)) {
+ valid = false;
+ break CHECK;
+ }
+
+ // right child
+ if (!(subExpr.getChild(1) instanceof StringLiteral)) {
+ valid = false;
+ break CHECK;
+ }
+
+ String value = subExpr.getChild(1).getStringValue();
+ if (Strings.isNullOrEmpty(value)) {
+ valid = false;
+ break CHECK;
+ }
+
+ this.stateValue = value.toUpperCase();
+ try {
+ StatisticsJob.JobState.valueOf(this.stateValue);
+ } catch (Exception e) {
+ valid = false;
+ }
+ }
+
+ if (!valid) {
+ throw new AnalysisException("Where clause should looks like: " +
+ "STATE = \"PENDING|SCHEDULING|RUNNING|FINISHED|FAILED|CANCELLED\", " +
+ "or compound predicate with operator AND");
+ }
+ }
+
+ private int analyzeColumn(String columnName) throws AnalysisException {
+ for (String title : TITLE_NAMES) {
+ if (title.equalsIgnoreCase(columnName)) {
+ return TITLE_NAMES.indexOf(title);
+ }
+ }
+ throw new AnalysisException("Title name[" + columnName + "] does not exist");
+ }
+
+ @Override
+ public String toSql() {
+ StringBuilder sb = new StringBuilder();
+ sb.append("SHOW ANALYZE ");
+ if (this.table != null) {
+ sb.append("`").append(this.table.getName()).append("`");
+ }
+
+ if (this.whereClause != null) {
+ sb.append(" WHERE ").append(this.whereClause.toSql());
+ }
+
+ // Order By clause
+ if (this.orderByElements != null) {
+ sb.append(" ORDER BY ");
+ for (int i = 0; i < this.orderByElements.size(); ++i) {
+ sb.append(this.orderByElements.get(i).getExpr().toSql());
+ sb.append((this.orderByElements.get(i).getIsAsc()) ? " ASC" : " DESC");
+ sb.append((i + 1 != this.orderByElements.size()) ? ", " : "");
Review Comment:
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] weizhengte commented on a diff in pull request #8862: [feature-wip](statistics) step5: show statistics job information
Posted by GitBox <gi...@apache.org>.
weizhengte commented on code in PR #8862:
URL: https://github.com/apache/incubator-doris/pull/8862#discussion_r846894236
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowAnalyzeStmt.java:
##########
@@ -0,0 +1,340 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Catalog;
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.catalog.Table;
+import org.apache.doris.cluster.ClusterNamespace;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.OrderByPair;
+import org.apache.doris.mysql.privilege.PaloAuth;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSetMetaData;
+import org.apache.doris.statistics.StatisticsJob;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableList;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * ShowAnalyzeStmt is used to show statistics job info.
+ * <p>
+ * Grammar:
+ * SHOW ANALYZE
+ * [TABLE | ID]
+ * [
+ * WHERE
+ * [STATE = ["PENDING"|"SCHEDULING"|"RUNNING"|"FINISHED"|"FAILED"|"CANCELLED"]]
+ * ]
+ * [ORDER BY ...]
+ * [LIMIT limit][OFFSET offset];
+ */
+public class ShowAnalyzeStmt extends ShowStmt {
+ private static final String STATE_NAME = "state";
+ private static final ImmutableList<String> TITLE_NAMES = new ImmutableList.Builder<String>()
+ .add("id")
+ .add("create_time")
+ .add("start_time")
+ .add("finish_time")
+ .add("error_msg")
+ .add("scope")
+ .add("progress")
+ .add("state")
+ .build();
+
+ private List<Long> jobIds;
+ private TableName dbTableName;
+ private Expr whereClause;
+ private LimitElement limitElement;
+ private List<OrderByElement> orderByElements;
+
+ // after analyzed
+ private Database db;
+ private Table table;
+ private String stateValue;
+ private ArrayList<OrderByPair> orderByPairs;
+
+ public ShowAnalyzeStmt() {
+ }
+
+ public ShowAnalyzeStmt(List<Long> jobIds) {
+ this.jobIds = jobIds;
+ }
+
+ public ShowAnalyzeStmt(TableName dbTableName, Expr whereClause, List<OrderByElement> orderByElements, LimitElement limitElement) {
+ this.dbTableName = dbTableName;
+ this.whereClause = whereClause;
+ this.orderByElements = orderByElements;
+ this.limitElement = limitElement;
+ }
+
+ public List<Long> getJobIds() {
+ return this.jobIds;
+ }
+
+ public Database getDb() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The db name must be obtained after the parsing is complete");
+ return this.db;
+ }
+
+ public Table getTable() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.table;
+ }
+
+ public String getStateValue() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.stateValue;
+ }
+
+ public ArrayList<OrderByPair> getOrderByPairs() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.orderByPairs;
+ }
+
+ public long getLimit() {
+ if (this.limitElement != null && this.limitElement.hasLimit()) {
+ return this.limitElement.getLimit();
+ }
+ return -1L;
+ }
+
+ public long getOffset() {
+ if (this.limitElement != null && this.limitElement.hasOffset()) {
+ return this.limitElement.getOffset();
+ }
+ return -1L;
+ }
+
+ @Override
+ public void analyze(Analyzer analyzer) throws AnalysisException, UserException {
+ super.analyze(analyzer);
+
+ String dbName = null;
+ String tblName = null;
+ if (this.dbTableName != null) {
+ dbName = this.dbTableName.getDb();
+ tblName = this.dbTableName.getTbl();
+ }
+
+ if (Strings.isNullOrEmpty(dbName)) {
+ dbName = analyzer.getDefaultDb();
+ } else {
+ dbName = ClusterNamespace.getFullName(analyzer.getClusterName(), dbName);
+ }
+
+ if (Strings.isNullOrEmpty(dbName)) {
+ ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR);
+ }
+ this.db = analyzer.getCatalog().getDbOrAnalysisException(dbName);
+
+ if (Strings.isNullOrEmpty(tblName)) {
+ this.table = null;
+ List<Table> tables = this.db.getTables();
+ for (Table table : tables) {
+ checkShowAnalyzePriv(dbName, table.getName());
+ }
+ } else {
+ this.table = this.db.getTableOrAnalysisException(tblName);
+ checkShowAnalyzePriv(dbName, this.table.getName());
+ }
+
+ // analyze where clause if not null
+ if (this.whereClause != null) {
+ if (this.whereClause instanceof CompoundPredicate) {
+ CompoundPredicate cp = (CompoundPredicate) this.whereClause;
+ if (cp.getOp() != CompoundPredicate.Operator.AND) {
+ throw new AnalysisException("Only allow compound predicate with operator AND");
+ }
+ // check whether left.columnName equals to right.columnName
+ checkPredicateName(cp.getChild(0), cp.getChild(1));
+ analyzeSubPredicate(cp.getChild(0));
+ analyzeSubPredicate(cp.getChild(1));
+ } else {
+ analyzeSubPredicate(this.whereClause);
+ }
+ }
+
+ // analyze order by
+ if (this.orderByElements != null && !this.orderByElements.isEmpty()) {
+ this.orderByPairs = new ArrayList<>();
+ for (OrderByElement orderByElement : this.orderByElements) {
+ if (!(orderByElement.getExpr() instanceof SlotRef)) {
+ throw new AnalysisException("Should order by column");
+ }
+ SlotRef slotRef = (SlotRef) orderByElement.getExpr();
+ int index = analyzeColumn(slotRef.getColumnName());
+ OrderByPair orderByPair = new OrderByPair(index, !orderByElement.getIsAsc());
+ this.orderByPairs.add(orderByPair);
+ }
+ }
+ }
+
+ @Override
+ public ShowResultSetMetaData getMetaData() {
Review Comment:
got it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] weizhengte commented on a diff in pull request #8862: [feature-wip](statistics) step5: show statistics job information
Posted by GitBox <gi...@apache.org>.
weizhengte commented on code in PR #8862:
URL: https://github.com/apache/incubator-doris/pull/8862#discussion_r846894707
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowAnalyzeStmt.java:
##########
@@ -0,0 +1,340 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Catalog;
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.catalog.Table;
+import org.apache.doris.cluster.ClusterNamespace;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.OrderByPair;
+import org.apache.doris.mysql.privilege.PaloAuth;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSetMetaData;
+import org.apache.doris.statistics.StatisticsJob;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableList;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * ShowAnalyzeStmt is used to show statistics job info.
+ * <p>
+ * Grammar:
+ * SHOW ANALYZE
+ * [TABLE | ID]
+ * [
+ * WHERE
+ * [STATE = ["PENDING"|"SCHEDULING"|"RUNNING"|"FINISHED"|"FAILED"|"CANCELLED"]]
+ * ]
+ * [ORDER BY ...]
+ * [LIMIT limit][OFFSET offset];
+ */
+public class ShowAnalyzeStmt extends ShowStmt {
+ private static final String STATE_NAME = "state";
+ private static final ImmutableList<String> TITLE_NAMES = new ImmutableList.Builder<String>()
+ .add("id")
+ .add("create_time")
+ .add("start_time")
+ .add("finish_time")
+ .add("error_msg")
+ .add("scope")
+ .add("progress")
+ .add("state")
+ .build();
+
+ private List<Long> jobIds;
+ private TableName dbTableName;
+ private Expr whereClause;
+ private LimitElement limitElement;
+ private List<OrderByElement> orderByElements;
+
+ // after analyzed
+ private Database db;
+ private Table table;
+ private String stateValue;
+ private ArrayList<OrderByPair> orderByPairs;
+
+ public ShowAnalyzeStmt() {
+ }
+
+ public ShowAnalyzeStmt(List<Long> jobIds) {
+ this.jobIds = jobIds;
+ }
+
+ public ShowAnalyzeStmt(TableName dbTableName, Expr whereClause, List<OrderByElement> orderByElements, LimitElement limitElement) {
+ this.dbTableName = dbTableName;
+ this.whereClause = whereClause;
+ this.orderByElements = orderByElements;
+ this.limitElement = limitElement;
+ }
+
+ public List<Long> getJobIds() {
+ return this.jobIds;
+ }
+
+ public Database getDb() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The db name must be obtained after the parsing is complete");
+ return this.db;
+ }
+
+ public Table getTable() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.table;
+ }
+
+ public String getStateValue() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.stateValue;
+ }
+
+ public ArrayList<OrderByPair> getOrderByPairs() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return this.orderByPairs;
+ }
+
+ public long getLimit() {
+ if (this.limitElement != null && this.limitElement.hasLimit()) {
+ return this.limitElement.getLimit();
+ }
+ return -1L;
+ }
+
+ public long getOffset() {
+ if (this.limitElement != null && this.limitElement.hasOffset()) {
+ return this.limitElement.getOffset();
+ }
+ return -1L;
+ }
+
+ @Override
+ public void analyze(Analyzer analyzer) throws AnalysisException, UserException {
+ super.analyze(analyzer);
+
+ String dbName = null;
+ String tblName = null;
+ if (this.dbTableName != null) {
+ dbName = this.dbTableName.getDb();
+ tblName = this.dbTableName.getTbl();
+ }
+
+ if (Strings.isNullOrEmpty(dbName)) {
+ dbName = analyzer.getDefaultDb();
+ } else {
+ dbName = ClusterNamespace.getFullName(analyzer.getClusterName(), dbName);
+ }
+
+ if (Strings.isNullOrEmpty(dbName)) {
+ ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR);
+ }
+ this.db = analyzer.getCatalog().getDbOrAnalysisException(dbName);
+
+ if (Strings.isNullOrEmpty(tblName)) {
+ this.table = null;
+ List<Table> tables = this.db.getTables();
+ for (Table table : tables) {
+ checkShowAnalyzePriv(dbName, table.getName());
+ }
+ } else {
+ this.table = this.db.getTableOrAnalysisException(tblName);
+ checkShowAnalyzePriv(dbName, this.table.getName());
+ }
+
+ // analyze where clause if not null
+ if (this.whereClause != null) {
+ if (this.whereClause instanceof CompoundPredicate) {
+ CompoundPredicate cp = (CompoundPredicate) this.whereClause;
+ if (cp.getOp() != CompoundPredicate.Operator.AND) {
+ throw new AnalysisException("Only allow compound predicate with operator AND");
+ }
+ // check whether left.columnName equals to right.columnName
+ checkPredicateName(cp.getChild(0), cp.getChild(1));
+ analyzeSubPredicate(cp.getChild(0));
+ analyzeSubPredicate(cp.getChild(1));
+ } else {
+ analyzeSubPredicate(this.whereClause);
+ }
+ }
+
+ // analyze order by
+ if (this.orderByElements != null && !this.orderByElements.isEmpty()) {
+ this.orderByPairs = new ArrayList<>();
+ for (OrderByElement orderByElement : this.orderByElements) {
+ if (!(orderByElement.getExpr() instanceof SlotRef)) {
+ throw new AnalysisException("Should order by column");
+ }
+ SlotRef slotRef = (SlotRef) orderByElement.getExpr();
+ int index = analyzeColumn(slotRef.getColumnName());
+ OrderByPair orderByPair = new OrderByPair(index, !orderByElement.getIsAsc());
+ this.orderByPairs.add(orderByPair);
+ }
+ }
+ }
+
+ @Override
+ public ShowResultSetMetaData getMetaData() {
+ ShowResultSetMetaData.Builder builder = ShowResultSetMetaData.builder();
+ for (String title : TITLE_NAMES) {
+ builder.addColumn(new Column(title, ScalarType.createVarchar(128)));
+ }
+ return builder.build();
+ }
+
+ @Override
+ public RedirectStatus getRedirectStatus() {
+ return RedirectStatus.FORWARD_NO_SYNC;
+ }
+
+ private void checkShowAnalyzePriv(String dbName, String tblName) throws AnalysisException {
+ PaloAuth auth = Catalog.getCurrentCatalog().getAuth();
+ if (!auth.checkTblPriv(ConnectContext.get(), dbName, tblName, PrivPredicate.SHOW)) {
+ ErrorReport.reportAnalysisException(
+ ErrorCode.ERR_TABLEACCESS_DENIED_ERROR,
+ "SHOW ANALYZE",
+ ConnectContext.get().getQualifiedUser(),
+ ConnectContext.get().getRemoteIP(),
+ dbName + ": " + tblName);
+ }
+ }
+
+ private void checkPredicateName(Expr leftChild, Expr rightChild) throws AnalysisException {
+ String leftChildColumnName = ((SlotRef) leftChild.getChild(0)).getColumnName();
+ String rightChildColumnName = ((SlotRef) rightChild.getChild(0)).getColumnName();
+ if (leftChildColumnName.equals(rightChildColumnName)) {
+ throw new AnalysisException("column names on both sides of operator AND should be different");
+ }
+ }
+
+ private void analyzeSubPredicate(Expr subExpr) throws AnalysisException {
+ if (subExpr == null) {
+ return;
+ }
+
+ boolean valid = true;
+
+ CHECK:
+ {
+ if (subExpr instanceof BinaryPredicate) {
+ BinaryPredicate binaryPredicate = (BinaryPredicate) subExpr;
+ if (binaryPredicate.getOp() != BinaryPredicate.Operator.EQ) {
+ valid = false;
+ break CHECK;
+ }
+ } else {
+ valid = false;
+ break CHECK;
+ }
+
+ // left child
+ if (!(subExpr.getChild(0) instanceof SlotRef)) {
+ valid = false;
+ break CHECK;
+ }
+ String leftKey = ((SlotRef) subExpr.getChild(0)).getColumnName();
+ if (!STATE_NAME.equalsIgnoreCase(leftKey)) {
+ valid = false;
+ break CHECK;
+ }
+
+ // right child
+ if (!(subExpr.getChild(1) instanceof StringLiteral)) {
+ valid = false;
+ break CHECK;
+ }
+
+ String value = subExpr.getChild(1).getStringValue();
+ if (Strings.isNullOrEmpty(value)) {
+ valid = false;
+ break CHECK;
+ }
+
+ this.stateValue = value.toUpperCase();
+ try {
+ StatisticsJob.JobState.valueOf(this.stateValue);
+ } catch (Exception e) {
+ valid = false;
+ }
+ }
+
+ if (!valid) {
+ throw new AnalysisException("Where clause should looks like: " +
+ "STATE = \"PENDING|SCHEDULING|RUNNING|FINISHED|FAILED|CANCELLED\", " +
+ "or compound predicate with operator AND");
+ }
+ }
+
+ private int analyzeColumn(String columnName) throws AnalysisException {
+ for (String title : TITLE_NAMES) {
+ if (title.equalsIgnoreCase(columnName)) {
+ return TITLE_NAMES.indexOf(title);
+ }
+ }
+ throw new AnalysisException("Title name[" + columnName + "] does not exist");
+ }
+
+ @Override
+ public String toSql() {
+ StringBuilder sb = new StringBuilder();
+ sb.append("SHOW ANALYZE ");
+ if (this.table != null) {
+ sb.append("`").append(this.table.getName()).append("`");
Review Comment:
It might be better if added~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morningman merged pull request #8862: [feature-wip](statistics) step5: show statistics job information
Posted by GitBox <gi...@apache.org>.
morningman merged PR #8862:
URL: https://github.com/apache/doris/pull/8862
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] jackwener commented on a diff in pull request #8862: [feature-wip](statistics) step5: show statistics job information
Posted by GitBox <gi...@apache.org>.
jackwener commented on code in PR #8862:
URL: https://github.com/apache/incubator-doris/pull/8862#discussion_r881366908
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowAnalyzeStmt.java:
##########
@@ -0,0 +1,354 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Catalog;
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.catalog.Table;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.OrderByPair;
+import org.apache.doris.mysql.privilege.PaloAuth;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSetMetaData;
+import org.apache.doris.statistics.StatisticsJob;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Sets;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * ShowAnalyzeStmt is used to show statistics job info.
+ * syntax:
+ * SHOW ANALYZE
+ * [TABLE | ID]
+ * [
+ * WHERE
+ * [STATE = ["PENDING"|"SCHEDULING"|"RUNNING"|"FINISHED"|"FAILED"|"CANCELLED"]]
+ * ]
+ * [ORDER BY ...]
+ * [LIMIT limit][OFFSET offset];
+ */
+public class ShowAnalyzeStmt extends ShowStmt {
+ private static final String STATE_NAME = "state";
+ private static final ImmutableList<String> TITLE_NAMES = new ImmutableList.Builder<String>()
+ .add("id")
+ .add("create_time")
+ .add("start_time")
+ .add("finish_time")
+ .add("error_msg")
+ .add("scope")
+ .add("progress")
+ .add("state")
+ .build();
+
+ private List<Long> jobIds;
+ private TableName dbTableName;
+ private Expr whereClause;
+ private LimitElement limitElement;
+ private List<OrderByElement> orderByElements;
+
+ // after analyzed
+ private long dbId;
+ private final Set<Long> tblIds = Sets.newHashSet();
+
+ private String stateValue;
+ private ArrayList<OrderByPair> orderByPairs;
+
+ public ShowAnalyzeStmt() {
+ }
+
+ public ShowAnalyzeStmt(List<Long> jobIds) {
+ this.jobIds = jobIds;
+ }
+
+ public ShowAnalyzeStmt(TableName dbTableName,
+ Expr whereClause,
+ List<OrderByElement> orderByElements,
+ LimitElement limitElement) {
+ this.dbTableName = dbTableName;
+ this.whereClause = whereClause;
+ this.orderByElements = orderByElements;
+ this.limitElement = limitElement;
+ }
+
+ public List<Long> getJobIds() {
+ return jobIds;
+ }
+
+ public long getDbId() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The dbId must be obtained after the parsing is complete");
+ return dbId;
+ }
+
+ public Set<Long> getTblIds() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The dbId must be obtained after the parsing is complete");
+ return tblIds;
+ }
+
+ public String getStateValue() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return stateValue;
+ }
+
+ public ArrayList<OrderByPair> getOrderByPairs() {
+ Preconditions.checkArgument(isAnalyzed(),
+ "The tbl name must be obtained after the parsing is complete");
+ return orderByPairs;
+ }
+
+ public long getLimit() {
+ if (limitElement != null && limitElement.hasLimit()) {
+ return limitElement.getLimit();
+ }
+ return -1L;
+ }
+
+ public long getOffset() {
+ if (limitElement != null && limitElement.hasOffset()) {
+ return limitElement.getOffset();
+ }
+ return -1L;
+ }
+
+ @Override
+ public void analyze(Analyzer analyzer) throws AnalysisException, UserException {
+ super.analyze(analyzer);
+
+ if (dbTableName != null) {
+ dbTableName.analyze(analyzer);
+ String dbName = dbTableName.getDb();
+ String tblName = dbTableName.getTbl();
+ checkShowAnalyzePriv(dbName, tblName);
+
+ Database db = analyzer.getCatalog().getDbOrAnalysisException(dbName);
+ Table table = db.getTableOrAnalysisException(tblName);
+
+ dbId = db.getId();
+ tblIds.add(table.getId());
+ } else {
+ // analyze the current default db
+ String dbName = analyzer.getDefaultDb();
+ if (Strings.isNullOrEmpty(dbName)) {
+ ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR);
+ }
+
+ Database db = analyzer.getCatalog().getDbOrAnalysisException(dbName);
+
+ db.readLock();
+ try {
+ List<Table> tables = db.getTables();
+ for (Table table : tables) {
+ checkShowAnalyzePriv(dbName, table.getName());
+ }
+
+ dbId = db.getId();
+ for (Table table : tables) {
+ long tblId = table.getId();
+ tblIds.add(tblId);
+ }
+ } finally {
+ db.readUnlock();
+ }
+ }
+
+ // analyze where clause if not null
+ if (whereClause != null) {
+ if (whereClause instanceof CompoundPredicate) {
+ CompoundPredicate cp = (CompoundPredicate) whereClause;
+ if (cp.getOp() != CompoundPredicate.Operator.AND) {
+ throw new AnalysisException("Only allow compound predicate with operator AND");
+ }
+ // check whether left.columnName equals to right.columnName
+ checkPredicateName(cp.getChild(0), cp.getChild(1));
+ analyzeSubPredicate(cp.getChild(0));
+ analyzeSubPredicate(cp.getChild(1));
+ } else {
+ analyzeSubPredicate(whereClause);
+ }
+ }
+
+ // analyze order by
+ if (orderByElements != null && !orderByElements.isEmpty()) {
+ orderByPairs = new ArrayList<>();
+ for (OrderByElement orderByElement : orderByElements) {
+ if (!(orderByElement.getExpr() instanceof SlotRef)) {
+ throw new AnalysisException("Should order by column");
+ }
+ SlotRef slotRef = (SlotRef) orderByElement.getExpr();
+ int index = analyzeColumn(slotRef.getColumnName());
+ OrderByPair orderByPair = new OrderByPair(index, !orderByElement.getIsAsc());
+ orderByPairs.add(orderByPair);
+ }
+ }
+ }
+
+ @Override
+ public ShowResultSetMetaData getMetaData() {
+ ShowResultSetMetaData.Builder builder = ShowResultSetMetaData.builder();
+ for (String title : TITLE_NAMES) {
+ builder.addColumn(new Column(title, ScalarType.createVarchar(128)));
+ }
+ return builder.build();
+ }
+
+ @Override
+ public RedirectStatus getRedirectStatus() {
+ return RedirectStatus.FORWARD_NO_SYNC;
+ }
+
+ private void checkShowAnalyzePriv(String dbName, String tblName) throws AnalysisException {
+ PaloAuth auth = Catalog.getCurrentCatalog().getAuth();
+ if (!auth.checkTblPriv(ConnectContext.get(), dbName, tblName, PrivPredicate.SHOW)) {
+ ErrorReport.reportAnalysisException(
+ ErrorCode.ERR_TABLEACCESS_DENIED_ERROR,
+ "SHOW ANALYZE",
+ ConnectContext.get().getQualifiedUser(),
+ ConnectContext.get().getRemoteIP(),
+ dbName + ": " + tblName);
+ }
+ }
+
+ private void checkPredicateName(Expr leftChild, Expr rightChild) throws AnalysisException {
+ String leftChildColumnName = ((SlotRef) leftChild.getChild(0)).getColumnName();
+ String rightChildColumnName = ((SlotRef) rightChild.getChild(0)).getColumnName();
+ if (leftChildColumnName.equals(rightChildColumnName)) {
+ throw new AnalysisException("column names on both sides of operator AND should be different");
+ }
+ }
+
+ private void analyzeSubPredicate(Expr subExpr) throws AnalysisException {
+ if (subExpr == null) {
+ return;
+ }
+
+ boolean valid = true;
+
+ CHECK:
+ {
+ if (subExpr instanceof BinaryPredicate) {
+ BinaryPredicate binaryPredicate = (BinaryPredicate) subExpr;
+ if (binaryPredicate.getOp() != BinaryPredicate.Operator.EQ) {
+ valid = false;
+ break CHECK;
+ }
+ } else {
+ valid = false;
+ break CHECK;
+ }
+
+ // left child
+ if (!(subExpr.getChild(0) instanceof SlotRef)) {
+ valid = false;
+ break CHECK;
+ }
+ String leftKey = ((SlotRef) subExpr.getChild(0)).getColumnName();
+ if (!STATE_NAME.equalsIgnoreCase(leftKey)) {
+ valid = false;
+ break CHECK;
+ }
+
+ // right child
+ if (!(subExpr.getChild(1) instanceof StringLiteral)) {
+ valid = false;
+ break CHECK;
+ }
+
+ String value = subExpr.getChild(1).getStringValue();
+ if (Strings.isNullOrEmpty(value)) {
+ valid = false;
+ break CHECK;
+ }
+
+ stateValue = value.toUpperCase();
+ try {
+ StatisticsJob.JobState.valueOf(stateValue);
+ } catch (Exception e) {
+ valid = false;
+ }
+ }
+
+ if (!valid) {
+ throw new AnalysisException("Where clause should looks like: " +
+ "STATE = \"PENDING|SCHEDULING|RUNNING|FINISHED|FAILED|CANCELLED\", " +
+ "or compound predicate with operator AND");
Review Comment:
"or compound predicate with operator AND"
->
"binary predicate must be OR"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #8862: [feature-wip](statistics) step5: show statistics job information
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #8862:
URL: https://github.com/apache/doris/pull/8862#issuecomment-1203557870
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org