You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu> on 2026/06/08 10:13:49 UTC
[XL] Change in asterixdb[master]: [ASTERIXDB-3183][COMP] Add query plan cache
From Rithwik Koul <ri...@couchbase.com>:
Rithwik Koul has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21328?usp=email )
Change subject: [ASTERIXDB-3183][COMP] Add query plan cache
......................................................................
[ASTERIXDB-3183][COMP] Add query plan cache
- user model changes: yes
- storage format changes: no
- interface changes: yes
Details:
The query plan cache reduces compile times by storing job specs for
old queries. This allows skipping the rewriting and compilation
stages of repeated queries. The cache is resolved via the
ICcApplicationContext interface so it works in both the AsterixDB and
Couchbase Analytics runtimes.
Change-Id: I116e5fa9d16e5c4a5cc6dfbdf8db230221086a9d
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/SqlppCompilationProvider.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/SessionConfig.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
A asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/PlanCacheApiServlet.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryResultApiServlet.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceRequestParameters.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryStatusApiServlet.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/RebalanceApiServlet.java
A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/QueryCache.java
A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/QueryCacheKey.java
A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/QueryCacheValue.java
A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/clear/ClearCachePredicate.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cc/CcApplicationContext.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ResponseMetrics.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/MetricsPrinter.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java
M asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionWithCacheTest.java
M asterixdb/asterix-app/src/test/resources/runtimets/api.xml
A asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.001.post.http
A asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.002.post.http
A asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.003.post.http
A asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.004.delete.http
A asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.005.post.http
A asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.006.delete.http
A asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.007.delete.http
M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/avg_mixed/avg_mixed.5.query.sqlpp
M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cache-residency/cache-residency.001.regexjson
M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cache-residency/cache-residency.002.regexjson
M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm
M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm
M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm
A asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.001.regexjson
A asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.002.regexjson
A asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.003.regexjson
A asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.004.json
A asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.005.regexjson
A asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.006.json
A asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.007.json
M asterixdb/asterix-app/src/test/resources/runtimets/results/api/request-param-validation-400-BAD/request-param-validation-400-BAD.01.regexjson
M asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.03.regexadm
M asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.04.regexadm
M asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.05.regexadm
M asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.06.regexadm
M asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.07.regexadm
M asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.08.regexadm
M asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.03.regexadm
M asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.04.regexadm
M asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.05.regexadm
M asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.07.regexadm
M asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.08.regexadm
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/annotations/ExternalSubpathAnnotation.java
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/CompilerProperties.java
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/ICcApplicationContext.java
M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/Servlets.java
M asterixdb/asterix-doc/src/site/markdown/api.md
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/annotation/ExcludeFromSelectStarAnnotation.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppAstPrintVisitor.java
A asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppQueryCachePrintVisitor.java
M asterixdb/asterix-test-framework/src/main/java/org/apache/asterix/testframework/context/TestCaseContext.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/BroadcastExpressionAnnotation.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/HashJoinExpressionAnnotation.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/JoinProductivityAnnotation.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/PredicateCardinalityAnnotation.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/config/AlgebricksConfig.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/rewriter/base/PhysicalOptimizationConfig.java
M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
72 files changed, 1,235 insertions(+), 20 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/28/21328/1
diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/SqlppCompilationProvider.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/SqlppCompilationProvider.java
index c54a531..4c2263e 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/SqlppCompilationProvider.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/SqlppCompilationProvider.java
@@ -103,8 +103,8 @@
CompilerProperties.COMPILER_CBO_TEST_KEY, CompilerProperties.COMPILER_FORCE_JOIN_ORDER_KEY,
CompilerProperties.COMPILER_QUERY_PLAN_SHAPE_KEY, CompilerProperties.COMPILER_MIN_MEMORY_ALLOCATION_KEY,
CompilerProperties.COMPILER_COLUMN_FILTER_KEY, CompilerProperties.COMPILER_BATCH_LOOKUP_KEY,
- CompilerProperties.COMPILER_FRAMESIZE_KEY, FunctionUtil.IMPORT_PRIVATE_FUNCTIONS,
- CompilerProperties.COMPILER_DISJUNCTION_HASH_THRESHOLD,
+ CompilerProperties.COMPILER_QUERY_PLAN_CACHE_KEY, CompilerProperties.COMPILER_FRAMESIZE_KEY,
+ FunctionUtil.IMPORT_PRIVATE_FUNCTIONS, CompilerProperties.COMPILER_DISJUNCTION_HASH_THRESHOLD,
CompilerProperties.COMPILER_MAX_VARIABLE_OCCURRENCES_INLINING_KEY,
CompilerProperties.COMPILER_DELTALAKE_FILESPLITS_KEY,
CompilerProperties.COMPILER_OPTIMIZE_EXPRESSION_MAX_ARGS_KEY, FuzzyUtils.SIM_FUNCTION_PROP_NAME,
diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java
index 90bfc37..6b2724f 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java
@@ -131,7 +131,7 @@
}
class Stats implements Serializable {
- private static final long serialVersionUID = 5885273238208454612L;
+ private static final long serialVersionUID = 6834062614264666810L;
public enum ProfileType {
COUNTS("counts"),
@@ -168,6 +168,7 @@
private long cloudReadRequestsCount;
private long cloudPagesReadCount;
private long cloudPagesPersistedCount;
+ private boolean wasFetchedFromPlanCache;
public long getCount() {
return count;
@@ -274,6 +275,14 @@
public long getCloudPagesPersistedCount() {
return cloudPagesPersistedCount;
}
+
+ public boolean wasFetchedFromPlanCache() {
+ return wasFetchedFromPlanCache;
+ }
+
+ public void setWasFetchedFromPlanCache(boolean wasFetchedFromPlanCache) {
+ this.wasFetchedFromPlanCache = wasFetchedFromPlanCache;
+ }
}
class Profile implements Serializable {
diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/SessionConfig.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/SessionConfig.java
index 89301ac..29dc501 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/SessionConfig.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/SessionConfig.java
@@ -149,6 +149,11 @@
public static final String FORMAT_INDENT_JSON = "indent-json";
/**
+ * Whether to bypass the query plan cache.
+ */
+ public static final String SKIP_QUERYPLAN_CACHE = "skip-plancache";
+
+ /**
* Format flag: quote records in the results array.
*/
public static final String FORMAT_QUOTE_RECORD = "quote-record";
@@ -168,6 +173,7 @@
private long maxWarnings;
private long timeout;
private boolean includeHost;
+ private boolean skipPlanCache;
// Flags.
private final Map<String, Boolean> flags;
@@ -213,6 +219,22 @@
this.clientType = ClientType.ASTERIX;
this.hyracksJobFormat = jobFormat;
this.includeHost = false;
+ this.skipPlanCache = false;
+ }
+
+ public SessionConfig(SessionConfig config) {
+ this.fmt = config.fmt;
+ this.optimize = config.optimize;
+ this.executeQuery = config.executeQuery;
+ this.generateJobSpec = config.generateJobSpec;
+ this.flags = new HashMap<>(config.flags);
+ this.planFormat = config.planFormat;
+ this.hyracksJobFormat = config.hyracksJobFormat;
+ this.clientType = config.clientType;
+ this.maxWarnings = config.maxWarnings;
+ this.timeout = config.timeout;
+ this.includeHost = config.includeHost;
+ this.skipPlanCache = config.skipPlanCache;
}
/**
@@ -315,6 +337,14 @@
this.generateJobSpec = generateJobSpec;
}
+ public boolean isSkipPlanCache() {
+ return skipPlanCache;
+ }
+
+ public void setSkipPlanCache(boolean skipPlanCache) {
+ this.skipPlanCache = skipPlanCache;
+ }
+
/**
* Specify all out-of-band settings at once. For convenience of older code.
*/
@@ -350,4 +380,29 @@
Boolean value = flags.get(flag);
return value == null ? false : value.booleanValue();
}
+
+ @Override
+ public boolean equals(Object o) {
+ if (!(o instanceof SessionConfig)) {
+ return false;
+ }
+ if (o == this) {
+ return true;
+ }
+ SessionConfig other = (SessionConfig) o;
+ return Objects.equals(other.fmt, fmt) && Objects.equals(other.optimize, optimize)
+ && Objects.equals(other.executeQuery, executeQuery)
+ && Objects.equals(other.generateJobSpec, generateJobSpec) && Objects.equals(other.flags, flags)
+ && Objects.equals(other.planFormat, planFormat)
+ && Objects.equals(other.hyracksJobFormat, hyracksJobFormat)
+ && Objects.equals(other.clientType, clientType) && Objects.equals(other.maxWarnings, maxWarnings)
+ && Objects.equals(other.timeout, timeout) && Objects.equals(other.includeHost, includeHost)
+ && Objects.equals(other.skipPlanCache, skipPlanCache);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(fmt, optimize, executeQuery, generateJobSpec, flags, planFormat, hyracksJobFormat,
+ clientType, maxWarnings, timeout, includeHost, skipPlanCache);
+ }
}
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
index ad5b7f9..2812bbf 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
@@ -626,6 +626,14 @@
}
}
+ public String generateExpressionTreeString(IReturningStatement statement) throws CompilationException {
+ final StringWriter stringWriter = new StringWriter();
+ try (PrintWriter writer = new PrintWriter(stringWriter)) {
+ statement.accept(astPrintVisitorFactory.createLangVisitor(writer), 0);
+ return stringWriter.toString();
+ }
+ }
+
private void generateRewrittenExpressionTree(IReturningStatement statement) throws CompilationException {
final StringWriter stringWriter = new StringWriter();
try (PrintWriter writer = new PrintWriter(stringWriter)) {
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
index 620c75d..8db5cbf 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
@@ -222,6 +222,7 @@
stats.setCloudReadRequestsCount(responseStats.getCloudReadRequestsCount());
stats.setCloudPagesReadCount(responseStats.getCloudPagesReadCount());
stats.setCloudPagesPersistedCount(responseStats.getCloudPagesPersistedCount());
+ stats.setWasFetchedFromPlanCache(responseStats.wasFetchedFromPlanCache());
}
private static void updatePropertiesFromCC(IStatementExecutor.StatementProperties statementProperties,
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/PlanCacheApiServlet.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/PlanCacheApiServlet.java
new file mode 100644
index 0000000..657089a
--- /dev/null
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/PlanCacheApiServlet.java
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.asterix.api.http.server;
+
+import static org.apache.asterix.api.http.server.ServletConstants.ASTERIX_APP_CONTEXT_INFO_ATTR;
+
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.util.concurrent.ConcurrentMap;
+
+import org.apache.asterix.common.dataflow.ICcApplicationContext;
+import org.apache.hyracks.http.api.IServletRequest;
+import org.apache.hyracks.http.api.IServletResponse;
+import org.apache.hyracks.http.server.AbstractServlet;
+import org.apache.logging.log4j.Level;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import com.fasterxml.jackson.databind.node.ObjectNode;
+
+import io.netty.handler.codec.http.HttpResponseStatus;
+
+public class PlanCacheApiServlet extends AbstractServlet {
+ private static final Logger LOGGER = LogManager.getLogger();
+
+ public PlanCacheApiServlet(ConcurrentMap<String, Object> ctx, String[] paths) {
+ super(ctx, paths);
+ }
+
+ @Override
+ protected void delete(IServletRequest request, IServletResponse response) throws IOException {
+ ICcApplicationContext appConfig = (ICcApplicationContext) ctx.get(ASTERIX_APP_CONTEXT_INFO_ATTR);
+ int numberOfClearedCacheEntries = appConfig.getQueryCache().size();
+ appConfig.getQueryCache().clear();
+ try {
+ response.setStatus(HttpResponseStatus.OK);
+ PrintWriter out = response.writer();
+ ObjectNode jsonResponse = OBJECT_MAPPER.createObjectNode();
+ jsonResponse.put("results",
+ String.format("%d query plan cache entries have been cleared.", numberOfClearedCacheEntries));
+ jsonResponse.put("entriesCleared", numberOfClearedCacheEntries);
+ out.write(jsonResponse.toString());
+
+ } catch (Exception e) {
+ LOGGER.log(Level.WARN, "unexpected exception thrown from DELETE PLAN_CACHE", e);
+ // response: INTERNAL SERVER ERROR
+ response.setStatus(HttpResponseStatus.INTERNAL_SERVER_ERROR);
+ }
+ }
+}
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryResultApiServlet.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryResultApiServlet.java
index 97a4b7f..cd7dbff 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryResultApiServlet.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryResultApiServlet.java
@@ -184,7 +184,7 @@
metadata.getJobDuration(), stats.getCount(), stats.getSize(), metadata.getProcessedObjects(), 0,
metadata.getTotalWarningsCount(), metadata.getCompileTimeNanos(), stats.getQueueWaitTimeNanos(),
stats.getBufferCacheHitRatio(), stats.getBufferCachePageReadCount(), stats.getCloudReadRequestsCount(),
- stats.getCloudPagesReadCount(), stats.getCloudPagesPersistedCount());
+ stats.getCloudPagesReadCount(), stats.getCloudPagesPersistedCount(), stats.wasFetchedFromPlanCache());
}
/**
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceRequestParameters.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceRequestParameters.java
index 70fe6d7..5664b24 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceRequestParameters.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceRequestParameters.java
@@ -66,6 +66,7 @@
CLIENT_TYPE("client-type"),
DATAVERSE("dataverse"),
PRETTY("pretty"),
+ SKIP_PLANCACHE("skip-plancache"),
MODE("mode"),
TIMEOUT("timeout"),
PLAN_FORMAT("plan-format"),
@@ -143,6 +144,7 @@
private Map<String, String> optionalParams = null;
private Map<String, JsonNode> statementParams = null;
private boolean pretty = false;
+ private boolean skipQueryPlanCache = false;
private boolean expressionTree = false;
private boolean parseOnly = false; // don't execute; simply check for syntax correctness and named parameters.
private boolean compileOnly = false; // don't execute; compile only.
@@ -230,6 +232,14 @@
this.pretty = pretty;
}
+ public boolean isSkipQueryPlanCache() {
+ return skipQueryPlanCache;
+ }
+
+ public void setSkipQueryPlanCache(boolean skipQueryPlanCache) {
+ this.skipQueryPlanCache = skipQueryPlanCache;
+ }
+
public String getClientContextID() {
return clientContextID;
}
@@ -437,6 +447,7 @@
object.put("statement", statement != null
? LogRedactionUtil.statement(JSONUtil.escape(new StringBuilder(), statement).toString()) : null);
object.put("pretty", pretty);
+ object.put("skipPlanCache", skipQueryPlanCache);
object.put("mode", mode.getName());
object.put("clientContextID", clientContextID);
object.put("clientType", clientType.toString());
@@ -539,6 +550,7 @@
setResultTtlInMillis(parseTime(req, Parameter.RESULT_TTL.str(), valGetter, getResultTtlInMillis()));
setPretty(parseBoolean(req, Parameter.PRETTY.str(), valGetter, isPretty()));
+ setSkipQueryPlanCache(parseBoolean(req, Parameter.SKIP_PLANCACHE.str(), valGetter, isSkipQueryPlanCache()));
setExpressionTree(parseBoolean(req, Parameter.EXPRESSION_TREE.str(), valGetter, isExpressionTree()));
setRewrittenExpressionTree(
parseBoolean(req, Parameter.REWRITTEN_EXPRESSION_TREE.str(), valGetter, isRewrittenExpressionTree()));
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
index 23e3d2b..71b3bb9 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
@@ -317,7 +317,7 @@
stats.getCount(), stats.getSize(), stats.getProcessedObjects(), errorCount,
stats.getTotalWarningsCount(), stats.getCompileTimeNanos(), stats.getQueueWaitTimeNanos(),
stats.getBufferCacheHitRatio(), stats.getBufferCachePageReadCount(), stats.getCloudReadRequestsCount(),
- stats.getCloudPagesReadCount(), stats.getCloudPagesPersistedCount());
+ stats.getCloudPagesReadCount(), stats.getCloudPagesPersistedCount(), stats.wasFetchedFromPlanCache());
if (ResultDelivery.ASYNC != delivery) {
responsePrinter.addFooterPrinter(new StatusPrinter(executionState.getResultStatus()));
responsePrinter.addFooterPrinter(new MetricsPrinter(metrics, resultCharset));
@@ -426,6 +426,7 @@
sessionConfig.set(SessionConfig.OOB_OPTIMIZED_LOGICAL_PLAN, param.isOptimizedLogicalPlan());
sessionConfig.set(SessionConfig.OOB_HYRACKS_JOB, param.isJob());
sessionConfig.set(SessionConfig.FORMAT_INDENT_JSON, param.isPretty());
+ sessionConfig.set(SessionConfig.SKIP_QUERYPLAN_CACHE, param.isSkipQueryPlanCache());
sessionConfig.set(SessionConfig.FORMAT_QUOTE_RECORD,
format != SessionConfig.OutputFormat.CLEAN_JSON && format != SessionConfig.OutputFormat.LOSSLESS_JSON
&& format != SessionConfig.OutputFormat.LOSSLESS_ADM_JSON);
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryStatusApiServlet.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryStatusApiServlet.java
index 9467fa0..7eeb0eb 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryStatusApiServlet.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryStatusApiServlet.java
@@ -194,7 +194,7 @@
ResponseMetrics metrics = ResponseMetrics.of(
metadata.getCompileTimeNanos() + metadata.getQueueWaitTimeNanos() + metadata.getJobDuration(),
metadata.getJobDuration(), 0, 0, metadata.getProcessedObjects(), 0, metadata.getTotalWarningsCount(),
- metadata.getCompileTimeNanos(), metadata.getQueueWaitTimeNanos(), 0, 0, 0, 0, 0);
+ metadata.getCompileTimeNanos(), metadata.getQueueWaitTimeNanos(), 0, 0, 0, 0, 0, false);
printer.addFooterPrinter(new MetricsPrinter(metrics, HttpUtil.getPreferredCharset(request),
Set.of(MetricsPrinter.Metrics.ELAPSED_TIME, MetricsPrinter.Metrics.EXECUTION_TIME,
MetricsPrinter.Metrics.QUEUE_WAIT_TIME, MetricsPrinter.Metrics.COMPILE_TIME,
@@ -235,7 +235,7 @@
long executionTimeNanos = status == ResultStatus.RUNNING ? elapsedTimeNanos : 0;
ResponseMetrics metrics = ResponseMetrics.of(elapsedTimeNanos, executionTimeNanos, 0, 0, 0, 0, 0,
- compileTimeNanos, TimeUnit.MILLISECONDS.toNanos(queueWaitTimeMillis), 0, 0, 0, 0, 0);
+ compileTimeNanos, TimeUnit.MILLISECONDS.toNanos(queueWaitTimeMillis), 0, 0, 0, 0, 0, false);
printer.addFooterPrinter(new MetricsPrinter(metrics, HttpUtil.getPreferredCharset(request),
Set.of(MetricsPrinter.Metrics.ELAPSED_TIME, MetricsPrinter.Metrics.EXECUTION_TIME,
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/RebalanceApiServlet.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/RebalanceApiServlet.java
index 0977e46..470b313 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/RebalanceApiServlet.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/RebalanceApiServlet.java
@@ -190,6 +190,9 @@
// Sets the content type.
HttpUtil.setContentType(response, HttpUtil.ContentType.APPLICATION_JSON, StandardCharsets.UTF_8);
+ // Clear the query plan cache
+ appCtx.getQueryCache().clear();
+
if (datasetName == null) {
// Rebalances datasets in a given dataverse or all non-metadata datasets.
Iterable<Dataset> datasets = dataverseName == null ? getAllDatasetsForRebalance()
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/QueryCache.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/QueryCache.java
new file mode 100644
index 0000000..efab160
--- /dev/null
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/QueryCache.java
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.asterix.app.cache;
+
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.commons.collections4.map.LRUMap;
+
+public class QueryCache implements Map<QueryCacheKey, QueryCacheValue> {
+ private final Map<QueryCacheKey, QueryCacheValue> map;
+
+ // TODO: Add data structures for fine-grained cache invalidation
+
+ public QueryCache(int maxLength) {
+ map = Collections.synchronizedMap(new LRUMap<>(maxLength));
+ }
+
+ @Override
+ public int size() {
+ return map.size();
+ }
+
+ @Override
+ public boolean isEmpty() {
+ return map.isEmpty();
+ }
+
+ @Override
+ public boolean containsKey(Object key) {
+ return map.containsKey(key);
+ }
+
+ @Override
+ public boolean containsValue(Object value) {
+ return map.containsValue(value);
+ }
+
+ @Override
+ public QueryCacheValue get(Object key) {
+ return map.get(key);
+ }
+
+ @Override
+ public QueryCacheValue put(QueryCacheKey key, QueryCacheValue value) {
+ return map.put(key, value);
+ }
+
+ @Override
+ public QueryCacheValue remove(Object key) {
+ return map.remove(key);
+ }
+
+ @Override
+ public void putAll(Map<? extends QueryCacheKey, ? extends QueryCacheValue> m) {
+ map.putAll(m);
+ }
+
+ @Override
+ public void clear() {
+ map.clear();
+ }
+
+ @Override
+ public Set<QueryCacheKey> keySet() {
+ return map.keySet();
+ }
+
+ @Override
+ public Collection<QueryCacheValue> values() {
+ return map.values();
+ }
+
+ @Override
+ public Set<Entry<QueryCacheKey, QueryCacheValue>> entrySet() {
+ return map.entrySet();
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder buffer = new StringBuilder();
+
+ int cnt = 1;
+ for (Map.Entry<QueryCacheKey, QueryCacheValue> entry : map.entrySet()) {
+ buffer.append("Entry ").append(cnt).append(" query:\n");
+ buffer.append(entry.getKey().getQueryString()).append("\n");
+ buffer.append("Entry ").append(cnt).append(" spec:\n");
+ if (entry.getValue().getSpec() != null) {
+ buffer.append(entry.getValue().getSpec().toString()).append("\n\n");
+ }
+ cnt++;
+ }
+
+ return buffer.toString();
+ }
+}
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/QueryCacheKey.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/QueryCacheKey.java
new file mode 100644
index 0000000..9805a88
--- /dev/null
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/QueryCacheKey.java
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.asterix.app.cache;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+
+import org.apache.asterix.common.metadata.Namespace;
+import org.apache.asterix.metadata.declared.MetadataProvider;
+import org.apache.asterix.translator.SessionConfig;
+
+public class QueryCacheKey {
+ private final String queryString;
+ private final SessionConfig sessionConfig;
+ private final Map<String, Object> config;
+ private final long resultSetId;
+ private final Namespace defaultNamespace;
+
+ public QueryCacheKey(String queryString, SessionConfig sessionConfig, MetadataProvider metadataProvider) {
+ this.queryString = queryString;
+ this.sessionConfig = new SessionConfig(sessionConfig);
+ this.config = new HashMap<>(metadataProvider.getConfig());
+ this.resultSetId = metadataProvider.getResultSetId().getId();
+ this.defaultNamespace = metadataProvider.getDefaultNamespace();
+ }
+
+ public String getQueryString() {
+ return queryString;
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (!(o instanceof QueryCacheKey)) {
+ return false;
+ }
+ if (o == this) {
+ return true;
+ }
+ QueryCacheKey other = (QueryCacheKey) o;
+ return Objects.equals(other.queryString, queryString) && Objects.equals(other.sessionConfig, sessionConfig)
+ && Objects.equals(other.config, config) && Objects.equals(other.resultSetId, resultSetId)
+ && Objects.equals(other.defaultNamespace, defaultNamespace);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(queryString, sessionConfig, config, resultSetId, defaultNamespace);
+ }
+}
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/QueryCacheValue.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/QueryCacheValue.java
new file mode 100644
index 0000000..b0a13c7
--- /dev/null
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/QueryCacheValue.java
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.asterix.app.cache;
+
+import java.util.Set;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReentrantLock;
+
+import org.apache.asterix.translator.ExecutionPlans;
+import org.apache.hyracks.api.exceptions.Warning;
+import org.apache.hyracks.api.job.JobSpecification;
+
+public class QueryCacheValue {
+ private final JobSpecification spec;
+ private final Set<Warning> warnings;
+ private final ExecutionPlans executionPlans;
+ private final Lock lock;
+
+ public QueryCacheValue(JobSpecification spec, Set<Warning> warnings, ExecutionPlans executionPlans) {
+ this.spec = spec;
+ this.warnings = warnings;
+ this.executionPlans = executionPlans;
+ this.lock = new ReentrantLock();
+ }
+
+ public JobSpecification getSpec() {
+ return spec;
+ }
+
+ public Set<Warning> getWarnings() {
+ return warnings;
+ }
+
+ public Lock getLock() {
+ return lock;
+ }
+
+ public ExecutionPlans getExecutionPlans() {
+ return executionPlans;
+ }
+}
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/clear/ClearCachePredicate.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/clear/ClearCachePredicate.java
new file mode 100644
index 0000000..567cd83
--- /dev/null
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cache/clear/ClearCachePredicate.java
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.asterix.app.cache.clear;
+
+import java.util.function.Predicate;
+
+import org.apache.asterix.lang.common.base.Statement;
+
+public class ClearCachePredicate implements Predicate<Statement.Kind> {
+ @Override
+ public boolean test(Statement.Kind kind) {
+ switch (kind) {
+ case CREATE_INDEX:
+ case DATABASE_DROP:
+ case DATAVERSE_DROP:
+ case DATASET_DROP:
+ case INDEX_DROP:
+ case FULL_TEXT_FILTER_DROP:
+ case FULL_TEXT_CONFIG_DROP:
+ case NODEGROUP_DROP:
+ case FUNCTION_DROP:
+ case LIBRARY_DROP:
+ case SYNONYM_DROP:
+ case VIEW_DROP:
+ case DROP_FEED:
+ case DISCONNECT_FEED:
+ case ANALYZE:
+ case ANALYZE_DROP:
+ return true;
+ default:
+ return false;
+ }
+ }
+}
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cc/CcApplicationContext.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cc/CcApplicationContext.java
index 0126b6e..d8c34ba 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cc/CcApplicationContext.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/cc/CcApplicationContext.java
@@ -24,6 +24,7 @@
import java.util.concurrent.locks.ReentrantReadWriteLock;
import java.util.function.Supplier;
+import org.apache.asterix.app.cache.QueryCache;
import org.apache.asterix.app.external.ExternalStatsTracker;
import org.apache.asterix.app.result.ResultReader;
import org.apache.asterix.common.api.IConfigValidator;
@@ -79,6 +80,7 @@
import org.apache.hyracks.api.job.IJobLifecycleListener;
import org.apache.hyracks.api.result.IResultSet;
import org.apache.hyracks.client.result.ResultSet;
+import org.apache.hyracks.control.common.controllers.CCConfig;
import org.apache.hyracks.control.nc.io.IOManager;
import org.apache.hyracks.ipc.impl.HyracksConnection;
import org.apache.hyracks.storage.common.IStorageManager;
@@ -123,6 +125,7 @@
private final IRequestTracker requestTracker;
private final IConfigValidator configValidator;
private final IAdapterFactoryService adapterFactoryService;
+ private final QueryCache cache;
private final ReentrantReadWriteLock compilationLock = new ReentrantReadWriteLock(true);
private final IDataPartitioningProvider dataPartitioningProvider;
private final IGlobalTxManager globalTxManager;
@@ -175,6 +178,7 @@
requestTracker = new RequestTracker(this);
configValidator = configValidatorFactory.create();
this.adapterFactoryService = adapterFactoryService;
+ this.cache = new QueryCache(ccServiceCtx.getAppConfig().getInt(CCConfig.Option.QUERY_PLAN_CACHE_CAPACITY));
this.namespacePathResolver = namespacePathResolver;
this.namespaceResolver = namespaceResolver;
this.globalTxManager = globalTxManager;
@@ -424,4 +428,9 @@
public IExternalStatsTracker getExternalStatsTracker() {
return externalStatsTracker;
}
+
+ @Override
+ public QueryCache getQueryCache() {
+ return cache;
+ }
}
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ResponseMetrics.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ResponseMetrics.java
index 6ef195a..34af10e 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ResponseMetrics.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ResponseMetrics.java
@@ -35,6 +35,7 @@
private long cloudReadRequestsCount;
private long cloudPagesReadCount;
private long cloudPagesPersistedCount;
+ private boolean wasFetchedFromPlanCache;
private ResponseMetrics() {
}
@@ -42,7 +43,7 @@
public static ResponseMetrics of(long elapsedTimeNanos, long executionTimeNanos, long resultCount, long resultSize,
long processedObjects, long errorCount, long warnCount, long compileTimeNanos, long queueWaitTimeNanos,
double bufferCacheHitRatio, long bufferCachePageReadCount, long cloudRequestsCount,
- long cloudPagesReadCount, long cloudPagesPersistedCount) {
+ long cloudPagesReadCount, long cloudPagesPersistedCount, boolean wasFetchedFromPlanCache) {
ResponseMetrics metrics = new ResponseMetrics();
metrics.elapsedTime = elapsedTimeNanos;
metrics.executionTime = executionTimeNanos;
@@ -58,6 +59,7 @@
metrics.cloudReadRequestsCount = cloudRequestsCount;
metrics.cloudPagesReadCount = cloudPagesReadCount;
metrics.cloudPagesPersistedCount = cloudPagesPersistedCount;
+ metrics.wasFetchedFromPlanCache = wasFetchedFromPlanCache;
return metrics;
}
@@ -116,4 +118,8 @@
public long getCloudPagesPersistedCount() {
return cloudPagesPersistedCount;
}
+
+ public boolean wasFetchedFromPlanCache() {
+ return wasFetchedFromPlanCache;
+ }
}
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/MetricsPrinter.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/MetricsPrinter.java
index c05afdc..6defd70 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/MetricsPrinter.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/MetricsPrinter.java
@@ -46,7 +46,8 @@
BUFFERCACHE_PAGEREAD_COUNT("bufferCachePageReadCount"),
REMOTE_STORAGE_REQUESTS_COUNT("remoteStorageRequestsCount"),
REMOTE_STORAGE_PAGES_READ_COUNT("remoteStoragePagesReadCount"),
- REMOTE_PAGES_PERSISTED_COUNT("remoteStoragePagesPersistedCount");
+ REMOTE_PAGES_PERSISTED_COUNT("remoteStoragePagesPersistedCount"),
+ FETCHED_FROM_PLANCACHE("wasFetchedFromPlanCache");
private final String str;
@@ -140,6 +141,9 @@
entries.add(new LongEntry(Metrics.RESULT_COUNT, metrics.getResultCount()));
if (isSelected(Metrics.RESULT_SIZE))
entries.add(new LongEntry(Metrics.RESULT_SIZE, metrics.getResultSize()));
+ if (isSelected(Metrics.FETCHED_FROM_PLANCACHE))
+ entries.add(new StringEntry(Metrics.FETCHED_FROM_PLANCACHE,
+ Boolean.toString(metrics.wasFetchedFromPlanCache())));
if (isSelected(Metrics.PROCESSED_OBJECTS_COUNT))
entries.add(new LongEntry(Metrics.PROCESSED_OBJECTS_COUNT, metrics.getProcessedObjects()));
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
index df52f9c..29b264c 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
@@ -28,6 +28,8 @@
import java.io.FileInputStream;
import java.io.InputStream;
+import java.io.PrintWriter;
+import java.io.StringWriter;
import java.rmi.RemoteException;
import java.util.ArrayList;
import java.util.Collection;
@@ -48,6 +50,7 @@
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
+import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import java.util.stream.Collectors;
@@ -63,11 +66,16 @@
import org.apache.asterix.app.active.ActiveEntityEventsListener;
import org.apache.asterix.app.active.ActiveNotificationHandler;
import org.apache.asterix.app.active.FeedEventsListener;
+import org.apache.asterix.app.cache.QueryCache;
+import org.apache.asterix.app.cache.QueryCacheKey;
+import org.apache.asterix.app.cache.QueryCacheValue;
+import org.apache.asterix.app.cache.clear.ClearCachePredicate;
import org.apache.asterix.app.external.ExternalLibraryJobUtils;
import org.apache.asterix.app.result.ResultHandle;
import org.apache.asterix.app.result.ResultReader;
import org.apache.asterix.app.result.fields.ResultHandlePrinter;
import org.apache.asterix.app.result.fields.ResultsPrinter;
+import org.apache.asterix.app.result.fields.SignaturePrinter;
import org.apache.asterix.app.translator.handlers.CRSStatementHandler;
import org.apache.asterix.app.translator.handlers.IcebergCatalogStatementHandler;
import org.apache.asterix.app.translator.helpers.IcebergStatementValidationHelper;
@@ -81,6 +89,7 @@
import org.apache.asterix.common.api.IResponsePrinter;
import org.apache.asterix.common.cluster.IClusterStateManager;
import org.apache.asterix.common.cluster.IGlobalTxManager;
+import org.apache.asterix.common.config.CompilerProperties;
import org.apache.asterix.common.config.DatasetConfig;
import org.apache.asterix.common.config.DatasetConfig.DatasetType;
import org.apache.asterix.common.config.DatasetConfig.IndexType;
@@ -192,6 +201,7 @@
import org.apache.asterix.lang.common.util.LangRecordParseUtil;
import org.apache.asterix.lang.common.util.ViewUtil;
import org.apache.asterix.lang.sqlpp.rewrites.SqlppQueryRewriter;
+import org.apache.asterix.lang.sqlpp.visitor.SqlppQueryCachePrintVisitor;
import org.apache.asterix.metadata.IDatasetDetails;
import org.apache.asterix.metadata.MetadataManager;
import org.apache.asterix.metadata.MetadataTransactionContext;
@@ -334,6 +344,8 @@
protected final IResponsePrinter responsePrinter;
protected final WarningCollector warningCollector;
protected final ReentrantReadWriteLock compilationLock;
+ protected final QueryCache queryPlanCache;
+ protected final ClearCachePredicate clearCachePredicate;
protected final IGlobalTxManager globalTxManager;
protected final INamespaceResolver namespaceResolver;
@@ -355,6 +367,11 @@
this.responsePrinter = responsePrinter;
this.warningCollector = new WarningCollector();
this.compilationLock = appCtx.getCompilationLock();
+ // Obtain the plan cache via the interface (returns it as a Map) and cast to the concrete type. Casting the
+ // interface result (not appCtx itself) works in both the AsterixDB runtime and the Couchbase Analytics runtime,
+ // where appCtx is a delegating AnalyticsCcApplicationContext rather than a CcApplicationContext.
+ this.queryPlanCache = (QueryCache) appCtx.getQueryCache();
+ this.clearCachePredicate = new ClearCachePredicate();
if (appCtx.getServiceContext().getAppConfig().getBoolean(CCConfig.Option.ENFORCE_FRAME_WRITER_PROTOCOL)) {
this.jobFlags.add(JobFlag.ENFORCE_CONTRACT);
}
@@ -489,6 +506,7 @@
break;
case SYNONYM_DROP:
handleDropSynonymStatement(metadataProvider, stmt, requestParameters);
+ queryPlanCache.clear();
break;
case CREATE_VIEW:
handleCreateViewStatement(metadataProvider, stmt, stmtRewriter, requestParameters,
@@ -496,6 +514,7 @@
break;
case VIEW_DROP:
handleViewDropStatement(metadataProvider, stmt, requestParameters);
+ queryPlanCache.clear();
break;
case LOAD:
if (stats.getProfileType() == Stats.ProfileType.FULL) {
@@ -608,6 +627,9 @@
throw new CompilationException(ErrorCode.COMPILATION_ILLEGAL_STATE, stmt.getSourceLocation(),
"Unexpected statement: " + kind);
}
+ if (clearCachePredicate.test(kind)) {
+ queryPlanCache.clear();
+ }
}
} catch (Exception ex) {
this.appCtx.getRequestTracker().incrementFailedRequests();
@@ -5570,6 +5592,8 @@
compilationLock.readLock().unlock();
}
};
+ final Pair<QueryCacheKey, QueryCacheValue> toCache = new Pair<>(null, null);
+ final Lock[] toUnlock = new Lock[1];
final IStatementCompiler compiler = () -> {
long compileStart = System.nanoTime();
MetadataTransactionContext mdTxnCtx = MetadataManager.INSTANCE.beginTransaction();
@@ -5579,8 +5603,75 @@
org.apache.asterix.translator.ResultMetadata resultMetadata =
new org.apache.asterix.translator.ResultMetadata(sessionConfig.fmt());
resultMetadata.setCreateTimeMillis(clientRequest.getRequestTimeMillis());
- final JobSpecification jobSpec = rewriteCompileQuery(hcc, metadataProvider, query, null, stmtParams,
- requestParameters, resultMetadata);
+ // check caching conditions
+ boolean isPlanCacheEnabled =
+ metadataProvider.getBooleanProperty(CompilerProperties.COMPILER_QUERY_PLAN_CACHE_KEY,
+ appCtx.getCompilerProperties().isQueryPlanCacheEnabled());
+ boolean ignoreCache = !isPlanCacheEnabled || sessionConfig.is(SessionConfig.SKIP_QUERYPLAN_CACHE)
+ || query.isExplain() || requestParameters.getStatementParameters() != null
+ || !sessionConfig.isGenerateJobSpec() || sessionConfig.is(SessionConfig.OOB_EXPR_TREE)
+ || sessionConfig.is(SessionConfig.OOB_REWRITTEN_EXPR_TREE)
+ || sessionConfig.is(SessionConfig.OOB_LOGICAL_PLAN)
+ || sessionConfig.is(SessionConfig.OOB_OPTIMIZED_LOGICAL_PLAN)
+ || sessionConfig.is(SessionConfig.OOB_HYRACKS_JOB);
+
+ final JobSpecification jobSpec;
+ if (ignoreCache) {
+ jobSpec = rewriteCompileQuery(hcc, metadataProvider, query, null, stmtParams, requestParameters,
+ resultMetadata);
+ stats.setWasFetchedFromPlanCache(false);
+ } else {
+ // TODO: Use raw AST object for cache lookups instead of query string
+ final StringWriter stringWriter = new StringWriter();
+ try (PrintWriter writer = new PrintWriter(stringWriter)) {
+ query.accept(new SqlppQueryCachePrintVisitor(writer), 0);
+ }
+ final String queryString = stringWriter.toString();
+ final QueryCacheKey cacheKey = new QueryCacheKey(queryString, sessionConfig, metadataProvider);
+ final QueryCacheValue cacheValue = queryPlanCache.get(cacheKey);
+ // try to acquire lock on query cache entry, since same job cannot be run from multiple threads
+ if (cacheValue != null && cacheValue.getLock().tryLock()) {
+ toUnlock[0] = cacheValue.getLock();
+ jobSpec = cacheValue.getSpec();
+ stats.setWasFetchedFromPlanCache(true);
+
+ // Add skipped warnings and execution-plans from cache value.
+ WarningUtil.mergeWarnings(cacheValue.getWarnings(), warningCollector);
+ ExecutionPlans workingExecutionPlans = apiFramework.getExecutionPlans();
+ ExecutionPlans cachedExecutionPlans = cacheValue.getExecutionPlans();
+ workingExecutionPlans.setExpressionTree(cachedExecutionPlans.getExpressionTree());
+ workingExecutionPlans
+ .setRewrittenExpressionTree(cachedExecutionPlans.getRewrittenExpressionTree());
+ workingExecutionPlans.setLogicalPlan(cachedExecutionPlans.getLogicalPlan());
+ workingExecutionPlans.setOptimizedLogicalPlan(cachedExecutionPlans.getOptimizedLogicalPlan());
+ workingExecutionPlans.setJob(cachedExecutionPlans.getJob());
+ workingExecutionPlans.setSignature(cachedExecutionPlans.getSignature());
+ workingExecutionPlans.setStatementCategory(cachedExecutionPlans.getStatementCategory());
+ workingExecutionPlans.setStatementParameters(cachedExecutionPlans.getStatementParameters());
+ workingExecutionPlans.setExplainOnly(cachedExecutionPlans.isExplainOnly());
+ if (requestParameters.isPrintSignature()) {
+ // Also make sure we aren't missing out on printing our signature.
+ responsePrinter.addResultPrinter(SignaturePrinter.newInstance(workingExecutionPlans));
+ }
+
+ } else {
+ Set<Warning> oldWarnings = new LinkedHashSet<>();
+ warningCollector.getWarnings(oldWarnings, sessionConfig.getMaxWarnings());
+ jobSpec = rewriteCompileQuery(hcc, metadataProvider, query, null, stmtParams, requestParameters,
+ resultMetadata);
+ if (cacheValue == null) {
+ // Warnings and execution-plans (the OOB string fields) are saved with our job.
+ ExecutionPlans executionPlans = apiFramework.getExecutionPlans();
+ Set<Warning> newWarnings = new LinkedHashSet<>();
+ warningCollector.getWarnings(newWarnings, sessionConfig.getMaxWarnings());
+ newWarnings.removeAll(oldWarnings);
+ toCache.setFirst(cacheKey);
+ toCache.setSecond(new QueryCacheValue(jobSpec, newWarnings, executionPlans));
+ }
+ stats.setWasFetchedFromPlanCache(false);
+ }
+ } //
+
// update stats with count of compile-time warnings. needs to be adapted for multi-statement.
stats.updateTotalWarningsCount(warningCollector.getTotalWarningsCount());
afterCompile();
@@ -5598,8 +5689,19 @@
throw e;
}
};
- deliverResult(hcc, resultSet, compiler, metadataProvider, locker, resultDelivery, outMetadata, stats,
- requestParameters, true, null, clientRequest, JobKind.USER_QUERY);
+ try {
+ deliverResult(hcc, resultSet, compiler, metadataProvider, locker, resultDelivery, outMetadata, stats,
+ requestParameters, true, null, clientRequest, JobKind.USER_QUERY);
+ } finally {
+ // release lock on job spec from cache after job has finished running
+ if (toUnlock[0] != null) {
+ toUnlock[0].unlock();
+ }
+ }
+ // insert into query cache only after running job so that queries raising runtime errors are not cached
+ if (toCache.getFirst() != null) {
+ queryPlanCache.put(toCache.getFirst(), toCache.getSecond());
+ }
}
private void deliverResult(IHyracksClientConnection hcc, IResultSet resultSet, IStatementCompiler compiler,
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java
index 6d17fef..a28699b 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java
@@ -50,6 +50,7 @@
import org.apache.asterix.api.http.server.ConnectorApiServlet;
import org.apache.asterix.api.http.server.DiagnosticsApiServlet;
import org.apache.asterix.api.http.server.NodeControllerDetailsApiServlet;
+import org.apache.asterix.api.http.server.PlanCacheApiServlet;
import org.apache.asterix.api.http.server.QueryResultApiServlet;
import org.apache.asterix.api.http.server.QueryServiceServlet;
import org.apache.asterix.api.http.server.QueryStatusApiServlet;
@@ -355,6 +356,7 @@
addServlet(jsonAPIServer, Servlets.CONNECTOR);
addServlet(jsonAPIServer, Servlets.SHUTDOWN);
addServlet(jsonAPIServer, Servlets.VERSION);
+ addServlet(jsonAPIServer, Servlets.PLAN_CACHE);
addServlet(jsonAPIServer, Servlets.CLUSTER_STATE);
addServlet(jsonAPIServer, Servlets.REBALANCE);
addServlet(jsonAPIServer, Servlets.CLUSTER_STATE_NODE_DETAIL); // must not precede add of CLUSTER_STATE
@@ -401,6 +403,8 @@
return new ShutdownApiServlet(appCtx, ctx, paths);
case Servlets.VERSION:
return new VersionApiServlet(ctx, paths);
+ case Servlets.PLAN_CACHE:
+ return new PlanCacheApiServlet(ctx, paths);
case Servlets.CLUSTER_STATE:
return new ClusterApiServlet(appCtx, ctx, paths);
case Servlets.CLUSTER_STATE_NODE_DETAIL:
diff --git a/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java b/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
index 0d86782..3fbef15 100644
--- a/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
+++ b/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
@@ -2267,8 +2267,20 @@
try {
boolean loopCmd = testFile.getName().endsWith(".loop.cmd");
if (!testFile.getName().startsWith(DIAGNOSE)) {
- executeTestFile(testCaseCtx, ctx, variableCtx, statement, isDmlRecoveryTest, pb, cUnit,
- queryCount, expectedResultFileCtxs, testFile, actualPath);
+ int loopIterations = 1;
+ // run query files 3 times to test the query plan cache
+ if (testCaseCtx.isTestCache() && testCaseCtx.isFileRepeated(ctx)) {
+ loopIterations = 3;
+ }
+ for (int i = 0; i < loopIterations; i++) {
+ BitSet warningsCopy = (BitSet) testCaseCtx.expectedWarnings.clone();
+ executeTestFile(testCaseCtx, ctx, variableCtx, statement, isDmlRecoveryTest, pb, cUnit,
+ queryCount, expectedResultFileCtxs, testFile, actualPath);
+ if (i < loopIterations - 1) {
+ queryCount.decrement();
+ testCaseCtx.expectedWarnings = warningsCopy;
+ }
+ }
}
if (loopCmd) {
// this was a loop file and we have exited the loop; reset the loop iteration
@@ -2842,12 +2854,21 @@
dropDatabase(db);
}
}
+ clearCache();
} catch (Throwable th) {
th.printStackTrace();
throw th;
}
}
+ protected void clearCache() throws Exception {
+ HttpUriRequest method = constructDeleteMethod(getEndpoint(Servlets.PLAN_CACHE), Collections.emptyList());
+ HttpResponse response = executeHttpRequest(method);
+ if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK) {
+ throw new Exception("HTTP error " + response.getStatusLine().getStatusCode() + ":\n" + response);
+ }
+ }
+
protected void listUserDefinedDataverses(List<Pair<String, DataverseName>> outDataverses) throws Exception {
String query =
"select dv.DatabaseName, dv.DataverseName from Metadata.`Dataverse` as dv order by dv.DataverseName";
diff --git a/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionWithCacheTest.java b/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionWithCacheTest.java
new file mode 100644
index 0000000..e3669cb
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionWithCacheTest.java
@@ -0,0 +1,101 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.asterix.test.runtime;
+
+import java.net.InetAddress;
+import java.net.InetSocketAddress;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.asterix.common.api.INcApplicationContext;
+import org.apache.asterix.test.common.TestExecutor;
+import org.apache.asterix.testframework.context.TestCaseContext;
+import org.apache.hyracks.control.nc.NodeControllerService;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameters;
+
+/**
+ * Runs the SQL++ runtime tests with the storage parallelism, running each query 3 times to test the query plan cache.
+ */
+@RunWith(Parameterized.class)
+public class SqlppExecutionWithCacheTest {
+ protected static final String TEST_CONFIG_FILE_NAME = "src/main/resources/cc.conf";
+ protected static final Set<String> BLACKLISTED_REPEAT_FILES = new HashSet<>();
+
+ static {
+ // TODO (GLENN): Maybe add the paths here?
+ BLACKLISTED_REPEAT_FILES.add("upsert-case-returning.3.query.sqlpp");
+ BLACKLISTED_REPEAT_FILES.add("insert-return-records.3.query.sqlpp");
+ BLACKLISTED_REPEAT_FILES.add("atomic-statements-2.02.query.sqlpp");
+ }
+
+ @BeforeClass
+ public static void setUp() throws Exception {
+ final TestExecutor testExecutor = new TestExecutor();
+ LangExecutionUtil.setUp(TEST_CONFIG_FILE_NAME, testExecutor);
+ setNcEndpoints(testExecutor);
+ }
+
+ @AfterClass
+ public static void tearDown() throws Exception {
+ LangExecutionUtil.tearDown();
+ }
+
+ @Parameters(name = "SqlppExecutionWithCacheTest {index}: {0}")
+ public static Collection<Object[]> tests() throws Exception {
+ return LangExecutionUtil.tests("only_sqlpp.xml", "testsuite_sqlpp.xml");
+ }
+
+ protected TestCaseContext tcCtx;
+
+ public SqlppExecutionWithCacheTest(TestCaseContext tcCtx) {
+ this.tcCtx = tcCtx;
+ }
+
+ @Test
+ public void test() throws Exception {
+ this.tcCtx.setRepeatFilePredicate(testFileContext -> {
+ if (BLACKLISTED_REPEAT_FILES.contains(testFileContext.getFile().getName())) {
+ return false;
+ }
+ return testFileContext.getType().equals("query");
+ });
+ LangExecutionUtil.test(this.tcCtx);
+ }
+
+ private static void setNcEndpoints(TestExecutor testExecutor) {
+ final NodeControllerService[] ncs = ExecutionTestUtil.integrationUtil.ncs;
+ final Map<String, InetSocketAddress> ncEndPoints = new HashMap<>();
+ final String ip = InetAddress.getLoopbackAddress().getHostAddress();
+ for (NodeControllerService nc : ncs) {
+ final String nodeId = nc.getId();
+ final INcApplicationContext appCtx = (INcApplicationContext) nc.getApplicationContext();
+ int apiPort = appCtx.getExternalProperties().getNcApiPort();
+ ncEndPoints.put(nodeId, InetSocketAddress.createUnresolved(ip, apiPort));
+ }
+ testExecutor.setNcEndPoints(ncEndPoints);
+ }
+}
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/api.xml b/asterixdb/asterix-app/src/test/resources/runtimets/api.xml
index c960475..a98103c 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/api.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/api.xml
@@ -129,5 +129,10 @@
<output-dir compare="Text">net-diagnostics</output-dir>
</compilation-unit>
</test-case>
+ <test-case FilePath="api">
+ <compilation-unit name="plan_cache_clear">
+ <output-dir compare="Text">plan_cache_clear</output-dir>
+ </compilation-unit>
+ </test-case>
</test-group>
</test-suite>
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.001.post.http b/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.001.post.http
new file mode 100644
index 0000000..7a17eed
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.001.post.http
@@ -0,0 +1,20 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/query/service
+--body={"statement": "FROM [1, 2] AS v SELECT v;"}
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.002.post.http b/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.002.post.http
new file mode 100644
index 0000000..7a17eed
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.002.post.http
@@ -0,0 +1,20 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/query/service
+--body={"statement": "FROM [1, 2] AS v SELECT v;"}
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.003.post.http b/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.003.post.http
new file mode 100644
index 0000000..820f83e
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.003.post.http
@@ -0,0 +1,20 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/query/service
+--body={"statement": "FROM [1, 2] AS v SELECT v;","skip-plancache":"true"}
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.004.delete.http b/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.004.delete.http
new file mode 100644
index 0000000..157ceeb
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.004.delete.http
@@ -0,0 +1,19 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/admin/plancache
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.005.post.http b/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.005.post.http
new file mode 100644
index 0000000..7a17eed
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.005.post.http
@@ -0,0 +1,20 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/query/service
+--body={"statement": "FROM [1, 2] AS v SELECT v;"}
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.006.delete.http b/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.006.delete.http
new file mode 100644
index 0000000..157ceeb
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.006.delete.http
@@ -0,0 +1,19 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/admin/plancache
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.007.delete.http b/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.007.delete.http
new file mode 100644
index 0000000..157ceeb
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries/api/plan_cache_clear/plan_cache_clear.007.delete.http
@@ -0,0 +1,19 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/admin/plancache
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/avg_mixed/avg_mixed.5.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/avg_mixed/avg_mixed.5.query.sqlpp
index 71b22e7..0d4993e 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/avg_mixed/avg_mixed.5.query.sqlpp
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/avg_mixed/avg_mixed.5.query.sqlpp
@@ -22,10 +22,10 @@
*/
-- param max-warnings:json=100
+-- param skip-plancache:string=true
use test;
-
select d1.id, avg(v) a
from d1, d1.v
/* +hash */ group by d1.id
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cache-residency/cache-residency.001.regexjson b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cache-residency/cache-residency.001.regexjson
index 628e8b0..bb4eebc 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cache-residency/cache-residency.001.regexjson
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cache-residency/cache-residency.001.regexjson
@@ -14,6 +14,7 @@
"queueWaitTime": "R{.*}",
"resultCount": 2,
"resultSize": "R{.*}",
+ "wasFetchedFromPlanCache": "R{true|false}",
"processedObjects": 0
}
}
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cache-residency/cache-residency.002.regexjson b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cache-residency/cache-residency.002.regexjson
index d478e8a..8c62147 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cache-residency/cache-residency.002.regexjson
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cache-residency/cache-residency.002.regexjson
@@ -14,6 +14,7 @@
"queueWaitTime": "R{.*}",
"resultCount": 1,
"resultSize": "R{.*}",
+ "wasFetchedFromPlanCache": "R{true|false}",
"processedObjects": "R{.*}",
"bufferCacheHitRatio": "100.00%",
"bufferCachePageReadCount": "R{.*}"
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm
index 10c1856..ab2fff8 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm
@@ -74,6 +74,7 @@
"compiler.ordered.fields" : false,
"compiler\.parallelism" : 0,
"compiler\.parquet\.filesplits" : false,
+ "compiler\.queryplan\.cache" : true,
"compiler.queryplanshape" : "zigzag",
"compiler.runtime.memory.overhead" : 5,
"compiler\.sort\.parallel" : false,
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm
index 22c4fc8..63524aa 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm
@@ -74,6 +74,7 @@
"compiler.ordered.fields" : false,
"compiler\.parallelism" : -1,
"compiler\.parquet\.filesplits" : false,
+ "compiler\.queryplan\.cache" : true,
"compiler.queryplanshape" : "zigzag",
"compiler.runtime.memory.overhead" : 5,
"compiler\.sort\.parallel" : true,
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm
index a36d3b5..c015b30 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm
@@ -74,6 +74,7 @@
"compiler.ordered.fields" : false,
"compiler\.parallelism" : 3,
"compiler\.parquet\.filesplits" : false,
+ "compiler\.queryplan\.cache" : true,
"compiler.queryplanshape" : "zigzag",
"compiler.runtime.memory.overhead" : 5,
"compiler\.sort\.parallel" : true,
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.001.regexjson b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.001.regexjson
new file mode 100644
index 0000000..b0fae76
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.001.regexjson
@@ -0,0 +1,20 @@
+{
+ "requestID": "R{.*}",
+ "signature": {
+ "*": "*"
+ },
+ "type": "application/x-adm",
+ "results": [ "{ \"v\": 1 }\n", "{ \"v\": 2 }\n" ],
+ "plans": "R{.*}",
+ "status": "success",
+ "metrics": {
+ "elapsedTime": "R{.*}",
+ "executionTime": "R{.*}",
+ "compileTime": "R{.*}",
+ "queueWaitTime": "R{.*}",
+ "resultCount": 2,
+ "resultSize": 32,
+ "wasFetchedFromPlanCache": "false",
+ "processedObjects": 0
+ }
+}
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.002.regexjson b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.002.regexjson
new file mode 100644
index 0000000..edf3f2c
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.002.regexjson
@@ -0,0 +1,20 @@
+{
+ "requestID": "R{.*}",
+ "signature": {
+ "*": "*"
+ },
+ "type": "application/x-adm",
+ "results": [ "{ \"v\": 1 }\n", "{ \"v\": 2 }\n" ],
+ "plans": "R{.*}",
+ "status": "success",
+ "metrics": {
+ "elapsedTime": "R{.*}",
+ "executionTime": "R{.*}",
+ "compileTime": "R{.*}",
+ "queueWaitTime": "R{.*}",
+ "resultCount": 2,
+ "resultSize": 32,
+ "wasFetchedFromPlanCache": "true",
+ "processedObjects": 0
+ }
+}
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.003.regexjson b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.003.regexjson
new file mode 100644
index 0000000..b0fae76
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.003.regexjson
@@ -0,0 +1,20 @@
+{
+ "requestID": "R{.*}",
+ "signature": {
+ "*": "*"
+ },
+ "type": "application/x-adm",
+ "results": [ "{ \"v\": 1 }\n", "{ \"v\": 2 }\n" ],
+ "plans": "R{.*}",
+ "status": "success",
+ "metrics": {
+ "elapsedTime": "R{.*}",
+ "executionTime": "R{.*}",
+ "compileTime": "R{.*}",
+ "queueWaitTime": "R{.*}",
+ "resultCount": 2,
+ "resultSize": 32,
+ "wasFetchedFromPlanCache": "false",
+ "processedObjects": 0
+ }
+}
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.004.json b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.004.json
new file mode 100644
index 0000000..f6b353d
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.004.json
@@ -0,0 +1 @@
+{"results":"1 query plan cache entries have been cleared.","entriesCleared":1}
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.005.regexjson b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.005.regexjson
new file mode 100644
index 0000000..b0fae76
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.005.regexjson
@@ -0,0 +1,20 @@
+{
+ "requestID": "R{.*}",
+ "signature": {
+ "*": "*"
+ },
+ "type": "application/x-adm",
+ "results": [ "{ \"v\": 1 }\n", "{ \"v\": 2 }\n" ],
+ "plans": "R{.*}",
+ "status": "success",
+ "metrics": {
+ "elapsedTime": "R{.*}",
+ "executionTime": "R{.*}",
+ "compileTime": "R{.*}",
+ "queueWaitTime": "R{.*}",
+ "resultCount": 2,
+ "resultSize": 32,
+ "wasFetchedFromPlanCache": "false",
+ "processedObjects": 0
+ }
+}
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.006.json b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.006.json
new file mode 100644
index 0000000..f6b353d
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.006.json
@@ -0,0 +1 @@
+{"results":"1 query plan cache entries have been cleared.","entriesCleared":1}
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.007.json b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.007.json
new file mode 100644
index 0000000..3b7a4da
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/plan_cache_clear/plan_cache_clear.007.json
@@ -0,0 +1 @@
+{"results":"0 query plan cache entries have been cleared.","entriesCleared":0}
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/request-param-validation-400-BAD/request-param-validation-400-BAD.01.regexjson b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/request-param-validation-400-BAD/request-param-validation-400-BAD.01.regexjson
index ba3c970..a50217d 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/api/request-param-validation-400-BAD/request-param-validation-400-BAD.01.regexjson
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/api/request-param-validation-400-BAD/request-param-validation-400-BAD.01.regexjson
@@ -10,6 +10,7 @@
"queueWaitTime": "R{.*}",
"resultCount": 0,
"resultSize": 0,
+ "wasFetchedFromPlanCache": "R{true|false}",
"processedObjects": 0,
"bufferCacheHitRatio": "0.00%",
"bufferCachePageReadCount": 0,
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.03.regexadm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.03.regexadm
index 5c091b4..60621e2 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.03.regexadm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.03.regexadm
@@ -21,6 +21,7 @@
\s*\Q"queueWaitTime": "\E[^"]+\Q",\E
\s*\Q"resultCount": \E[0-9]+\Q,\E
\s*\Q"resultSize": \E[0-9]+\Q,\E
+\s*\Q"wasFetchedFromPlanCache": "\E(true|false)\Q",\E
\s*\Q"processedObjects": \E[0-9]+\Q,\E
\s*\Q"warningCount": 12\E
\s*\Q}\E
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.04.regexadm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.04.regexadm
index a0d6601..649e69b 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.04.regexadm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.04.regexadm
@@ -14,6 +14,7 @@
\s*\Q"queueWaitTime": "\E[^"]+\Q",\E
\s*\Q"resultCount": \E[0-9]+\Q,\E
\s*\Q"resultSize": \E[0-9]+\Q,\E
+\s*\Q"wasFetchedFromPlanCache": "\E(true|false)\Q",\E
\s*\Q"processedObjects": \E[0-9]+\Q,\E
\s*\Q"warningCount": 10\E
\s*\Q}\E
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.05.regexadm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.05.regexadm
index ac945a7..4d66fde 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.05.regexadm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.05.regexadm
@@ -14,6 +14,7 @@
\s*\Q"queueWaitTime": "\E[^"]+\Q",\E
\s*\Q"resultCount": \E[0-9]+\Q,\E
\s*\Q"resultSize": \E[0-9]+\Q,\E
+\s*\Q"wasFetchedFromPlanCache": "\E(true|false)\Q",\E
\s*\Q"processedObjects": \E[0-9]+\Q,\E
\s*\Q"warningCount": 11\E
\s*\Q}\E
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.06.regexadm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.06.regexadm
index f168154..a7f007e 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.06.regexadm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.06.regexadm
@@ -16,6 +16,7 @@
\s*\Q"queueWaitTime": "\E[^"]+\Q",\E
\s*\Q"resultCount": \E[0-9]+\Q,\E
\s*\Q"resultSize": \E[0-9]+\Q,\E
+\s*\Q"wasFetchedFromPlanCache": "\E(true|false)\Q",\E
\s*\Q"processedObjects": \E[0-9]+\Q,\E
\s*\Q"bufferCacheHitRatio": "\E[^"]+\Q",\E
\s*\Q"bufferCachePageReadCount": \E[0-9]+\Q,\E
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.07.regexadm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.07.regexadm
index ac945a7..4d66fde 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.07.regexadm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.07.regexadm
@@ -14,6 +14,7 @@
\s*\Q"queueWaitTime": "\E[^"]+\Q",\E
\s*\Q"resultCount": \E[0-9]+\Q,\E
\s*\Q"resultSize": \E[0-9]+\Q,\E
+\s*\Q"wasFetchedFromPlanCache": "\E(true|false)\Q",\E
\s*\Q"processedObjects": \E[0-9]+\Q,\E
\s*\Q"warningCount": 11\E
\s*\Q}\E
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.08.regexadm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.08.regexadm
index 5c091b4..60621e2 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.08.regexadm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/warnings-limit/warnings-limit.08.regexadm
@@ -21,6 +21,7 @@
\s*\Q"queueWaitTime": "\E[^"]+\Q",\E
\s*\Q"resultCount": \E[0-9]+\Q,\E
\s*\Q"resultSize": \E[0-9]+\Q,\E
+\s*\Q"wasFetchedFromPlanCache": "\E(true|false)\Q",\E
\s*\Q"processedObjects": \E[0-9]+\Q,\E
\s*\Q"warningCount": 12\E
\s*\Q}\E
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.03.regexadm b/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.03.regexadm
index 69842df..4ae61e2 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.03.regexadm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.03.regexadm
@@ -21,6 +21,7 @@
\s*\Q"queueWaitTime": "\E[^"]+\Q",\E
\s*\Q"resultCount": \E[0-9]+\Q,\E
\s*\Q"resultSize": \E[0-9]+\Q,\E
+\s*\Q"wasFetchedFromPlanCache": "\E(true|false)\Q",\E
\s*\Q"processedObjects": \E[0-9]+\Q,\E
\s*\Q"bufferCacheHitRatio": "\E[^"]+\Q",\E
\s*\Q"bufferCachePageReadCount": \E[0-9]+\Q,\E
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.04.regexadm b/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.04.regexadm
index 245357d..935433d 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.04.regexadm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.04.regexadm
@@ -14,6 +14,7 @@
\s*\Q"queueWaitTime": "\E[^"]+\Q",\E
\s*\Q"resultCount": \E[0-9]+\Q,\E
\s*\Q"resultSize": \E[0-9]+\Q,\E
+\s*\Q"wasFetchedFromPlanCache": "\E(true|false)\Q",\E
\s*\Q"processedObjects": \E[0-9]+\Q,\E
\s*\Q"bufferCacheHitRatio": "\E[^"]+\Q",\E
\s*\Q"bufferCachePageReadCount": \E[0-9]+\Q,\E
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.05.regexadm b/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.05.regexadm
index 28a9b78..7d48cc9 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.05.regexadm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.05.regexadm
@@ -14,6 +14,7 @@
\s*\Q"queueWaitTime": "\E[^"]+\Q",\E
\s*\Q"resultCount": \E[0-9]+\Q,\E
\s*\Q"resultSize": \E[0-9]+\Q,\E
+\s*\Q"wasFetchedFromPlanCache": "\E(true|false)\Q",\E
\s*\Q"processedObjects": \E[0-9]+\Q,\E
\s*\Q"bufferCacheHitRatio": "\E[^"]+\Q",\E
\s*\Q"bufferCachePageReadCount": \E[0-9]+\Q,\E
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.07.regexadm b/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.07.regexadm
index 28a9b78..7d48cc9 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.07.regexadm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.07.regexadm
@@ -14,6 +14,7 @@
\s*\Q"queueWaitTime": "\E[^"]+\Q",\E
\s*\Q"resultCount": \E[0-9]+\Q,\E
\s*\Q"resultSize": \E[0-9]+\Q,\E
+\s*\Q"wasFetchedFromPlanCache": "\E(true|false)\Q",\E
\s*\Q"processedObjects": \E[0-9]+\Q,\E
\s*\Q"bufferCacheHitRatio": "\E[^"]+\Q",\E
\s*\Q"bufferCachePageReadCount": \E[0-9]+\Q,\E
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.08.regexadm b/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.08.regexadm
index 69842df..4ae61e2 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.08.regexadm
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/warnings/warnings-limit/warnings-limit.08.regexadm
@@ -21,6 +21,7 @@
\s*\Q"queueWaitTime": "\E[^"]+\Q",\E
\s*\Q"resultCount": \E[0-9]+\Q,\E
\s*\Q"resultSize": \E[0-9]+\Q,\E
+\s*\Q"wasFetchedFromPlanCache": "\E(true|false)\Q",\E
\s*\Q"processedObjects": \E[0-9]+\Q,\E
\s*\Q"bufferCacheHitRatio": "\E[^"]+\Q",\E
\s*\Q"bufferCachePageReadCount": \E[0-9]+\Q,\E
diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/annotations/ExternalSubpathAnnotation.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/annotations/ExternalSubpathAnnotation.java
index e3e5ea3..364e432 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/annotations/ExternalSubpathAnnotation.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/annotations/ExternalSubpathAnnotation.java
@@ -22,6 +22,7 @@
public final class ExternalSubpathAnnotation implements IExpressionAnnotation {
+ public static final String HINT_STRING = "external-subpath";
private final String subPath;
public ExternalSubpathAnnotation(String subPath) {
@@ -31,4 +32,9 @@
public String getSubPath() {
return subPath;
}
+
+ @Override
+ public String toString() {
+ return HINT_STRING + ':' + subPath;
+ }
}
diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/CompilerProperties.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/CompilerProperties.java
index 073bd43..e5b5f36 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/CompilerProperties.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/CompilerProperties.java
@@ -180,7 +180,11 @@
COMPILER_HDFS_SPLIT_PARALLELISM(
INTEGER,
Runtime.getRuntime().availableProcessors(),
- "Number of threads to use for generating file splits for HDFS files");
+ "Number of threads to use for generating file splits for HDFS files"),
+ COMPILER_QUERYPLAN_CACHE(
+ BOOLEAN,
+ AlgebricksConfig.QUERY_PLAN_CACHE_DEFAULT,
+ "Enable/disable using the query plan cache.");
private final IOptionType type;
private final Object defaultValue;
@@ -279,6 +283,8 @@
public static final String COMPILER_ORDERED_FIELDS_KEY = Option.COMPILER_ORDERED_FIELDS.ini();
+ public static final String COMPILER_QUERY_PLAN_CACHE_KEY = Option.COMPILER_QUERYPLAN_CACHE.ini();
+
public static final int COMPILER_PARALLELISM_AS_STORAGE = 0;
public static final String COMPILER_DELTALAKE_FILESPLITS_KEY = Option.COMPILER_DELTALAKE_FILESPLITS.ini();
public static final String COMPILER_PARQUET_FILESPLITS_KEY = Option.COMPILER_PARQUET_FILESPLITS.ini();
@@ -450,4 +456,7 @@
return accessor.getInt(Option.COMPILER_HDFS_SPLIT_PARALLELISM);
}
+ public boolean isQueryPlanCacheEnabled() {
+ return accessor.getBoolean(Option.COMPILER_QUERYPLAN_CACHE);
+ }
}
diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/ICcApplicationContext.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/ICcApplicationContext.java
index 922c352..75c8427 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/ICcApplicationContext.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/ICcApplicationContext.java
@@ -18,6 +18,7 @@
*/
package org.apache.asterix.common.dataflow;
+import java.util.Map;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import org.apache.asterix.common.api.IApplicationContext;
@@ -49,7 +50,7 @@
* {@link org.apache.asterix.common.cluster.IGlobalRecoveryManager},
* {@link org.apache.asterix.common.library.ILibraryManager},
* {@link org.apache.asterix.common.transactions.IResourceIdManager}
- *
+ * <p>
* at the cluster controller side.
*/
public interface ICcApplicationContext extends IApplicationContext {
@@ -67,7 +68,7 @@
/**
* @return the global recovery manager which implements
- * {@link org.apache.asterix.common.cluster.IGlobalRecoveryManager}
+ * {@link org.apache.asterix.common.cluster.IGlobalRecoveryManager}
*/
IGlobalRecoveryManager getGlobalRecoveryManager();
@@ -171,4 +172,6 @@
IGlobalTxManager getGlobalTxManager();
IOManager getIoManager();
+
+ Map<?, ?> getQueryCache();
}
diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/Servlets.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/Servlets.java
index 5edc186..6dc0601 100644
--- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/Servlets.java
+++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/Servlets.java
@@ -27,6 +27,7 @@
public static final String REBALANCE = "/admin/rebalance";
public static final String SHUTDOWN = "/admin/shutdown";
public static final String VERSION = "/admin/version";
+ public static final String PLAN_CACHE = "/admin/plancache";
public static final String RUNNING_REQUESTS = "/admin/requests/running/*";
public static final String CLUSTER_STATE = "/admin/cluster/*";
public static final String CLUSTER_STATE_NODE_DETAIL = "/admin/cluster/node/*";
diff --git a/asterixdb/asterix-doc/src/site/markdown/api.md b/asterixdb/asterix-doc/src/site/markdown/api.md
index 584c424..9f45da7 100644
--- a/asterixdb/asterix-doc/src/site/markdown/api.md
+++ b/asterixdb/asterix-doc/src/site/markdown/api.md
@@ -24,6 +24,7 @@
* [Query Service API](#queryservice)
* [Query Status API](#querystatus)
* [Query Result API](#queryresult)
+* [Plan Cache API](#plancache)
## <a id="queryservice">POST /query/service</a><font size="4"> <a href="#toc">[Back to TOC]</a></font>
@@ -202,7 +203,7 @@
__Command__
- This example shows a request/reponse for the (opaque) result handle that was returned by the <a
+ This example shows a request/response for the (opaque) result handle that was returned by the <a
href="#deferred">deferred result delivery</a> example.
$ curl -v http://localhost:19002/query/service/result/7-0
@@ -222,3 +223,27 @@
[ { "$1": 1 }
]
+## <a id="plancache">DELETE /admin/plancache</a><font size="4"> <a href="#toc">[Back to TOC]</a></font>
+
+__Description__ Clears all entries in the query plan cache.
+The response is a JSON object that describes the number of entries that were cleared from the query plan cache.
+
+__Command__
+
+ $ curl -X DELETE -v http://localhost:19002/admin/plancache
+
+__Sample response__
+
+ > DELETE /admin/plancache HTTP/1.1
+ > Host: localhost:19002
+ > User-Agent: curl/8.4.0
+ > Accept: */*
+ >
+ < HTTP/1.1 200 OK
+ < connection: keep-alive
+ < content-length: 78
+ <
+ {
+ "results": "2 query plan cache entries have been cleared.",
+ "entriesCleared": 2
+ }
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/annotation/ExcludeFromSelectStarAnnotation.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/annotation/ExcludeFromSelectStarAnnotation.java
index a2998a7..d0e8f13 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/annotation/ExcludeFromSelectStarAnnotation.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/annotation/ExcludeFromSelectStarAnnotation.java
@@ -23,8 +23,15 @@
public final class ExcludeFromSelectStarAnnotation implements IExpressionAnnotation {
+ public static final String HINT_STRING = "exclude-from-select-star";
+
public static final ExcludeFromSelectStarAnnotation INSTANCE = new ExcludeFromSelectStarAnnotation();
private ExcludeFromSelectStarAnnotation() {
}
+
+ @Override
+ public String toString() {
+ return HINT_STRING;
+ }
}
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java
index b98151b..9cac5a0 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlCompatRewriteVisitor.java
@@ -509,6 +509,8 @@
private static class SqlCompatSelectExpressionCoercionAnnotation implements IExpressionAnnotation {
+ static final String HINT_STRING = "sql-compat-select-coercion";
+
static final SqlCompatSelectExpressionCoercionAnnotation NONE_NONE =
new SqlCompatSelectExpressionCoercionAnnotation(SqlCompatSelectCoercionKind.NONE,
SqlCompatSelectCoercionKind.NONE, null);
@@ -525,6 +527,12 @@
this.typeCoercion = Objects.requireNonNull(typeCoercion);
this.typeCoercionFieldName = typeCoercionFieldName;
}
+
+ @Override
+ public String toString() {
+ String result = HINT_STRING + ':' + cardinalityCoercion + ':' + typeCoercion;
+ return typeCoercionFieldName == null ? result : result + ':' + typeCoercionFieldName;
+ }
}
private enum SqlCompatSelectCoercionKind {
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppAstPrintVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppAstPrintVisitor.java
index 01cec35..027b270 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppAstPrintVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppAstPrintVisitor.java
@@ -483,6 +483,12 @@
printFieldList(step + 1, winExpr.getWindowFieldList());
}
}
+ if (winExpr.getFromLast() != null && winExpr.getFromLast()) {
+ out.println(skip(step + 1) + "FROM LAST");
+ }
+ if (winExpr.getIgnoreNulls() != null && winExpr.getIgnoreNulls()) {
+ out.println(skip(step + 1) + "IGNORE NULLS");
+ }
if (winExpr.hasAggregateFilterExpr()) {
out.println(skip(step + 1) + "FILTER (WHERE");
winExpr.getAggregateFilterExpr().accept(this, step + 2);
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppQueryCachePrintVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppQueryCachePrintVisitor.java
new file mode 100644
index 0000000..dd4eda8
--- /dev/null
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppQueryCachePrintVisitor.java
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.asterix.lang.sqlpp.visitor;
+
+import java.io.PrintWriter;
+
+import org.apache.asterix.common.exceptions.CompilationException;
+import org.apache.asterix.lang.common.base.AbstractExpression;
+import org.apache.asterix.lang.common.expression.CallExpr;
+import org.apache.asterix.lang.common.expression.FieldAccessor;
+import org.apache.asterix.lang.common.expression.IfExpr;
+import org.apache.asterix.lang.common.expression.IndexAccessor;
+import org.apache.asterix.lang.common.expression.ListConstructor;
+import org.apache.asterix.lang.common.expression.ListSliceExpression;
+import org.apache.asterix.lang.common.expression.LiteralExpr;
+import org.apache.asterix.lang.common.expression.OperatorExpr;
+import org.apache.asterix.lang.common.expression.QuantifiedExpression;
+import org.apache.asterix.lang.common.expression.RecordConstructor;
+import org.apache.asterix.lang.common.expression.UnaryExpr;
+import org.apache.asterix.lang.common.expression.VariableExpr;
+import org.apache.asterix.lang.sqlpp.expression.CaseExpression;
+import org.apache.asterix.lang.sqlpp.expression.SelectExpression;
+import org.apache.asterix.lang.sqlpp.expression.WindowExpression;
+import org.apache.hyracks.algebricks.core.algebra.expressions.IExpressionAnnotation;
+
+/**
+ * <b>All</b> AST nodes that extend {@link AbstractExpression} must print their hints here.
+ */
+public class SqlppQueryCachePrintVisitor extends SqlppAstPrintVisitor {
+ public SqlppQueryCachePrintVisitor(PrintWriter out) {
+ super(out);
+ }
+
+ public void printHints(AbstractExpression abstractExpr, Integer step) {
+ if (abstractExpr.hasHints()) {
+ for (IExpressionAnnotation hint : abstractExpr.getHints()) {
+ out.print(skip(step + 1) + hint.toString());
+ }
+ }
+ }
+
+ @Override
+ public Void visit(CallExpr callExpr, Integer step) throws CompilationException {
+ printHints(callExpr, step);
+ return super.visit(callExpr, step);
+ }
+
+ @Override
+ public Void visit(CaseExpression caseExpr, Integer step) throws CompilationException {
+ printHints(caseExpr, step);
+ return super.visit(caseExpr, step);
+ }
+
+ @Override
+ public Void visit(FieldAccessor fieldAccessor, Integer step) throws CompilationException {
+ printHints(fieldAccessor, step);
+ return super.visit(fieldAccessor, step);
+ }
+
+ @Override
+ public Void visit(IfExpr ifExpr, Integer step) throws CompilationException {
+ printHints(ifExpr, step);
+ return super.visit(ifExpr, step);
+ }
+
+ @Override
+ public Void visit(IndexAccessor indexAccessor, Integer step) throws CompilationException {
+ printHints(indexAccessor, step);
+ return super.visit(indexAccessor, step);
+ }
+
+ @Override
+ public Void visit(ListConstructor listConstructor, Integer step) throws CompilationException {
+ printHints(listConstructor, step);
+ return super.visit(listConstructor, step);
+ }
+
+ @Override
+ public Void visit(ListSliceExpression listSliceExpression, Integer step) throws CompilationException {
+ printHints(listSliceExpression, step);
+ return super.visit(listSliceExpression, step);
+ }
+
+ @Override
+ public Void visit(LiteralExpr literalExpr, Integer step) {
+ printHints(literalExpr, step);
+ return super.visit(literalExpr, step);
+ }
+
+ @Override
+ public Void visit(OperatorExpr operatorExpr, Integer step) throws CompilationException {
+ printHints(operatorExpr, step);
+ return super.visit(operatorExpr, step);
+ }
+
+ @Override
+ public Void visit(QuantifiedExpression quantifiedExpression, Integer step) throws CompilationException {
+ printHints(quantifiedExpression, step);
+ return super.visit(quantifiedExpression, step);
+ }
+
+ @Override
+ public Void visit(RecordConstructor recordConstructor, Integer step) throws CompilationException {
+ printHints(recordConstructor, step);
+ return super.visit(recordConstructor, step);
+ }
+
+ @Override
+ public Void visit(SelectExpression selectExpression, Integer step) throws CompilationException {
+ printHints(selectExpression, step);
+ return super.visit(selectExpression, step);
+ }
+
+ @Override
+ public Void visit(UnaryExpr unaryExpr, Integer step) throws CompilationException {
+ printHints(unaryExpr, step);
+ return super.visit(unaryExpr, step);
+ }
+
+ @Override
+ public Void visit(VariableExpr variableExpr, Integer step) {
+ printHints(variableExpr, step);
+ return super.visit(variableExpr, step);
+ }
+
+ @Override
+ public Void visit(WindowExpression windowExpression, Integer step) throws CompilationException {
+ printHints(windowExpression, step);
+ return super.visit(windowExpression, step);
+ }
+}
diff --git a/asterixdb/asterix-test-framework/src/main/java/org/apache/asterix/testframework/context/TestCaseContext.java b/asterixdb/asterix-test-framework/src/main/java/org/apache/asterix/testframework/context/TestCaseContext.java
index eded7445..00e6afa 100644
--- a/asterixdb/asterix-test-framework/src/main/java/org/apache/asterix/testframework/context/TestCaseContext.java
+++ b/asterixdb/asterix-test-framework/src/main/java/org/apache/asterix/testframework/context/TestCaseContext.java
@@ -29,6 +29,7 @@
import java.util.Map;
import java.util.Set;
import java.util.TreeSet;
+import java.util.function.Predicate;
import java.util.regex.Pattern;
import org.apache.asterix.testframework.template.TemplateHelper;
@@ -56,6 +57,10 @@
public List<String> expectedErrors;
public BitSet expectedWarnings;
+ // For testing the query plan cache.
+ public boolean isTestCache = false;
+ public Predicate<TestFileContext> repeatFilePredicate;
+
public TestCaseContext(File tsRoot, TestSuite testSuite, TestGroup[] testGroups, TestCase testCase) {
this.tsRoot = tsRoot;
this.testSuite = testSuite;
@@ -83,6 +88,19 @@
return testCase.getRepeat().intValue();
}
+ public boolean isTestCache() {
+ return isTestCache;
+ }
+
+ public boolean isFileRepeated(TestFileContext testFileContext) {
+ return repeatFilePredicate.test(testFileContext);
+ }
+
+ public void setRepeatFilePredicate(Predicate<TestFileContext> repeatFilePredicate) {
+ this.isTestCache = true;
+ this.repeatFilePredicate = repeatFilePredicate;
+ }
+
public void put(String key, Object object) {
if (kv == null) {
kv = new HashMap<>();
diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/BroadcastExpressionAnnotation.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/BroadcastExpressionAnnotation.java
index f1b5093..2dcec9f 100644
--- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/BroadcastExpressionAnnotation.java
+++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/BroadcastExpressionAnnotation.java
@@ -22,6 +22,8 @@
public final class BroadcastExpressionAnnotation implements IExpressionAnnotation {
+ public static final String HINT_STRING = "broadcast";
+
public enum BroadcastSide {
LEFT,
RIGHT;
@@ -71,4 +73,9 @@
}
return n;
}
+
+ @Override
+ public String toString() {
+ return name == null ? HINT_STRING : HINT_STRING + ':' + name;
+ }
}
diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/HashJoinExpressionAnnotation.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/HashJoinExpressionAnnotation.java
index 88d9dd5..7a063f5 100644
--- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/HashJoinExpressionAnnotation.java
+++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/HashJoinExpressionAnnotation.java
@@ -25,6 +25,8 @@
public class HashJoinExpressionAnnotation implements IExpressionAnnotation {
+ public static final String HINT_STRING = "hash-join";
+
public enum BuildOrProbe {
BUILD,
PROBE
@@ -76,4 +78,19 @@
}
return n;
}
+
+ @Override
+ public String toString() {
+ String result = HINT_STRING;
+ if (name != null) {
+ result += ':' + name;
+ }
+ if (buildOrProbe != null) {
+ result += ':' + buildOrProbe.toString();
+ }
+ if (side != null) {
+ result += ':' + side.toString();
+ }
+ return result;
+ }
}
\ No newline at end of file
diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/JoinProductivityAnnotation.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/JoinProductivityAnnotation.java
index 0191b85..0cba019 100644
--- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/JoinProductivityAnnotation.java
+++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/JoinProductivityAnnotation.java
@@ -21,6 +21,7 @@
public class JoinProductivityAnnotation implements IExpressionAnnotation {
+ public static final String HINT_STRING = "join-productivity";
private final double productivity;
private final String leftSideDataSet;
@@ -36,4 +37,10 @@
public String getLeftSideDataSet() {
return leftSideDataSet;
}
+
+ @Override
+ public String toString() {
+ String result = HINT_STRING + ':' + productivity;
+ return leftSideDataSet == null ? result : result + ':' + leftSideDataSet;
+ }
}
\ No newline at end of file
diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/PredicateCardinalityAnnotation.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/PredicateCardinalityAnnotation.java
index e172449..155783b 100644
--- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/PredicateCardinalityAnnotation.java
+++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/PredicateCardinalityAnnotation.java
@@ -21,6 +21,8 @@
public class PredicateCardinalityAnnotation implements IExpressionAnnotation {
+ public static final String HINT_STRING = "predicate-cardinality";
+
private final double selectivity;
public PredicateCardinalityAnnotation(double selectivity) {
@@ -30,4 +32,9 @@
public double getSelectivity() {
return selectivity;
}
+
+ @Override
+ public String toString() {
+ return HINT_STRING + ':' + selectivity;
+ }
}
diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/config/AlgebricksConfig.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/config/AlgebricksConfig.java
index b87cb92..146d66d 100644
--- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/config/AlgebricksConfig.java
+++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/config/AlgebricksConfig.java
@@ -53,4 +53,5 @@
public static final int MAX_EXPRESSION_TREE_SIZE_DEFAULT = 100;
public static final int COMMON_EXPRESSION_LIMIT_DEFAULT = 100;
public static final int HDFS_SPLIT_PARALLEL_DEFAULT = Runtime.getRuntime().availableProcessors();
+ public static final boolean QUERY_PLAN_CACHE_DEFAULT = true;
}
diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/rewriter/base/PhysicalOptimizationConfig.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/rewriter/base/PhysicalOptimizationConfig.java
index 1aa821b..b626cd0 100644
--- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/rewriter/base/PhysicalOptimizationConfig.java
+++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/rewriter/base/PhysicalOptimizationConfig.java
@@ -70,6 +70,7 @@
private static final String COMMON_EXPRESSION_LIMIT = "EXTRACT_COMMON_EXPRESSION_LIMIT";
private static final String ORDER_FIELDS = "ORDERED_FIELDS";
+ private static final String QUERY_PLAN_CACHE = "QUERY_PLAN_CACHE";
private final Properties properties = new Properties();
@@ -401,6 +402,14 @@
return getBoolean(COLUMN_FILTER, AlgebricksConfig.COLUMN_FILTER_DEFAULT);
}
+ public void setQueryPlanCache(boolean queryPlanCache) {
+ setBoolean(QUERY_PLAN_CACHE, queryPlanCache);
+ }
+
+ public boolean isQueryPlanCacheEnabled() {
+ return getBoolean(QUERY_PLAN_CACHE, AlgebricksConfig.QUERY_PLAN_CACHE_DEFAULT);
+ }
+
public void setExtensionProperty(String property, Object value) {
if (value instanceof Integer) {
setInt(property, (Integer) value);
diff --git a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
index 18825da..68424ed 100644
--- a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
+++ b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
@@ -73,6 +73,7 @@
CLUSTER_TOPOLOGY(STRING),
JOB_QUEUE_CLASS(STRING, "org.apache.hyracks.control.cc.scheduler.FIFOJobQueue"),
JOB_QUEUE_CAPACITY(POSITIVE_INTEGER, 4096),
+ QUERY_PLAN_CACHE_CAPACITY(NONNEGATIVE_INTEGER, 1024), // TODO (GLENN): base default capacity off of available memory?
JOB_MANAGER_CLASS(STRING, "org.apache.hyracks.control.cc.job.JobManager"),
ENFORCE_FRAME_WRITER_PROTOCOL(BOOLEAN, false),
CORES_MULTIPLIER(POSITIVE_INTEGER, 3),
@@ -195,6 +196,8 @@
return "Specify the implementation class name for the job queue";
case JOB_QUEUE_CAPACITY:
return "The maximum number of jobs to queue before rejecting new jobs";
+ case QUERY_PLAN_CACHE_CAPACITY:
+ return "The maximum size of the query plan cache before replacing old entries";
case JOB_MANAGER_CLASS:
return "Specify the implementation class name for the job manager";
case ENFORCE_FRAME_WRITER_PROTOCOL:
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21328?usp=email
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email
Gerrit-MessageType: newchange
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I116e5fa9d16e5c4a5cc6dfbdf8db230221086a9d
Gerrit-Change-Number: 21328
Gerrit-PatchSet: 1
Gerrit-Owner: Rithwik Koul <ri...@couchbase.com>