You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@asterixdb.apache.org by al...@apache.org on 2026/09/17 22:25:31 UTC
(asterixdb) branch master updated: [ASTERIXDB-3864][ASTERIXDB-3865][COMP] Serve filtered ANN queries from the vector index
This is an automated email from the ASF dual-hosted git repository.
asf-gitbox-commits pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/asterixdb.git
The following commit(s) were added to refs/heads/master by this push:
new 579030e41d [ASTERIXDB-3864][ASTERIXDB-3865][COMP] Serve filtered ANN queries from the vector index
579030e41d is described below
commit 579030e41db5819872621756fd7fc76708a210c4
Author: Ali Alsuliman <al...@gmail.com>
AuthorDate: Thu Sep 17 10:23:47 2026 -0700
[ASTERIXDB-3864][ASTERIXDB-3865][COMP] Serve filtered ANN queries from the vector index
A WHERE over a vector index's INCLUDE columns disabled the index-only
ANN plan, so such a query fetched and reranked every candidate from
the primary index -- the cost INCLUDE exists to avoid. The gate asked
whether the predicate was primary-key derived, not whether the chosen
index covered it.
The index-only branch now rebinds the predicate onto the index's
INCLUDE columns and leaves it in its SELECT;
PushFilterIntoVectorSearchRule moves it into the vector search during
the physical rewrites, as it already does for the lookup-and-rerank
plan. Creating the select condition any earlier exposes it to rules
that type an operator's expressions against its input environment,
which an expression reading the operator's own outputs breaks.
Also repairs three CREATE INDEX validation defects met along the way:
a boolean INCLUDE column rejected by a rule meant for BTree keys, a
rejection message that named BTree whatever the index type was, and a
DDL-declared type on an INCLUDE field failing with an unrelated error.
Ext-ref: MB-74038
Co-Authored-By: Claude Opus 5 <no...@anthropic.com>
Change-Id: Iaca95fb11f3d629c56dfa1d73a497e981d69ea09
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21814
Integration-Tests: Jenkins <je...@fulliautomatix.ics.uci.edu>
Reviewed-by: Ian Maxon <im...@apache.org>
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
Reviewed-by: Ali Alsuliman <al...@gmail.com>
---
.../operators/physical/VectorSearchPOperator.java | 16 +-
.../rules/PushFilterIntoVectorSearchRule.java | 359 ++-------------
.../rules/VectorIncludeFilterPushdown.java | 487 +++++++++++++++++++++
.../rules/am/IntroduceTopKAccessMethodRule.java | 100 ++++-
.../rules/am/VectorIndexAccessMethod.java | 53 +++
.../asterix/translator/util/ValidateUtil.java | 82 ++--
.../asterix/app/translator/QueryTranslator.java | 9 +
.../queries_sqlpp/vector/VectorQueries.xml | 11 +
.../create-index-vtree-composite-pk.10.plans.sqlpp | 33 ++
.../create-index-vtree-composite-pk.11.plans.sqlpp | 35 ++
.../create-index-vtree-composite-pk.12.query.sqlpp | 32 ++
.../create-index-vtree-composite-pk.13.plans.sqlpp | 31 ++
.../create-index-vtree-composite-pk.14.query.sqlpp | 32 ++
.../create-index-vtree-composite-pk.15.plans.sqlpp | 31 ++
.../create-index-vtree-composite-pk.7.query.sqlpp | 28 ++
.../create-index-vtree-composite-pk.8.plans.sqlpp | 32 ++
.../create-index-vtree-composite-pk.9.query.sqlpp | 29 ++
...te-index-vtree-include-filter-multi.1.ddl.sqlpp | 37 ++
...index-vtree-include-filter-multi.10.query.sqlpp | 30 ++
...index-vtree-include-filter-multi.11.query.sqlpp | 29 ++
...index-vtree-include-filter-multi.12.plans.sqlpp | 33 ++
...index-vtree-include-filter-multi.13.plans.sqlpp | 31 ++
...index-vtree-include-filter-multi.14.plans.sqlpp | 31 ++
...index-vtree-include-filter-multi.2.update.sqlpp | 55 +++
...te-index-vtree-include-filter-multi.3.ddl.sqlpp | 21 +
...te-index-vtree-include-filter-multi.4.ddl.sqlpp | 25 ++
...-index-vtree-include-filter-multi.5.query.sqlpp | 26 ++
...-index-vtree-include-filter-multi.6.query.sqlpp | 28 ++
...-index-vtree-include-filter-multi.7.plans.sqlpp | 33 ++
...-index-vtree-include-filter-multi.8.query.sqlpp | 28 ++
...-index-vtree-include-filter-multi.9.query.sqlpp | 29 ++
...e-index-vtree-include-filter-multi.99.ddl.sqlpp | 20 +
...create-index-vtree-include-filter.7.plans.sqlpp | 38 ++
.../create-index-vtree-include-typed.1.ddl.sqlpp | 30 ++
...create-index-vtree-include-typed.2.update.sqlpp | 25 ++
.../create-index-vtree-include-typed.3.ddl.sqlpp | 20 +
.../create-index-vtree-include-typed.4.ddl.sqlpp | 27 ++
.../create-index-vtree-include-typed.99.ddl.sqlpp | 20 +
.../create-index-vtree-composite-pk.10.plan | 32 ++
.../create-index-vtree-composite-pk.11.plan | 38 ++
.../create-index-vtree-composite-pk.12.adm | 3 +
.../create-index-vtree-composite-pk.13.plan | 30 ++
.../create-index-vtree-composite-pk.14.adm | 3 +
.../create-index-vtree-composite-pk.15.plan | 32 ++
.../create-index-vtree-composite-pk.7.adm | 3 +
.../create-index-vtree-composite-pk.8.plan | 30 ++
.../create-index-vtree-composite-pk.9.adm | 3 +
.../create-index-vtree-include-filter-multi.10.adm | 1 +
.../create-index-vtree-include-filter-multi.11.adm | 3 +
...create-index-vtree-include-filter-multi.12.plan | 45 ++
...create-index-vtree-include-filter-multi.13.plan | 30 ++
...create-index-vtree-include-filter-multi.14.plan | 30 ++
.../create-index-vtree-include-filter-multi.5.adm | 3 +
.../create-index-vtree-include-filter-multi.6.adm | 3 +
.../create-index-vtree-include-filter-multi.7.plan | 30 ++
.../create-index-vtree-include-filter-multi.8.adm | 1 +
.../create-index-vtree-include-filter-multi.9.adm | 3 +
.../create-index-vtree-include-filter.7.plan | 30 ++
58 files changed, 1982 insertions(+), 387 deletions(-)
diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/VectorSearchPOperator.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/VectorSearchPOperator.java
index 477ec96ff4..5186f97938 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/VectorSearchPOperator.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/VectorSearchPOperator.java
@@ -32,7 +32,7 @@ import org.apache.asterix.metadata.entities.Dataset;
import org.apache.asterix.metadata.entities.Index;
import org.apache.asterix.om.functions.BuiltinFunctions;
import org.apache.asterix.om.types.IAType;
-import org.apache.asterix.optimizer.rules.PushFilterIntoVectorSearchRule;
+import org.apache.asterix.optimizer.rules.VectorIncludeFilterPushdown;
import org.apache.asterix.optimizer.rules.am.VectorJobGenParams;
import org.apache.hyracks.algebricks.common.constraints.AlgebricksPartitionConstraint;
import org.apache.hyracks.algebricks.common.exceptions.AlgebricksException;
@@ -166,12 +166,12 @@ public class VectorSearchPOperator extends IndexSearchPOperator {
// Get filter variable to physical field index mapping from annotation
@SuppressWarnings("unchecked")
Map<LogicalVariable, Integer> filterVarToFieldIndex = (Map<LogicalVariable, Integer>) unnestMapOp
- .getAnnotations().get(PushFilterIntoVectorSearchRule.VECTOR_FILTER_VAR_MAPPING);
+ .getAnnotations().get(VectorIncludeFilterPushdown.VECTOR_FILTER_VAR_MAPPING);
// Get filter variable types from annotation
@SuppressWarnings("unchecked")
Map<LogicalVariable, IAType> filterVarTypes = (Map<LogicalVariable, IAType>) unnestMapOp
- .getAnnotations().get(PushFilterIntoVectorSearchRule.VECTOR_FILTER_VAR_TYPES);
+ .getAnnotations().get(VectorIncludeFilterPushdown.VECTOR_FILTER_VAR_TYPES);
// Create filter schema with direct mapping for filter-only variables
// numSecondaryKeys: offset from physical tuple start to PK field
@@ -185,6 +185,16 @@ public class VectorSearchPOperator extends IndexSearchPOperator {
tupleFilterFactory = mp.createTupleFilterFactory(new IOperatorSchema[] { filterSchema }, filterTypeEnv,
unnestMapOp.getSelectCondition().getValue(), context);
+ } else if (VectorIncludeFilterPushdown.hasDeclaredFilterVariables(unnestMapOp)) {
+ // The index-only plan declares the INCLUDE columns its predicate reads as outputs of this
+ // unnest-map while leaving the predicate in a SELECT above, for PushFilterIntoVectorSearchRule
+ // to move into the select condition. Reaching job generation with the columns declared but no
+ // condition means that never happened -- something relocated or consumed the SELECT. The
+ // runtime emits only [pk..., dist], so the declared columns would be read as fields that were
+ // never written. Fail here rather than emit that.
+ throw new CompilationException(ErrorCode.COMPILATION_ILLEGAL_STATE, unnestMap.getSourceLocation(),
+ "the vector index-only plan declared INCLUDE filter columns on the index search of "
+ + jobGenParams.getIndexName() + " but no filter condition reached it");
}
}
diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFilterIntoVectorSearchRule.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFilterIntoVectorSearchRule.java
index a21f9c287b..f9752ea24c 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFilterIntoVectorSearchRule.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFilterIntoVectorSearchRule.java
@@ -18,11 +18,8 @@
*/
package org.apache.asterix.optimizer.rules;
-import java.util.ArrayList;
-import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
-import java.util.Map;
import java.util.Set;
import org.apache.asterix.common.config.DatasetConfig.IndexType;
@@ -31,11 +28,7 @@ import org.apache.asterix.metadata.entities.Dataset;
import org.apache.asterix.metadata.entities.Index;
import org.apache.asterix.om.functions.BuiltinFunctions;
import org.apache.asterix.om.types.ARecordType;
-import org.apache.asterix.om.types.ATypeTag;
-import org.apache.asterix.om.types.BuiltinType;
-import org.apache.asterix.om.types.IAType;
import org.apache.asterix.optimizer.rules.am.AccessMethodJobGenParams;
-import org.apache.asterix.optimizer.rules.am.AccessMethodUtils;
import org.apache.commons.lang3.mutable.Mutable;
import org.apache.commons.lang3.mutable.MutableObject;
import org.apache.hyracks.algebricks.common.exceptions.AlgebricksException;
@@ -46,16 +39,10 @@ import org.apache.hyracks.algebricks.core.algebra.base.LogicalExpressionTag;
import org.apache.hyracks.algebricks.core.algebra.base.LogicalOperatorTag;
import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable;
import org.apache.hyracks.algebricks.core.algebra.expressions.AbstractFunctionCallExpression;
-import org.apache.hyracks.algebricks.core.algebra.expressions.ScalarFunctionCallExpression;
-import org.apache.hyracks.algebricks.core.algebra.expressions.VariableReferenceExpression;
-import org.apache.hyracks.algebricks.core.algebra.functions.FunctionIdentifier;
-import org.apache.hyracks.algebricks.core.algebra.operators.logical.AssignOperator;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.SelectOperator;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.UnnestMapOperator;
import org.apache.hyracks.algebricks.core.algebra.util.OperatorPropertiesUtil;
import org.apache.hyracks.algebricks.core.rewriter.base.IAlgebraicRewriteRule;
-import org.apache.hyracks.algebricks.rewriter.rules.InlineVariablesRule;
-import org.apache.hyracks.storage.am.vector.utils.VTreeDataTupleAccessor;
import org.apache.hyracks.util.annotations.AiProvenance;
/**
@@ -86,18 +73,6 @@ import org.apache.hyracks.util.annotations.AiProvenance;
*/
public class PushFilterIntoVectorSearchRule implements IAlgebraicRewriteRule {
- /**
- * Annotation key for filter variable to physical field index mapping.
- * Value type: Map<LogicalVariable, Integer>
- */
- public static final String VECTOR_FILTER_VAR_MAPPING = "VECTOR_FILTER_VAR_MAPPING";
-
- /**
- * Annotation key for filter variable to type mapping.
- * Value type: Map<LogicalVariable, IAType>
- */
- public static final String VECTOR_FILTER_VAR_TYPES = "VECTOR_FILTER_VAR_TYPES";
-
@Override
public boolean rewritePost(Mutable<ILogicalOperator> opRef, IOptimizationContext context) {
return false;
@@ -122,135 +97,45 @@ public class PushFilterIntoVectorSearchRule implements IAlgebraicRewriteRule {
if (searchInfo == null) {
return false;
}
- if (searchInfo.includeFieldNames() == null || searchInfo.includeFieldNames().isEmpty()) {
- return false;
- }
-
- // Inline ASSIGNs into the select condition so that filter references hit field-access
- // expressions on the source record directly.
- ILogicalExpression selectCondition = selectOp.getCondition().getValue();
- MutableObject<ILogicalExpression> conditionRef = new MutableObject<>(selectCondition.cloneExpression());
-
- Set<LogicalVariable> usedVariables = new HashSet<>();
- conditionRef.getValue().getUsedVariables(usedVariables);
-
- InlineVariablesRule.InlineVariablesVisitor inlineVisitor = null;
- Map<LogicalVariable, ILogicalExpression> varAssignRhs = new HashMap<>();
- for (ILogicalOperator child = selectOp.getInputs().get(0).getValue(); child
- .getOperatorTag() == LogicalOperatorTag.ASSIGN; child = child.getInputs().get(0).getValue()) {
- varAssignRhs.clear();
- AssignOperator assignOp = (AssignOperator) child;
- extractInlinableVariables(assignOp, usedVariables, varAssignRhs);
-
- if (!varAssignRhs.isEmpty()) {
- if (inlineVisitor == null) {
- inlineVisitor = new InlineVariablesRule.InlineVariablesVisitor(varAssignRhs, null);
- inlineVisitor.setContext(context);
- inlineVisitor.setOperator(selectOp);
- }
- if (!inlineVisitor.transform(conditionRef)) {
- break;
- }
- usedVariables.clear();
- conditionRef.getValue().getUsedVariables(usedVariables);
- }
- }
-
- // Collect the FULL field paths the filter reads, resolved against the record variable(s) this vector
- // search feeds. Matching on full paths (not leaf names) is what keeps `WHERE m.b.year > 2000` from
- // being pushed against an `INCLUDE (a.year)` column, and restricting the base to this subtree's
- // record variable(s) is what keeps a second record (from a join or a LET) from being redirected into
- // the index's INCLUDE field.
- Set<List<String>> filterFieldPaths = new HashSet<>();
- extractFieldPaths(conditionRef.getValue(), searchInfo, filterFieldPaths);
-
- if (filterFieldPaths.isEmpty()) {
- return false;
- }
-
- // Bail out if the filter references any field that is not in the index's INCLUDE list.
- Map<List<String>, Integer> includeFieldIndex = buildIncludeFieldIndex(searchInfo.includeFieldNames());
- // Two identical INCLUDE paths would collapse the map and bind the filter to whichever was inserted
- // last; bail rather than risk pushing a predicate that resolves to the wrong physical field.
- if (includeFieldIndex.size() < searchInfo.includeFieldNames().size()) {
- return false;
- }
- for (List<String> fieldPath : filterFieldPaths) {
- if (!includeFieldIndex.containsKey(fieldPath)) {
+ // Index-only shape: the access-method phase already declared the INCLUDE columns on the unnest-map
+ // and rebound the predicate to them, leaving it here as an ordinary SELECT so that the rules in
+ // between saw an ordinary plan. Nothing is left to resolve -- just move it into the select
+ // condition, which is the one place the runtime reads it from.
+ if (VectorIncludeFilterPushdown.hasDeclaredFilterVariables(searchInfo.vectorUnnest())) {
+ Set<LogicalVariable> conditionVars = new HashSet<>();
+ selectOp.getCondition().getValue().getUsedVariables(conditionVars);
+ if (!searchInfo.vectorUnnest().getVariables().containsAll(conditionVars)) {
return false;
}
- }
-
- // Create fresh logical variables for INCLUDE fields referenced by the filter. These variables
- // are produced by VECTOR_INDEX_UNNEST solely for filter evaluation.
- Map<List<String>, LogicalVariable> fieldToNewVar = new HashMap<>();
- Map<LogicalVariable, Integer> filterVarToFieldIndex = new HashMap<>();
- Map<LogicalVariable, IAType> filterVarToType = new HashMap<>();
-
- // Physical tuple format depends on quantization:
- // Non-quantized: [distance, centroidId, pk_0..pk_{N-1}, include_fields...]
- // Quantized: [distance, centroidId, qDist, qEmbed, pk_0..pk_{N-1}, include_fields...]
- // INCLUDE fields start after the secondary keys and ALL primary key columns.
- int numSecondaryKeys = searchInfo.isQuantized() ? VTreeDataTupleAccessor.Q_NUM_SECONDARY_FIELDS
- : VTreeDataTupleAccessor.NQ_NUM_SECONDARY_FIELDS;
- int includeFieldPhysicalIndex = numSecondaryKeys + searchInfo.numPrimaryKeys();
-
- for (List<String> fieldPath : searchInfo.includeFieldNames()) {
- if (filterFieldPaths.contains(fieldPath)) {
- LogicalVariable newVar = context.newVar();
- fieldToNewVar.put(fieldPath, newVar);
- filterVarToFieldIndex.put(newVar, includeFieldPhysicalIndex);
-
- // For open-schema fields not in the type declaration, getSubFieldType returns null;
- // default to ANY so the type environment can still resolve the variable type.
- IAType fieldType = searchInfo.recordType().getSubFieldType(fieldPath);
- if (fieldType == null) {
- fieldType = BuiltinType.ANY;
- }
- filterVarToType.put(newVar, fieldType);
- }
-
- includeFieldPhysicalIndex++;
- }
-
- // Rewrite field-access expressions in the condition to reference the new INCLUDE variables.
- ILogicalExpression rewrittenCondition = rewriteFieldAccess(conditionRef.getValue(), fieldToNewVar, searchInfo);
-
- // Completeness guard: the pushed condition must reference ONLY the freshly-created INCLUDE
- // variables. The earlier checks act on field names that extractFieldNames recognizes; an
- // access pattern it does not recognize, or a non-inlinable ASSIGN output, can leave a
- // reference to the source record (produced ABOVE the vector unnest-map) in the condition.
- // Embedding such a condition would reference a variable that is not in scope at the unnest --
- // an invalid plan. Bail and leave the SELECT untouched rather than emit a broken plan.
- Set<LogicalVariable> rewrittenUsed = new HashSet<>();
- rewrittenCondition.getUsedVariables(rewrittenUsed);
- if (!new HashSet<>(fieldToNewVar.values()).containsAll(rewrittenUsed)) {
+ searchInfo.vectorUnnest().setSelectCondition(new MutableObject<>(selectOp.getCondition().getValue()));
+ return dropSelect(opRef, selectOp, searchInfo.vectorUnnest(), op, context);
+ }
+
+ // The whole decision — which field paths the predicate reads, whether the index's INCLUDE list
+ // covers them, and what the rewritten predicate looks like — lives in VectorIncludeFilterPushdown,
+ // shared with the index-only gate in IntroduceTopKAccessMethodRule. See that class for why the two
+ // must not be separate implementations.
+ VectorIncludeFilterPushdown.IndexContext idx =
+ new VectorIncludeFilterPushdown.IndexContext(searchInfo.includeFieldNames(), searchInfo.recordType(),
+ searchInfo.isQuantized(), searchInfo.numPrimaryKeys(), searchInfo.recordVars());
+ VectorIncludeFilterPushdown.PushedIncludeFilter pushed = VectorIncludeFilterPushdown
+ .analyze(selectOp.getCondition().getValue(), selectOp, idx, context, context::newVar);
+ if (pushed == null) {
return false;
}
- // Hand the mappings to VectorSearchPOperator so it can construct the filter schema.
- searchInfo.vectorUnnest().getAnnotations().put(VECTOR_FILTER_VAR_MAPPING, filterVarToFieldIndex);
- searchInfo.vectorUnnest().getAnnotations().put(VECTOR_FILTER_VAR_TYPES, filterVarToType);
-
- // Register filter variables as produced by the unnest-map so sanity checks recognize them.
- for (Map.Entry<List<String>, LogicalVariable> entry : fieldToNewVar.entrySet()) {
- LogicalVariable var = entry.getValue();
- IAType type = filterVarToType.get(var);
- searchInfo.vectorUnnest().getVariables().add(var);
- searchInfo.vectorUnnest().getVariableTypes().add(type);
- }
-
- searchInfo.vectorUnnest().setSelectCondition(new MutableObject<>(rewrittenCondition));
+ VectorIncludeFilterPushdown.apply(searchInfo.vectorUnnest(), pushed);
+ return dropSelect(opRef, selectOp, searchInfo.vectorUnnest(), op, context);
+ }
- // Drop the SELECT now that the predicate has been embedded in the vector index search.
+ /** Remove the SELECT whose predicate now lives in the vector search, and retype what changed. */
+ private boolean dropSelect(Mutable<ILogicalOperator> opRef, SelectOperator selectOp, UnnestMapOperator vectorUnnest,
+ ILogicalOperator visitedOp, IOptimizationContext context) throws AlgebricksException {
opRef.setValue(selectOp.getInputs().get(0).getValue());
-
- context.addToDontApplySet(this, op);
-
- context.computeAndSetTypeEnvironmentForOperator(searchInfo.vectorUnnest());
+ context.addToDontApplySet(this, visitedOp);
+ context.computeAndSetTypeEnvironmentForOperator(vectorUnnest);
OperatorPropertiesUtil.typeOpRec(opRef, context);
-
return true;
}
@@ -345,188 +230,4 @@ public class PushFilterIntoVectorSearchRule implements IAlgebraicRewriteRule {
return new VectorSearchInfo(unnest, details.getIncludeFieldNames(), recordType,
details.getVectorParameters().isQuantized(), dataset.getPrimaryKeys().size(), recordVars);
}
-
- /**
- * Extracts variables from ASSIGN that can be inlined.
- */
- private void extractInlinableVariables(AssignOperator assignOp, Set<LogicalVariable> targetVars,
- Map<LogicalVariable, ILogicalExpression> outMap) {
- List<LogicalVariable> vars = assignOp.getVariables();
- List<Mutable<ILogicalExpression>> exprs = assignOp.getExpressions();
-
- for (int i = 0; i < vars.size(); i++) {
- LogicalVariable var = vars.get(i);
- if (targetVars.contains(var)) {
- ILogicalExpression expr = exprs.get(i).getValue();
- if (expr.isFunctional()) {
- outMap.put(var, expr);
- }
- }
- }
- }
-
- /**
- * Builds a map from an INCLUDE field's full path to its index in the INCLUDE list.
- * <p>
- * Keyed by the whole path, not the leaf name: {@code INCLUDE (a.year)} must not match a filter on
- * {@code b.year}. {@link List} equality gives exactly path equality.
- */
- private Map<List<String>, Integer> buildIncludeFieldIndex(List<List<String>> includeFieldNames) {
- Map<List<String>, Integer> result = new HashMap<>();
- for (int i = 0; i < includeFieldNames.size(); i++) {
- result.put(includeFieldNames.get(i), i);
- }
- return result;
- }
-
- /**
- * Collects the full field paths the filter reads from this vector search's record variable(s).
- * <p>
- * Only accesses rooted at one of {@code searchInfo.recordVars()} are collected. An access on any other
- * record is deliberately ignored here, so it survives the rewrite and trips the completeness guard in
- * {@link #rewritePre} — the rule then leaves the SELECT alone instead of silently evaluating someone
- * else's predicate against an INCLUDE column.
- */
- private void extractFieldPaths(ILogicalExpression expr, VectorSearchInfo searchInfo, Set<List<String>> fieldPaths)
- throws AlgebricksException {
- if (expr.getExpressionTag() != LogicalExpressionTag.FUNCTION_CALL) {
- return;
- }
-
- AbstractFunctionCallExpression funcExpr = (AbstractFunctionCallExpression) expr;
- List<String> path = resolveFieldPath(funcExpr, searchInfo);
- if (path != null) {
- fieldPaths.add(path);
- // Do not recurse: the nested accesses under this one are the prefix of the path just added.
- return;
- }
-
- // Recurse into arguments
- for (Mutable<ILogicalExpression> arg : funcExpr.getArguments()) {
- extractFieldPaths(arg.getValue(), searchInfo, fieldPaths);
- }
- }
-
- /**
- * Resolve a (possibly nested) field-access expression to its full path relative to one of this vector
- * search's record variables, or {@code null} if it is not such an access.
- * <p>
- * {@code FIELD_ACCESS_BY_INDEX}'s integer is an index into the field names of the type of ITS OWN base
- * expression, so for a nested access the base's type has to be resolved first (descending the record
- * type as the recursion descends) — resolving every level against the top-level record type produces a
- * bogus name, which in the worst case collides with a real INCLUDE path. Note that
- * {@code ByNameToByIndexFieldAccessRule} has already converted accesses on a declared type to
- * {@code FIELD_ACCESS_BY_INDEX} by the time this rule runs, so the nested case is the common one.
- */
- @AiProvenance(agent = AiProvenance.Agent.CLAUDE_OPUS_5, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
- private List<String> resolveFieldPath(AbstractFunctionCallExpression funcExpr, VectorSearchInfo searchInfo)
- throws AlgebricksException {
- FunctionIdentifier fid = funcExpr.getFunctionIdentifier();
- boolean byName = fid.equals(BuiltinFunctions.FIELD_ACCESS_BY_NAME);
- boolean byIndex = fid.equals(BuiltinFunctions.FIELD_ACCESS_BY_INDEX);
- if ((!byName && !byIndex) || funcExpr.getArguments().size() < 2) {
- return null;
- }
-
- // Resolve the base: either this search's record variable (empty prefix) or an enclosing field access.
- ILogicalExpression base = funcExpr.getArguments().get(0).getValue();
- List<String> prefix;
- if (base.getExpressionTag() == LogicalExpressionTag.VARIABLE) {
- LogicalVariable baseVar = ((VariableReferenceExpression) base).getVariableReference();
- if (!searchInfo.recordVars().contains(baseVar)) {
- return null;
- }
- prefix = List.of();
- } else if (base.getExpressionTag() == LogicalExpressionTag.FUNCTION_CALL) {
- prefix = resolveFieldPath((AbstractFunctionCallExpression) base, searchInfo);
- if (prefix == null) {
- return null;
- }
- } else {
- return null;
- }
-
- String fieldName;
- if (byName) {
- fieldName = AccessMethodUtils.getStringConstant(funcExpr.getArguments().get(1));
- } else {
- Integer idx = AccessMethodUtils.getInt32Constant(funcExpr.getArguments().get(1));
- ARecordType baseType = resolveRecordTypeAt(searchInfo.recordType(), prefix);
- if (idx == null || baseType == null) {
- return null;
- }
- String[] names = baseType.getFieldNames();
- fieldName = idx >= 0 && idx < names.length ? names[idx] : null;
- }
- if (fieldName == null) {
- return null;
- }
-
- List<String> path = new ArrayList<>(prefix);
- path.add(fieldName);
- return path;
- }
-
- /**
- * The record type reached by following {@code path} from {@code recordType}, or {@code null} if the path
- * does not resolve to a record type (open field, non-record type, unknown path).
- */
- private ARecordType resolveRecordTypeAt(ARecordType recordType, List<String> path) throws AlgebricksException {
- if (recordType == null) {
- return null;
- }
- if (path.isEmpty()) {
- return recordType;
- }
- IAType type = recordType.getSubFieldType(path);
- if (type != null && type.getTypeTag() == ATypeTag.OBJECT) {
- return (ARecordType) type;
- }
- return null;
- }
-
- /**
- * Rewrites field-access expressions to use new INCLUDE field variables.
- * Example: gt($row.getField(2), 2000) -> gt($year, 2000)
- */
- private ILogicalExpression rewriteFieldAccess(ILogicalExpression expr,
- Map<List<String>, LogicalVariable> fieldToVar, VectorSearchInfo searchInfo) throws AlgebricksException {
-
- if (expr.getExpressionTag() != LogicalExpressionTag.FUNCTION_CALL) {
- return expr;
- }
-
- AbstractFunctionCallExpression funcExpr = (AbstractFunctionCallExpression) expr;
-
- // Replace a field access with a variable reference only when its FULL path — resolved against this
- // search's record variable — is one of the INCLUDE fields we created a variable for.
- List<String> path = resolveFieldPath(funcExpr, searchInfo);
- if (path != null && fieldToVar.containsKey(path)) {
- LogicalVariable newVar = fieldToVar.get(path);
- VariableReferenceExpression varRef = new VariableReferenceExpression(newVar);
- varRef.setSourceLocation(funcExpr.getSourceLocation());
- return varRef;
- }
-
- // Recurse into arguments
- List<Mutable<ILogicalExpression>> newArgs = new ArrayList<>();
- boolean changed = false;
-
- for (Mutable<ILogicalExpression> argRef : funcExpr.getArguments()) {
- ILogicalExpression newArg = rewriteFieldAccess(argRef.getValue(), fieldToVar, searchInfo);
- newArgs.add(new MutableObject<>(newArg));
- if (newArg != argRef.getValue()) {
- changed = true;
- }
- }
-
- if (changed) {
- ScalarFunctionCallExpression newFunc =
- new ScalarFunctionCallExpression(funcExpr.getFunctionInfo(), newArgs);
- newFunc.setSourceLocation(funcExpr.getSourceLocation());
- return newFunc;
- }
-
- return expr;
- }
}
diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/VectorIncludeFilterPushdown.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/VectorIncludeFilterPushdown.java
new file mode 100644
index 0000000000..c2e019daf2
--- /dev/null
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/VectorIncludeFilterPushdown.java
@@ -0,0 +1,487 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.asterix.optimizer.rules;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Supplier;
+
+import org.apache.asterix.om.functions.BuiltinFunctions;
+import org.apache.asterix.om.types.ARecordType;
+import org.apache.asterix.om.types.ATypeTag;
+import org.apache.asterix.om.types.BuiltinType;
+import org.apache.asterix.om.types.IAType;
+import org.apache.asterix.optimizer.rules.am.AccessMethodUtils;
+import org.apache.commons.lang3.mutable.Mutable;
+import org.apache.commons.lang3.mutable.MutableObject;
+import org.apache.hyracks.algebricks.common.exceptions.AlgebricksException;
+import org.apache.hyracks.algebricks.core.algebra.base.ILogicalExpression;
+import org.apache.hyracks.algebricks.core.algebra.base.ILogicalOperator;
+import org.apache.hyracks.algebricks.core.algebra.base.IOptimizationContext;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalExpressionTag;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalOperatorTag;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable;
+import org.apache.hyracks.algebricks.core.algebra.expressions.AbstractFunctionCallExpression;
+import org.apache.hyracks.algebricks.core.algebra.expressions.ScalarFunctionCallExpression;
+import org.apache.hyracks.algebricks.core.algebra.expressions.VariableReferenceExpression;
+import org.apache.hyracks.algebricks.core.algebra.functions.FunctionIdentifier;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.AssignOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.UnnestMapOperator;
+import org.apache.hyracks.algebricks.rewriter.rules.InlineVariablesRule;
+import org.apache.hyracks.storage.am.vector.utils.VTreeDataTupleAccessor;
+import org.apache.hyracks.util.annotations.AiProvenance;
+
+/**
+ * Decides whether a {@code WHERE} predicate can be evaluated entirely from a vector index's
+ * {@code INCLUDE} columns, and if so rewrites it to read those columns out of the secondary tuple.
+ * <p>
+ * Two callers share this analysis, and they must never disagree:
+ * <ul>
+ * <li>{@link PushFilterIntoVectorSearchRule}, which moves the predicate into the vector search during
+ * {@code physicalRewritesTopLevel}. It does this for both plan shapes.</li>
+ * <li>{@code IntroduceTopKAccessMethodRule} / {@code VectorIndexAccessMethod}, which need the same
+ * answer during the {@code accessMethod} phase to decide whether the index-only plan can carry the
+ * predicate, and to rebind it onto the index's INCLUDE columns. That decision is irreversible —
+ * {@code indexOnly} is serialized into the index-search function arguments before the index-only
+ * branch runs — so a gate that admitted a predicate this analysis then declined would leave the
+ * branch unable to rebind it, with no way back to lookup-and-rerank. Sharing one routine is what
+ * makes the two impossible to drift apart.</li>
+ * </ul>
+ */
+@AiProvenance(agent = AiProvenance.Agent.CLAUDE_OPUS_5, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.REFACTORED, notes = "Extracted from PushFilterIntoVectorSearchRule so the index-only gate and the pushdown "
+ + "share one decision")
+public final class VectorIncludeFilterPushdown {
+
+ private VectorIncludeFilterPushdown() {
+ }
+
+ /**
+ * Annotation key for the filter variable to physical field index mapping.
+ * Value type: Map<LogicalVariable, Integer>
+ */
+ public static final String VECTOR_FILTER_VAR_MAPPING = "VECTOR_FILTER_VAR_MAPPING";
+
+ /**
+ * Annotation key for the filter variable to type mapping.
+ * Value type: Map<LogicalVariable, IAType>
+ */
+ public static final String VECTOR_FILTER_VAR_TYPES = "VECTOR_FILTER_VAR_TYPES";
+
+ /**
+ * Register the INCLUDE columns a rewritten predicate reads as variables produced by the vector index
+ * unnest-map, and hand {@link org.apache.asterix.algebra.operators.physical.VectorSearchPOperator} their
+ * physical field indexes via the annotations above.
+ * <p>
+ * This only declares the columns; the predicate itself stays wherever the caller put it. The index-only
+ * plan uses that: it binds its predicate to these variables but leaves it in a {@code SELECT} above the
+ * unnest-map, so the predicate remains an ordinary expression over ordinary variables for every rule
+ * that runs before {@link PushFilterIntoVectorSearchRule} moves it in.
+ */
+ public static void declareFilterVariables(UnnestMapOperator vectorUnnest, PushedIncludeFilter pushed) {
+ vectorUnnest.getAnnotations().put(VECTOR_FILTER_VAR_MAPPING, pushed.varToFieldIndex());
+ vectorUnnest.getAnnotations().put(VECTOR_FILTER_VAR_TYPES, pushed.varTypes());
+
+ for (Map.Entry<LogicalVariable, IAType> entry : pushed.varTypes().entrySet()) {
+ vectorUnnest.getVariables().add(entry.getKey());
+ vectorUnnest.getVariableTypes().add(entry.getValue());
+ }
+ }
+
+ /**
+ * Declare the INCLUDE columns and move the predicate into the unnest-map's select condition, which is
+ * where the runtime reads it from. Only {@link PushFilterIntoVectorSearchRule} calls this, during the
+ * physical rewrites: a select condition reads variables the operator produces rather than variables from
+ * its input, which is not what an expression on an operator normally means, so it is created as late as
+ * possible to keep it out of the way of the rules that walk expressions.
+ */
+ public static void apply(UnnestMapOperator vectorUnnest, PushedIncludeFilter pushed) {
+ declareFilterVariables(vectorUnnest, pushed);
+ vectorUnnest.setSelectCondition(new MutableObject<>(pushed.condition()));
+ }
+
+ /**
+ * Whether {@code vectorUnnest} already has its INCLUDE filter variables declared -- i.e. the index-only
+ * plan bound a predicate to them during the access-method phase and left it in a SELECT for
+ * {@link PushFilterIntoVectorSearchRule} to move in.
+ */
+ public static boolean hasDeclaredFilterVariables(UnnestMapOperator vectorUnnest) {
+ return vectorUnnest.getAnnotations().containsKey(VECTOR_FILTER_VAR_MAPPING);
+ }
+
+ /**
+ * A predicate that has been fully rewritten against a vector index's INCLUDE columns.
+ *
+ * @param condition the rewritten predicate, referencing only the variables in {@code varToFieldIndex}
+ * @param varToFieldIndex each fresh filter variable's physical field index in the secondary tuple
+ * @param varTypes each fresh filter variable's type, for the filter's type environment
+ */
+ public record PushedIncludeFilter(ILogicalExpression condition, Map<LogicalVariable, Integer> varToFieldIndex,
+ Map<LogicalVariable, IAType> varTypes) {
+ }
+
+ /**
+ * The index-side inputs the analysis needs. Grouped so the two call sites, which obtain them from
+ * different places (a metadata lookup keyed off the plan versus the already-chosen index), cannot pass
+ * them in a different order.
+ *
+ * @param includeFieldNames the index's INCLUDE field paths, in index order
+ * @param recordType the dataset's record type, used to resolve {@code FIELD_ACCESS_BY_INDEX} and to
+ * type each filter variable
+ * @param isQuantized whether the index is quantized, which sets the secondary-field count
+ * @param numPrimaryKeys the dataset's primary key count
+ * @param recordVars the variables holding the record(s) this vector search feeds; a field access must
+ * be rooted at one of these to be a pushdown candidate
+ */
+ public record IndexContext(List<List<String>> includeFieldNames, ARecordType recordType, boolean isQuantized,
+ int numPrimaryKeys, Set<LogicalVariable> recordVars) {
+ }
+
+ /**
+ * Whether {@code condition} can be pushed, without allocating any variables or touching the
+ * optimization context. Used by the cost model and by the index-only gate.
+ */
+ public static boolean isPushable(ILogicalExpression condition, ILogicalOperator selectOp, IndexContext idx,
+ IOptimizationContext context) throws AlgebricksException {
+ return analyze(condition, selectOp, idx, context, new ThrowawayVariableSupplier()) != null;
+ }
+
+ /**
+ * Rewrites {@code condition} to read the index's INCLUDE columns, or returns {@code null} when any part
+ * of it cannot be served that way.
+ *
+ * @param condition the {@code SELECT} condition to push
+ * @param selectOp the {@code SELECT} carrying {@code condition}; the {@code ASSIGN} chain below it is
+ * inlined into the condition so that field accesses on the record are visible
+ * @param varSupplier allocates the fresh filter variables — {@code context::newVar} when the result
+ * will be spliced into the plan, a throwaway supplier when only the verdict matters
+ */
+ public static PushedIncludeFilter analyze(ILogicalExpression condition, ILogicalOperator selectOp, IndexContext idx,
+ IOptimizationContext context, Supplier<LogicalVariable> varSupplier) throws AlgebricksException {
+ if (condition == null || idx.includeFieldNames() == null || idx.includeFieldNames().isEmpty()) {
+ return null;
+ }
+
+ MutableObject<ILogicalExpression> conditionRef = new MutableObject<>(condition.cloneExpression());
+ inlineAssigns(conditionRef, selectOp, context);
+
+ // Collect the FULL field paths the filter reads, resolved against the record variable(s) this vector
+ // search feeds. Matching on full paths (not leaf names) is what keeps `WHERE m.b.year > 2000` from
+ // being pushed against an `INCLUDE (a.year)` column, and restricting the base to this subtree's
+ // record variable(s) is what keeps a second record (from a join or a LET) from being redirected into
+ // the index's INCLUDE field.
+ Set<List<String>> filterFieldPaths = new HashSet<>();
+ extractFieldPaths(conditionRef.getValue(), idx, filterFieldPaths);
+ if (filterFieldPaths.isEmpty()) {
+ return null;
+ }
+
+ // Bail out if the filter references any field that is not in the index's INCLUDE list.
+ Map<List<String>, Integer> includeFieldIndex = buildIncludeFieldIndex(idx.includeFieldNames());
+ // Two identical INCLUDE paths would collapse the map and bind the filter to whichever was inserted
+ // last; bail rather than risk pushing a predicate that resolves to the wrong physical field.
+ if (includeFieldIndex.size() < idx.includeFieldNames().size()) {
+ return null;
+ }
+ for (List<String> fieldPath : filterFieldPaths) {
+ if (!includeFieldIndex.containsKey(fieldPath)) {
+ return null;
+ }
+ }
+
+ // Physical tuple format depends on quantization:
+ // Non-quantized: [distance, centroidId, pk_0..pk_{N-1}, include_fields...]
+ // Quantized: [distance, centroidId, qDist, qEmbed, pk_0..pk_{N-1}, include_fields...]
+ // INCLUDE fields start after the secondary keys and ALL primary key columns.
+ int numSecondaryKeys = idx.isQuantized() ? VTreeDataTupleAccessor.Q_NUM_SECONDARY_FIELDS
+ : VTreeDataTupleAccessor.NQ_NUM_SECONDARY_FIELDS;
+ int includeFieldPhysicalIndex = numSecondaryKeys + idx.numPrimaryKeys();
+
+ Map<List<String>, LogicalVariable> fieldToNewVar = new HashMap<>();
+ // Insertion-ordered (INCLUDE order): the caller appends these to the unnest-map's variable list,
+ // so a hash order would make the emitted plan differ run to run for a multi-field predicate.
+ Map<LogicalVariable, Integer> filterVarToFieldIndex = new LinkedHashMap<>();
+ Map<LogicalVariable, IAType> filterVarToType = new LinkedHashMap<>();
+
+ for (List<String> fieldPath : idx.includeFieldNames()) {
+ if (filterFieldPaths.contains(fieldPath)) {
+ LogicalVariable newVar = varSupplier.get();
+ fieldToNewVar.put(fieldPath, newVar);
+ filterVarToFieldIndex.put(newVar, includeFieldPhysicalIndex);
+
+ // For open-schema fields not in the type declaration, getSubFieldType returns null;
+ // default to ANY so the type environment can still resolve the variable type.
+ IAType fieldType = idx.recordType().getSubFieldType(fieldPath);
+ if (fieldType == null) {
+ fieldType = BuiltinType.ANY;
+ }
+ filterVarToType.put(newVar, fieldType);
+ }
+ includeFieldPhysicalIndex++;
+ }
+
+ ILogicalExpression rewritten = rewriteFieldAccess(conditionRef.getValue(), fieldToNewVar, idx);
+
+ // Completeness guard: the pushed condition must reference ONLY the freshly-created INCLUDE
+ // variables. The earlier checks act on field paths that resolveFieldPath recognizes; an access
+ // pattern it does not recognize, or a non-inlinable ASSIGN output, can leave a reference to the
+ // source record in the condition. Embedding such a condition would reference a variable that is not
+ // in scope at the unnest -- an invalid plan. Decline instead of emitting one.
+ Set<LogicalVariable> rewrittenUsed = new HashSet<>();
+ rewritten.getUsedVariables(rewrittenUsed);
+ if (!new HashSet<>(fieldToNewVar.values()).containsAll(rewrittenUsed)) {
+ return null;
+ }
+
+ return new PushedIncludeFilter(rewritten, filterVarToFieldIndex, filterVarToType);
+ }
+
+ /**
+ * Inline the {@code ASSIGN} chain below the {@code SELECT} into the condition, so that filter references
+ * hit field-access expressions on the source record directly rather than an intermediate variable.
+ */
+ private static void inlineAssigns(MutableObject<ILogicalExpression> conditionRef, ILogicalOperator selectOp,
+ IOptimizationContext context) throws AlgebricksException {
+ if (selectOp == null || selectOp.getInputs().isEmpty()) {
+ return;
+ }
+ Set<LogicalVariable> usedVariables = new HashSet<>();
+ conditionRef.getValue().getUsedVariables(usedVariables);
+
+ InlineVariablesRule.InlineVariablesVisitor inlineVisitor = null;
+ Map<LogicalVariable, ILogicalExpression> varAssignRhs = new HashMap<>();
+
+ for (ILogicalOperator child = selectOp.getInputs().get(0).getValue(); child
+ .getOperatorTag() == LogicalOperatorTag.ASSIGN; child = child.getInputs().get(0).getValue()) {
+ varAssignRhs.clear();
+ extractInlinableVariables((AssignOperator) child, usedVariables, varAssignRhs);
+
+ if (!varAssignRhs.isEmpty()) {
+ if (inlineVisitor == null) {
+ inlineVisitor = new InlineVariablesRule.InlineVariablesVisitor(varAssignRhs, null);
+ inlineVisitor.setContext(context);
+ // The visitor checks rhs variables against the live-in of this operator; without it the
+ // live-variable lookup has no operator to ask.
+ inlineVisitor.setOperator(selectOp);
+ }
+ if (!inlineVisitor.transform(conditionRef)) {
+ break;
+ }
+ usedVariables.clear();
+ conditionRef.getValue().getUsedVariables(usedVariables);
+ }
+ if (child.getInputs().isEmpty()) {
+ break;
+ }
+ }
+ }
+
+ private static void extractInlinableVariables(AssignOperator assignOp, Set<LogicalVariable> targetVars,
+ Map<LogicalVariable, ILogicalExpression> outMap) {
+ List<LogicalVariable> vars = assignOp.getVariables();
+ List<Mutable<ILogicalExpression>> exprs = assignOp.getExpressions();
+
+ for (int i = 0; i < vars.size(); i++) {
+ LogicalVariable var = vars.get(i);
+ if (targetVars.contains(var)) {
+ ILogicalExpression expr = exprs.get(i).getValue();
+ if (expr.isFunctional()) {
+ outMap.put(var, expr);
+ }
+ }
+ }
+ }
+
+ /**
+ * Builds a map from an INCLUDE field's full path to its index in the INCLUDE list.
+ * <p>
+ * Keyed by the whole path, not the leaf name: {@code INCLUDE (a.year)} must not match a filter on
+ * {@code b.year}. {@link List} equality gives exactly path equality.
+ */
+ private static Map<List<String>, Integer> buildIncludeFieldIndex(List<List<String>> includeFieldNames) {
+ Map<List<String>, Integer> result = new HashMap<>();
+ for (int i = 0; i < includeFieldNames.size(); i++) {
+ result.put(includeFieldNames.get(i), i);
+ }
+ return result;
+ }
+
+ /**
+ * Collects the full field paths the filter reads from this vector search's record variable(s).
+ * <p>
+ * Only accesses rooted at one of {@code idx.recordVars()} are collected. An access on any other record
+ * is deliberately ignored here, so it survives the rewrite and trips the completeness guard in
+ * {@link #analyze} — which then declines instead of silently evaluating someone else's predicate
+ * against an INCLUDE column.
+ */
+ private static void extractFieldPaths(ILogicalExpression expr, IndexContext idx, Set<List<String>> fieldPaths)
+ throws AlgebricksException {
+ if (expr.getExpressionTag() != LogicalExpressionTag.FUNCTION_CALL) {
+ return;
+ }
+
+ AbstractFunctionCallExpression funcExpr = (AbstractFunctionCallExpression) expr;
+ List<String> path = resolveFieldPath(funcExpr, idx);
+ if (path != null) {
+ fieldPaths.add(path);
+ // Do not recurse: the nested accesses under this one are the prefix of the path just added.
+ return;
+ }
+
+ for (Mutable<ILogicalExpression> arg : funcExpr.getArguments()) {
+ extractFieldPaths(arg.getValue(), idx, fieldPaths);
+ }
+ }
+
+ /**
+ * Resolve a (possibly nested) field-access expression to its full path relative to one of this vector
+ * search's record variables, or {@code null} if it is not such an access.
+ * <p>
+ * {@code FIELD_ACCESS_BY_INDEX}'s integer is an index into the field names of the type of ITS OWN base
+ * expression, so for a nested access the base's type has to be resolved first (descending the record
+ * type as the recursion descends) — resolving every level against the top-level record type produces a
+ * bogus name, which in the worst case collides with a real INCLUDE path. Both forms are handled because
+ * this runs in two phases: before {@code ByNameToByIndexFieldAccessRule} (the index-only gate) accesses
+ * are by name, after it (the physical pushdown) a declared type's accesses are by index.
+ */
+ private static List<String> resolveFieldPath(AbstractFunctionCallExpression funcExpr, IndexContext idx)
+ throws AlgebricksException {
+ FunctionIdentifier fid = funcExpr.getFunctionIdentifier();
+ boolean byName = fid.equals(BuiltinFunctions.FIELD_ACCESS_BY_NAME);
+ boolean byIndex = fid.equals(BuiltinFunctions.FIELD_ACCESS_BY_INDEX);
+ if ((!byName && !byIndex) || funcExpr.getArguments().size() < 2) {
+ return null;
+ }
+
+ ILogicalExpression base = funcExpr.getArguments().get(0).getValue();
+ List<String> prefix;
+ if (base.getExpressionTag() == LogicalExpressionTag.VARIABLE) {
+ LogicalVariable baseVar = ((VariableReferenceExpression) base).getVariableReference();
+ if (!idx.recordVars().contains(baseVar)) {
+ return null;
+ }
+ prefix = List.of();
+ } else if (base.getExpressionTag() == LogicalExpressionTag.FUNCTION_CALL) {
+ prefix = resolveFieldPath((AbstractFunctionCallExpression) base, idx);
+ if (prefix == null) {
+ return null;
+ }
+ } else {
+ return null;
+ }
+
+ String fieldName;
+ if (byName) {
+ fieldName = AccessMethodUtils.getStringConstant(funcExpr.getArguments().get(1));
+ } else {
+ Integer fieldIdx = AccessMethodUtils.getInt32Constant(funcExpr.getArguments().get(1));
+ ARecordType baseType = resolveRecordTypeAt(idx.recordType(), prefix);
+ if (fieldIdx == null || baseType == null) {
+ return null;
+ }
+ String[] names = baseType.getFieldNames();
+ fieldName = fieldIdx >= 0 && fieldIdx < names.length ? names[fieldIdx] : null;
+ }
+ if (fieldName == null) {
+ return null;
+ }
+
+ List<String> path = new ArrayList<>(prefix);
+ path.add(fieldName);
+ return path;
+ }
+
+ /**
+ * The record type reached by following {@code path} from {@code recordType}, or {@code null} if the path
+ * does not resolve to a record type (open field, non-record type, unknown path).
+ */
+ private static ARecordType resolveRecordTypeAt(ARecordType recordType, List<String> path)
+ throws AlgebricksException {
+ if (recordType == null) {
+ return null;
+ }
+ if (path.isEmpty()) {
+ return recordType;
+ }
+ IAType type = recordType.getSubFieldType(path);
+ if (type != null && type.getTypeTag() == ATypeTag.OBJECT) {
+ return (ARecordType) type;
+ }
+ return null;
+ }
+
+ /**
+ * Rewrites field-access expressions to use the new INCLUDE field variables.
+ * Example: {@code gt($row.getField(2), 2000)} -> {@code gt($year, 2000)}
+ */
+ private static ILogicalExpression rewriteFieldAccess(ILogicalExpression expr,
+ Map<List<String>, LogicalVariable> fieldToVar, IndexContext idx) throws AlgebricksException {
+ if (expr.getExpressionTag() != LogicalExpressionTag.FUNCTION_CALL) {
+ return expr;
+ }
+
+ AbstractFunctionCallExpression funcExpr = (AbstractFunctionCallExpression) expr;
+
+ // Replace a field access with a variable reference only when its FULL path — resolved against this
+ // search's record variable — is one of the INCLUDE fields we created a variable for.
+ List<String> path = resolveFieldPath(funcExpr, idx);
+ if (path != null && fieldToVar.containsKey(path)) {
+ LogicalVariable newVar = fieldToVar.get(path);
+ VariableReferenceExpression varRef = new VariableReferenceExpression(newVar);
+ varRef.setSourceLocation(funcExpr.getSourceLocation());
+ return varRef;
+ }
+
+ List<Mutable<ILogicalExpression>> newArgs = new ArrayList<>();
+ boolean changed = false;
+ for (Mutable<ILogicalExpression> argRef : funcExpr.getArguments()) {
+ ILogicalExpression newArg = rewriteFieldAccess(argRef.getValue(), fieldToVar, idx);
+ newArgs.add(new MutableObject<>(newArg));
+ if (newArg != argRef.getValue()) {
+ changed = true;
+ }
+ }
+
+ if (changed) {
+ ScalarFunctionCallExpression newFunc =
+ new ScalarFunctionCallExpression(funcExpr.getFunctionInfo(), newArgs);
+ newFunc.setSourceLocation(funcExpr.getSourceLocation());
+ return newFunc;
+ }
+
+ return expr;
+ }
+
+ /**
+ * Hands out variables that are never spliced into a plan, for the callers that only need the verdict.
+ * Negative ids keep them distinguishable from real ones if they ever leak into a log line.
+ */
+ private static final class ThrowawayVariableSupplier implements Supplier<LogicalVariable> {
+ private int nextId = -1;
+
+ @Override
+ public LogicalVariable get() {
+ return new LogicalVariable(nextId--);
+ }
+ }
+}
diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceTopKAccessMethodRule.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceTopKAccessMethodRule.java
index 001f3471fd..a02d2dc643 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceTopKAccessMethodRule.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceTopKAccessMethodRule.java
@@ -44,6 +44,7 @@ import org.apache.asterix.om.functions.BuiltinFunctions;
import org.apache.asterix.om.types.IAType;
import org.apache.asterix.om.utils.ConstantExpressionUtil;
import org.apache.asterix.optimizer.cost.VectorIndexGeometry;
+import org.apache.asterix.optimizer.rules.VectorIncludeFilterPushdown;
import org.apache.commons.lang3.mutable.Mutable;
import org.apache.commons.lang3.mutable.MutableObject;
import org.apache.hyracks.algebricks.common.exceptions.AlgebricksException;
@@ -84,6 +85,34 @@ import org.apache.logging.log4j.Logger;
* → Falls back to exact KNN search (exhaustive distance computation on all tuples)
*
* The ORDER BY ANN_DISTANCE operator handles distance computation in both cases.
+ *
+ * <h2>Why this rule does not push the WHERE into the index search</h2>
+ *
+ * A {@code WHERE} over the index's {@code INCLUDE} columns is evaluated by the vector cursor itself, which
+ * reads it from the unnest-map's {@code selectCondition}. This rule never sets that condition, even for the
+ * index-only plan, which it could: it stops at rebinding the predicate and leaves it in its {@code SELECT}
+ * for {@link org.apache.asterix.optimizer.rules.PushFilterIntoVectorSearchRule} to move in during
+ * {@code physicalRewritesTopLevel}. That split is deliberate.
+ * <p>
+ * A {@code selectCondition} is the one expression on an operator that reads variables the operator
+ * <em>produces</em> rather than variables from its input. Rules that walk an operator's expressions
+ * reasonably type them against the input environment, so a condition present while the logical rewrites are
+ * still running breaks them. Two rules in {@code buildPlanCleanupRuleCollection} — which runs immediately
+ * after this one — do exactly that and fail on it:
+ * {@code SetClosedRecordConstructorsRule} ("Could not infer type for variable") and
+ * {@code InjectTypeCastForFunctionArgumentsRule} (NPE in the type computer, for a predicate containing
+ * {@code switch-case} or one of the {@code if-missing}/{@code if-null} family). Both are latent rather than
+ * broken today precisely because every producer of a {@code selectCondition} —
+ * {@code PushFilterIntoVectorSearchRule} and {@code PushLimitIntoPrimarySearchRule} — runs in the physical
+ * rewrites, after the last rule that would trip over one.
+ * <p>
+ * What this rule must still do early is rebind the predicate. The index-only branch drops the primary
+ * lookup, so the record variable the {@code WHERE} reads ceases to exist and
+ * {@code VectorIndexAccessMethod#neutralizeDanglingExpressions} would collapse the predicate to
+ * {@code select(missing)} and drop every row. So the branch declares the {@code INCLUDE} columns as outputs
+ * of the index-search unnest-map and rewrites the predicate to read those variables instead — after which
+ * it is an ordinary predicate over ordinary variables, in an ordinary {@code SELECT}, and every rule between
+ * here and the physical rewrites sees a plan it already understands.
*/
public class IntroduceTopKAccessMethodRule extends AbstractIntroduceAccessMethodRule {
@@ -852,9 +881,12 @@ public class IntroduceTopKAccessMethodRule extends AbstractIntroduceAccessMethod
// The search collects this many candidates either way; an index-only plan then reads the distance
// off the cursor rather than fetching and reranking them.
double candidateCard = (double) topK * queryKMultiplier;
- double fetchedCard = isProjectionPkOnly() ? 0 : candidateCard;
for (Pair<IAccessMethod, Index> candidate : chosenIndexes) {
+ // Whether the primary lookup is skipped is per-index: two indexes can differ in whether their
+ // INCLUDE list covers this query's WHERE, and that is the difference between fetching every
+ // candidate and fetching none.
+ double fetchedCard = isProjectionPkOnly(candidate.second, context) ? 0 : candidateCard;
Index.VectorIndexDetails details = (Index.VectorIndexDetails) candidate.second.getIndexDetails();
VectorIndexGeometry geometry = new VectorIndexGeometry(details.getVectorParameters(),
details.getIncludeFieldTypes(), primaryKeyTypes, datasetCardinality, numPartitions, pageSize,
@@ -896,7 +928,7 @@ public class IntroduceTopKAccessMethodRule extends AbstractIntroduceAccessMethod
// when every variable consumed above the LIMIT is PK-derived, the assembled record from the
// primary BTree lookup is never needed, so the vector index emits (pk, dist) and the plan
// skips the lookup + rerank entirely.
- boolean indexOnly = isProjectionPkOnly();
+ boolean indexOnly = isProjectionPkOnly(vectorIndex, context);
// Build the index-search subplan (UNNEST-MAP over vector index). selectOp is passed so the
// access method can attach a selectCondition for filter pushdown when applicable.
@@ -937,7 +969,7 @@ public class IntroduceTopKAccessMethodRule extends AbstractIntroduceAccessMethod
* <p>Conservatively returns {@code false} for composite or nested PK paths, external data sources, or
* unfamiliar plan shapes.
*/
- protected boolean isProjectionPkOnly() {
+ protected boolean isProjectionPkOnly(Index vectorIndex, IOptimizationContext context) throws AlgebricksException {
// Index-only skips the primary BTree lookup and emits (pk, dist) straight from the secondary VTree.
// Correct even with deletes: the VTree search cursor reconciles anti-matter (delete) tuples itself,
// so a PK-only ANN query over a dataset with deletes returns no deleted PKs. Enabled by default; the
@@ -1009,24 +1041,70 @@ public class IntroduceTopKAccessMethodRule extends AbstractIntroduceAccessMethod
}
}
- // A WHERE below the LIMIT that needs a non-PK record field (e.g. an INCLUDE-field filter such as
- // WHERE m.year > 2000) cannot be served by the index-only plan: the record variable is dead, so
- // VectorIndexAccessMethod#neutralizeDanglingExpressions collapses the predicate to select(missing)
- // and drops every row. The liveOut scan above only covers the projection above the LIMIT, so a
- // SELECT below the LIMIT is invisible to it. Require every SELECT-condition variable in the subtree
- // to be PK-safe; otherwise fall back to lookup-and-rerank, which materializes the record and lets
- // PushFilterIntoVectorSearchRule push the INCLUDE filter into the vector cursor correctly.
+ // A WHERE below the LIMIT is invisible to the liveOut scan above, which only covers the projection
+ // above the LIMIT. It can still be served index-only in two ways: its variables are PK-derived, or
+ // the predicate reads nothing but INCLUDE columns of the chosen index, in which case the index-only
+ // branch rebinds it onto those columns instead of leaving a SELECT above a dead record variable
+ // (which VectorIndexAccessMethod#neutralizeDanglingExpressions would collapse to select(missing),
+ // dropping every row). Anything else falls back to lookup-and-rerank.
Set<LogicalVariable> filterVars = new HashSet<>();
collectSelectConditionVars(subTree.getRoot(), filterVars);
for (LogicalVariable v : filterVars) {
if (!isVarPkSafe(v, bindings, pkCtx, visiting)) {
- LOGGER.trace("isProjectionPkOnly: WHERE condition var {} is not PK-safe; not index-only", v);
+ if (isFilterPushableToInclude(vectorIndex, context)) {
+ return true;
+ }
+ LOGGER.trace("isProjectionPkOnly: WHERE condition var {} is neither PK-safe nor an INCLUDE "
+ + "field of {}; not index-only", v, vectorIndex.getIndexName());
return false;
}
}
return true;
}
+ /**
+ * Whether this subtree's {@code SELECT} predicate can be evaluated entirely from {@code vectorIndex}'s
+ * INCLUDE columns, and can therefore be rebound onto them by the index-only branch.
+ * <p>
+ * The verdict must match what {@code VectorIndexAccessMethod} will actually manage, because by the time
+ * that branch runs {@code indexOnly} has already been serialized into the index-search arguments and
+ * cannot be withdrawn. Both go through {@link VectorIncludeFilterPushdown} for exactly that reason; this
+ * call allocates no variables.
+ */
+ protected boolean isFilterPushableToInclude(Index vectorIndex, IOptimizationContext context)
+ throws AlgebricksException {
+ if (selectOp == null || vectorIndex == null || vectorIndex.getIndexType() != IndexType.VTREE) {
+ return false;
+ }
+ VectorIncludeFilterPushdown.IndexContext idx = buildIncludeFilterContext(vectorIndex);
+ return idx != null
+ && VectorIncludeFilterPushdown.isPushable(selectOp.getCondition().getValue(), selectOp, idx, context);
+ }
+
+ /**
+ * Assembles the INCLUDE-pushdown inputs from this subtree: the chosen index's INCLUDE list and
+ * quantization, the dataset record type, and the record variable(s) the {@code DataSourceScan}
+ * produces — the only bases a pushable field access may be rooted at.
+ */
+ protected VectorIncludeFilterPushdown.IndexContext buildIncludeFilterContext(Index vectorIndex) {
+ if (subTree == null || subTree.getDataset() == null || subTree.getDataSourceRef() == null) {
+ return null;
+ }
+ ILogicalOperator dataSourceOpRaw = subTree.getDataSourceRef().getValue();
+ if (!(dataSourceOpRaw instanceof AbstractScanOperator)) {
+ return null;
+ }
+ List<LogicalVariable> dsVars = ((AbstractScanOperator) dataSourceOpRaw).getVariables();
+ int numPK = subTree.getDataset().getPrimaryKeys().size();
+ if (dsVars.size() <= numPK) {
+ return null;
+ }
+ Index.VectorIndexDetails details = (Index.VectorIndexDetails) vectorIndex.getIndexDetails();
+ return new VectorIncludeFilterPushdown.IndexContext(details.getIncludeFieldNames(), subTree.getRecordType(),
+ details.getVectorParameters().isQuantized(), numPK,
+ new HashSet<>(dsVars.subList(numPK, dsVars.size())));
+ }
+
/**
* Groups the pieces {@link #isVarPkSafe} and {@link #isExprPkSafe} need to decide whether a
* {@code field-access-by-name} target is PK-safe: the data-source PK variables, the combined
diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/VectorIndexAccessMethod.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/VectorIndexAccessMethod.java
index 3f8983e109..da3b9c69c6 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/VectorIndexAccessMethod.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/VectorIndexAccessMethod.java
@@ -48,6 +48,7 @@ import org.apache.asterix.om.types.ARecordType;
import org.apache.asterix.om.types.ATypeTag;
import org.apache.asterix.om.types.BuiltinType;
import org.apache.asterix.om.types.IAType;
+import org.apache.asterix.optimizer.rules.VectorIncludeFilterPushdown;
import org.apache.commons.lang3.mutable.Mutable;
import org.apache.commons.lang3.mutable.MutableObject;
import org.apache.hyracks.algebricks.common.exceptions.AlgebricksException;
@@ -363,6 +364,11 @@ public class VectorIndexAccessMethod implements IAccessMethod {
distVarRef.setSourceLocation(orderOp.getSourceLocation());
orderOp.getOrderExpressions().get(0).second.setValue(distVarRef);
+ // Rebind the WHERE onto the index's INCLUDE columns. There is no primary lookup here to
+ // evaluate it against, and the record variable is about to be neutralized, so a predicate
+ // left reading the record would collapse to select(missing) and drop every row.
+ bindIncludeFilterToSearch(unnestMap, selectOp, chosenIndex, dataset, recordType, dataSourceOp, context);
+
// Build the variable substitution map: old PK vars → new PK vars. The secondary
// UnnestMap allocates fresh context.newVar() PK variables, so anything that
// referenced the old PK variable from the DataSourceScan needs to be redirected.
@@ -720,6 +726,53 @@ public class VectorIndexAccessMethod implements IAccessMethod {
}
}
+ /**
+ * Bind a {@code WHERE} over the index's {@code INCLUDE} columns to the vector search of an index-only
+ * plan: declare the columns it reads as outputs of the index-search unnest-map and rewrite the
+ * predicate to read them, so the cursor can filter candidates itself and no primary lookup is needed to
+ * evaluate it. The predicate stays in its {@code SELECT} until the physical rewrites.
+ * <p>
+ * A no-op when the query has no {@code WHERE}. Otherwise the predicate must be fully bindable:
+ * {@code IntroduceTopKAccessMethodRule.isProjectionPkOnly} only admits the index-only plan when
+ * {@link VectorIncludeFilterPushdown} says it is, and by the time we get here {@code indexOnly} has
+ * already been serialized into the index-search arguments and cannot be withdrawn. A decline at this
+ * point is therefore the two having drifted apart, which is a broken invariant rather than an
+ * unsupported plan shape — fail loudly instead of emitting a plan that silently drops every row.
+ */
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_OPUS_5, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED, notes = "Index-only ANN plans bind their INCLUDE-field predicate")
+ private static void bindIncludeFilterToSearch(UnnestMapOperator unnestMap, SelectOperator selectOp,
+ Index chosenIndex, Dataset dataset, ARecordType recordType, AbstractDataSourceOperator dataSourceOp,
+ IOptimizationContext context) throws AlgebricksException {
+ if (selectOp == null) {
+ return;
+ }
+ Index.VectorIndexDetails details = (Index.VectorIndexDetails) chosenIndex.getIndexDetails();
+ int numPK = dataset.getPrimaryKeys().size();
+ List<LogicalVariable> dsVars = dataSourceOp.getVariables();
+ // Everything after the primary keys is the dataset record and, when present, the meta record: the
+ // only bases a pushable field access may be rooted at.
+ Set<LogicalVariable> recordVars = new HashSet<>(dsVars.subList(Math.min(numPK, dsVars.size()), dsVars.size()));
+
+ VectorIncludeFilterPushdown.IndexContext idx =
+ new VectorIncludeFilterPushdown.IndexContext(details.getIncludeFieldNames(), recordType,
+ details.getVectorParameters().isQuantized(), numPK, recordVars);
+ VectorIncludeFilterPushdown.PushedIncludeFilter pushed = VectorIncludeFilterPushdown
+ .analyze(selectOp.getCondition().getValue(), selectOp, idx, context, context::newVar);
+ if (pushed == null) {
+ throw new CompilationException(ErrorCode.COMPILATION_ILLEGAL_STATE,
+ "the vector index-only plan was chosen for a WHERE that cannot be pushed into the "
+ + "INCLUDE fields of index " + chosenIndex.getIndexName());
+ }
+ VectorIncludeFilterPushdown.declareFilterVariables(unnestMap, pushed);
+ // Rebind the SELECT to the INCLUDE columns instead of moving the predicate into the unnest-map's
+ // select condition. The record variable it used to read is about to be neutralized, so it cannot
+ // stay as it is -- but once it reads variables the unnest-map produces it is an ordinary predicate
+ // over ordinary variables, which every rule between here and the physical rewrites can handle.
+ // PushFilterIntoVectorSearchRule moves it into the select condition at the same point it does for
+ // the lookup-and-rerank plan.
+ selectOp.getCondition().setValue(pushed.condition());
+ }
+
/**
* Replace any ASSIGN expression whose subtree (transitively) mentions a dead variable with a MISSING
* constant. Seeded with the now-gone dataset record variable(s) (data record and, if present, meta
diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/util/ValidateUtil.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/util/ValidateUtil.java
index 593a1bf251..a5a1f70690 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/util/ValidateUtil.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/util/ValidateUtil.java
@@ -279,16 +279,19 @@ public class ValidateUtil {
case DAYTIMEDURATION:
break;
case ANY:
+ // ANY is allowed for BTREE and VTREE (include fields) indexes
if (indexType == IndexType.BTREE || indexType == IndexType.VTREE) {
- // ANY is allowed for BTREE and VECTOR (include fields) indexes
break;
}
+ throw illegalIndexFieldType(indexType, fieldType, displayFieldName, sourceLoc);
+ case BOOLEAN:
+ // for VTREE INCLUDE fields, the storage layer has a type trait and comparator for them
+ if (indexType == IndexType.VTREE) {
+ break;
+ }
+ throw illegalIndexFieldType(indexType, fieldType, displayFieldName, sourceLoc);
default:
- throw new CompilationException(ErrorCode.COMPILATION_ERROR, sourceLoc,
- "The field '"
- + LogRedactionUtil.userData(RecordUtil.toFullyQualifiedName(displayFieldName))
- + "' which is of type " + fieldType.getTypeTag()
- + " cannot be indexed using the BTree index.");
+ throw illegalIndexFieldType(indexType, fieldType, displayFieldName, sourceLoc);
}
break;
case RTREE:
@@ -301,55 +304,22 @@ public class ValidateUtil {
case GEOMETRY:
break;
default:
- throw new CompilationException(ErrorCode.COMPILATION_ERROR, sourceLoc,
- "The field '"
- + LogRedactionUtil.userData(RecordUtil.toFullyQualifiedName(displayFieldName))
- + "' which is of type " + fieldType.getTypeTag()
- + " cannot be indexed using the RTree index.");
+ throw illegalIndexFieldType(indexType, fieldType, displayFieldName, sourceLoc);
}
break;
- case LENGTH_PARTITIONED_NGRAM_INVIX:
+ case LENGTH_PARTITIONED_NGRAM_INVIX, SINGLE_PARTITION_NGRAM_INVIX:
if (fieldType.getTypeTag() != ATypeTag.STRING) {
- throw new CompilationException(ErrorCode.COMPILATION_ERROR, sourceLoc,
- "The field '" + LogRedactionUtil.userData(RecordUtil.toFullyQualifiedName(displayFieldName))
- + "' which is of type " + fieldType.getTypeTag()
- + " cannot be indexed using the Length Partitioned N-Gram index.");
+ throw illegalIndexFieldType(indexType, fieldType, displayFieldName, sourceLoc);
}
break;
- case LENGTH_PARTITIONED_WORD_INVIX:
+ case LENGTH_PARTITIONED_WORD_INVIX, SINGLE_PARTITION_WORD_INVIX:
switch (fieldType.getTypeTag()) {
case STRING:
case MULTISET:
case ARRAY:
break;
default:
- throw new CompilationException(ErrorCode.COMPILATION_ERROR, sourceLoc,
- "The field '"
- + LogRedactionUtil.userData(RecordUtil.toFullyQualifiedName(displayFieldName))
- + "' which is of type " + fieldType.getTypeTag()
- + " cannot be indexed using the Length Partitioned Keyword index.");
- }
- break;
- case SINGLE_PARTITION_NGRAM_INVIX:
- if (fieldType.getTypeTag() != ATypeTag.STRING) {
- throw new CompilationException(ErrorCode.COMPILATION_ERROR, sourceLoc,
- "The field '" + LogRedactionUtil.userData(RecordUtil.toFullyQualifiedName(displayFieldName))
- + "' which is of type " + fieldType.getTypeTag()
- + " cannot be indexed using the N-Gram index.");
- }
- break;
- case SINGLE_PARTITION_WORD_INVIX:
- switch (fieldType.getTypeTag()) {
- case STRING:
- case MULTISET:
- case ARRAY:
- break;
- default:
- throw new CompilationException(ErrorCode.COMPILATION_ERROR, sourceLoc,
- "The field '"
- + LogRedactionUtil.userData(RecordUtil.toFullyQualifiedName(displayFieldName))
- + "' which is of type " + fieldType.getTypeTag()
- + " cannot be indexed using the Keyword index.");
+ throw illegalIndexFieldType(indexType, fieldType, displayFieldName, sourceLoc);
}
break;
default:
@@ -358,6 +328,30 @@ public class ValidateUtil {
}
}
+ private static CompilationException illegalIndexFieldType(IndexType indexType, IAType fieldType,
+ List<String> displayFieldName, SourceLocation sourceLoc) {
+ String fieldName = LogRedactionUtil.userData(RecordUtil.toFullyQualifiedName(displayFieldName));
+ String rejection = indexType == IndexType.VTREE ? "used as an INCLUDE field of a vector index"
+ : "indexed using the " + indexTypeDisplayName(indexType) + " index";
+ return new CompilationException(ErrorCode.COMPILATION_ERROR, sourceLoc, "The field '" + fieldName
+ + "' which is of type " + fieldType.getTypeTag() + " cannot be " + rejection + ".");
+ }
+
+ /** The index type as it is written in user-facing messages. */
+ private static String indexTypeDisplayName(IndexType indexType) {
+ return switch (indexType) {
+ case BTREE -> "BTree";
+ case RTREE -> "RTree";
+ case ARRAY -> "Array";
+ case VTREE -> "Vector";
+ case LENGTH_PARTITIONED_NGRAM_INVIX -> "Length Partitioned N-Gram";
+ case LENGTH_PARTITIONED_WORD_INVIX -> "Length Partitioned Keyword";
+ case SINGLE_PARTITION_NGRAM_INVIX -> "N-Gram";
+ case SINGLE_PARTITION_WORD_INVIX -> "Keyword";
+ default -> String.valueOf(indexType);
+ };
+ }
+
/**
* Validates the key fields that will be used as either primary or foreign keys of a view.
*/
diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
index 98db7755ab..1d45930383 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
@@ -1735,6 +1735,15 @@ public class QueryTranslator extends AbstractLangTranslator implements IStatemen
fieldTypeMissable = projectTypeMissable;
} else {
// the type of the indexed field is explicitly specified in the DDL
+ if (indexType == IndexType.VTREE) {
+ // INCLUDE fields types comes from the schema (or is ANY for an undeclared field)
+ // so DDL should not declare any type
+ throw new CompilationException(ErrorCode.COMPILATION_ERROR,
+ indexedElement.getSourceLocation(),
+ "Cannot specify the type of the INCLUDE field '"
+ + LogRedactionUtil.userData(RecordUtil.toFullyQualifiedName(projectPath))
+ + "' of a vector index (TYPE VTREE)");
+ }
Map<TypeSignature, IAType> typeMap = TypeTranslator.computeTypes(databaseName, dataverseName,
indexName, projectTypeExpr.getType(), databaseName, dataverseName, mdTxnCtx);
TypeSignature typeSignature = new TypeSignature(databaseName, dataverseName, indexName);
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/VectorQueries.xml b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/VectorQueries.xml
index b2dbabd40a..37a7f27e2f 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/VectorQueries.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/VectorQueries.xml
@@ -47,6 +47,17 @@
<output-dir compare="Clean-JSON">create-index-vtree-include-filter</output-dir>
</compilation-unit>
</test-case>
+ <test-case FilePath="vector">
+ <compilation-unit name="create-index-vtree-include-typed">
+ <output-dir compare="Text">create-index-vtree-include-typed</output-dir>
+ <expected-error>Cannot specify the type of the INCLUDE field 'year' of a vector index (TYPE VTREE)</expected-error>
+ </compilation-unit>
+ </test-case>
+ <test-case FilePath="vector">
+ <compilation-unit name="create-index-vtree-include-filter-multi">
+ <output-dir compare="Clean-JSON">create-index-vtree-include-filter-multi</output-dir>
+ </compilation-unit>
+ </test-case>
<test-case FilePath="vector">
<compilation-unit name="create-index-vtree-include-nested">
<output-dir compare="Clean-JSON">create-index-vtree-include-nested</output-dir>
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.10.plans.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.10.plans.sqlpp
new file mode 100644
index 0000000000..83c7702204
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.10.plans.sqlpp
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+-- param compile-only:string=true
+-- param optimized-logical-plan:string=true
+
+// CBO cost estimates vary with the cluster shape (partition count / core count), which
+// differs between a local run and CI. Disable the CBO so the plan is stable everywhere.
+SET `compiler.cbo` "false";
+
+// The .9 query as a plan assertion: the vector unnest-map carries "condition (...)" AND there is no
+// BTREE_SEARCH above it. Contrast with .5, whose only difference is projecting the non-indexed `title`.
+SELECT VALUE [m.id1, m.id2]
+FROM testcpk.MovieCpk m
+WHERE m.year > 2000
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.11.plans.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.11.plans.sqlpp
new file mode 100644
index 0000000000..9038a903e3
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.11.plans.sqlpp
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+-- param compile-only:string=true
+-- param optimized-logical-plan:string=true
+
+// CBO cost estimates vary with the cluster shape (partition count / core count), which
+// differs between a local run and CI. Disable the CBO so the plan is stable everywhere.
+SET `compiler.cbo` "false";
+
+// The fourth corner of the matrix: lookup-and-rerank with NO filter. Projecting `title` keeps the
+// primary BTREE_SEARCH, and with no WHERE the vector unnest-map carries neither a condition nor any
+// INCLUDE variable -- it declares exactly the two PK columns the runtime writes. Together with .8, .10
+// and .5 this pins down how the unnest-map's declared outputs vary with the two independent choices,
+// index-only and filtered.
+SELECT VALUE [m.id1, m.id2, m.title]
+FROM testcpk.MovieCpk m
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.12.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.12.query.sqlpp
new file mode 100644
index 0000000000..1ba7f00a98
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.12.query.sqlpp
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE testcpk;
+
+// WHERE on the PRIMARY KEY only. This never reaches the vector index: IntroduceSelectAccessMethodRule
+// runs before IntroduceTopKAccessMethodRule and claims a PK predicate for a primary-index range search,
+// after which the distance is computed exactly over the matching rows. Correct, and usually the better
+// plan for a selective key predicate -- but it means a PK predicate takes the query off the ANN path
+// entirely, which is worth pinning down. id1 = 1 keeps the near-origin group; nearest 3 are (1,1), (1,2),
+// (1,3).
+SELECT VALUE [m.id1, m.id2]
+FROM MovieCpk m
+WHERE m.id1 = 1
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.13.plans.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.13.plans.sqlpp
new file mode 100644
index 0000000000..81e28fe017
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.13.plans.sqlpp
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+-- param compile-only:string=true
+-- param optimized-logical-plan:string=true
+SET `compiler.cbo` "false";
+
+// Plan for .12. Asserts the access path, which is the point of the test: a BTREE_SEARCH on the primary
+// index driven by id1 = 1, an exact euclidean-distance assign above it, and no vector index-search
+// anywhere.
+SELECT VALUE [m.id1, m.id2]
+FROM testcpk.MovieCpk m
+WHERE m.id1 = 1
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.14.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.14.query.sqlpp
new file mode 100644
index 0000000000..fb9ef7d174
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.14.query.sqlpp
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE testcpk;
+
+// WHERE spanning BOTH the primary key and an INCLUDE field. The PK conjunct wins the same way it does
+// in .12: id1 = 1 becomes a primary-index range search and `year > 2000` is left as a residual SELECT
+// above it, so the INCLUDE column is filtered from the materialized record rather than pushed into a
+// vector cursor. Mixing a PK column into the predicate therefore costs the ANN path even though the
+// other conjunct is exactly what INCLUDE is for. id1 = 1 keeps the near-origin group and year > 2000
+// keeps id2 11..15 within it; nearest 3 are (1,11), (1,12), (1,13).
+SELECT VALUE [m.id1, m.id2]
+FROM MovieCpk m
+WHERE m.id1 = 1 AND m.year > 2000
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.15.plans.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.15.plans.sqlpp
new file mode 100644
index 0000000000..f366bc1e4a
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.15.plans.sqlpp
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+-- param compile-only:string=true
+-- param optimized-logical-plan:string=true
+SET `compiler.cbo` "false";
+
+// Plan for .14: BTREE_SEARCH on the primary index for the PK conjunct, STREAM_SELECT above it for the
+// INCLUDE conjunct, exact distance, and no vector index-search. Contrast with .10, where the same
+// INCLUDE predicate alone reaches the index-only plan and is evaluated by the vector cursor.
+SELECT VALUE [m.id1, m.id2]
+FROM testcpk.MovieCpk m
+WHERE m.id1 = 1 AND m.year > 2000
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.7.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.7.query.sqlpp
new file mode 100644
index 0000000000..1e14c41671
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.7.query.sqlpp
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE testcpk;
+
+// Index-only over a COMPOSITE primary key, no filter. Projecting only the two PK columns keeps every
+// live-out variable PK-derived, so the plan skips the primary lookup and ranks on the distance the
+// cursor reports. Nearest 3 to [0,0,0,0] are (1,1), (1,2), (1,3) at distances 0.1, 0.2, 0.3.
+SELECT VALUE [m.id1, m.id2]
+FROM MovieCpk m
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.8.plans.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.8.plans.sqlpp
new file mode 100644
index 0000000000..08fe3a814a
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.8.plans.sqlpp
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+-- param compile-only:string=true
+-- param optimized-logical-plan:string=true
+
+// CBO cost estimates vary with the cluster shape (partition count / core count), which
+// differs between a local run and CI. Disable the CBO so the plan is stable everywhere.
+SET `compiler.cbo` "false";
+
+// The .7 query as a plan assertion: index-only over a composite PK emits [pk1, pk2, dist] and carries
+// no BTREE_SEARCH.
+SELECT VALUE [m.id1, m.id2]
+FROM testcpk.MovieCpk m
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.9.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.9.query.sqlpp
new file mode 100644
index 0000000000..241d149b22
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.9.query.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE testcpk;
+
+// Index-only over a composite PK WITH a pushed INCLUDE filter -- the shape the INCLUDE feature exists
+// for. Same filtered top-3 as .6, which reaches it by lookup-and-rerank because it also projects
+// `title`; here nothing but the PK columns is projected, so the primary lookup is skipped entirely.
+SELECT VALUE [m.id1, m.id2]
+FROM MovieCpk m
+WHERE m.year > 2000
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.1.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.1.ddl.sqlpp
new file mode 100644
index 0000000000..a71f501039
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.1.ddl.sqlpp
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+// A conjunction over several INCLUDE fields, the shape a real filtered ANN query has. One filter
+// variable per INCLUDE field is created, and each must be bound to its own physical column of the
+// secondary tuple -- a single-field test cannot catch two of them being swapped. `title` additionally
+// covers a string column, whose stored bytes compare differently from the integer ones, and
+// `verified` covers a boolean one -- a type that is not legal as a BTREE or ARRAY key but is fine as a
+// payload column that is only ever read to evaluate a pushed predicate.
+DROP DATAVERSE test IF EXISTS;
+CREATE DATAVERSE test;
+USE test;
+CREATE TYPE MovieType AS {
+ id: int,
+ year: int,
+ rating: int,
+ title: string,
+ verified: boolean,
+ embedding: [double]
+};
+CREATE DATASET MovieMulti(MovieType) PRIMARY KEY id;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.10.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.10.query.sqlpp
new file mode 100644
index 0000000000..0a54e6aa38
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.10.query.sqlpp
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE test;
+// A record constructor inside the pushed predicate. It is not a shape anyone writes on purpose, but it
+// is reachable, and it is the case that proves a pushed condition is NOT limited to comparisons over
+// storage columns: SetClosedRecordConstructorsRule has real work to do inside one, so the rule has to be
+// able to type it (against the unnest-map's OUTPUT environment, where the INCLUDE columns live) rather
+// than pass over it. Only id 6 has year 2006.
+SELECT VALUE m.id
+FROM MovieMulti m
+WHERE {"y": m.year} = {"y": 2006}
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.11.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.11.query.sqlpp
new file mode 100644
index 0000000000..6fd759c24d
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.11.query.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE test;
+// if_null() in the predicate: a function InjectTypeCastForFunctionArgumentsRule rewrites, typing it
+// against the operator's INPUT environment. Harmless while the predicate is still a SELECT over the
+// unnest-map's output variables; it is the shape that broke when the predicate was moved into the
+// unnest-map's select condition during the access-method phase.
+SELECT VALUE m.id
+FROM MovieMulti m
+WHERE if_null(m.year, 0) > 2005 AND m.rating = 5
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.12.plans.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.12.plans.sqlpp
new file mode 100644
index 0000000000..bd44dda06d
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.12.plans.sqlpp
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+-- param compile-only:string=true
+-- param optimized-logical-plan:string=true
+SET `compiler.cbo` "false";
+
+// A correlated subquery over a non-PK field puts a SUBPLAN between the ORDER and the data source. The
+// variables it reads are invisible to collectSelectConditionVars/collectAssignBindings, which walk only
+// getInputs() and not nested plans -- but that is unreachable rather than unhandled: the subtree pattern
+// is SELECT* (ASSIGN|UNNEST)* DATASOURCE, so a SUBPLAN there stops initFromSubTree and no vector index is
+// chosen at all. Asserted here so that widening the subtree pattern later cannot silently start feeding
+// nested-plan variables to a scan that never emits them.
+SELECT m.id, (SELECT VALUE t.v FROM [{"v": m.rating}] t) AS sub
+FROM test.MovieMulti m
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.13.plans.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.13.plans.sqlpp
new file mode 100644
index 0000000000..ba894ee638
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.13.plans.sqlpp
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+-- param compile-only:string=true
+-- param optimized-logical-plan:string=true
+SET `compiler.cbo` "false";
+
+// The .10 query as a plan assertion. The .10 result is the same whichever plan is chosen, so only this
+// pins the claim the record constructor is there to make: a pushed condition may contain an expression
+// a later rule has to type, and the index-only plan still carries it.
+SELECT VALUE m.id
+FROM test.MovieMulti m
+WHERE {"y": m.year} = {"y": 2006}
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.14.plans.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.14.plans.sqlpp
new file mode 100644
index 0000000000..9df21b96a7
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.14.plans.sqlpp
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+-- param compile-only:string=true
+-- param optimized-logical-plan:string=true
+SET `compiler.cbo` "false";
+
+// The .11 query as a plan assertion. Same reason as .13: if_null() in the predicate is the shape that
+// broke when the select condition was created during the access-method phase, and a regression that
+// dropped it back to lookup-and-rerank would leave the .11 result untouched.
+SELECT VALUE m.id
+FROM test.MovieMulti m
+WHERE if_null(m.year, 0) > 2005 AND m.rating = 5
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.2.update.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.2.update.sqlpp
new file mode 100644
index 0000000000..5392573186
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.2.update.sqlpp
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE test;
+// year = 2000 + id; rating is 5 for even ids and 1 for odd ids; verified is true when id is a
+// multiple of 3. So WHERE year > 2005 AND rating = 5 keeps the even ids 6..30, whose 3 nearest to
+// [0,0,0,0] are 6, 8, 10.
+INSERT INTO MovieMulti ([
+ {"id": 1, "year": 2001, "rating": 1, "title": "t1", "verified": false, "embedding": [0.0, 0.0, 0.0, 0.1]},
+ {"id": 2, "year": 2002, "rating": 5, "title": "t2", "verified": false, "embedding": [0.0, 0.0, 0.0, 0.2]},
+ {"id": 3, "year": 2003, "rating": 1, "title": "t3", "verified": true, "embedding": [0.0, 0.0, 0.0, 0.3]},
+ {"id": 4, "year": 2004, "rating": 5, "title": "t4", "verified": false, "embedding": [0.0, 0.0, 0.0, 0.4]},
+ {"id": 5, "year": 2005, "rating": 1, "title": "t5", "verified": false, "embedding": [0.0, 0.0, 0.0, 0.5]},
+ {"id": 6, "year": 2006, "rating": 5, "title": "t6", "verified": true, "embedding": [0.0, 0.0, 0.0, 0.6]},
+ {"id": 7, "year": 2007, "rating": 1, "title": "t7", "verified": false, "embedding": [0.0, 0.0, 0.0, 0.7]},
+ {"id": 8, "year": 2008, "rating": 5, "title": "t8", "verified": false, "embedding": [0.0, 0.0, 0.0, 0.8]},
+ {"id": 9, "year": 2009, "rating": 1, "title": "t9", "verified": true, "embedding": [0.0, 0.0, 0.0, 0.9]},
+ {"id": 10, "year": 2010, "rating": 5, "title": "t10", "verified": false, "embedding": [0.0, 0.0, 0.0, 1.0]},
+ {"id": 11, "year": 2011, "rating": 1, "title": "t11", "verified": false, "embedding": [0.0, 0.0, 0.0, 1.1]},
+ {"id": 12, "year": 2012, "rating": 5, "title": "t12", "verified": true, "embedding": [0.0, 0.0, 0.0, 1.2]},
+ {"id": 13, "year": 2013, "rating": 1, "title": "t13", "verified": false, "embedding": [0.0, 0.0, 0.0, 1.3]},
+ {"id": 14, "year": 2014, "rating": 5, "title": "t14", "verified": false, "embedding": [0.0, 0.0, 0.0, 1.4]},
+ {"id": 15, "year": 2015, "rating": 1, "title": "t15", "verified": true, "embedding": [0.0, 0.0, 0.0, 1.5]},
+ {"id": 16, "year": 2016, "rating": 5, "title": "t16", "verified": false, "embedding": [10.0, 10.0, 10.0, 0.1]},
+ {"id": 17, "year": 2017, "rating": 1, "title": "t17", "verified": false, "embedding": [10.0, 10.0, 10.0, 0.2]},
+ {"id": 18, "year": 2018, "rating": 5, "title": "t18", "verified": true, "embedding": [10.0, 10.0, 10.0, 0.3]},
+ {"id": 19, "year": 2019, "rating": 1, "title": "t19", "verified": false, "embedding": [10.0, 10.0, 10.0, 0.4]},
+ {"id": 20, "year": 2020, "rating": 5, "title": "t20", "verified": false, "embedding": [10.0, 10.0, 10.0, 0.5]},
+ {"id": 21, "year": 2021, "rating": 1, "title": "t21", "verified": true, "embedding": [10.0, 10.0, 10.0, 0.6]},
+ {"id": 22, "year": 2022, "rating": 5, "title": "t22", "verified": false, "embedding": [10.0, 10.0, 10.0, 0.7]},
+ {"id": 23, "year": 2023, "rating": 1, "title": "t23", "verified": false, "embedding": [10.0, 10.0, 10.0, 0.8]},
+ {"id": 24, "year": 2024, "rating": 5, "title": "t24", "verified": true, "embedding": [10.0, 10.0, 10.0, 0.9]},
+ {"id": 25, "year": 2025, "rating": 1, "title": "t25", "verified": false, "embedding": [10.0, 10.0, 10.0, 1.0]},
+ {"id": 26, "year": 2026, "rating": 5, "title": "t26", "verified": false, "embedding": [10.0, 10.0, 10.0, 1.1]},
+ {"id": 27, "year": 2027, "rating": 1, "title": "t27", "verified": true, "embedding": [10.0, 10.0, 10.0, 1.2]},
+ {"id": 28, "year": 2028, "rating": 5, "title": "t28", "verified": false, "embedding": [10.0, 10.0, 10.0, 1.3]},
+ {"id": 29, "year": 2029, "rating": 1, "title": "t29", "verified": false, "embedding": [10.0, 10.0, 10.0, 1.4]},
+ {"id": 30, "year": 2030, "rating": 5, "title": "t30", "verified": true, "embedding": [10.0, 10.0, 10.0, 1.5]}
+]);
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.3.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.3.ddl.sqlpp
new file mode 100644
index 0000000000..9e0d4e9994
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.3.ddl.sqlpp
@@ -0,0 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+// A sample index is required before creating a vector index (quantization training scans it).
+ANALYZE DATASET test.MovieMulti WITH {"sample-seed": 1000};
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.4.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.4.ddl.sqlpp
new file mode 100644
index 0000000000..76214d4dc8
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.4.ddl.sqlpp
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE test;
+CREATE INDEX idx_emb
+ ON MovieMulti(embedding VECTOR)
+ INCLUDE (year, rating, title, verified)
+ TYPE VTREE
+ WITH { "dimension": 4, "similarity": "euclidean", "num_clusters": 2, "train_list_fraction": 1.0, "seed": 42 };
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.5.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.5.query.sqlpp
new file mode 100644
index 0000000000..36c1db6567
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.5.query.sqlpp
@@ -0,0 +1,26 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE test;
+// min_probe_fraction 1.0 scans every cluster, so the expected result is clustering-independent.
+SELECT VALUE m.id
+FROM MovieMulti m
+WHERE m.year > 2005 AND m.rating = 5
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.6.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.6.query.sqlpp
new file mode 100644
index 0000000000..49939df9fc
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.6.query.sqlpp
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE test;
+// Flipping only the second predicate must flip the answer to the odd ids 7, 9, 11. A pushdown that
+// bound the two filter variables to each other's physical column would still satisfy .5 by luck; it
+// cannot satisfy both .5 and this.
+SELECT VALUE m.id
+FROM MovieMulti m
+WHERE m.year > 2005 AND m.rating = 1
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.7.plans.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.7.plans.sqlpp
new file mode 100644
index 0000000000..df498c987d
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.7.plans.sqlpp
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+-- param compile-only:string=true
+-- param optimized-logical-plan:string=true
+
+// CBO cost estimates vary with the cluster shape (partition count / core count), which
+// differs between a local run and CI. Disable the CBO so the plan is stable everywhere.
+SET `compiler.cbo` "false";
+
+// The .5 query as a plan assertion: a two-field conjunction over INCLUDE columns must still reach the
+// index-only plan -- vector unnest-map carrying "condition (...)", no BTREE_SEARCH above it.
+SELECT VALUE m.id
+FROM test.MovieMulti m
+WHERE m.year > 2005 AND m.rating = 5
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.8.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.8.query.sqlpp
new file mode 100644
index 0000000000..81c561c3f3
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.8.query.sqlpp
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE test;
+// A STRING INCLUDE column: the pushed predicate is evaluated by the vector cursor over the stored
+// column bytes, so a string comparison exercises a different comparator from the integer ones above.
+// title = "t<id>", so this matches exactly one row and LIMIT 3 returns it alone.
+SELECT VALUE m.id
+FROM MovieMulti m
+WHERE m.title = "t8"
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.9.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.9.query.sqlpp
new file mode 100644
index 0000000000..290c6e6660
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.9.query.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE test;
+// A BOOLEAN INCLUDE column, the shape the filtered-ANN perf workload uses (verified_purchase = true
+// AND rating = 5). Boolean is not a legal BTREE or ARRAY key, but an INCLUDE column is not a key.
+// verified is true when id is a multiple of 3, so this keeps 6, 12, 18, 24, 30 -- nearest 3 are
+// 6, 12, 18.
+SELECT VALUE m.id
+FROM MovieMulti m
+WHERE m.verified = true AND m.rating = 5
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.99.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.99.ddl.sqlpp
new file mode 100644
index 0000000000..20dc6fde5e
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.99.ddl.sqlpp
@@ -0,0 +1,20 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+DROP DATAVERSE test IF EXISTS;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter/create-index-vtree-include-filter.7.plans.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter/create-index-vtree-include-filter.7.plans.sqlpp
new file mode 100644
index 0000000000..e8111b02b7
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter/create-index-vtree-include-filter.7.plans.sqlpp
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+-- param compile-only:string=true
+-- param optimized-logical-plan:string=true
+
+// CBO cost estimates vary with the cluster shape (partition count / core count), which
+// differs between a local run and CI. Disable the CBO so the plan is stable everywhere.
+SET `compiler.cbo` "false";
+
+// The .5 query again, as a plan assertion. A PK-only projection whose WHERE reads nothing but an
+// INCLUDE field must take the INDEX-ONLY plan: the vector unnest-map carries "condition (...)" AND
+// there is no BTREE_SEARCH above it, because the predicate is evaluated by the vector cursor and the
+// distance comes off the cursor rather than from a refetched record. Before this, any WHERE on an
+// INCLUDE field forced lookup-and-rerank, which fetches and reranks every candidate.
+USE test;
+
+SELECT VALUE m.id
+FROM MovieSmall m
+WHERE m.year > 2005
+ORDER BY ann_distance(m.embedding, [0.0, 0.0, 0.0, 0.0], "euclidean", 1.0)
+LIMIT 3;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.1.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.1.ddl.sqlpp
new file mode 100644
index 0000000000..7167c7f25c
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.1.ddl.sqlpp
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+// An INCLUDE field is a payload column, never a key: its type comes from the record schema, or is ANY
+// when the field is undeclared. The DDL therefore has no business declaring one, and saying so must be
+// rejected with a message that names the INCLUDE field -- not with the generic "non-enforced typed
+// index" error that the shared key-validation path used to produce.
+DROP DATAVERSE test IF EXISTS;
+CREATE DATAVERSE test;
+USE test;
+CREATE TYPE MovieType AS OPEN {
+ id: int
+};
+CREATE DATASET MovieOpen(MovieType) PRIMARY KEY id;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.2.update.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.2.update.sqlpp
new file mode 100644
index 0000000000..b9d05b48f4
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.2.update.sqlpp
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE test;
+INSERT INTO MovieOpen ([
+ {"id": 1, "year": 2001, "embedding": [0.0, 0.0, 0.0, 0.1]},
+ {"id": 2, "year": 2002, "embedding": [0.0, 0.0, 0.0, 0.2]},
+ {"id": 3, "year": 2003, "embedding": [0.0, 0.0, 0.0, 0.3]}
+]);
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.3.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.3.ddl.sqlpp
new file mode 100644
index 0000000000..0f7733d3e0
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.3.ddl.sqlpp
@@ -0,0 +1,20 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+ANALYZE DATASET test.MovieOpen WITH {"sample-seed": 1000};
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.4.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.4.ddl.sqlpp
new file mode 100644
index 0000000000..bd53765921
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.4.ddl.sqlpp
@@ -0,0 +1,27 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+USE test;
+// 'year' is undeclared, which is exactly the case where a user reaches for a type annotation. It is
+// still not accepted: the INCLUDE field resolves to ANY from the schema on its own.
+CREATE INDEX idx_emb
+ ON MovieOpen(embedding VECTOR)
+ INCLUDE (year: int)
+ TYPE VTREE
+ WITH { "dimension": 4, "similarity": "euclidean", "num_clusters": 2, "train_list_fraction": 1.0, "seed": 42 };
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.99.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.99.ddl.sqlpp
new file mode 100644
index 0000000000..20dc6fde5e
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.99.ddl.sqlpp
@@ -0,0 +1,20 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+DROP DATAVERSE test IF EXISTS;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.10.plan b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.10.plan
new file mode 100644
index 0000000000..792ed52311
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.10.plan
@@ -0,0 +1,32 @@
+distribute result [$$20]
+-- DISTRIBUTE_RESULT |UNPARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |UNPARTITIONED|
+ project ([$$20])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- SORT_MERGE_EXCHANGE [$$38(ASC) ] |PARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ order (topK: 3) (ASC, $$38)
+ -- STABLE_SORT [topK: 3] [$$38(ASC)] |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$20] <- [ordered-list-constructor($$36, $$37)] project: [$$20, $$38]
+ -- ASSIGN |PARTITIONED|
+ project ([$$36, $$37, $$38])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ unnest-map [$$36, $$37, $$38, $$39] <- index-search("idx_cpk_emb", 8, "Default", "testcpk", "MovieCpk", false, false, true, 5, $$31, $$32, $$33, $$34, $$35) condition (gt($$39, 2000))
+ -- VECTOR_SEARCH |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$31, $$32, $$33, $$34, $$35] <- [array: [ 0.0, 0.0, 0.0, 0.0 ], 3, "euclidean", 1.0, 1]
+ -- ASSIGN |PARTITIONED|
+ empty-tuple-source
+ -- EMPTY_TUPLE_SOURCE |PARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.11.plan b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.11.plan
new file mode 100644
index 0000000000..c8a635dbda
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.11.plan
@@ -0,0 +1,38 @@
+distribute result [$$19]
+-- DISTRIBUTE_RESULT |UNPARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |UNPARTITIONED|
+ project ([$$19])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- SORT_MERGE_EXCHANGE [$$23(ASC) ] |PARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ order (topK: 3) (ASC, $$23)
+ -- STABLE_SORT [topK: 3] [$$23(ASC)] |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$23, $$19] <- [euclidean-distance($$m.getField(4), array: [ 0.0, 0.0, 0.0, 0.0 ]), ordered-list-constructor($$24, $$25, $$m.getField(3))] project: [$$19, $$23]
+ -- ASSIGN |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ unnest-map [$$24, $$25, $$m] <- index-search("MovieCpk", 0, "Default", "testcpk", "MovieCpk", false, false, 2, $$35, $$36, 2, $$35, $$36, true, true, true)
+ -- BTREE_SEARCH |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ order (ASC, $$35) (ASC, $$36)
+ -- STABLE_SORT [$$35(ASC), $$36(ASC)] |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ unnest-map [$$35, $$36] <- index-search("idx_cpk_emb", 8, "Default", "testcpk", "MovieCpk", false, false, false, 5, $$30, $$31, $$32, $$33, $$34)
+ -- VECTOR_SEARCH |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$30, $$31, $$32, $$33, $$34] <- [array: [ 0.0, 0.0, 0.0, 0.0 ], 3, "euclidean", 1.0, 1]
+ -- ASSIGN |PARTITIONED|
+ empty-tuple-source
+ -- EMPTY_TUPLE_SOURCE |PARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.12.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.12.adm
new file mode 100644
index 0000000000..5afbf246d8
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.12.adm
@@ -0,0 +1,3 @@
+[ 1, 1 ]
+[ 1, 2 ]
+[ 1, 3 ]
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.13.plan b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.13.plan
new file mode 100644
index 0000000000..5012387e63
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.13.plan
@@ -0,0 +1,30 @@
+distribute result [$$20]
+-- DISTRIBUTE_RESULT |UNPARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |UNPARTITIONED|
+ project ([$$20])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- SORT_MERGE_EXCHANGE [$$24(ASC) ] |PARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ order (topK: 3) (ASC, $$24)
+ -- STABLE_SORT [topK: 3] [$$24(ASC)] |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$24, $$20] <- [euclidean-distance($$m.getField(4), array: [ 0.0, 0.0, 0.0, 0.0 ]), ordered-list-constructor($$26, $$27)] project: [$$20, $$24]
+ -- ASSIGN |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ unnest-map [$$26, $$27, $$m] <- index-search("MovieCpk", 0, "Default", "testcpk", "MovieCpk", false, false, 1, $$30, 1, $$31, true, true, true)
+ -- BTREE_SEARCH |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$30, $$31] <- [1, 1]
+ -- ASSIGN |PARTITIONED|
+ empty-tuple-source
+ -- EMPTY_TUPLE_SOURCE |PARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.14.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.14.adm
new file mode 100644
index 0000000000..b7f48b1d77
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.14.adm
@@ -0,0 +1,3 @@
+[ 1, 11 ]
+[ 1, 12 ]
+[ 1, 13 ]
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.15.plan b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.15.plan
new file mode 100644
index 0000000000..ffde4b90b6
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.15.plan
@@ -0,0 +1,32 @@
+distribute result [$$23]
+-- DISTRIBUTE_RESULT |UNPARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |UNPARTITIONED|
+ project ([$$23])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- SORT_MERGE_EXCHANGE [$$27(ASC) ] |PARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ order (topK: 3) (ASC, $$27)
+ -- STABLE_SORT [topK: 3] [$$27(ASC)] |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$27, $$23] <- [euclidean-distance($$m.getField(4), array: [ 0.0, 0.0, 0.0, 0.0 ]), ordered-list-constructor($$29, $$30)] project: [$$23, $$27]
+ -- ASSIGN |PARTITIONED|
+ select (gt($$m.getField(2), 2000))
+ -- STREAM_SELECT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ unnest-map [$$29, $$30, $$m] <- index-search("MovieCpk", 0, "Default", "testcpk", "MovieCpk", false, false, 1, $$34, 1, $$35, true, true, true)
+ -- BTREE_SEARCH |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$34, $$35] <- [1, 1]
+ -- ASSIGN |PARTITIONED|
+ empty-tuple-source
+ -- EMPTY_TUPLE_SOURCE |PARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.7.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.7.adm
new file mode 100644
index 0000000000..5afbf246d8
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.7.adm
@@ -0,0 +1,3 @@
+[ 1, 1 ]
+[ 1, 2 ]
+[ 1, 3 ]
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.8.plan b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.8.plan
new file mode 100644
index 0000000000..fb08757c27
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.8.plan
@@ -0,0 +1,30 @@
+distribute result [$$18]
+-- DISTRIBUTE_RESULT |UNPARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |UNPARTITIONED|
+ project ([$$18])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- SORT_MERGE_EXCHANGE [$$35(ASC) ] |PARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ order (topK: 3) (ASC, $$35)
+ -- STABLE_SORT [topK: 3] [$$35(ASC)] |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$18] <- [ordered-list-constructor($$33, $$34)] project: [$$18, $$35]
+ -- ASSIGN |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ unnest-map [$$33, $$34, $$35] <- index-search("idx_cpk_emb", 8, "Default", "testcpk", "MovieCpk", false, false, true, 5, $$28, $$29, $$30, $$31, $$32)
+ -- VECTOR_SEARCH |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$28, $$29, $$30, $$31, $$32] <- [array: [ 0.0, 0.0, 0.0, 0.0 ], 3, "euclidean", 1.0, 1]
+ -- ASSIGN |PARTITIONED|
+ empty-tuple-source
+ -- EMPTY_TUPLE_SOURCE |PARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.9.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.9.adm
new file mode 100644
index 0000000000..b7f48b1d77
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.9.adm
@@ -0,0 +1,3 @@
+[ 1, 11 ]
+[ 1, 12 ]
+[ 1, 13 ]
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.10.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.10.adm
new file mode 100644
index 0000000000..1e8b314962
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.10.adm
@@ -0,0 +1 @@
+6
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.11.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.11.adm
new file mode 100644
index 0000000000..2798b58c58
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.11.adm
@@ -0,0 +1,3 @@
+6
+8
+10
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.12.plan b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.12.plan
new file mode 100644
index 0000000000..1030ab2f58
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.12.plan
@@ -0,0 +1,45 @@
+distribute result [$$37]
+-- DISTRIBUTE_RESULT |UNPARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |UNPARTITIONED|
+ project ([$$37])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- SORT_MERGE_EXCHANGE [$$41(ASC) ] |PARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ order (topK: 3) (ASC, $$41)
+ -- STABLE_SORT [topK: 3] [$$41(ASC)] |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$41, $$37] <- [euclidean-distance($$45, array: [ 0.0, 0.0, 0.0, 0.0 ]), {"id": $$42, "sub": $$36}] project: [$$37, $$41]
+ -- ASSIGN |PARTITIONED|
+ project ([$$45, $$42, $$36])
+ -- STREAM_PROJECT |PARTITIONED|
+ subplan {
+ aggregate [$$36] <- [listify($$35)] [cardinality: 0.0, doc-size: 0.0, op-cost: 0.0, total-cost: 0.0]
+ -- AGGREGATE |LOCAL|
+ assign [$$35] <- [$$t.getField(0)] [cardinality: 0.0, doc-size: 0.0, op-cost: 0.0, total-cost: 0.0]
+ -- ASSIGN |LOCAL|
+ unnest $$t <- scan-collection(ordered-list-constructor({"v": $$43})) project: [$$t] [cardinality: 0.0, doc-size: 0.0, op-cost: 0.0, total-cost: 0.0]
+ -- UNNEST |LOCAL|
+ project ([$$43]) [cardinality: 0.0, doc-size: 0.0, op-cost: 0.0, total-cost: 0.0]
+ -- STREAM_PROJECT |LOCAL|
+ nested tuple source [cardinality: 0.0, doc-size: 0.0, op-cost: 0.0, total-cost: 0.0]
+ -- NESTED_TUPLE_SOURCE |LOCAL|
+ }
+ -- SUBPLAN |PARTITIONED|
+ assign [$$45, $$43] <- [$$m.getField(5), $$m.getField(2)] project: [$$45, $$42, $$43]
+ -- ASSIGN |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ data-scan []<-[$$42, $$m] <- test.MovieMulti
+ -- DATASOURCE_SCAN |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ empty-tuple-source
+ -- EMPTY_TUPLE_SOURCE |PARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.13.plan b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.13.plan
new file mode 100644
index 0000000000..92081a3c99
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.13.plan
@@ -0,0 +1,30 @@
+distribute result [$$33]
+-- DISTRIBUTE_RESULT |UNPARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |UNPARTITIONED|
+ project ([$$33])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- SORT_MERGE_EXCHANGE [$$34(ASC) ] |PARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ order (topK: 3) (ASC, $$34)
+ -- STABLE_SORT [topK: 3] [$$34(ASC)] |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ project ([$$33, $$34])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ unnest-map [$$33, $$34, $$35] <- index-search("idx_emb", 8, "Default", "test", "MovieMulti", false, false, true, 5, $$28, $$29, $$30, $$31, $$32) condition (eq({"y": $$35}, { "y": 2006 }))
+ -- VECTOR_SEARCH |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$28, $$29, $$30, $$31, $$32] <- [array: [ 0.0, 0.0, 0.0, 0.0 ], 3, "euclidean", 1.0, 1]
+ -- ASSIGN |PARTITIONED|
+ empty-tuple-source
+ -- EMPTY_TUPLE_SOURCE |PARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.14.plan b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.14.plan
new file mode 100644
index 0000000000..241fd98442
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.14.plan
@@ -0,0 +1,30 @@
+distribute result [$$36]
+-- DISTRIBUTE_RESULT |UNPARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |UNPARTITIONED|
+ project ([$$36])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- SORT_MERGE_EXCHANGE [$$37(ASC) ] |PARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ order (topK: 3) (ASC, $$37)
+ -- STABLE_SORT [topK: 3] [$$37(ASC)] |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ project ([$$36, $$37])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ unnest-map [$$36, $$37, $$38, $$39] <- index-search("idx_emb", 8, "Default", "test", "MovieMulti", false, false, true, 5, $$31, $$32, $$33, $$34, $$35) condition (and(gt(if-null($$38, 0), 2005), eq($$39, 5)))
+ -- VECTOR_SEARCH |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$31, $$32, $$33, $$34, $$35] <- [array: [ 0.0, 0.0, 0.0, 0.0 ], 3, "euclidean", 1.0, 1]
+ -- ASSIGN |PARTITIONED|
+ empty-tuple-source
+ -- EMPTY_TUPLE_SOURCE |PARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.5.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.5.adm
new file mode 100644
index 0000000000..2798b58c58
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.5.adm
@@ -0,0 +1,3 @@
+6
+8
+10
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.6.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.6.adm
new file mode 100644
index 0000000000..a49d5eb13a
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.6.adm
@@ -0,0 +1,3 @@
+7
+9
+11
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.7.plan b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.7.plan
new file mode 100644
index 0000000000..cb3862b9e0
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.7.plan
@@ -0,0 +1,30 @@
+distribute result [$$35]
+-- DISTRIBUTE_RESULT |UNPARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |UNPARTITIONED|
+ project ([$$35])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- SORT_MERGE_EXCHANGE [$$36(ASC) ] |PARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ order (topK: 3) (ASC, $$36)
+ -- STABLE_SORT [topK: 3] [$$36(ASC)] |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ project ([$$35, $$36])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ unnest-map [$$35, $$36, $$37, $$38] <- index-search("idx_emb", 8, "Default", "test", "MovieMulti", false, false, true, 5, $$30, $$31, $$32, $$33, $$34) condition (and(gt($$37, 2005), eq($$38, 5)))
+ -- VECTOR_SEARCH |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$30, $$31, $$32, $$33, $$34] <- [array: [ 0.0, 0.0, 0.0, 0.0 ], 3, "euclidean", 1.0, 1]
+ -- ASSIGN |PARTITIONED|
+ empty-tuple-source
+ -- EMPTY_TUPLE_SOURCE |PARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.8.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.8.adm
new file mode 100644
index 0000000000..45a4fb75db
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.8.adm
@@ -0,0 +1 @@
+8
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.9.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.9.adm
new file mode 100644
index 0000000000..3f56dbb3a1
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.9.adm
@@ -0,0 +1,3 @@
+6
+12
+18
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter/create-index-vtree-include-filter.7.plan b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter/create-index-vtree-include-filter.7.plan
new file mode 100644
index 0000000000..09ac41d52e
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter/create-index-vtree-include-filter.7.plan
@@ -0,0 +1,30 @@
+distribute result [$$31]
+-- DISTRIBUTE_RESULT |UNPARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |UNPARTITIONED|
+ project ([$$31])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- SORT_MERGE_EXCHANGE [$$32(ASC) ] |PARTITIONED|
+ limit 3
+ -- STREAM_LIMIT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ order (topK: 3) (ASC, $$32)
+ -- STABLE_SORT [topK: 3] [$$32(ASC)] |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ project ([$$31, $$32])
+ -- STREAM_PROJECT |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ unnest-map [$$31, $$32, $$33] <- index-search("idx_emb", 8, "Default", "test", "MovieSmall", false, false, true, 5, $$26, $$27, $$28, $$29, $$30) condition (gt($$33, 2005))
+ -- VECTOR_SEARCH |PARTITIONED|
+ exchange
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ assign [$$26, $$27, $$28, $$29, $$30] <- [array: [ 0.0, 0.0, 0.0, 0.0 ], 3, "euclidean", 1.0, 1]
+ -- ASSIGN |PARTITIONED|
+ empty-tuple-source
+ -- EMPTY_TUPLE_SOURCE |PARTITIONED|