[XL] Change in asterixdb[master]: [ASTERIXDB-3864][ASTERIXDB-3865][COMP] Serve filtered ANN queries fro...

[AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>]

From Ali Alsuliman <al...@gmail.com>:

Attention is currently required from: Ali Alsuliman.

Hello Anon. E. Moose #1000171, Jenkins, 

I'd like you to reexamine a change. Please visit

    https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21814?usp=email

to look at the new patch set (#2).

The following approvals got outdated and were removed:
Contrib-2 by Anon. E. Moose #1000171, Integration-Tests+1 by Jenkins


Change subject: [ASTERIXDB-3864][ASTERIXDB-3865][COMP] Serve filtered ANN queries from the vector index
......................................................................

[ASTERIXDB-3864][ASTERIXDB-3865][COMP] Serve filtered ANN queries from the vector index

A WHERE over a vector index's INCLUDE columns disabled the index-only
ANN plan, so such a query fetched and reranked every candidate from
the primary index -- the cost INCLUDE exists to avoid. The gate asked
whether the predicate was primary-key derived, not whether the chosen
index covered it.

The index-only branch now rebinds the predicate onto the index's
INCLUDE columns and leaves it in its SELECT;
PushFilterIntoVectorSearchRule moves it into the vector search during
the physical rewrites, as it already does for the lookup-and-rerank
plan. Creating the select condition any earlier exposes it to rules
that type an operator's expressions against its input environment,
which an expression reading the operator's own outputs breaks.

Also repairs three CREATE INDEX validation defects met along the way:
a boolean INCLUDE column rejected by a rule meant for BTree keys, a
rejection message that named BTree whatever the index type was, and a
DDL-declared type on an INCLUDE field failing with an unrelated error.

Ext-ref: MB-74038

Co-Authored-By: Claude Opus 5 <no...@anthropic.com>
Change-Id: Iaca95fb11f3d629c56dfa1d73a497e981d69ea09
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/VectorSearchPOperator.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFilterIntoVectorSearchRule.java
A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/VectorIncludeFilterPushdown.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceTopKAccessMethodRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/VectorIndexAccessMethod.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/util/ValidateUtil.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/VectorQueries.xml
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.10.plans.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.11.plans.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.12.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.13.plans.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.14.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.15.plans.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.7.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.8.plans.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.9.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.10.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.11.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.12.plans.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.4.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.5.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.6.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.7.plans.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.8.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.9.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.99.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-filter/create-index-vtree-include-filter.7.plans.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.3.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.4.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/vector/create-index-vtree-include-typed/create-index-vtree-include-typed.99.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.10.plan
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.11.plan
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.12.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.13.plan
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.14.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.15.plan
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.7.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.8.plan
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-composite-pk/create-index-vtree-composite-pk.9.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.10.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.11.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.12.plan
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.5.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.6.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.7.plan
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.8.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter-multi/create-index-vtree-include-filter-multi.9.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/vector/create-index-vtree-include-filter/create-index-vtree-include-filter.7.plan
54 files changed, 1,858 insertions(+), 385 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/14/21814/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21814?usp=email
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email

Gerrit-MessageType: newpatchset
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Iaca95fb11f3d629c56dfa1d73a497e981d69ea09
Gerrit-Change-Number: 21814
Gerrit-PatchSet: 2
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Attention: Ali Alsuliman <al...@gmail.com>