You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by GitBox <gi...@apache.org> on 2022/06/17 17:14:41 UTC
[GitHub] [solr] cpoerschke commented on a diff in pull request #904: SOLR-16246: Introduced pre-filtering in KnnQParser
cpoerschke commented on code in PR #904:
URL: https://github.com/apache/solr/pull/904#discussion_r900359934
##########
solr/core/src/java/org/apache/solr/search/neural/KnnQParser.java:
##########
@@ -72,7 +81,49 @@ public Query parse() {
DenseVectorField denseVectorType = (DenseVectorField) fieldType;
float[] parsedVectorToSearch = parseVector(vectorToSearch, denseVectorType.getDimension());
- return denseVectorType.getKnnVectorQuery(schemaField.getName(), parsedVectorToSearch, topK);
+
+ return getFilterQuery()
+ .map(
+ filterQuery ->
+ denseVectorType.getKnnVectorQuery(
+ schemaField.getName(), parsedVectorToSearch, topK, filterQuery))
+ .orElse(
+ denseVectorType.getKnnVectorQuery(schemaField.getName(), parsedVectorToSearch, topK));
+ }
+
+ private Optional<? extends Query> getFilterQuery() throws SolrException {
+ if (!isFilter()) {
+ String[] filterQueries = req.getParams().getParams(CommonParams.FQ);
+ return ofNullable(filterQueries).filter(fq -> fq.length != 0).map(this::computeFilterQuery);
+ }
+ return Optional.empty();
+ }
+
+ private Query computeFilterQuery(String[] filterQueries) throws SolrException {
+
+ List<Query> filters = new ArrayList<>(filterQueries.length);
+
+ for (String filterQuery : filterQueries) {
+ if (filterQuery != null && filterQuery.trim().length() != 0) {
+ QParser filterParser;
+ try {
+ filterParser = QParser.getParser(filterQuery, super.getReq());
Review Comment:
subjective: https://github.com/apache/solr/blob/releases/solr/9.0.0/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java#L199 and other places have similar code, perhaps a utility method could be factored out somehow to reduce code duplication e.g. in https://github.com/apache/solr/blob/releases/solr/9.0.0/solr/core/src/java/org/apache/solr/search/QueryUtils.java something like `public static List<Query> parseFilterQueries(String[] filterQueries, SolrQueryRequest req)` or so.
##########
solr/core/src/java/org/apache/solr/search/neural/KnnQParser.java:
##########
@@ -72,7 +81,49 @@ public Query parse() {
DenseVectorField denseVectorType = (DenseVectorField) fieldType;
float[] parsedVectorToSearch = parseVector(vectorToSearch, denseVectorType.getDimension());
- return denseVectorType.getKnnVectorQuery(schemaField.getName(), parsedVectorToSearch, topK);
+
+ return getFilterQuery()
+ .map(
+ filterQuery ->
+ denseVectorType.getKnnVectorQuery(
+ schemaField.getName(), parsedVectorToSearch, topK, filterQuery))
+ .orElse(
+ denseVectorType.getKnnVectorQuery(schemaField.getName(), parsedVectorToSearch, topK));
+ }
+
+ private Optional<? extends Query> getFilterQuery() throws SolrException {
+ if (!isFilter()) {
+ String[] filterQueries = req.getParams().getParams(CommonParams.FQ);
+ return ofNullable(filterQueries).filter(fq -> fq.length != 0).map(this::computeFilterQuery);
+ }
+ return Optional.empty();
+ }
+
+ private Query computeFilterQuery(String[] filterQueries) throws SolrException {
+
+ List<Query> filters = new ArrayList<>(filterQueries.length);
+
+ for (String filterQuery : filterQueries) {
+ if (filterQuery != null && filterQuery.trim().length() != 0) {
+ QParser filterParser;
+ try {
+ filterParser = QParser.getParser(filterQuery, super.getReq());
+ filterParser.setIsFilter(true);
+ filters.add(filterParser.getQuery());
+ } catch (SyntaxError e) {
+ throw new SolrException(
+ SolrException.ErrorCode.BAD_REQUEST,
+ "error in parsing filter queries:" + e.getMessage());
+ }
+ }
+ }
+
+ BooleanQuery.Builder builder = new BooleanQuery.Builder();
+ for (Query query : filters) {
+ builder.add(query, BooleanClause.Occur.FILTER);
+ }
+
+ return builder.build();
Review Comment:
If `filters` contained only one element, could that be returned directly?
##########
solr/core/src/java/org/apache/solr/schema/DenseVectorField.java:
##########
@@ -276,6 +276,11 @@ public Query getKnnVectorQuery(String fieldName, float[] vectorToSearch, int top
return new KnnVectorQuery(fieldName, vectorToSearch, topK);
}
+ public Query getKnnVectorQuery(
Review Comment:
Maybe keep the original one for at least one release e.g. mark it as deprecated now for 9.1 and remove in a future 9.2 or higher release with backwards compatibility in mind.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org