You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2020/01/13 12:37:32 UTC
[GitHub] [lucene-solr] a-siuniaev opened a new pull request #1161:
SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON Facet
a-siuniaev opened a new pull request #1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON Facet
URL: https://github.com/apache/lucene-solr/pull/1161
# Description
Added uniqueBlockQuery(parent:true) aggregation for JSON Facet, which is an optimized alternative to uniqueBlock(_root_).
# Solution
Is implemented following the same type of strategy as uniqueBlock, but in this case we check next bit in parents' bitset - which is faster than getting _root_ values.
# Tests
Covered by the tests analogous to uniqueBlock tests.
# Checklist
- [ ] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability.
- [ ] I have created a Jira issue and added the issue ID to my pull request title.
- [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended)
- [ ] I have developed this patch against the `master` branch.
- [ ] I have run `ant precommit` and the appropriate test suite.
- [ ] I have added tests for my changes.
- [ ] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request
#1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON
Facet
Posted by GitBox <gi...@apache.org>.
mkhludnev commented on a change in pull request #1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON Facet
URL: https://github.com/apache/lucene-solr/pull/1161#discussion_r366721693
##########
File path: solr/core/src/java/org/apache/solr/search/facet/UniqueBlockFieldAgg.java
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.search.facet;
+
+import java.io.IOException;
+
+import org.apache.solr.schema.SchemaField;
+
+public class UniqueBlockFieldAgg extends UniqueBlockAgg {
+
+ private static final class UniqueBlockFieldSlotAcc extends UniqueBlockSlotAcc {
Review comment:
is it really necessary?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] a-siuniaev commented on a change in pull request
#1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON
Facet
Posted by GitBox <gi...@apache.org>.
a-siuniaev commented on a change in pull request #1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON Facet
URL: https://github.com/apache/lucene-solr/pull/1161#discussion_r366806423
##########
File path: solr/core/src/java/org/apache/solr/search/facet/UniqueBlockFieldAgg.java
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.search.facet;
+
+import java.io.IOException;
+
+import org.apache.solr.schema.SchemaField;
+
+public class UniqueBlockFieldAgg extends UniqueBlockAgg {
+
+ private static final class UniqueBlockFieldSlotAcc extends UniqueBlockSlotAcc {
Review comment:
Agree.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request
#1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON
Facet
Posted by GitBox <gi...@apache.org>.
mkhludnev commented on a change in pull request #1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON Facet
URL: https://github.com/apache/lucene-solr/pull/1161#discussion_r366721270
##########
File path: solr/core/src/java/org/apache/solr/search/FunctionQParser.java
##########
@@ -231,7 +232,59 @@ public String parseArg() throws SyntaxError {
return val;
}
-
+ /**
+ * Parse argument in the same way as parseArg() but also parse the argument name
+ * if written in following syntax: argumentName=argument.
+ *
+ * @return Immutable entry with name as the key and argument as value. In case where there's no name, key is null.
+ * @throws SyntaxError in case when argument is not ended by ) or ,
+ */
+ public SimpleImmutableEntry<String, String> parseNamedArg() throws SyntaxError {
Review comment:
1. Usually it's nicer to declare interface ie Entry<>.
1. I've thought that Yonik whiteness about something already existed? Doesn't it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request
#1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON
Facet
Posted by GitBox <gi...@apache.org>.
mkhludnev commented on a change in pull request #1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON Facet
URL: https://github.com/apache/lucene-solr/pull/1161#discussion_r365842780
##########
File path: solr/core/src/test/org/apache/solr/search/facet/TestJsonFacetsWithNestedObjects.java
##########
@@ -360,4 +360,30 @@ public void testUniqueBlock() throws Exception {
"}"
);
}
+
+ public void testUniqueBlockQuery() throws Exception {
+ final Client client = Client.localClient();
+ ModifiableSolrParams p = params("rows","0");
+ client.testJQ(params(p, "q", "{!parent tag=top which=type_s:book v=$childquery}"
+ , "childquery", "comment_t:*"
+ , "fl", "id", "fl" , "title_t"
+ , "json.facet", "{" +
+ " types: {" +
+ " domain: { blockChildren:\"type_s:book\"" +
+ " }," +
+ " type:terms," +
+ " field:type_s,"
+ + " limit:-1," +
+ " facet: {" +
+ " in_books: \"uniqueBlockQuery(type_s:book)\" }" +
+ " }" +
+ "}" )
+
+ , "response=={numFound:2,start:0,docs:[]}"
+ , "facets=={ count:2," +
+ "types:{" +
+ " buckets:[ {val:review, count:5, in_books:2} ]}" +
Review comment:
This kind of smoke test is fine. I;m a little bit anxious from the code overall. Can you think about bigger scale random test making side by side comparison? Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request
#1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON
Facet
Posted by GitBox <gi...@apache.org>.
mkhludnev commented on a change in pull request #1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON Facet
URL: https://github.com/apache/lucene-solr/pull/1161#discussion_r365840526
##########
File path: solr/core/src/java/org/apache/solr/search/ValueSourceParser.java
##########
@@ -975,6 +976,13 @@ public ValueSource parse(FunctionQParser fp) throws SyntaxError {
}
});
+ addParser("agg_uniqueBlockQuery", new ValueSourceParser() {
+ @Override
+ public ValueSource parse(FunctionQParser fp) throws SyntaxError {
Review comment:
Hello, @a-siuniaev. Thanks for the patch. Wouldn't you mind to explore [Yonik's idea](https://issues.apache.org/jira/browse/SOLR-12325?focusedCommentId=16649083&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16649083) about overloading the same name with the different argument type (query)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request
#1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON
Facet
Posted by GitBox <gi...@apache.org>.
mkhludnev commented on a change in pull request #1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON Facet
URL: https://github.com/apache/lucene-solr/pull/1161#discussion_r365841249
##########
File path: solr/core/src/java/org/apache/solr/search/facet/UniqueBlockQueryAgg.java
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search.facet;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Objects;
+import java.util.function.IntFunction;
+
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.search.Query;
+import org.apache.lucene.util.BitSet;
+
+import static org.apache.solr.search.join.BlockJoinParentQParser.getCachedFilter;
+
+public class UniqueBlockQueryAgg extends AggValueSource {
Review comment:
This reminds me uniqueBlock's code. Can you unify them somehow via extracting ancestors probably?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] a-siuniaev commented on a change in pull request
#1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON
Facet
Posted by GitBox <gi...@apache.org>.
a-siuniaev commented on a change in pull request #1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON Facet
URL: https://github.com/apache/lucene-solr/pull/1161#discussion_r366312200
##########
File path: solr/core/src/java/org/apache/solr/search/ValueSourceParser.java
##########
@@ -975,6 +976,13 @@ public ValueSource parse(FunctionQParser fp) throws SyntaxError {
}
});
+ addParser("agg_uniqueBlockQuery", new ValueSourceParser() {
+ @Override
+ public ValueSource parse(FunctionQParser fp) throws SyntaxError {
Review comment:
Added parseNamedArg to FunctionQParser.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request
#1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON
Facet
Posted by GitBox <gi...@apache.org>.
mkhludnev commented on a change in pull request #1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON Facet
URL: https://github.com/apache/lucene-solr/pull/1161#discussion_r365842011
##########
File path: solr/core/src/test/org/apache/solr/search/facet/TestJsonFacets.java
##########
@@ -3183,6 +3183,41 @@ public void testUniquesForMethod() throws Exception {
" ]}" +
"}"
);
+
+ //the same test with uniqueBlockQuery agg function
+ client.testJQ(params(p, "q", "type_s:page"
+ , "json.facet", "{" +
+ " types: {" +
+ " type:terms," +
+ " field:type_s," +
+ " limit:-1," +
+ " facet: {" +
+ " in_books: \"unique(_root_)\" }"+
+ " }," +
+ " pages: {" +
+ " type:terms," +
+ " field:page_s," +
+ " limit:-1," +
+ " facet: {" +
+ " in_books: \"uniqueBlockQuery(type_s:book)\" }"+
+ " }" +
+ "}" )
+
+ , "response=={numFound:6,start:0,docs:[]}"
+ , "facets=={ count:6," +
+ "types:{" +
+ " buckets:[ {val:page, count:6, in_books:2} ]}" +
+ "pages:{" +
+ " buckets:[ " +
+ " {val:a, count:1, in_books:1}," +
Review comment:
asserting equal counts across children and parents is little bit pointless. The later, should be smaller than former.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] a-siuniaev commented on a change in pull request
#1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON
Facet
Posted by GitBox <gi...@apache.org>.
a-siuniaev commented on a change in pull request #1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON Facet
URL: https://github.com/apache/lucene-solr/pull/1161#discussion_r366312261
##########
File path: solr/core/src/java/org/apache/solr/search/facet/UniqueBlockQueryAgg.java
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search.facet;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Objects;
+import java.util.function.IntFunction;
+
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.search.Query;
+import org.apache.lucene.util.BitSet;
+
+import static org.apache.solr.search.join.BlockJoinParentQParser.getCachedFilter;
+
+public class UniqueBlockQueryAgg extends AggValueSource {
Review comment:
Extracted.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene-solr] a-siuniaev commented on a change in pull request
#1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON
Facet
Posted by GitBox <gi...@apache.org>.
a-siuniaev commented on a change in pull request #1161: SOLR-12325 Added uniqueBlockQuery(parent:true) aggregation for JSON Facet
URL: https://github.com/apache/lucene-solr/pull/1161#discussion_r366787064
##########
File path: solr/core/src/java/org/apache/solr/search/FunctionQParser.java
##########
@@ -231,7 +232,59 @@ public String parseArg() throws SyntaxError {
return val;
}
-
+ /**
+ * Parse argument in the same way as parseArg() but also parse the argument name
+ * if written in following syntax: argumentName=argument.
+ *
+ * @return Immutable entry with name as the key and argument as value. In case where there's no name, key is null.
+ * @throws SyntaxError in case when argument is not ended by ) or ,
+ */
+ public SimpleImmutableEntry<String, String> parseNamedArg() throws SyntaxError {
Review comment:
1. Agree.
2. [Here](https://issues.apache.org/jira/browse/SOLR-12325?focusedCommentId=16649083&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16649083) he said "I wish we had..." - i.e. it did't exist at least at that time. And I can't find it now...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org