You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by GitBox <gi...@apache.org> on 2022/08/08 13:17:10 UTC

[GitHub] [jackrabbit-oak] nfsantos opened a new pull request, #657: OAK-9881 - Unreachable code in the logic that processes like constraints in Elastic

nfsantos opened a new pull request, #657:
URL: https://github.com/apache/jackrabbit-oak/pull/657

   Fix unreachable code in Elastic plugin when processing like constraints. The intention of the unreachable code is to transform constraints of the form "like abc%" into ElasticSearch prefix queries instead of using the more expansive wildcard queries. This commit enables this optimization.
   
   Move a couple of Lucene tests for the like constraint to oak-search module, so it is run in both Elastic and Lucene. One of these tests is ignored for Elastic because it is failing (see OAK-9885).
   
   **Note: The OAK issue identifies unreachable code in both the Lucene and Elastic modules (same code, copy'n'paste). This PR fixes only the unreachable code in Elastic.**


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] nfsantos commented on a diff in pull request #657: OAK-9881 - Unreachable code in the logic that processes like constraints in Elastic

Posted by GitBox <gi...@apache.org>.
nfsantos commented on code in PR #657:
URL: https://github.com/apache/jackrabbit-oak/pull/657#discussion_r942448782


##########
oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndex.java:
##########
@@ -765,6 +765,8 @@ private static void addNonFullTextConstraints(List<Query> qs,
             }
 
             if (isLike) {
+                // This code may be unreachable. Adding a log statement to try to identify usages of this code
+                LOG.info("Transforming 'like' constraint {} for restriction {}", first, pr);

Review Comment:
   Thanks for the clarification that the LuceneIndex class is used for compatVersion 1, that explains why I could not hit this part of the code. 
   Is compatVersion 1 still being used? As I assume the intention is to move away from this code, then I guess it's not worth to fix it. My suggestion is to remove the log statement (since we already know when this code is used) and leave a comment mentioning that the logic is wrong, but that we will not fix it. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] nit0906 commented on a diff in pull request #657: OAK-9881 - Unreachable code in the logic that processes like constraints in Elastic

Posted by GitBox <gi...@apache.org>.
nit0906 commented on code in PR #657:
URL: https://github.com/apache/jackrabbit-oak/pull/657#discussion_r942009423


##########
oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndex.java:
##########
@@ -765,6 +765,8 @@ private static void addNonFullTextConstraints(List<Query> qs,
             }
 
             if (isLike) {
+                // This code may be unreachable. Adding a log statement to try to identify usages of this code
+                LOG.info("Transforming 'like' constraint {} for restriction {}", first, pr);

Review Comment:
   Was this usage identified from a test ? If yes, I think we can remove this from the PR ? If not - then I think trying to add a test here would be good.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] nfsantos commented on a diff in pull request #657: OAK-9881 - Unreachable code in the logic that processes like constraints in Elastic

Posted by GitBox <gi...@apache.org>.
nfsantos commented on code in PR #657:
URL: https://github.com/apache/jackrabbit-oak/pull/657#discussion_r940965614


##########
oak-search-elastic/src/main/java/org/apache/jackrabbit/oak/plugins/index/elastic/query/ElasticRequestHandler.java:
##########
@@ -746,15 +746,14 @@ private static Query nodeName(Filter.PropertyRestriction pr) {
         throw new IllegalStateException("For nodeName queries only EQUALS and LIKE are supported " + pr);
     }
 
-    private static Query like(String name, String first) {
-        first = first.replace('%', WildcardQuery.WILDCARD_STRING);
+    private static Query like(String name, String first) {first = first.replace('%', WildcardQuery.WILDCARD_STRING);
         first = first.replace('_', WildcardQuery.WILDCARD_CHAR);
 
-        int indexOfWS = first.indexOf(WildcardQuery.WILDCARD_STRING);
-        int indexOfWC = first.indexOf(WildcardQuery.WILDCARD_CHAR);
-        int len = first.length();
+        // If the query ends in a wildcard string (*) and has no other wildcard characters, use a prefix match query
+        boolean hasSingleWildcardStringAtEnd = first.indexOf(WildcardQuery.WILDCARD_STRING) == first.length() - 1;
+        boolean doesNotContainWildcardChar = first.indexOf(WildcardQuery.WILDCARD_CHAR) == -1;

Review Comment:
   I have no strong opinion here. I chose comparing with `-1` because it follows more closely the specification of `indexOf`, which states that it returns -1 if it does not find the character, so I believe it is more clear. `indexOf` should never return values less than `-1`, that would be a very serious problem from which we would likely not be able to recover. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] thomasmueller commented on a diff in pull request #657: OAK-9881 - Unreachable code in the logic that processes like constraints in Elastic

Posted by GitBox <gi...@apache.org>.
thomasmueller commented on code in PR #657:
URL: https://github.com/apache/jackrabbit-oak/pull/657#discussion_r940367285


##########
oak-search-elastic/src/main/java/org/apache/jackrabbit/oak/plugins/index/elastic/query/ElasticRequestHandler.java:
##########
@@ -746,15 +746,14 @@ private static Query nodeName(Filter.PropertyRestriction pr) {
         throw new IllegalStateException("For nodeName queries only EQUALS and LIKE are supported " + pr);
     }
 
-    private static Query like(String name, String first) {
-        first = first.replace('%', WildcardQuery.WILDCARD_STRING);
+    private static Query like(String name, String first) {first = first.replace('%', WildcardQuery.WILDCARD_STRING);
         first = first.replace('_', WildcardQuery.WILDCARD_CHAR);
 
-        int indexOfWS = first.indexOf(WildcardQuery.WILDCARD_STRING);
-        int indexOfWC = first.indexOf(WildcardQuery.WILDCARD_CHAR);
-        int len = first.length();
+        // If the query ends in a wildcard string (*) and has no other wildcard characters, use a prefix match query
+        boolean hasSingleWildcardStringAtEnd = first.indexOf(WildcardQuery.WILDCARD_STRING) == first.length() - 1;
+        boolean doesNotContainWildcardChar = first.indexOf(WildcardQuery.WILDCARD_CHAR) == -1;

Review Comment:
   The escape character problem applies here as well.



##########
oak-search-elastic/src/main/java/org/apache/jackrabbit/oak/plugins/index/elastic/query/ElasticRequestHandler.java:
##########
@@ -746,15 +746,14 @@ private static Query nodeName(Filter.PropertyRestriction pr) {
         throw new IllegalStateException("For nodeName queries only EQUALS and LIKE are supported " + pr);
     }
 
-    private static Query like(String name, String first) {
-        first = first.replace('%', WildcardQuery.WILDCARD_STRING);
+    private static Query like(String name, String first) {first = first.replace('%', WildcardQuery.WILDCARD_STRING);
         first = first.replace('_', WildcardQuery.WILDCARD_CHAR);
 
-        int indexOfWS = first.indexOf(WildcardQuery.WILDCARD_STRING);
-        int indexOfWC = first.indexOf(WildcardQuery.WILDCARD_CHAR);
-        int len = first.length();
+        // If the query ends in a wildcard string (*) and has no other wildcard characters, use a prefix match query
+        boolean hasSingleWildcardStringAtEnd = first.indexOf(WildcardQuery.WILDCARD_STRING) == first.length() - 1;

Review Comment:
   What about the escape character... At first glance, we could additional check that `first` doesn't end with `\*`. But actually it's a bit more complicated, we would need to check for `\\*` as well, and then `\\\*`. Hm, it's complicated... 
   
   If we want to support it ourselves (that is, not use a method of Lucene code; not sure if that even exists), then maybe it's best to first remove all escape characters and replace the character that follows escape characters with 'x'. Then, we can use the code you have. But not replace in `filter` itself -- instead in a copy of `filter` used just for this test.



##########
oak-search-elastic/src/main/java/org/apache/jackrabbit/oak/plugins/index/elastic/query/ElasticRequestHandler.java:
##########
@@ -746,15 +746,14 @@ private static Query nodeName(Filter.PropertyRestriction pr) {
         throw new IllegalStateException("For nodeName queries only EQUALS and LIKE are supported " + pr);
     }
 
-    private static Query like(String name, String first) {
-        first = first.replace('%', WildcardQuery.WILDCARD_STRING);
+    private static Query like(String name, String first) {first = first.replace('%', WildcardQuery.WILDCARD_STRING);
         first = first.replace('_', WildcardQuery.WILDCARD_CHAR);
 
-        int indexOfWS = first.indexOf(WildcardQuery.WILDCARD_STRING);
-        int indexOfWC = first.indexOf(WildcardQuery.WILDCARD_CHAR);
-        int len = first.length();
+        // If the query ends in a wildcard string (*) and has no other wildcard characters, use a prefix match query
+        boolean hasSingleWildcardStringAtEnd = first.indexOf(WildcardQuery.WILDCARD_STRING) == first.length() - 1;
+        boolean doesNotContainWildcardChar = first.indexOf(WildcardQuery.WILDCARD_CHAR) == -1;

Review Comment:
   I usually use `first.indexOf(WildcardQuery.WILDCARD_CHAR) < 0` but `== -1` might be better



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] nit0906 commented on a diff in pull request #657: OAK-9881 - Unreachable code in the logic that processes like constraints in Elastic

Posted by GitBox <gi...@apache.org>.
nit0906 commented on code in PR #657:
URL: https://github.com/apache/jackrabbit-oak/pull/657#discussion_r942455212


##########
oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndex.java:
##########
@@ -765,6 +765,8 @@ private static void addNonFullTextConstraints(List<Query> qs,
             }
 
             if (isLike) {
+                // This code may be unreachable. Adding a log statement to try to identify usages of this code
+                LOG.info("Transforming 'like' constraint {} for restriction {}", first, pr);

Review Comment:
   I don't think it's being used. This is where the compat version is being set https://github.com/apache/jackrabbit-oak/blob/trunk/oak-search/src/main/java/org/apache/jackrabbit/oak/plugins/index/search/util/IndexDefinitionBuilder.java#L87. and it sets it to 2. 
   
   I think this code is just for backward compatibility for any repositories that probably still use an older version of lucene.
   
   
   
   >My suggestion is to remove the log statement (since we already know when this code is used) and leave a comment mentioning that the logic is wrong, but that we will not fix it.
   
   +1



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] nfsantos commented on a diff in pull request #657: OAK-9881 - Unreachable code in the logic that processes like constraints in Elastic

Posted by GitBox <gi...@apache.org>.
nfsantos commented on code in PR #657:
URL: https://github.com/apache/jackrabbit-oak/pull/657#discussion_r940963346


##########
oak-search-elastic/src/main/java/org/apache/jackrabbit/oak/plugins/index/elastic/query/ElasticRequestHandler.java:
##########
@@ -746,15 +746,14 @@ private static Query nodeName(Filter.PropertyRestriction pr) {
         throw new IllegalStateException("For nodeName queries only EQUALS and LIKE are supported " + pr);
     }
 
-    private static Query like(String name, String first) {
-        first = first.replace('%', WildcardQuery.WILDCARD_STRING);
+    private static Query like(String name, String first) {first = first.replace('%', WildcardQuery.WILDCARD_STRING);
         first = first.replace('_', WildcardQuery.WILDCARD_CHAR);
 
-        int indexOfWS = first.indexOf(WildcardQuery.WILDCARD_STRING);
-        int indexOfWC = first.indexOf(WildcardQuery.WILDCARD_CHAR);
-        int len = first.length();
+        // If the query ends in a wildcard string (*) and has no other wildcard characters, use a prefix match query
+        boolean hasSingleWildcardStringAtEnd = first.indexOf(WildcardQuery.WILDCARD_STRING) == first.length() - 1;

Review Comment:
   I was planning of handling the escape characters issue separately from this PR, in this separate issue: https://issues.apache.org/jira/browse/OAK-9885 
   
   The Lucene logic that handles like queries with wildcards is here, in the class `LucenePropertyIndex`: 
   https://github.com/apache/jackrabbit-oak/blob/27a7a9ffa0e78e5b258626aef24066eb58efc559/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LucenePropertyIndex.java#L1161-L1198
   I confirmed that this code is being used in a debugging session. 
   
   There is similar logic in `LuceneIndex` class, but I suspect it is unused, maybe leftovers from a refactoring. The logic in LuceneIndex does seem incomplete, it even has a TODO asking what to do about escaped characters.
   https://github.com/apache/jackrabbit-oak/blob/34edee29cf65e44f18cc0548d176ab005de6dea6/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndex.java#L755-L791
   
   Although it's not an ideal solution, we could just copy the correct code from Lucene to Elastic. Alternatively, we could refactor the wildcard handling logic to some common package where we can reuse it.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] thomasmueller merged pull request #657: OAK-9881 - Unreachable code in the logic that processes like constraints in Elastic

Posted by GitBox <gi...@apache.org>.
thomasmueller merged PR #657:
URL: https://github.com/apache/jackrabbit-oak/pull/657


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] nit0906 commented on a diff in pull request #657: OAK-9881 - Unreachable code in the logic that processes like constraints in Elastic

Posted by GitBox <gi...@apache.org>.
nit0906 commented on code in PR #657:
URL: https://github.com/apache/jackrabbit-oak/pull/657#discussion_r942013257


##########
oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndex.java:
##########
@@ -765,6 +765,8 @@ private static void addNonFullTextConstraints(List<Query> qs,
             }
 
             if (isLike) {
+                // This code may be unreachable. Adding a log statement to try to identify usages of this code
+                LOG.info("Transforming 'like' constraint {} for restriction {}", first, pr);

Review Comment:
   Ah - I just noticed this is LuceneIndex. This class is used for backward compatability of compatVersion 1 lucene index. 
   I am not sure if there are any tests with lucene indexes of compatVersion 1 . But that could be used to hit this code. As per me, this should be lower priority. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] nfsantos commented on a diff in pull request #657: OAK-9881 - Unreachable code in the logic that processes like constraints in Elastic

Posted by GitBox <gi...@apache.org>.
nfsantos commented on code in PR #657:
URL: https://github.com/apache/jackrabbit-oak/pull/657#discussion_r940965614


##########
oak-search-elastic/src/main/java/org/apache/jackrabbit/oak/plugins/index/elastic/query/ElasticRequestHandler.java:
##########
@@ -746,15 +746,14 @@ private static Query nodeName(Filter.PropertyRestriction pr) {
         throw new IllegalStateException("For nodeName queries only EQUALS and LIKE are supported " + pr);
     }
 
-    private static Query like(String name, String first) {
-        first = first.replace('%', WildcardQuery.WILDCARD_STRING);
+    private static Query like(String name, String first) {first = first.replace('%', WildcardQuery.WILDCARD_STRING);
         first = first.replace('_', WildcardQuery.WILDCARD_CHAR);
 
-        int indexOfWS = first.indexOf(WildcardQuery.WILDCARD_STRING);
-        int indexOfWC = first.indexOf(WildcardQuery.WILDCARD_CHAR);
-        int len = first.length();
+        // If the query ends in a wildcard string (*) and has no other wildcard characters, use a prefix match query
+        boolean hasSingleWildcardStringAtEnd = first.indexOf(WildcardQuery.WILDCARD_STRING) == first.length() - 1;
+        boolean doesNotContainWildcardChar = first.indexOf(WildcardQuery.WILDCARD_CHAR) == -1;

Review Comment:
   I have no strong opinion here. I chose comparing with `-1` because it follows more closely the specification of `indexOf`, which states that it returns -1 if it does not find the character, so I believe it is more clear. `indexOf` should never return values less than `-1`, that would be a very serious problem from which we would like not be able to recover. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org