You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by GitBox <gi...@apache.org> on 2022/08/17 15:40:40 UTC

[GitHub] [jackrabbit-oak] fabriziofortino opened a new pull request, #670: OAK-9875-ut: full-text queries on date fields

fabriziofortino opened a new pull request, #670:
URL: https://github.com/apache/jackrabbit-oak/pull/670

   New unit test with full-text query on date field. Ignored in Elastic, it will be addressed with https://issues.apache.org/jira/browse/OAK-9875


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] thomasmueller commented on a diff in pull request #670: OAK-9875-ut: full-text queries on date fields

Posted by GitBox <gi...@apache.org>.
thomasmueller commented on code in PR #670:
URL: https://github.com/apache/jackrabbit-oak/pull/670#discussion_r949340989


##########
oak-search-elastic/src/main/java/org/apache/jackrabbit/oak/plugins/index/elastic/query/async/ElasticAsyncQueryException.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.index.elastic.query.async;
+
+import co.elastic.clients.elasticsearch._types.ElasticsearchException;
+
+public class ElasticAsyncQueryException extends RuntimeException {
+
+    private final String esQuery;
+
+    public ElasticAsyncQueryException(Throwable cause, String esQuery) {
+        super(cause);
+
+        this.esQuery = esQuery;
+    }
+
+    @Override
+    public String getMessage() {
+        StringBuilder sb = new StringBuilder();
+        sb.append(String.format("Error retrieving data for ES query:\n\t%s", esQuery));
+        if (getCause() instanceof ElasticsearchException) {
+            ElasticsearchException esException = (ElasticsearchException) getCause();
+            sb.append("\n:: Elasticsearch response status: ").append(esException.status()).append(", message: ").append(esException.getMessage());
+            esException.error().rootCause().forEach(ec -> sb.append("\n\t* caused by ").append(ec.type()).append(": ").append(ec.reason()));

Review Comment:
   I wonder if we need to protect against error() being null, rootCause() returning null, and so on.



##########
oak-search-elastic/src/main/java/org/apache/jackrabbit/oak/plugins/index/elastic/query/async/ElasticResultRowAsyncIterator.java:
##########
@@ -336,13 +336,10 @@ public void onFailure(Throwable t) {
                     System.currentTimeMillis() - searchStartTime);
             // Check in case errorRef is already set - this seems unlikely since we close the scanner once we hit failure.
             // But still, in case this do happen, we will log a warn.
-            Throwable error = errorRef.getAndSet(t);
+            Throwable error = errorRef.getAndSet(new ElasticAsyncQueryException(t, ElasticIndexUtils.toString(query)));
             if (error != null) {
                 LOG.warn("Error reference for async iterator was previously set to {}. It has now been reset to new error {}", error.getMessage(), t.getMessage());
             }
-
-            LOG.error("Error retrieving data for jcr query [{}] :: Corresponding ES query {} : closing scanner, notifying listeners",

Review Comment:
   So we no longer log this, why? Isn't there a risk that we don't log it at all in some cases? Unless I'm mistaken this code was changed recently, because we didn't have sufficient log messages...



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] nfsantos commented on a diff in pull request #670: OAK-9875-ut: full-text queries on date fields

Posted by GitBox <gi...@apache.org>.
nfsantos commented on code in PR #670:
URL: https://github.com/apache/jackrabbit-oak/pull/670#discussion_r949286227


##########
oak-search/src/test/java/org/apache/jackrabbit/oak/plugins/index/FullTextIndexCommonTest.java:
##########
@@ -45,12 +48,12 @@ public void defaultAnalyzer() throws Exception {
         root.commit();
 
         assertEventually(() -> {
-            assertQuery("//*[jcr:contains(@analyzed_field, 'SUN.JPG')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(@analyzed_field, 'Sun')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(@analyzed_field, 'jpg')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(., 'SUN.jpg')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(., 'sun')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(., 'jpg')] ", XPATH, Collections.singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(@analyzed_field, 'SUN.JPG')] ", XPATH, singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(@analyzed_field, 'Sun')] ", XPATH, singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(@analyzed_field, 'jpg')] ", XPATH, singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(., 'SUN.jpg')] ", XPATH, singletonList("/test/a"));

Review Comment:
   Ok, I'll do that.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] nit0906 commented on a diff in pull request #670: OAK-9875-ut: full-text queries on date fields

Posted by GitBox <gi...@apache.org>.
nit0906 commented on code in PR #670:
URL: https://github.com/apache/jackrabbit-oak/pull/670#discussion_r948611277


##########
oak-search/src/test/java/org/apache/jackrabbit/oak/plugins/index/FullTextIndexCommonTest.java:
##########
@@ -62,13 +65,26 @@ public void defaultAnalyzerHonourSplitOptions() throws Exception {
         root.commit();
 
         assertEventually(() -> {
-            assertQuery("//*[jcr:contains(@analyzed_field, '1234')] ", XPATH, Collections.emptyList());
-            assertQuery("//*[jcr:contains(@analyzed_field, 'abcd')] ", XPATH, Collections.emptyList());
-            assertQuery("//*[jcr:contains(@analyzed_field, '5678')] ", XPATH, Collections.emptyList());
-            assertQuery("//*[jcr:contains(@analyzed_field, '1234abCd5678')] ", XPATH, Collections.singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(@analyzed_field, '1234')] ", XPATH, emptyList());
+            assertQuery("//*[jcr:contains(@analyzed_field, 'abcd')] ", XPATH, emptyList());
+            assertQuery("//*[jcr:contains(@analyzed_field, '5678')] ", XPATH, emptyList());
+            assertQuery("//*[jcr:contains(@analyzed_field, '1234abCd5678')] ", XPATH, singletonList("/test/a"));
         });
     }
 
+    @Test
+    public void searchInDateField() throws Exception {
+        Tree test = setup();
+
+        Tree a = test.addChild("a");
+        a.setProperty("analyzed_field", "1234abCd5678");
+        a.setProperty("date_prop", "2014-04-01T09:58:03.231Z", Type.DATE);
+        root.commit();
+
+        assertEventually(() ->
+                assertQuery("//*[jcr:contains(., '01T09')] ", XPATH, singletonList("/test/a")));

Review Comment:
   Can we add one more assertion case where expected result would be 2 nodes - one satisfied from a date type prop and other from String type.
   
   for example - we add another node test/b with analyzed_field=231Z and then search on `//*[jcr:contains(., '231Z')`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] fabriziofortino commented on a diff in pull request #670: OAK-9875-ut: full-text queries on date fields

Posted by GitBox <gi...@apache.org>.
fabriziofortino commented on code in PR #670:
URL: https://github.com/apache/jackrabbit-oak/pull/670#discussion_r949989809


##########
oak-search-elastic/src/main/java/org/apache/jackrabbit/oak/plugins/index/elastic/query/async/ElasticResultRowAsyncIterator.java:
##########
@@ -336,13 +336,10 @@ public void onFailure(Throwable t) {
                     System.currentTimeMillis() - searchStartTime);
             // Check in case errorRef is already set - this seems unlikely since we close the scanner once we hit failure.
             // But still, in case this do happen, we will log a warn.
-            Throwable error = errorRef.getAndSet(t);
+            Throwable error = errorRef.getAndSet(new ElasticAsyncQueryException(t, ElasticIndexUtils.toString(query)));
             if (error != null) {
                 LOG.warn("Error reference for async iterator was previously set to {}. It has now been reset to new error {}", error.getMessage(), t.getMessage());
             }
-
-            LOG.error("Error retrieving data for jcr query [{}] :: Corresponding ES query {} : closing scanner, notifying listeners",

Review Comment:
   we were logging it twice with slightly different content. The newly introduced exception shows a message with the entire content. It is printed in `hasNext()`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] nit0906 commented on a diff in pull request #670: OAK-9875-ut: full-text queries on date fields

Posted by GitBox <gi...@apache.org>.
nit0906 commented on code in PR #670:
URL: https://github.com/apache/jackrabbit-oak/pull/670#discussion_r948618621


##########
oak-search/src/test/java/org/apache/jackrabbit/oak/plugins/index/FullTextIndexCommonTest.java:
##########
@@ -45,12 +48,12 @@ public void defaultAnalyzer() throws Exception {
         root.commit();
 
         assertEventually(() -> {
-            assertQuery("//*[jcr:contains(@analyzed_field, 'SUN.JPG')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(@analyzed_field, 'Sun')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(@analyzed_field, 'jpg')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(., 'SUN.jpg')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(., 'sun')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(., 'jpg')] ", XPATH, Collections.singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(@analyzed_field, 'SUN.JPG')] ", XPATH, singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(@analyzed_field, 'Sun')] ", XPATH, singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(@analyzed_field, 'jpg')] ", XPATH, singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(., 'SUN.jpg')] ", XPATH, singletonList("/test/a"));

Review Comment:
   This test seems to be failing at this assertion in this PR checks here https://ci-builds.apache.org/job/Jackrabbit/job/oak-trunk-pr/job/PR-670/1/testReport/junit/org.apache.jackrabbit.oak.plugins.index.elastic/ElasticFullTextIndexCommonTest/oak_search_elastic___defaultAnalyzer/?cloudbees-analytics-link=scm-reporting%2Ftests%2Ffailed 
   
   Seems like the original issue with the exception
   `failed to create query: failed to parse date field [sun] with format [strict_date_optional_time||epoch_millis]: [failed to parse date field [sun] with format [strict_date_optional_time||epoch_millis]] 
   `



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] fabriziofortino closed pull request #670: OAK-9875-ut: full-text queries on date fields

Posted by GitBox <gi...@apache.org>.
fabriziofortino closed pull request #670: OAK-9875-ut: full-text queries on date fields
URL: https://github.com/apache/jackrabbit-oak/pull/670


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] fabriziofortino commented on a diff in pull request #670: OAK-9875-ut: full-text queries on date fields

Posted by GitBox <gi...@apache.org>.
fabriziofortino commented on code in PR #670:
URL: https://github.com/apache/jackrabbit-oak/pull/670#discussion_r949228992


##########
oak-search/src/test/java/org/apache/jackrabbit/oak/plugins/index/FullTextIndexCommonTest.java:
##########
@@ -45,12 +48,12 @@ public void defaultAnalyzer() throws Exception {
         root.commit();
 
         assertEventually(() -> {
-            assertQuery("//*[jcr:contains(@analyzed_field, 'SUN.JPG')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(@analyzed_field, 'Sun')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(@analyzed_field, 'jpg')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(., 'SUN.jpg')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(., 'sun')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(., 'jpg')] ", XPATH, Collections.singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(@analyzed_field, 'SUN.JPG')] ", XPATH, singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(@analyzed_field, 'Sun')] ", XPATH, singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(@analyzed_field, 'jpg')] ", XPATH, singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(., 'SUN.jpg')] ", XPATH, singletonList("/test/a"));

Review Comment:
   @nit0906 I did not notice this because the test console output does not include log messages.
   
   I have converted this PR to `draft` and I do not plan to merge it. I have improved a bit the logging in case of failure starting from the work done in https://github.com/apache/jackrabbit-oak/pull/649/files#diff-ac2f41f4e63cb259a78a57e764daed1a88e5a1d225161398ee16cc7db60a4034
   
   I have mainly centralized the message in a new Exception avoiding printing the different information in multiple places.
   
   @nfsantos please review this and incorporate this work in the main OAK-9875 PR. Thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] fabriziofortino commented on pull request #670: OAK-9875-ut: full-text queries on date fields

Posted by GitBox <gi...@apache.org>.
fabriziofortino commented on PR #670:
URL: https://github.com/apache/jackrabbit-oak/pull/670#issuecomment-1231919019

   changes integrated in #649 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] fabriziofortino commented on a diff in pull request #670: OAK-9875-ut: full-text queries on date fields

Posted by GitBox <gi...@apache.org>.
fabriziofortino commented on code in PR #670:
URL: https://github.com/apache/jackrabbit-oak/pull/670#discussion_r949216374


##########
oak-search/src/test/java/org/apache/jackrabbit/oak/plugins/index/FullTextIndexCommonTest.java:
##########
@@ -45,12 +48,12 @@ public void defaultAnalyzer() throws Exception {
         root.commit();
 
         assertEventually(() -> {
-            assertQuery("//*[jcr:contains(@analyzed_field, 'SUN.JPG')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(@analyzed_field, 'Sun')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(@analyzed_field, 'jpg')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(., 'SUN.jpg')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(., 'sun')] ", XPATH, Collections.singletonList("/test/a"));
-            assertQuery("//*[jcr:contains(., 'jpg')] ", XPATH, Collections.singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(@analyzed_field, 'SUN.JPG')] ", XPATH, singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(@analyzed_field, 'Sun')] ", XPATH, singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(@analyzed_field, 'jpg')] ", XPATH, singletonList("/test/a"));
+            assertQuery("//*[jcr:contains(., 'SUN.jpg')] ", XPATH, singletonList("/test/a"));

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [jackrabbit-oak] fabriziofortino commented on a diff in pull request #670: OAK-9875-ut: full-text queries on date fields

Posted by GitBox <gi...@apache.org>.
fabriziofortino commented on code in PR #670:
URL: https://github.com/apache/jackrabbit-oak/pull/670#discussion_r949993389


##########
oak-search-elastic/src/main/java/org/apache/jackrabbit/oak/plugins/index/elastic/query/async/ElasticAsyncQueryException.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.index.elastic.query.async;
+
+import co.elastic.clients.elasticsearch._types.ElasticsearchException;
+
+public class ElasticAsyncQueryException extends RuntimeException {
+
+    private final String esQuery;
+
+    public ElasticAsyncQueryException(Throwable cause, String esQuery) {
+        super(cause);
+
+        this.esQuery = esQuery;
+    }
+
+    @Override
+    public String getMessage() {
+        StringBuilder sb = new StringBuilder();
+        sb.append(String.format("Error retrieving data for ES query:\n\t%s", esQuery));
+        if (getCause() instanceof ElasticsearchException) {
+            ElasticsearchException esException = (ElasticsearchException) getCause();
+            sb.append("\n:: Elasticsearch response status: ").append(esException.status()).append(", message: ").append(esException.getMessage());
+            esException.error().rootCause().forEach(ec -> sb.append("\n\t* caused by ").append(ec.type()).append(": ").append(ec.reason()));

Review Comment:
   I checked the Elastic source code and both methods cannot return null



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org