You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by "Julian Reschke (Jira)" <ji...@apache.org> on 2022/09/08 13:09:00 UTC
[jira] [Commented] (SLING-11439) resource resolver: fails to detect aborted vanity path query
[ https://issues.apache.org/jira/browse/SLING-11439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17601815#comment-17601815 ]
Julian Reschke commented on SLING-11439:
----------------------------------------
[~thomasm] re https://issues.apache.org/jira/browse/SLING-11439?focusedCommentId=17564308&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17564308:
{noformat}
This is not actually needed. Instead, we can iterate over the result, counting the entries in the loop itself, and stop when needed. E.g.
lastEntry = "";
while (true) {
it = query("select [sling:vanityPath], [sling:redirect], [sling:redirectStatus] " +
"from [nt:base] where first([sling:vanityPath]) >= $lastEntry " +
"order by first([sling:vanityPath]), [jcr:path]", lastEntry);
for (int i = 0; it.hasNext(); i++) {
String x = it.next().get("[sling:vanityPath]");
if (i > 1000 && !x.equals(lastEntry)) {
// next page
lastEntry = x;
break;
}
... remember x (in the Bloom filter) ...
}
}
BTW we need "first(..)" because sling:vanityPath can be a multi-valued property.
{noformat}
That doesn't seem to work. In my test I see that org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException is thrown on getting the iterator for the query result:
{noformat}
08.09.2022 14:19:38.945 *ERROR* [FelixDispatchQueue] org.apache.sling.resourceresolver.impl.CommonResourceResolverFactoryImpl activate: Cannot access repository, failed setting up Mapping Support
org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.fetchNextPossiblyDuplicate(ContentMirrorStoreStrategy.java:459) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.fetchNext(ContentMirrorStoreStrategy.java:402) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.next(ContentMirrorStoreStrategy.java:488) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.next(ContentMirrorStoreStrategy.java:338) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at com.google.common.collect.Iterators$5.next(Iterators.java:553) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:43) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.hasNext(FulltextIndex.java:419) [org.apache.jackrabbit.oak-lucene:1.44.0.T20220901123248-ccd9028]
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at org.apache.jackrabbit.oak.plugins.index.cursor.PathCursor.hasNext(PathCursor.java:70) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:452) [org.apache.jackrabbit.oak-lucene:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:529) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:522) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:933) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:960) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.init(FilterIterators.java:207) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.hasNext(FilterIterators.java:241) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.QueryImpl.getRows(QueryImpl.java:631) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.ResultImpl$1.iterator(ResultImpl.java:72) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$4.<init>(QueryResultImpl.java:177) [org.apache.jackrabbit.oak-jcr:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl.getNodes(QueryResultImpl.java:174) [org.apache.jackrabbit.oak-jcr:1.44.0.T20220901123248-ccd9028]
at org.apache.sling.jcr.resource.internal.helper.jcr.BasicQueryLanguageProvider.findResources(BasicQueryLanguageProvider.java:83) [org.apache.sling.jcr.resource:3.2.0]
at org.apache.sling.resourceresolver.impl.providers.stateful.AuthenticatedResourceProvider.findResources(AuthenticatedResourceProvider.java:285) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
at org.apache.sling.resourceresolver.impl.helper.ResourceResolverControl.findResources(ResourceResolverControl.java:574) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
at org.apache.sling.resourceresolver.impl.ResourceResolverImpl.findResources(ResourceResolverImpl.java:574) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
at org.apache.sling.resourceresolver.impl.mapping.MapEntries.queryAllVanityPaths(MapEntries.java:1282) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
{noformat}
So it's not possible just to read a "small" number of results, and then to restart the query.
> resource resolver: fails to detect aborted vanity path query
> ------------------------------------------------------------
>
> Key: SLING-11439
> URL: https://issues.apache.org/jira/browse/SLING-11439
> Project: Sling
> Issue Type: Bug
> Components: ResourceResolver
> Reporter: Julian Reschke
> Priority: Major
>
> With the introduction of the Oak query limit, JCR queries may get aborted when the result set size exceeds a certain value (currently by default 100000).
> However:
> https://github.com/apache/sling-org-apache-sling-jcr-resource/blob/604332e9be17378276685033bdbce54994dad8c1/src/main/java/org/apache/sling/jcr/resource/internal/helper/jcr/JcrNodeResourceIterator.java#L115-L134
> So Apache Sling JCR Resource's API hides that exception, and thus resource resolver will happily startup with an incomplete cache.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)