You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-issues@jackrabbit.apache.org by "Julian Reschke (Jira)" <ji...@apache.org> on 2022/09/09 13:28:00 UTC
[jira] [Updated] (OAK-9929) OPTION (LIMIT ...) doesn't prevent RuntimeNodeTraversalException
[ https://issues.apache.org/jira/browse/OAK-9929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julian Reschke updated OAK-9929:
--------------------------------
Description:
It appears that the concept of paging doesn't help when the result of the unpaged query exceeds the query limit.
With >100K vanity paths and this query:
{noformat}
SELECT [sling:vanityPath], [sling:redirect], [sling:redirectStatus] FROM [nt:base] WHERE NOT isdescendantnode('/jcr:system') AND [sling:vanityPath] IS NOT NULL ORDER BY FIRST([sling:vanityPath]), [jcr:path] OPTION (LIMIT 1000)
{noformat}
I get the following exeption:
{noformat}08.09.2022 14:19:38.945 *ERROR* [FelixDispatchQueue] org.apache.sling.resourceresolver.impl.CommonResourceResolverFactoryImpl activate: Cannot access repository, failed setting up Mapping Support
org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.fetchNextPossiblyDuplicate(ContentMirrorStoreStrategy.java:459) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.fetchNext(ContentMirrorStoreStrategy.java:402) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.next(ContentMirrorStoreStrategy.java:488) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.next(ContentMirrorStoreStrategy.java:338) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at com.google.common.collect.Iterators$5.next(Iterators.java:553) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:43) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.hasNext(FulltextIndex.java:419) [org.apache.jackrabbit.oak-lucene:1.44.0.T20220901123248-ccd9028]
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at org.apache.jackrabbit.oak.plugins.index.cursor.PathCursor.hasNext(PathCursor.java:70) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:452) [org.apache.jackrabbit.oak-lucene:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:529) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:522) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:933) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:960) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.init(FilterIterators.java:207) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.hasNext(FilterIterators.java:241) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.QueryImpl.getRows(QueryImpl.java:631) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.ResultImpl$1.iterator(ResultImpl.java:72) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$4.<init>(QueryResultImpl.java:177) [org.apache.jackrabbit.oak-jcr:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl.getNodes(QueryResultImpl.java:174) [org.apache.jackrabbit.oak-jcr:1.44.0.T20220901123248-ccd9028]
at org.apache.sling.jcr.resource.internal.helper.jcr.BasicQueryLanguageProvider.findResources(BasicQueryLanguageProvider.java:83) [org.apache.sling.jcr.resource:3.2.0]
at org.apache.sling.resourceresolver.impl.providers.stateful.AuthenticatedResourceProvider.findResources(AuthenticatedResourceProvider.java:285) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
at org.apache.sling.resourceresolver.impl.helper.ResourceResolverControl.findResources(ResourceResolverControl.java:574) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
at org.apache.sling.resourceresolver.impl.ResourceResolverImpl.findResources(ResourceResolverImpl.java:574) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
at org.apache.sling.resourceresolver.impl.mapping.MapEntries.queryAllVanityPaths(MapEntries.java:1282) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
{noformat}
That is, the querying code doesn't even get a query result it could iterate on.
(Please let me know if more information is needed to reproduce)
was:
It appeats that the concept of paging doesn't help when the result of the unpaged query exceeds the query limit.
With >100K vanity paths and this query:
~~~
SELECT [sling:vanityPath], [sling:redirect], [sling:redirectStatus] FROM [nt:base] WHERE NOT isdescendantnode('/jcr:system') AND [sling:vanityPath] IS NOT NULL ORDER BY FIRST([sling:vanityPath]), [jcr:path] OPTION (LIMIT 1000)
~~~
I get the following exeption:
~~~
08.09.2022 14:19:38.945 *ERROR* [FelixDispatchQueue] org.apache.sling.resourceresolver.impl.CommonResourceResolverFactoryImpl activate: Cannot access repository, failed setting up Mapping Support
org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.fetchNextPossiblyDuplicate(ContentMirrorStoreStrategy.java:459) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.fetchNext(ContentMirrorStoreStrategy.java:402) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.next(ContentMirrorStoreStrategy.java:488) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.next(ContentMirrorStoreStrategy.java:338) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at com.google.common.collect.Iterators$5.next(Iterators.java:553) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:43) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.hasNext(FulltextIndex.java:419) [org.apache.jackrabbit.oak-lucene:1.44.0.T20220901123248-ccd9028]
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
at org.apache.jackrabbit.oak.plugins.index.cursor.PathCursor.hasNext(PathCursor.java:70) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:452) [org.apache.jackrabbit.oak-lucene:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:529) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:522) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:933) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:960) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.init(FilterIterators.java:207) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.hasNext(FilterIterators.java:241) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.QueryImpl.getRows(QueryImpl.java:631) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.query.ResultImpl$1.iterator(ResultImpl.java:72) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$4.<init>(QueryResultImpl.java:177) [org.apache.jackrabbit.oak-jcr:1.44.0.T20220901123248-ccd9028]
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl.getNodes(QueryResultImpl.java:174) [org.apache.jackrabbit.oak-jcr:1.44.0.T20220901123248-ccd9028]
at org.apache.sling.jcr.resource.internal.helper.jcr.BasicQueryLanguageProvider.findResources(BasicQueryLanguageProvider.java:83) [org.apache.sling.jcr.resource:3.2.0]
at org.apache.sling.resourceresolver.impl.providers.stateful.AuthenticatedResourceProvider.findResources(AuthenticatedResourceProvider.java:285) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
at org.apache.sling.resourceresolver.impl.helper.ResourceResolverControl.findResources(ResourceResolverControl.java:574) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
at org.apache.sling.resourceresolver.impl.ResourceResolverImpl.findResources(ResourceResolverImpl.java:574) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
at org.apache.sling.resourceresolver.impl.mapping.MapEntries.queryAllVanityPaths(MapEntries.java:1282) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
~~~
That is, the querying code doesn't even get a query result it could iterate on.
(Please let me know if more information is needed to reproduce)
> OPTION (LIMIT ...) doesn't prevent RuntimeNodeTraversalException
> ----------------------------------------------------------------
>
> Key: OAK-9929
> URL: https://issues.apache.org/jira/browse/OAK-9929
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: search
> Affects Versions: 1.44.0
> Reporter: Julian Reschke
> Priority: Major
>
> It appears that the concept of paging doesn't help when the result of the unpaged query exceeds the query limit.
> With >100K vanity paths and this query:
> {noformat}
> SELECT [sling:vanityPath], [sling:redirect], [sling:redirectStatus] FROM [nt:base] WHERE NOT isdescendantnode('/jcr:system') AND [sling:vanityPath] IS NOT NULL ORDER BY FIRST([sling:vanityPath]), [jcr:path] OPTION (LIMIT 1000)
> {noformat}
> I get the following exeption:
> {noformat}08.09.2022 14:19:38.945 *ERROR* [FelixDispatchQueue] org.apache.sling.resourceresolver.impl.CommonResourceResolverFactoryImpl activate: Cannot access repository, failed setting up Mapping Support
> org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
> at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.fetchNextPossiblyDuplicate(ContentMirrorStoreStrategy.java:459) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.fetchNext(ContentMirrorStoreStrategy.java:402) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.next(ContentMirrorStoreStrategy.java:488) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.next(ContentMirrorStoreStrategy.java:338) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
> at com.google.common.collect.Iterators$5.next(Iterators.java:553) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
> at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
> at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
> at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
> at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:43) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
> at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
> at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.hasNext(FulltextIndex.java:419) [org.apache.jackrabbit.oak-lucene:1.44.0.T20220901123248-ccd9028]
> at com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
> at org.apache.jackrabbit.oak.plugins.index.cursor.PathCursor.hasNext(PathCursor.java:70) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:452) [org.apache.jackrabbit.oak-lucene:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:529) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:522) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:933) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:960) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.init(FilterIterators.java:207) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.query.FilterIterators$SortIterator.hasNext(FilterIterators.java:241) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.query.QueryImpl.getRows(QueryImpl.java:631) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.query.ResultImpl$1.iterator(ResultImpl.java:72) [org.apache.jackrabbit.oak-core:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$4.<init>(QueryResultImpl.java:177) [org.apache.jackrabbit.oak-jcr:1.44.0.T20220901123248-ccd9028]
> at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl.getNodes(QueryResultImpl.java:174) [org.apache.jackrabbit.oak-jcr:1.44.0.T20220901123248-ccd9028]
> at org.apache.sling.jcr.resource.internal.helper.jcr.BasicQueryLanguageProvider.findResources(BasicQueryLanguageProvider.java:83) [org.apache.sling.jcr.resource:3.2.0]
> at org.apache.sling.resourceresolver.impl.providers.stateful.AuthenticatedResourceProvider.findResources(AuthenticatedResourceProvider.java:285) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
> at org.apache.sling.resourceresolver.impl.helper.ResourceResolverControl.findResources(ResourceResolverControl.java:574) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
> at org.apache.sling.resourceresolver.impl.ResourceResolverImpl.findResources(ResourceResolverImpl.java:574) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
> at org.apache.sling.resourceresolver.impl.mapping.MapEntries.queryAllVanityPaths(MapEntries.java:1282) [org.apache.sling.resourceresolver:1.10.1.SNAPSHOT]
> {noformat}
> That is, the querying code doesn't even get a query result it could iterate on.
> (Please let me know if more information is needed to reproduce)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)