You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by "Joerg Hoh (Jira)" <ji...@apache.org> on 2025/08/28 10:41:00 UTC
[jira] [Updated] (JCRVLT-809) DocViewImporter: logIgnoredProtectedProperties can slow down package import
[ https://issues.apache.org/jira/browse/JCRVLT-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joerg Hoh updated JCRVLT-809:
-----------------------------
Description:
*NOTE*: I leave this initial idea as is, but the improvement is done differently; see the comments below and the Github PR.
In the case of Sling distribution the packages being created and imported are controlled by the framework, and IIUC protected properties are not part of the package; for that reason {{DocViewImporter.logIgnoredProtectedProperties}} will not write any WARN message.
When checking the details of the import of a complex package (with almost 100k nodes and low to medium amount of properties per node), I came frequently across this type of stack, where it's checked if properties are protected. Commenting the 2 calls to {{DocViewImporter.logIgnoredProtectedProperties}} gave me an improvement of the import time in the range of 10%.
For that I think that we should provide a chance to disable these checks, preferably via the API, so that it can configured per package I want to import. Disabling it globally might not be helpful, as we might have cases, where we cannot give the guarantee about no protected properties being part of the package(s).
{noformat}
at org.apache.commons.collections4.iterators.FilterIterator.hasNext(FilterIterator.java:105)
at org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.getDeclaredPropertyDefinitions(NodeTypeImpl.java:189)
at org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.internalGetPropertyDefinitions(NodeTypeImpl.java:521)
at org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.getPropertyDefinitions(NodeTypeImpl.java:316)
at org.apache.jackrabbit.vault.util.EffectiveNodeType.lambda$getApplicablePropertyDefinition$1(EffectiveNodeType.java:111)
at org.apache.jackrabbit.vault.util.EffectiveNodeType$$Lambda/0x000000080194cb00.apply(Unknown Source)
at java.util.stream.ReferencePipeline$7$1.accept(java.base@21.0.2/ReferencePipeline.java:273)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(java.base@21.0.2/ArrayList.java:1708)
at java.util.stream.AbstractPipeline.copyInto(java.base@21.0.2/AbstractPipeline.java:509)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@21.0.2/AbstractPipeline.java:499)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(java.base@21.0.2/ReduceOps.java:921)
at java.util.stream.AbstractPipeline.evaluate(java.base@21.0.2/AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(java.base@21.0.2/ReferencePipeline.java:682)
at org.apache.jackrabbit.vault.util.EffectiveNodeType.getApplicablePropertyDefinition(EffectiveNodeType.java:111)
at org.apache.jackrabbit.vault.util.EffectiveNodeType.getApplicablePropertyDefinition(EffectiveNodeType.java:107)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.isPropertyProtected(DocViewImporter.java:1235)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.lambda$logIgnoredProtectedProperties$7(DocViewImporter.java:1217)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter$$Lambda/0x000000080194c688.accept(Unknown Source)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(java.base@21.0.2/ForEachOps.java:184)
at java.util.stream.ReferencePipeline$2$1.accept(java.base@21.0.2/ReferencePipeline.java:179)
at java.util.Iterator.forEachRemaining(java.base@21.0.2/Iterator.java:133)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(java.base@21.0.2/Spliterators.java:1939)
at java.util.stream.AbstractPipeline.copyInto(java.base@21.0.2/AbstractPipeline.java:509)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@21.0.2/AbstractPipeline.java:499)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(java.base@21.0.2/ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(java.base@21.0.2/ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(java.base@21.0.2/AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(java.base@21.0.2/ReferencePipeline.java:596)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.logIgnoredProtectedProperties(DocViewImporter.java:1215)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.createNewNode(DocViewImporter.java:1181)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.addNode(DocViewImporter.java:931)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.startDocViewNode(DocViewImporter.java:410)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewSAXHandler.startElement(DocViewSAXHandler.java:353)
{noformat}
Alternatively I would also be happy about an optimized version of "DocViewImporter.isPropertyProtected", which does not need resolve every property all the time from scratch.
Note: Optimizing {{isPropertyProtected}} might be the prefered way, as it is also invoked via the {{DocViewImporter.setUnprotectedProperties}} code path, which I also see very frequently in my dumps.
was:
In the case of Sling distribution the packages being created and imported are controlled by the framework, and IIUC protected properties are not part of the package; for that reason {{DocViewImporter.logIgnoredProtectedProperties}} will not write any WARN message.
When checking the details of the import of a complex package (with almost 100k nodes and low to medium amount of properties per node), I came frequently across this type of stack, where it's checked if properties are protected. Commenting the 2 calls to {{DocViewImporter.logIgnoredProtectedProperties}} gave me an improvement of the import time in the range of 10%.
For that I think that we should provide a chance to disable these checks, preferably via the API, so that it can configured per package I want to import. Disabling it globally might not be helpful, as we might have cases, where we cannot give the guarantee about no protected properties being part of the package(s).
{noformat}
at org.apache.commons.collections4.iterators.FilterIterator.hasNext(FilterIterator.java:105)
at org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.getDeclaredPropertyDefinitions(NodeTypeImpl.java:189)
at org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.internalGetPropertyDefinitions(NodeTypeImpl.java:521)
at org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.getPropertyDefinitions(NodeTypeImpl.java:316)
at org.apache.jackrabbit.vault.util.EffectiveNodeType.lambda$getApplicablePropertyDefinition$1(EffectiveNodeType.java:111)
at org.apache.jackrabbit.vault.util.EffectiveNodeType$$Lambda/0x000000080194cb00.apply(Unknown Source)
at java.util.stream.ReferencePipeline$7$1.accept(java.base@21.0.2/ReferencePipeline.java:273)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(java.base@21.0.2/ArrayList.java:1708)
at java.util.stream.AbstractPipeline.copyInto(java.base@21.0.2/AbstractPipeline.java:509)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@21.0.2/AbstractPipeline.java:499)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(java.base@21.0.2/ReduceOps.java:921)
at java.util.stream.AbstractPipeline.evaluate(java.base@21.0.2/AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(java.base@21.0.2/ReferencePipeline.java:682)
at org.apache.jackrabbit.vault.util.EffectiveNodeType.getApplicablePropertyDefinition(EffectiveNodeType.java:111)
at org.apache.jackrabbit.vault.util.EffectiveNodeType.getApplicablePropertyDefinition(EffectiveNodeType.java:107)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.isPropertyProtected(DocViewImporter.java:1235)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.lambda$logIgnoredProtectedProperties$7(DocViewImporter.java:1217)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter$$Lambda/0x000000080194c688.accept(Unknown Source)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(java.base@21.0.2/ForEachOps.java:184)
at java.util.stream.ReferencePipeline$2$1.accept(java.base@21.0.2/ReferencePipeline.java:179)
at java.util.Iterator.forEachRemaining(java.base@21.0.2/Iterator.java:133)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(java.base@21.0.2/Spliterators.java:1939)
at java.util.stream.AbstractPipeline.copyInto(java.base@21.0.2/AbstractPipeline.java:509)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@21.0.2/AbstractPipeline.java:499)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(java.base@21.0.2/ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(java.base@21.0.2/ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(java.base@21.0.2/AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(java.base@21.0.2/ReferencePipeline.java:596)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.logIgnoredProtectedProperties(DocViewImporter.java:1215)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.createNewNode(DocViewImporter.java:1181)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.addNode(DocViewImporter.java:931)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.startDocViewNode(DocViewImporter.java:410)
at org.apache.jackrabbit.vault.fs.impl.io.DocViewSAXHandler.startElement(DocViewSAXHandler.java:353)
{noformat}
Alternatively I would also be happy about an optimized version of "DocViewImporter.isPropertyProtected", which does not need resolve every property all the time from scratch.
Note: Optimizing {{isPropertyProtected}} might be the prefered way, as it is also invoked via the {{DocViewImporter.setUnprotectedProperties}} code path, which I also see very frequently in my dumps.
> DocViewImporter: logIgnoredProtectedProperties can slow down package import
> ---------------------------------------------------------------------------
>
> Key: JCRVLT-809
> URL: https://issues.apache.org/jira/browse/JCRVLT-809
> Project: Jackrabbit FileVault
> Issue Type: Improvement
> Components: Packaging
> Reporter: Joerg Hoh
> Assignee: Joerg Hoh
> Priority: Major
> Fix For: 4.0.0
>
>
> *NOTE*: I leave this initial idea as is, but the improvement is done differently; see the comments below and the Github PR.
> In the case of Sling distribution the packages being created and imported are controlled by the framework, and IIUC protected properties are not part of the package; for that reason {{DocViewImporter.logIgnoredProtectedProperties}} will not write any WARN message.
> When checking the details of the import of a complex package (with almost 100k nodes and low to medium amount of properties per node), I came frequently across this type of stack, where it's checked if properties are protected. Commenting the 2 calls to {{DocViewImporter.logIgnoredProtectedProperties}} gave me an improvement of the import time in the range of 10%.
> For that I think that we should provide a chance to disable these checks, preferably via the API, so that it can configured per package I want to import. Disabling it globally might not be helpful, as we might have cases, where we cannot give the guarantee about no protected properties being part of the package(s).
> {noformat}
> at org.apache.commons.collections4.iterators.FilterIterator.hasNext(FilterIterator.java:105)
> at org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.getDeclaredPropertyDefinitions(NodeTypeImpl.java:189)
> at org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.internalGetPropertyDefinitions(NodeTypeImpl.java:521)
> at org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.getPropertyDefinitions(NodeTypeImpl.java:316)
> at org.apache.jackrabbit.vault.util.EffectiveNodeType.lambda$getApplicablePropertyDefinition$1(EffectiveNodeType.java:111)
> at org.apache.jackrabbit.vault.util.EffectiveNodeType$$Lambda/0x000000080194cb00.apply(Unknown Source)
> at java.util.stream.ReferencePipeline$7$1.accept(java.base@21.0.2/ReferencePipeline.java:273)
> at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(java.base@21.0.2/ArrayList.java:1708)
> at java.util.stream.AbstractPipeline.copyInto(java.base@21.0.2/AbstractPipeline.java:509)
> at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@21.0.2/AbstractPipeline.java:499)
> at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(java.base@21.0.2/ReduceOps.java:921)
> at java.util.stream.AbstractPipeline.evaluate(java.base@21.0.2/AbstractPipeline.java:234)
> at java.util.stream.ReferencePipeline.collect(java.base@21.0.2/ReferencePipeline.java:682)
> at org.apache.jackrabbit.vault.util.EffectiveNodeType.getApplicablePropertyDefinition(EffectiveNodeType.java:111)
> at org.apache.jackrabbit.vault.util.EffectiveNodeType.getApplicablePropertyDefinition(EffectiveNodeType.java:107)
> at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.isPropertyProtected(DocViewImporter.java:1235)
> at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.lambda$logIgnoredProtectedProperties$7(DocViewImporter.java:1217)
> at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter$$Lambda/0x000000080194c688.accept(Unknown Source)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(java.base@21.0.2/ForEachOps.java:184)
> at java.util.stream.ReferencePipeline$2$1.accept(java.base@21.0.2/ReferencePipeline.java:179)
> at java.util.Iterator.forEachRemaining(java.base@21.0.2/Iterator.java:133)
> at java.util.Spliterators$IteratorSpliterator.forEachRemaining(java.base@21.0.2/Spliterators.java:1939)
> at java.util.stream.AbstractPipeline.copyInto(java.base@21.0.2/AbstractPipeline.java:509)
> at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@21.0.2/AbstractPipeline.java:499)
> at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(java.base@21.0.2/ForEachOps.java:151)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(java.base@21.0.2/ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(java.base@21.0.2/AbstractPipeline.java:234)
> at java.util.stream.ReferencePipeline.forEach(java.base@21.0.2/ReferencePipeline.java:596)
> at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.logIgnoredProtectedProperties(DocViewImporter.java:1215)
> at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.createNewNode(DocViewImporter.java:1181)
> at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.addNode(DocViewImporter.java:931)
> at org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.startDocViewNode(DocViewImporter.java:410)
> at org.apache.jackrabbit.vault.fs.impl.io.DocViewSAXHandler.startElement(DocViewSAXHandler.java:353)
> {noformat}
> Alternatively I would also be happy about an optimized version of "DocViewImporter.isPropertyProtected", which does not need resolve every property all the time from scratch.
> Note: Optimizing {{isPropertyProtected}} might be the prefered way, as it is also invoked via the {{DocViewImporter.setUnprotectedProperties}} code path, which I also see very frequently in my dumps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)